vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #431
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client: QEMU running (pid 49)14client # Formatting '/build/vm-state-client/tmp.cpQNTEpJIo', fmt=raw size=107374182415client # mke2fs 1.47.4 (6-Mar-2025)16client # Discarding device blocks: 0/262144 done17client # Creating filesystem with 262144 4k blocks and 65536 inodes18client # Filesystem UUID: 24e4ca9e-b8f8-476a-a338-54b19bb8c0ba19client # Superblock backups stored on blocks:20client # 32768, 98304, 163840, 22937621client # 22client # Allocating group tables: 0/8 done23client # Writing inode tables: 0/8 done24client # Creating journal (8192 blocks): done25client # Writing superblocks and filesystem accounting information: 0/8 done26client # 27worker2 # Disk image does not exist, creating the virtualisation disk image...28client # Virtualisation disk image created.29worker2 # Formatting '/build/vm-state-worker2/tmp.Ngy8f6n06k', fmt=raw size=107374182430worker2: QEMU running (pid 52)31worker2 # mke2fs 1.47.4 (6-Mar-2025)32worker2 # Discarding device blocks: 0/262144 done33worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes34worker2 # Filesystem UUID: 856b4ff4-2ac4-40cf-b6e1-1302eacb97c135worker2 # Superblock backups stored on blocks:36worker2 # 32768, 98304, 163840, 22937637worker2 # 38worker2 # Allocating group tables: 0/8 done39worker2 # Writing inode tables: 0/8 done40worker2 # Creating journal (8192 blocks): done41worker2 # Writing superblocks and filesystem accounting information: 0/8 done42worker2 # 43worker2 # Virtualisation disk image created.44worker1 # Disk image does not exist, creating the virtualisation disk image...45worker1: QEMU running (pid 50)46worker1 # Formatting '/build/vm-state-worker1/tmp.UFKWdFKoFg', fmt=raw size=107374182447worker1 # mke2fs 1.47.4 (6-Mar-2025)48worker1 # Discarding device blocks: 0/262144 done49worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes50worker1 # Filesystem UUID: 576e6f91-39e4-41e3-bdc8-9a532913ad6651lb: QEMU running (pid 51)52lb # Disk image does not exist, creating the virtualisation disk image...53worker1 # Superblock backups stored on blocks:54worker1 # 32768, 98304, 163840, 22937655lb # Formatting '/build/vm-state-lb/tmp.anjrOsYpOa', fmt=raw size=107374182456worker1 # 57lb # mke2fs 1.47.4 (6-Mar-2025)58worker1 # Allocating group tables: 0/8 done59lb # Discarding device blocks: 0/262144 done60worker1 # Writing inode tables: 0/8 done61lb # Creating filesystem with 262144 4k blocks and 65536 inodes62worker1 # Creating journal (8192 blocks): done63lb # Filesystem UUID: 2b64e7b9-c22e-4f4f-a368-7fcc96e9075164worker1 # Writing superblocks and filesystem accounting information: 0/8 done65lb # Superblock backups stored on blocks:66worker1 # 67lb # 32768, 98304, 163840, 22937668worker1 # Virtualisation disk image created.69lb # 70lb # Allocating group tables: 0/8 done71lb # Writing inode tables: 0/8 done72lb # Creating journal (8192 blocks): done73lb # Writing superblocks and filesystem accounting information: 0/8 done74lb # 75lb # Virtualisation disk image created.76(finished: start all VMs, in 0.23 seconds)77client: waiting for unit network-addresses-eth1.service78client: waiting for the VM to finish booting79client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83client # 84client # 85client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87client # 88client # 89client # 90client # 91worker2 # 92worker2 # 93client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0094worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0095lb # 96lb # 97worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...98worker2 # 99worker2 # 100worker2 # 101worker2 # 102client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...103client # 104client # 105lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00106worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00107lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108lb # 109lb # 110lb # 111lb # 112worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...113worker2 # 114worker2 # 115lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117lb # 118lb # 119worker1 # 120worker1 # 121worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123worker1 # 124worker1 # 125worker1 # 126worker1 # 127worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128client # Booting from ROM...129worker2 # Booting from ROM...130worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...131worker1 # 132worker1 # 133lb # Booting from ROM...134worker1 # Booting from ROM...135worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ypgsyd3xa1k1g9gyx6jjkphbn88igsr2-nixos-system-worker2-test/init regInfo=/nix/store/fffk2m7l3xw80vwc35mj5kbycghkcn0m-closure-info/registration console=ttyS0,115200n8 console=tty0137worker2 # [ 0.000000] BIOS-provided physical RAM map:138worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148worker2 # [ 0.000000] NX (Execute Disable) protection: active149worker2 # [ 0.000000] APIC: Static calls initialized150worker2 # [ 0.000000] SMBIOS 2.8 present.151client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026152worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014153worker2 # [ 0.000000] DMI: Memory slots populated: 1/1154worker2 # [ 0.000000] Hypervisor detected: KVM155worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000156worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00157worker2 # [ 0.000000] kvm-clock: using sched offset of 471373773 cycles158client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test/init regInfo=/nix/store/2ak3y085bh8fzf8b25fqh92fkzpsvy3w-closure-info/registration console=ttyS0,115200n8 console=tty0159client # [ 0.000000] BIOS-provided physical RAM map:160worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns161client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable162worker2 # [ 0.000004] tsc: Detected 2400.012 MHz processor163client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved164worker2 # [ 0.000803] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000165client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved166worker2 # [ 0.000830] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs167client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable168worker2 # [ 0.000832] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT169worker2 # [ 0.002723] found SMP MP-table at [mem 0x000f5450-0x000f545f]170client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved171worker2 # [ 0.002733] Using GB pages for direct mapping172client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved173worker2 # [ 0.002878] RAMDISK: [mem 0x5e367000-0x5ffcffff]174worker2 # [ 0.002887] ACPI: Early table checksum verification disabled175client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved176worker2 # [ 0.002889] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )177client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved178worker2 # [ 0.002892] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)179client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved180client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved181worker2 # [ 0.002896] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)182client # [ 0.000000] NX (Execute Disable) protection: active183worker2 # [ 0.002904] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)184client # [ 0.000000] APIC: Static calls initialized185client # [ 0.000000] SMBIOS 2.8 present.186worker2 # [ 0.002906] ACPI: FACS 0x000000005FFE0000 000040187worker2 # [ 0.002907] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)188client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014189client # [ 0.000000] DMI: Memory slots populated: 1/1190worker2 # [ 0.002909] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)191client # [ 0.000000] Hypervisor detected: KVM192client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000193worker2 # [ 0.002910] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)194client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00195worker2 # [ 0.002912] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)196client # [ 0.000001] kvm-clock: using sched offset of 467891780 cycles197worker2 # [ 0.002913] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]198lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026199client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns200worker2 # [ 0.002914] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]201client # [ 0.000005] tsc: Detected 2400.012 MHz processor202worker2 # [ 0.002915] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]203client # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000204worker2 # [ 0.002915] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]205client # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs206worker2 # [ 0.002916] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]207client # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT208lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m097agrx0jinhy67cn4afbshq893g59l-nixos-system-lb-test/init regInfo=/nix/store/6ryicbywj6r050v0hx7xqbv6gfi18m0f-closure-info/registration console=ttyS0,115200n8 console=tty0209worker2 # [ 0.002916] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]210client # [ 0.002741] found SMP MP-table at [mem 0x000f5450-0x000f545f]211lb # [ 0.000000] BIOS-provided physical RAM map:212client # [ 0.002752] Using GB pages for direct mapping213worker2 # [ 0.002916] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]214client # [ 0.002874] RAMDISK: [mem 0x5e366000-0x5ffcffff]215lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable216worker2 # [ 0.003141] No NUMA configuration found217client # [ 0.002883] ACPI: Early table checksum verification disabled218worker2 # [ 0.003142] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]219lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved220client # [ 0.002886] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )221worker2 # [ 0.003144] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]222lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved223worker2 # [ 0.005328] Zone ranges:224client # [ 0.002890] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)225lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable226worker2 # [ 0.005328] DMA [mem 0x0000000000001000-0x0000000000ffffff]227worker2 # [ 0.005330] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]228client # [ 0.002894] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)229lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved230worker2 # [ 0.005331] Normal empty231worker2 # [ 0.005332] Device empty232lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved233client # [ 0.002902] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)234worker2 # [ 0.005332] Movable zone start for each node235worker2 # [ 0.005333] Early memory node ranges236client # [ 0.002904] ACPI: FACS 0x000000005FFE0000 000040237lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved238worker2 # [ 0.005333] node 0: [mem 0x0000000000001000-0x000000000009efff]239lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved240client # [ 0.002905] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)241worker2 # [ 0.005334] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]242lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved243client # [ 0.002907] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)244worker2 # [ 0.005335] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]245lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved246worker2 # [ 0.005354] On node 0, zone DMA: 1 pages in unavailable ranges247client # [ 0.002908] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)248lb # [ 0.000000] NX (Execute Disable) protection: active249worker2 # [ 0.005608] On node 0, zone DMA: 97 pages in unavailable ranges250lb # [ 0.000000] APIC: Static calls initialized251lb # [ 0.000000] SMBIOS 2.8 present.252client # [ 0.002910] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)253worker2 # [ 0.033181] On node 0, zone DMA32: 40 pages in unavailable ranges254worker2 # [ 0.033641] ACPI: PM-Timer IO Port: 0x608255client # [ 0.002911] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]256lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014257worker2 # [ 0.033652] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])258client # [ 0.002912] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]259lb # [ 0.000000] DMI: Memory slots populated: 1/1260lb # [ 0.000000] Hypervisor detected: KVM261worker2 # [ 0.033678] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23262client # [ 0.002913] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]263lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000264worker2 # [ 0.033680] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)265client # [ 0.002913] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]266lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00267worker2 # [ 0.033682] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)268lb # [ 0.000000] kvm-clock: using sched offset of 478824065 cycles269client # [ 0.002914] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]270worker2 # [ 0.033683] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)271client # [ 0.002914] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]272worker2 # [ 0.033684] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)273lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns274client # [ 0.002915] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]275lb # [ 0.000005] tsc: Detected 2400.012 MHz processor276worker2 # [ 0.033685] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)277client # [ 0.003146] No NUMA configuration found278lb # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000279worker2 # [ 0.033687] ACPI: Using ACPI (MADT) for SMP configuration information280client # [ 0.003147] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]281worker2 # [ 0.033688] ACPI: HPET id: 0x8086a201 base: 0xfed00000282lb # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs283client # [ 0.003150] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]284worker2 # [ 0.033691] TSC deadline timer available285client # [ 0.005956] Zone ranges:286lb # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT287worker2 # [ 0.033695] CPU topo: Max. logical packages: 1288client # [ 0.005957] DMA [mem 0x0000000000001000-0x0000000000ffffff]289lb # [ 0.002734] found SMP MP-table at [mem 0x000f5450-0x000f545f]290worker2 # [ 0.033696] CPU topo: Max. logical dies: 1291client # [ 0.005959] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]292lb # [ 0.002746] Using GB pages for direct mapping293worker2 # [ 0.033697] CPU topo: Max. dies per package: 1294client # [ 0.005960] Normal empty295lb # [ 0.002835] RAMDISK: [mem 0x5e367000-0x5ffcffff]296worker2 # [ 0.033700] CPU topo: Max. threads per core: 1297client # [ 0.005961] Device empty298worker2 # [ 0.033700] CPU topo: Num. cores per package: 1299client # [ 0.005961] Movable zone start for each node300lb # [ 0.002843] ACPI: Early table checksum verification disabled301worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026302client # [ 0.005962] Early memory node ranges303worker2 # [ 0.033701] CPU topo: Num. threads per package: 1304lb # [ 0.002847] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )305client # [ 0.005962] node 0: [mem 0x0000000000001000-0x000000000009efff]306worker2 # [ 0.033701] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs307lb # [ 0.002851] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)308client # [ 0.005963] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]309worker2 # [ 0.033719] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()310client # [ 0.005965] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]311lb # [ 0.002855] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)312worker2 # [ 0.033748] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]313client # [ 0.005984] On node 0, zone DMA: 1 pages in unavailable ranges314lb # [ 0.002862] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)315worker2 # [ 0.033750] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]316client # [ 0.006258] On node 0, zone DMA: 97 pages in unavailable ranges317lb # [ 0.002864] ACPI: FACS 0x000000005FFE0000 000040318worker2 # [ 0.033751] [mem 0x60000000-0xafffffff] available for PCI devices319client # [ 0.034640] On node 0, zone DMA32: 40 pages in unavailable ranges320worker2 # [ 0.033753] Booting paravirtualized kernel on KVM321worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6bxdm9hq3wgx5n86ind647zs9wws5ym4-nixos-system-worker1-test/init regInfo=/nix/store/gknxg845hhwh6hyrwy765h8p7n4wy4an-closure-info/registration console=ttyS0,115200n8 console=tty0322client # [ 0.035100] ACPI: PM-Timer IO Port: 0x608323lb # [ 0.002866] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)324client # [ 0.035112] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])325worker1 # [ 0.000000] BIOS-provided physical RAM map:326lb # [ 0.002867] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)327worker2 # [ 0.033755] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns328client # [ 0.035141] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23329worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable330lb # [ 0.002869] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)331worker2 # [ 0.038207] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1332client # [ 0.035145] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)333worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved334worker2 # [ 0.040594] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152335lb # [ 0.002870] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)336client # [ 0.035146] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)337worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved338worker2 # [ 0.040636] kvm-guest: PV spinlocks disabled, single CPU339client # [ 0.035148] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)340lb # [ 0.002872] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]341worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable342client # [ 0.035149] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)343lb # [ 0.002873] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]344worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved345client # [ 0.035149] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)346lb # [ 0.002873] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]347worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved348client # [ 0.035152] ACPI: Using ACPI (MADT) for SMP configuration information349lb # [ 0.002874] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]350worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved351client # [ 0.035153] ACPI: HPET id: 0x8086a201 base: 0xfed00000352lb # [ 0.002874] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]353worker2 # [ 0.040638] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ypgsyd3xa1k1g9gyx6jjkphbn88igsr2-nixos-system-worker2-test/init regInfo=/nix/store/fffk2m7l3xw80vwc35mj5kbycghkcn0m-closure-info/registration console=ttyS0,115200n8 console=tty0354worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved355client # [ 0.035156] TSC deadline timer available356client # [ 0.035160] CPU topo: Max. logical packages: 1357lb # [ 0.002875] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]358worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved359client # [ 0.035161] CPU topo: Max. logical dies: 1360lb # [ 0.002875] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]361worker2 # [ 0.040728] Unknown kernel command line parameters "regInfo=/nix/store/fffk2m7l3xw80vwc35mj5kbycghkcn0m-closure-info/registration", will be passed to user space.362worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved363client # [ 0.035161] CPU topo: Max. dies per package: 1364worker2 # [ 0.040741] random: crng init done365lb # [ 0.003109] No NUMA configuration found366worker1 # [ 0.000000] NX (Execute Disable) protection: active367client # [ 0.035165] CPU topo: Max. threads per core: 1368worker1 # [ 0.000000] APIC: Static calls initialized369client # [ 0.035165] CPU topo: Num. cores per package: 1370lb # [ 0.003110] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]371worker2 # [ 0.040741] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes372worker1 # [ 0.000000] SMBIOS 2.8 present.373client # [ 0.035166] CPU topo: Num. threads per package: 1374lb # [ 0.003112] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]375worker2 # [ 0.042865] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)376lb # [ 0.005476] Zone ranges:377client # [ 0.035166] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs378worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014379lb # [ 0.005477] DMA [mem 0x0000000000001000-0x0000000000ffffff]380worker2 # [ 0.043966] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)381worker1 # [ 0.000000] DMI: Memory slots populated: 1/1382client # [ 0.035187] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()383worker2 # [ 0.043996] Fallback order for Node 0: 0384lb # [ 0.005479] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]385worker1 # [ 0.000000] Hypervisor detected: KVM386lb # [ 0.005480] Normal empty387client # [ 0.035219] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]388worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000389worker2 # [ 0.043998] Built 1 zonelists, mobility grouping on. Total pages: 393078390lb # [ 0.005481] Device empty391worker2 # [ 0.043999] Policy zone: DMA32392lb # [ 0.005481] Movable zone start for each node393worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00394client # [ 0.035221] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]395lb # [ 0.005482] Early memory node ranges396worker2 # [ 0.046440] mem auto-init: stack:all(zero), heap alloc:on, heap free:off397worker1 # [ 0.000001] kvm-clock: using sched offset of 525299404 cycles398client # [ 0.035222] [mem 0x60000000-0xafffffff] available for PCI devices399lb # [ 0.005483] node 0: [mem 0x0000000000001000-0x000000000009efff]400worker2 # [ 0.049503] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1401client # [ 0.035224] Booting paravirtualized kernel on KVM402lb # [ 0.005484] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]403worker2 # [ 0.052826] allocated 3145728 bytes of page_ext404worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns405worker2 # [ 0.062237] ftrace: allocating 48731 entries in 192 pages406worker1 # [ 0.000006] tsc: Detected 2400.012 MHz processor407lb # [ 0.005485] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]408client # [ 0.035227] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns409worker1 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000410worker2 # [ 0.062239] ftrace: allocated 192 pages with 2 groups411lb # [ 0.005506] On node 0, zone DMA: 1 pages in unavailable ranges412worker2 # [ 0.063060] Dynamic Preempt: lazy413client # [ 0.039733] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1414lb # [ 0.005782] On node 0, zone DMA: 97 pages in unavailable ranges415worker1 # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs416worker2 # [ 0.063217] rcu: Preemptible hierarchical RCU implementation.417worker2 # [ 0.063217] rcu: RCU event tracing is enabled.418lb # [ 0.034249] On node 0, zone DMA32: 40 pages in unavailable ranges419client # [ 0.042029] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152420client # [ 0.042076] kvm-guest: PV spinlocks disabled, single CPU421worker1 # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT422lb # [ 0.034716] ACPI: PM-Timer IO Port: 0x608423worker2 # [ 0.063218] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.424worker1 # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]425lb # [ 0.034730] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])426worker2 # [ 0.063219] Trampoline variant of Tasks RCU enabled.427worker1 # [ 0.002742] Using GB pages for direct mapping428worker2 # [ 0.063220] Rude variant of Tasks RCU enabled.429worker1 # [ 0.002818] RAMDISK: [mem 0x5e367000-0x5ffcffff]430lb # [ 0.034757] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23431worker2 # [ 0.063220] Tracing variant of Tasks RCU enabled.432worker1 # [ 0.002826] ACPI: Early table checksum verification disabled433lb # [ 0.034761] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)434worker1 # [ 0.002830] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )435worker2 # [ 0.063221] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.436lb # [ 0.034762] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)437client # [ 0.042078] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test/init regInfo=/nix/store/2ak3y085bh8fzf8b25fqh92fkzpsvy3w-closure-info/registration console=ttyS0,115200n8 console=tty0438worker2 # [ 0.063222] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1439worker1 # [ 0.002834] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)440lb # [ 0.034764] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)441lb # [ 0.034765] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)442worker2 # [ 0.063242] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.443worker1 # [ 0.002838] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)444client # [ 0.042174] Unknown kernel command line parameters "regInfo=/nix/store/2ak3y085bh8fzf8b25fqh92fkzpsvy3w-closure-info/registration", will be passed to user space.445lb # [ 0.034765] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)446client # [ 0.042186] random: crng init done447worker2 # [ 0.063243] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.448worker1 # [ 0.002846] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)449lb # [ 0.034768] ACPI: Using ACPI (MADT) for SMP configuration information450worker1 # [ 0.002848] ACPI: FACS 0x000000005FFE0000 000040451client # [ 0.042187] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes452lb # [ 0.034769] ACPI: HPET id: 0x8086a201 base: 0xfed00000453worker2 # [ 0.063244] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.454lb # [ 0.034774] TSC deadline timer available455worker1 # [ 0.002849] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)456client # [ 0.044429] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)457worker2 # [ 0.067534] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16458lb # [ 0.034779] CPU topo: Max. logical packages: 1459client # [ 0.045544] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)460lb # [ 0.034780] CPU topo: Max. logical dies: 1461worker1 # [ 0.002851] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)462worker2 # [ 0.067806] rcu: srcu_init: Setting srcu_struct sizes based on contention.463client # [ 0.045588] Fallback order for Node 0: 0464lb # [ 0.034780] CPU topo: Max. dies per package: 1465lb # [ 0.034784] CPU topo: Max. threads per core: 1466worker1 # [ 0.002852] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)467worker2 # [ 0.067812] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns468client # [ 0.045591] Built 1 zonelists, mobility grouping on. Total pages: 393078469lb # [ 0.034785] CPU topo: Num. cores per package: 1470client # [ 0.045592] Policy zone: DMA32471worker1 # [ 0.002854] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)472lb # [ 0.034785] CPU topo: Num. threads per package: 1473worker2 # [ 0.067967] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)474client # [ 0.048512] mem auto-init: stack:all(zero), heap alloc:on, heap free:off475worker2 # [ 0.071480] Console: colour VGA+ 80x25476worker1 # [ 0.002855] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]477lb # [ 0.034785] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs478client # [ 0.051848] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1479worker2 # [ 0.071484] printk: legacy console [tty0] enabled480client # [ 0.055378] allocated 3145728 bytes of page_ext481worker1 # [ 0.002856] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]482lb # [ 0.034810] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()483worker2 # [ 0.100897] printk: legacy console [ttyS0] enabled484client # [ 0.065736] ftrace: allocating 48731 entries in 192 pages485worker1 # [ 0.002857] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]486worker2 # [ 0.196917] ACPI: Core revision 20250807487lb # [ 0.034849] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]488client # [ 0.065739] ftrace: allocated 192 pages with 2 groups489client # [ 0.066646] Dynamic Preempt: lazy490worker1 # [ 0.002857] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]491lb # [ 0.034851] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]492client # [ 0.066825] rcu: Preemptible hierarchical RCU implementation.493worker1 # [ 0.002858] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]494lb # [ 0.034852] [mem 0x60000000-0xafffffff] available for PCI devices495worker2 # [ 0.197818] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns496client # [ 0.066826] rcu: RCU event tracing is enabled.497lb # [ 0.034854] Booting paravirtualized kernel on KVM498worker1 # [ 0.002858] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]499worker2 # [ 0.199298] APIC: Switch to symmetric I/O mode setup500client # [ 0.066827] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.501worker2 # [ 0.200213] x2apic enabled502worker1 # [ 0.002859] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]503client # [ 0.066828] Trampoline variant of Tasks RCU enabled.504lb # [ 0.034858] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns505worker1 # [ 0.003087] No NUMA configuration found506client # [ 0.066829] Rude variant of Tasks RCU enabled.507lb # [ 0.039337] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1508worker2 # [ 0.200942] APIC: Switched APIC routing to: physical x2apic509worker1 # [ 0.003088] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]510client # [ 0.066829] Tracing variant of Tasks RCU enabled.511lb # [ 0.041624] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152512worker1 # [ 0.003091] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]513worker1 # [ 0.005465] Zone ranges:514client # [ 0.066830] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.515lb # [ 0.041667] kvm-guest: PV spinlocks disabled, single CPU516worker2 # [ 0.202701] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1517client # [ 0.066831] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1518worker1 # [ 0.005467] DMA [mem 0x0000000000001000-0x0000000000ffffff]519worker1 # [ 0.005469] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]520client # [ 0.066852] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.521worker1 # [ 0.005470] Normal empty522worker1 # [ 0.005471] Device empty523worker2 # [ 0.203680] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns524worker1 # [ 0.005471] Movable zone start for each node525client # [ 0.066854] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.526worker1 # [ 0.005472] Early memory node ranges527worker2 # [ 0.206282] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)528worker1 # [ 0.005473] node 0: [mem 0x0000000000001000-0x000000000009efff]529lb # [ 0.041669] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m097agrx0jinhy67cn4afbshq893g59l-nixos-system-lb-test/init regInfo=/nix/store/6ryicbywj6r050v0hx7xqbv6gfi18m0f-closure-info/registration console=ttyS0,115200n8 console=tty0530client # [ 0.066855] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.531worker1 # [ 0.005474] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]532client # [ 0.071405] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16533worker2 # [ 0.207368] x86/cpu: User Mode Instruction Prevention (UMIP) activated534worker1 # [ 0.005475] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]535client # [ 0.071687] rcu: srcu_init: Setting srcu_struct sizes based on contention.536lb # [ 0.041769] Unknown kernel command line parameters "regInfo=/nix/store/6ryicbywj6r050v0hx7xqbv6gfi18m0f-closure-info/registration", will be passed to user space.537worker2 # [ 0.208436] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127538lb # [ 0.041782] random: crng init done539worker1 # [ 0.005495] On node 0, zone DMA: 1 pages in unavailable ranges540client # [ 0.071694] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns541worker1 # [ 0.005767] On node 0, zone DMA: 97 pages in unavailable ranges542lb # [ 0.041783] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes543worker2 # [ 0.209282] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0544worker1 # [ 0.033945] On node 0, zone DMA32: 40 pages in unavailable ranges545client # [ 0.071803] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)546lb # [ 0.044023] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)547worker1 # [ 0.034407] ACPI: PM-Timer IO Port: 0x608548client # [ 0.075424] Console: colour VGA+ 80x25549worker2 # [ 0.210285] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto550worker1 # [ 0.034420] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])551lb # [ 0.045457] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)552client # [ 0.075427] printk: legacy console [tty0] enabled553lb # [ 0.045490] Fallback order for Node 0: 0554client # [ 0.104904] printk: legacy console [ttyS0] enabled555worker1 # [ 0.034451] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23556worker2 # [ 0.211282] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl557lb # [ 0.045493] Built 1 zonelists, mobility grouping on. Total pages: 393078558worker1 # [ 0.034454] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)559client # [ 0.202051] ACPI: Core revision 20250807560lb # [ 0.045494] Policy zone: DMA32561worker2 # [ 0.212283] Transient Scheduler Attacks: Vulnerable: No microcode562worker1 # [ 0.034456] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)563lb # [ 0.048498] mem auto-init: stack:all(zero), heap alloc:on, heap free:off564worker1 # [ 0.034457] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)565client # [ 0.202918] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns566worker2 # [ 0.213282] Spectre V2 : Mitigation: Enhanced / Automatic IBRS567lb # [ 0.052016] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1568worker1 # [ 0.034458] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)569lb # [ 0.055573] allocated 3145728 bytes of page_ext570client # [ 0.204411] APIC: Switch to symmetric I/O mode setup571worker2 # [ 0.214282] Speculative Return Stack Overflow: Mitigation: Safe RET572lb # [ 0.065918] ftrace: allocating 48731 entries in 192 pages573worker1 # [ 0.034459] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)574client # [ 0.205385] x2apic enabled575lb # [ 0.065921] ftrace: allocated 192 pages with 2 groups576worker1 # [ 0.034461] ACPI: Using ACPI (MADT) for SMP configuration information577lb # [ 0.066808] Dynamic Preempt: lazy578client # [ 0.206164] APIC: Switched APIC routing to: physical x2apic579worker1 # [ 0.034462] ACPI: HPET id: 0x8086a201 base: 0xfed00000580lb # [ 0.066995] rcu: Preemptible hierarchical RCU implementation.581worker1 # [ 0.034466] TSC deadline timer available582lb # [ 0.066996] rcu: RCU event tracing is enabled.583worker1 # [ 0.034471] CPU topo: Max. logical packages: 1584worker1 # [ 0.034472] CPU topo: Max. logical dies: 1585lb # [ 0.066997] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.586client # [ 0.208002] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1587worker2 # [ 0.215282] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization588worker1 # [ 0.034472] CPU topo: Max. dies per package: 1589lb # [ 0.066998] Trampoline variant of Tasks RCU enabled.590worker1 # [ 0.034476] CPU topo: Max. threads per core: 1591lb # [ 0.066999] Rude variant of Tasks RCU enabled.592worker2 # [ 0.217289] Spectre V2 : Enabling IBPB for BPF593worker1 # [ 0.034477] CPU topo: Num. cores per package: 1594lb # [ 0.066999] Tracing variant of Tasks RCU enabled.595client # [ 0.208961] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns596worker1 # [ 0.034477] CPU topo: Num. threads per package: 1597worker2 # [ 0.217996] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier598lb # [ 0.067000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.599worker1 # [ 0.034478] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs600client # [ 0.210606] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)601lb # [ 0.067001] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1602worker2 # [ 0.219282] active return thunk: srso_alias_return_thunk603worker1 # [ 0.034501] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()604client # [ 0.211691] x86/cpu: User Mode Instruction Prevention (UMIP) activated605lb # [ 0.067023] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.606worker1 # [ 0.034537] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]607worker2 # [ 0.220303] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'608client # [ 0.212730] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127609worker1 # [ 0.034538] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]610lb # [ 0.067025] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.611worker2 # [ 0.221282] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'612worker1 # [ 0.034540] [mem 0x60000000-0xafffffff] available for PCI devices613client # [ 0.213603] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0614worker2 # [ 0.222282] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'615lb # [ 0.067026] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.616worker1 # [ 0.034542] Booting paravirtualized kernel on KVM617lb # [ 0.071734] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16618worker2 # [ 0.223282] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'619worker1 # [ 0.034546] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns620client # [ 0.214606] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto621lb # [ 0.072019] rcu: srcu_init: Setting srcu_struct sizes based on contention.622worker2 # [ 0.224282] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'623worker1 # [ 0.039009] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1624lb # [ 0.072026] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns625client # [ 0.216603] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl626worker2 # [ 0.225282] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'627worker1 # [ 0.041290] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152628worker1 # [ 0.041334] kvm-guest: PV spinlocks disabled, single CPU629client # [ 0.217603] Transient Scheduler Attacks: Vulnerable: No microcode630lb # [ 0.072147] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)631worker2 # [ 0.226282] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'632lb # [ 0.075678] Console: colour VGA+ 80x25633client # [ 0.218603] Spectre V2 : Mitigation: Enhanced / Automatic IBRS634lb # [ 0.075682] printk: legacy console [tty0] enabled635worker2 # [ 0.227282] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256636lb # [ 0.105086] printk: legacy console [ttyS0] enabled637client # [ 0.219603] Speculative Return Stack Overflow: Mitigation: Safe RET638worker2 # [ 0.228282] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64639lb # [ 0.201252] ACPI: Core revision 20250807640worker2 # [ 0.229282] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512641worker1 # [ 0.041336] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6bxdm9hq3wgx5n86ind647zs9wws5ym4-nixos-system-worker1-test/init regInfo=/nix/store/gknxg845hhwh6hyrwy765h8p7n4wy4an-closure-info/registration console=ttyS0,115200n8 console=tty0642lb # [ 0.202129] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns643worker2 # [ 0.230282] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024644lb # [ 0.203643] APIC: Switch to symmetric I/O mode setup645worker1 # [ 0.041435] Unknown kernel command line parameters "regInfo=/nix/store/gknxg845hhwh6hyrwy765h8p7n4wy4an-closure-info/registration", will be passed to user space.646worker2 # [ 0.231282] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8647worker1 # [ 0.041448] random: crng init done648lb # [ 0.204631] x2apic enabled649client # [ 0.220603] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization650worker1 # [ 0.041449] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes651client # [ 0.222611] Spectre V2 : Enabling IBPB for BPF652worker2 # [ 0.232282] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.653lb # [ 0.205412] APIC: Switched APIC routing to: physical x2apic654worker1 # [ 0.043719] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)655client # [ 0.223604] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier656worker1 # [ 0.045127] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)657lb # [ 0.207199] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1658worker1 # [ 0.045161] Fallback order for Node 0: 0659client # [ 0.224604] active return thunk: srso_alias_return_thunk660worker1 # [ 0.045164] Built 1 zonelists, mobility grouping on. Total pages: 393078661worker1 # [ 0.045165] Policy zone: DMA32662client # [ 0.225624] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'663lb # [ 0.208191] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns664worker1 # [ 0.047791] mem auto-init: stack:all(zero), heap alloc:on, heap free:off665client # [ 0.226603] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'666worker1 # [ 0.051293] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1667lb # [ 0.209907] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)668worker1 # [ 0.054813] allocated 3145728 bytes of page_ext669client # [ 0.227603] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'670worker1 # [ 0.065031] ftrace: allocating 48731 entries in 192 pages671lb # [ 0.210992] x86/cpu: User Mode Instruction Prevention (UMIP) activated672client # [ 0.228603] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'673worker1 # [ 0.065033] ftrace: allocated 192 pages with 2 groups674worker1 # [ 0.065910] Dynamic Preempt: lazy675client # [ 0.229603] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'676lb # [ 0.212080] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127677worker1 # [ 0.066077] rcu: Preemptible hierarchical RCU implementation.678worker1 # [ 0.066078] rcu: RCU event tracing is enabled.679lb # [ 0.213601] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0680client # [ 0.230603] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'681worker1 # [ 0.066079] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.682worker1 # [ 0.066081] Trampoline variant of Tasks RCU enabled.683client # [ 0.231602] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'684worker1 # [ 0.066081] Rude variant of Tasks RCU enabled.685lb # [ 0.214681] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto686worker1 # [ 0.066081] Tracing variant of Tasks RCU enabled.687client # [ 0.233603] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256688worker1 # [ 0.066082] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.689lb # [ 0.215903] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl690client # [ 0.234547] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64691worker1 # [ 0.066083] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1692lb # [ 0.216905] Transient Scheduler Attacks: Vulnerable: No microcode693client # [ 0.235331] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512694worker1 # [ 0.066098] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.695lb # [ 0.217903] Spectre V2 : Mitigation: Enhanced / Automatic IBRS696client # [ 0.236338] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024697worker1 # [ 0.066100] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.698lb # [ 0.218818] Speculative Return Stack Overflow: Mitigation: Safe RET699client # [ 0.237342] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8700worker1 # [ 0.066101] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.701client # [ 0.238341] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.702worker1 # [ 0.070808] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16703worker1 # [ 0.071097] rcu: srcu_init: Setting srcu_struct sizes based on contention.704worker1 # [ 0.071104] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns705lb # [ 0.219652] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization706lb # [ 0.220912] Spectre V2 : Enabling IBPB for BPF707worker1 # [ 0.071212] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)708worker1 # [ 0.074800] Console: colour VGA+ 80x25709worker1 # [ 0.074802] printk: legacy console [tty0] enabled710lb # [ 0.222442] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier711worker1 # [ 0.104293] printk: legacy console [ttyS0] enabled712lb # [ 0.222904] active return thunk: srso_alias_return_thunk713worker1 # [ 0.201799] ACPI: Core revision 20250807714lb # [ 0.224568] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'715worker1 # [ 0.202668] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns716lb # [ 0.225838] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'717worker1 # [ 0.204174] APIC: Switch to symmetric I/O mode setup718lb # [ 0.226903] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'719worker1 # [ 0.205115] x2apic enabled720lb # [ 0.227903] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'721worker1 # [ 0.205874] APIC: Switched APIC routing to: physical x2apic722lb # [ 0.228903] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'723worker1 # [ 0.207649] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1724lb # [ 0.229903] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'725lb # [ 0.230903] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'726worker1 # [ 0.208612] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns727lb # [ 0.231904] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256728worker1 # [ 0.210250] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)729lb # [ 0.232904] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64730worker1 # [ 0.211336] x86/cpu: User Mode Instruction Prevention (UMIP) activated731lb # [ 0.233903] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512732worker1 # [ 0.212410] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127733lb # [ 0.234903] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024734worker2 # [ 0.266969] Freeing SMP alternatives memory: 44K735worker1 # [ 0.213247] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0736lb # [ 0.235903] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8737worker2 # [ 0.267285] pid_max: default: 32768 minimum: 301738worker1 # [ 0.214250] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto739lb # [ 0.236903] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.740worker2 # [ 0.268376] LSM: initializing lsm=capability,landlock,yama,bpf,ima741worker2 # [ 0.269390] landlock: Up and running.742worker1 # [ 0.215247] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl743worker2 # [ 0.270283] Yama: becoming mindful.744worker1 # [ 0.216247] Transient Scheduler Attacks: Vulnerable: No microcode745worker2 # [ 0.271078] LSM support for eBPF active746worker1 # [ 0.217246] Spectre V2 : Mitigation: Enhanced / Automatic IBRS747worker2 # [ 0.271848] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)748worker1 # [ 0.218247] Speculative Return Stack Overflow: Mitigation: Safe RET749worker2 # [ 0.272319] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)750worker2 # [ 0.275204] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)751worker1 # [ 0.219246] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization752worker1 # [ 0.221255] Spectre V2 : Enabling IBPB for BPF753worker2 # [ 0.275822] Performance Events: Fam17h+ core perfctr, AMD PMU driver.754worker2 # [ 0.276286] ... version: 2755worker1 # [ 0.221982] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier756worker2 # [ 0.276958] ... bit width: 48757worker2 # [ 0.277284] ... generic counters: 6758worker1 # [ 0.223247] active return thunk: srso_alias_return_thunk759worker2 # [ 0.277960] ... generic bitmap: 000000000000003f760worker1 # [ 0.224268] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'761worker2 # [ 0.278293] ... fixed-purpose counters: 0762worker1 # [ 0.225246] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'763worker2 # [ 0.278962] ... fixed-purpose bitmap: 0000000000000000764worker2 # [ 0.279284] ... value mask: 0000ffffffffffff765worker1 # [ 0.226246] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'766client # [ 0.272618] Freeing SMP alternatives memory: 44K767client # [ 0.273437] pid_max: default: 32768 minimum: 301768worker2 # [ 0.280122] ... max period: 00007fffffffffff769worker1 # [ 0.227246] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'770worker2 # [ 0.280915] ... global_ctrl mask: 000000000000003f771client # [ 0.274243] LSM: initializing lsm=capability,landlock,yama,bpf,ima772worker1 # [ 0.228246] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'773worker2 # [ 0.281391] signal: max sigframe size: 3376774client # [ 0.274708] landlock: Up and running.775worker1 # [ 0.229247] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'776client # [ 0.275603] Yama: becoming mindful.777worker2 # [ 0.282141] rcu: Hierarchical SRCU implementation.778worker2 # [ 0.282833] rcu: Max phase no-delay instances is 400.779client # [ 0.276824] LSM support for eBPF active780worker1 # [ 0.230246] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'781worker1 # [ 0.231247] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256782client # [ 0.277649] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)783worker1 # [ 0.232247] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64784client # [ 0.278645] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)785worker1 # [ 0.233246] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512786worker1 # [ 0.234246] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024787worker1 # [ 0.235246] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8788client # [ 0.281206] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)789worker2 # [ 0.287782] smp: Bringing up secondary CPUs ...790worker2 # [ 0.288298] smp: Brought up 1 node, 1 CPU791worker1 # [ 0.236246] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.792client # [ 0.282163] Performance Events: Fam17h+ core perfctr, AMD PMU driver.793worker2 # [ 0.288945] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)794client # [ 0.282608] ... version: 2795client # [ 0.283327] ... bit width: 48796client # [ 0.283605] ... generic counters: 6797worker2 # [ 0.289461] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)798worker2 # [ 0.290491] devtmpfs: initialized799client # [ 0.284296] ... generic bitmap: 000000000000003f800client # [ 0.284657] ... fixed-purpose counters: 0801worker2 # [ 0.291280] x86/mm: Memory block size: 128MB802client # [ 0.285374] ... fixed-purpose bitmap: 0000000000000000803client # [ 0.285606] ... value mask: 0000ffffffffffff804worker2 # [ 0.292984] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)805client # [ 0.286500] ... max period: 00007fffffffffff806client # [ 0.287291] ... global_ctrl mask: 000000000000003f807worker2 # [ 0.294147] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).808client # [ 0.287740] signal: max sigframe size: 3376809worker2 # [ 0.295364] pinctrl core: initialized pinctrl subsystem810client # [ 0.288495] rcu: Hierarchical SRCU implementation.811lb # [ 0.271278] Freeing SMP alternatives memory: 44K812worker2 # [ 0.296454] PM: RTC time: 14:29:58, date: 2026-09-20813client # [ 0.289175] rcu: Max phase no-delay instances is 400.814lb # [ 0.271907] pid_max: default: 32768 minimum: 301815lb # [ 0.272930] LSM: initializing lsm=capability,landlock,yama,bpf,ima816lb # [ 0.273973] landlock: Up and running.817lb # [ 0.274579] Yama: becoming mindful.818worker2 # [ 0.299896] NET: Registered PF_NETLINK/PF_ROUTE protocol family819lb # [ 0.275125] LSM support for eBPF active820lb # [ 0.276015] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)821worker2 # [ 0.300795] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations822client # [ 0.294195] smp: Bringing up secondary CPUs ...823lb # [ 0.276945] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)824worker2 # [ 0.301304] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations825client # [ 0.294618] smp: Brought up 1 node, 1 CPU826client # [ 0.295276] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)827worker2 # [ 0.302547] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations828lb # [ 0.279691] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)829worker2 # [ 0.303299] audit: initializing netlink subsys (disabled)830client # [ 0.295747] Memory: 1454532K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)831worker2 # [ 0.304365] thermal_sys: Registered thermal governor 'fair_share'832lb # [ 0.280505] Performance Events: Fam17h+ core perfctr, AMD PMU driver.833client # [ 0.296927] devtmpfs: initialized834worker2 # [ 0.304367] thermal_sys: Registered thermal governor 'bang_bang'835lb # [ 0.280908] ... version: 2836client # [ 0.297815] x86/mm: Memory block size: 128MB837worker2 # [ 0.305277] thermal_sys: Registered thermal governor 'step_wise'838lb # [ 0.281614] ... bit width: 48839lb # [ 0.281906] ... generic counters: 6840worker2 # [ 0.306173] thermal_sys: Registered thermal governor 'user_space'841lb # [ 0.282587] ... generic bitmap: 000000000000003f842worker2 # [ 0.306964] audit: type=2000 audit(1789914598.702:1): state=initialized audit_enabled=0 res=1843client # [ 0.299570] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)844lb # [ 0.282955] ... fixed-purpose counters: 0845worker2 # [ 0.308287] thermal_sys: Registered thermal governor 'power_allocator'846lb # [ 0.283654] ... fixed-purpose bitmap: 0000000000000000847client # [ 0.300531] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).848worker2 # [ 0.308311] cpuidle: using governor menu849lb # [ 0.283906] ... value mask: 0000ffffffffffff850client # [ 0.301694] pinctrl core: initialized pinctrl subsystem851lb # [ 0.284777] ... max period: 00007fffffffffff852worker2 # [ 0.310456] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5853client # [ 0.302842] PM: RTC time: 14:29:58, date: 2026-09-20854lb # [ 0.285577] ... global_ctrl mask: 000000000000003f855lb # [ 0.286021] signal: max sigframe size: 3376856worker2 # [ 0.311508] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]857lb # [ 0.286777] rcu: Hierarchical SRCU implementation.858worker2 # [ 0.312287] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry859lb # [ 0.287468] rcu: Max phase no-delay instances is 400.860worker2 # [ 0.313296] PCI: Using configuration type 1 for base access861client # [ 0.306591] NET: Registered PF_NETLINK/PF_ROUTE protocol family862worker2 # [ 0.314337] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.863client # [ 0.308139] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations864worker1 # [ 0.270646] Freeing SMP alternatives memory: 44K865worker1 # [ 0.271250] pid_max: default: 32768 minimum: 301866client # [ 0.308630] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations867worker1 # [ 0.272276] LSM: initializing lsm=capability,landlock,yama,bpf,ima868lb # [ 0.292542] smp: Bringing up secondary CPUs ...869worker1 # [ 0.273353] landlock: Up and running.870lb # [ 0.292919] smp: Brought up 1 node, 1 CPU871client # [ 0.309898] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations872worker1 # [ 0.273974] Yama: becoming mindful.873client # [ 0.310615] audit: initializing netlink subsys (disabled)874lb # [ 0.293576] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)875worker1 # [ 0.274459] LSM support for eBPF active876client # [ 0.311607] audit: type=2000 audit(1789914598.718:1): state=initialized audit_enabled=0 res=1877worker1 # [ 0.275357] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)878worker2 # [ 0.321516] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages879lb # [ 0.294119] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)880client # [ 0.312760] thermal_sys: Registered thermal governor 'fair_share'881worker1 # [ 0.276287] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)882worker2 # [ 0.322286] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page883lb # [ 0.295175] devtmpfs: initialized884client # [ 0.312762] thermal_sys: Registered thermal governor 'bang_bang'885lb # [ 0.295969] x86/mm: Memory block size: 128MB886worker2 # [ 0.323263] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages887client # [ 0.313606] thermal_sys: Registered thermal governor 'step_wise'888client # [ 0.314520] thermal_sys: Registered thermal governor 'user_space'889worker1 # [ 0.279117] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)890lb # [ 0.297736] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)891client # [ 0.315315] thermal_sys: Registered thermal governor 'power_allocator'892client # [ 0.315618] cpuidle: using governor menu893worker1 # [ 0.279918] Performance Events: Fam17h+ core perfctr, AMD PMU driver.894worker1 # [ 0.280251] ... version: 2895lb # [ 0.298847] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).896worker1 # [ 0.280945] ... bit width: 48897client # [ 0.317822] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5898lb # [ 0.299990] pinctrl core: initialized pinctrl subsystem899worker1 # [ 0.281249] ... generic counters: 6900worker2 # [ 0.328286] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page901lb # [ 0.301147] PM: RTC time: 14:29:58, date: 2026-09-20902worker1 # [ 0.281943] ... generic bitmap: 000000000000003f903client # [ 0.318843] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]904worker1 # [ 0.282259] ... fixed-purpose counters: 0905client # [ 0.319608] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry906worker1 # [ 0.282930] ... fixed-purpose bitmap: 0000000000000000907client # [ 0.320617] PCI: Using configuration type 1 for base access908worker1 # [ 0.283249] ... value mask: 0000ffffffffffff909worker1 # [ 0.284112] ... max period: 00007fffffffffff910lb # [ 0.304862] NET: Registered PF_NETLINK/PF_ROUTE protocol family911client # [ 0.321725] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.912worker1 # [ 0.284903] ... global_ctrl mask: 000000000000003f913worker1 # [ 0.285376] signal: max sigframe size: 3376914lb # [ 0.306118] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations915worker1 # [ 0.286147] rcu: Hierarchical SRCU implementation.916worker1 # [ 0.286813] rcu: Max phase no-delay instances is 400.917lb # [ 0.306930] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations918worker2 # [ 0.336680] ACPI: Added _OSI(Module Device)919worker2 # [ 0.337286] ACPI: Added _OSI(Processor Device)920lb # [ 0.308224] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations921worker2 # [ 0.337984] ACPI: Added _OSI(Processor Aggregator Device)922lb # [ 0.308919] audit: initializing netlink subsys (disabled)923lb # [ 0.309907] audit: type=2000 audit(1789914598.737:1): state=initialized audit_enabled=0 res=1924client # [ 0.328622] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages925client # [ 0.329606] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page926lb # [ 0.311082] thermal_sys: Registered thermal governor 'fair_share'927worker1 # [ 0.291845] smp: Bringing up secondary CPUs ...928worker1 # [ 0.292263] smp: Brought up 1 node, 1 CPU929lb # [ 0.311084] thermal_sys: Registered thermal governor 'bang_bang'930worker1 # [ 0.292920] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)931lb # [ 0.311907] thermal_sys: Registered thermal governor 'step_wise'932lb # [ 0.312907] thermal_sys: Registered thermal governor 'user_space'933worker1 # [ 0.293481] Memory: 1454568K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)934lb # [ 0.313883] thermal_sys: Registered thermal governor 'power_allocator'935worker2 # [ 0.344840] ACPI: 1 ACPI AML tables successfully acquired and loaded936lb # [ 0.314708] cpuidle: using governor menu937worker1 # [ 0.294496] devtmpfs: initialized938client # [ 0.334607] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages939worker1 # [ 0.295269] x86/mm: Memory block size: 128MB940client # [ 0.335606] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page941lb # [ 0.316456] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5942lb # [ 0.317140] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]943worker1 # [ 0.297257] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)944worker2 # [ 0.348693] ACPI: Interpreter enabled945lb # [ 0.317908] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry946worker2 # [ 0.349299] ACPI: PM: (supports S0 S3 S4 S5)947worker1 # [ 0.298277] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).948lb # [ 0.318918] PCI: Using configuration type 1 for base access949worker2 # [ 0.349972] ACPI: Using IOAPIC for interrupt routing950worker1 # [ 0.299338] pinctrl core: initialized pinctrl subsystem951worker1 # [ 0.300460] PM: RTC time: 14:29:58, date: 2026-09-20952lb # [ 0.320028] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.953worker2 # [ 0.352330] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug954worker2 # [ 0.353285] PCI: Using E820 reservations for host bridge windows955worker1 # [ 0.304350] NET: Registered PF_NETLINK/PF_ROUTE protocol family956client # [ 0.346830] ACPI: Added _OSI(Module Device)957worker1 # [ 0.305773] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations958worker2 # [ 0.356331] ACPI: Enabled 2 GPEs in block 00 to 3F959client # [ 0.347556] ACPI: Added _OSI(Processor Device)960worker1 # [ 0.306272] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations961client # [ 0.348172] ACPI: Added _OSI(Processor Aggregator Device)962lb # [ 0.326935] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages963lb # [ 0.327907] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page964worker1 # [ 0.307554] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations965worker1 # [ 0.308261] audit: initializing netlink subsys (disabled)966worker1 # [ 0.309327] thermal_sys: Registered thermal governor 'fair_share'967worker1 # [ 0.309329] thermal_sys: Registered thermal governor 'bang_bang'968worker1 # [ 0.310250] thermal_sys: Registered thermal governor 'step_wise'969worker1 # [ 0.311159] thermal_sys: Registered thermal governor 'user_space'970lb # [ 0.332907] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages971client # [ 0.354357] ACPI: 1 ACPI AML tables successfully acquired and loaded972lb # [ 0.333907] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page973worker2 # [ 0.364897] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])974worker1 # [ 0.311948] audit: type=2000 audit(1789914598.760:1): state=initialized audit_enabled=0 res=1975worker1 # [ 0.313252] thermal_sys: Registered thermal governor 'power_allocator'976worker2 # [ 0.365291] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]977worker1 # [ 0.313277] cpuidle: using governor menu978worker2 # [ 0.366363] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]979worker1 # [ 0.315447] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5980client # [ 0.358027] ACPI: Interpreter enabled981worker2 # [ 0.367412] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]982client # [ 0.358621] ACPI: PM: (supports S0 S3 S4 S5)983worker2 # [ 0.368648] PCI host bridge to bus 0000:00984worker1 # [ 0.316484] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]985client # [ 0.359371] ACPI: Using IOAPIC for interrupt routing986worker2 # [ 0.369288] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]987worker1 # [ 0.317252] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry988worker1 # [ 0.318261] PCI: Using configuration type 1 for base access989worker2 # [ 0.370284] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]990worker2 # [ 0.371284] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]991worker1 # [ 0.319330] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.992client # [ 0.361666] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug993worker2 # [ 0.372284] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]994client # [ 0.362606] PCI: Using E820 reservations for host bridge windows995worker2 # [ 0.373284] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]996lb # [ 0.345146] ACPI: Added _OSI(Module Device)997worker2 # [ 0.374284] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]998lb # [ 0.345907] ACPI: Added _OSI(Processor Device)999worker2 # [ 0.375284] pci_bus 0000:00: root bus resource [bus 00-ff]1000client # [ 0.365702] ACPI: Enabled 2 GPEs in block 00 to 3F1001lb # [ 0.346681] ACPI: Added _OSI(Processor Aggregator Device)1002worker2 # [ 0.376233] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1003worker2 # [ 0.377661] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1004worker1 # [ 0.326331] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1005worker1 # [ 0.327250] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1006lb # [ 0.352606] ACPI: 1 ACPI AML tables successfully acquired and loaded1007client # [ 0.372996] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1008client # [ 0.373612] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1009worker1 # [ 0.332251] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1010worker1 # [ 0.333250] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1011client # [ 0.374686] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1012worker2 # [ 0.380308] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1013lb # [ 0.356333] ACPI: Interpreter enabled1014client # [ 0.375735] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1015worker2 # [ 0.381296] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1016lb # [ 0.356924] ACPI: PM: (supports S0 S3 S4 S5)1017client # [ 0.376956] PCI host bridge to bus 0000:001018worker2 # [ 0.382214] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1019lb # [ 0.357610] ACPI: Using IOAPIC for interrupt routing1020client # [ 0.377610] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1021worker2 # [ 0.383199] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1022client # [ 0.378606] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1023client # [ 0.379606] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1024lb # [ 0.359952] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1025worker2 # [ 0.385008] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1026lb # [ 0.360907] PCI: Using E820 reservations for host bridge windows1027client # [ 0.380605] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1028client # [ 0.381606] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1029worker2 # [ 0.387334] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1030client # [ 0.382605] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1031worker1 # [ 0.343006] ACPI: Added _OSI(Module Device)1032lb # [ 0.364029] ACPI: Enabled 2 GPEs in block 00 to 3F1033worker2 # [ 0.388151] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1034worker1 # [ 0.343251] ACPI: Added _OSI(Processor Device)1035client # [ 0.383606] pci_bus 0000:00: root bus resource [bus 00-ff]1036worker2 # [ 0.388976] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1037worker1 # [ 0.343953] ACPI: Added _OSI(Processor Aggregator Device)1038client # [ 0.384607] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1039worker2 # [ 0.389290] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1040client # [ 0.386072] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1041worker2 # [ 0.390917] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1042worker2 # [ 0.392292] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1043worker2 # [ 0.393107] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1044lb # [ 0.371468] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1045worker2 # [ 0.393988] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1046worker1 # [ 0.350813] ACPI: 1 ACPI AML tables successfully acquired and loaded1047lb # [ 0.371913] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1048lb # [ 0.373000] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1049worker2 # [ 0.394835] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1050lb # [ 0.374033] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1051worker1 # [ 0.354178] ACPI: Interpreter enabled1052client # [ 0.388687] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1053lb # [ 0.375288] PCI host bridge to bus 0000:001054worker1 # [ 0.354704] ACPI: PM: (supports S0 S3 S4 S5)1055client # [ 0.389617] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1056lb # [ 0.375911] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1057worker2 # [ 0.397292] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1058worker1 # [ 0.355250] ACPI: Using IOAPIC for interrupt routing1059client # [ 0.390569] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1060lb # [ 0.376906] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1061worker1 # [ 0.356106] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1062worker2 # [ 0.398095] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1063lb # [ 0.377906] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1064client # [ 0.391777] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1065worker2 # [ 0.399307] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1066lb # [ 0.378906] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1067worker2 # [ 0.400859] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1068lb # [ 0.379906] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1069client # [ 0.393522] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1070lb # [ 0.380906] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1071worker1 # [ 0.361250] PCI: Using E820 reservations for host bridge windows1072worker2 # [ 0.402292] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1073lb # [ 0.381907] pci_bus 0000:00: root bus resource [bus 00-ff]1074worker1 # [ 0.362349] ACPI: Enabled 2 GPEs in block 00 to 3F1075worker2 # [ 0.403103] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1076client # [ 0.396633] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1077lb # [ 0.382847] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1078worker2 # [ 0.403984] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1079client # [ 0.397444] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1080client # [ 0.398313] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1081lb # [ 0.384353] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1082worker2 # [ 0.404824] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1083client # [ 0.398611] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1084client # [ 0.400490] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1085worker2 # [ 0.407292] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1086worker2 # [ 0.408112] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1087worker2 # [ 0.408986] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1088client # [ 0.402613] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1089worker1 # [ 0.370850] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1090client # [ 0.403455] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1091worker2 # [ 0.409843] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1092worker1 # [ 0.371257] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1093client # [ 0.404332] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1094worker1 # [ 0.372326] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1095lb # [ 0.386990] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1096client # [ 0.405205] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1097worker2 # [ 0.411293] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1098worker1 # [ 0.373372] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1099lb # [ 0.387918] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1100worker2 # [ 0.412099] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1101worker1 # [ 0.374600] PCI host bridge to bus 0000:001102lb # [ 0.388897] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1103worker2 # [ 0.412986] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1104worker1 # [ 0.375255] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1105worker1 # [ 0.376250] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1106lb # [ 0.390224] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1107client # [ 0.407615] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1108worker2 # [ 0.414343] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1109worker1 # [ 0.377249] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1110client # [ 0.409204] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1111lb # [ 0.391677] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1112worker1 # [ 0.378249] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1113client # [ 0.409627] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1114worker2 # [ 0.416291] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1115worker1 # [ 0.379249] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1116worker2 # [ 0.417095] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1117client # [ 0.411175] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1118worker1 # [ 0.380249] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1119worker2 # [ 0.417979] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1120worker1 # [ 0.381249] pci_bus 0000:00: root bus resource [bus 00-ff]1121lb # [ 0.394936] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1122worker2 # [ 0.418290] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1123lb # [ 0.395775] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1124worker1 # [ 0.382251] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1125client # [ 0.412632] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1126lb # [ 0.396649] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1127worker2 # [ 0.419749] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1128client # [ 0.413455] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1129worker1 # [ 0.383697] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1130client # [ 0.414325] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1131lb # [ 0.396912] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1132worker2 # [ 0.421297] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1133lb # [ 0.398645] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1134worker2 # [ 0.422308] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1135client # [ 0.415151] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1136worker2 # [ 0.423820] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1137client # [ 0.416632] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1138lb # [ 0.401937] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1139client # [ 0.417462] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1140lb # [ 0.402780] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1141worker2 # [ 0.425309] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1142client # [ 0.418327] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1143worker2 # [ 0.426123] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1144lb # [ 0.403662] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1145worker2 # [ 0.426977] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1146worker1 # [ 0.386340] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1147client # [ 0.419170] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1148lb # [ 0.404500] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1149worker1 # [ 0.387262] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1150worker2 # [ 0.427836] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1151worker1 # [ 0.388218] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1152client # [ 0.421635] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1153lb # [ 0.406858] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1154worker1 # [ 0.389483] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1155worker2 # [ 0.430201] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1156lb # [ 0.407527] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1157client # [ 0.422479] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1158worker2 # [ 0.430887] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1159lb # [ 0.407928] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1160client # [ 0.423292] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1161worker1 # [ 0.391067] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1162worker2 # [ 0.431306] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1163lb # [ 0.409479] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1164client # [ 0.424233] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1165worker2 # [ 0.432893] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1166worker1 # [ 0.394282] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1167worker2 # [ 0.433926] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1168worker1 # [ 0.395129] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1169client # [ 0.425632] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1170worker2 # [ 0.434489] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1171client # [ 0.426470] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1172worker1 # [ 0.395984] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1173client # [ 0.427319] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1174worker2 # [ 0.435918] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1175lb # [ 0.411935] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1176worker1 # [ 0.396255] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1177client # [ 0.427612] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1178lb # [ 0.412913] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1179lb # [ 0.413911] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1180worker1 # [ 0.397973] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1181client # [ 0.429141] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1182lb # [ 0.415491] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1183worker1 # [ 0.401277] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1184client # [ 0.430618] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1185worker1 # [ 0.402125] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1186client # [ 0.431542] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1187lb # [ 0.417814] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1188worker1 # [ 0.402995] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1189lb # [ 0.418569] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1190client # [ 0.432964] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1191lb # [ 0.418928] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1192worker1 # [ 0.403832] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1193client # [ 0.434638] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1194client # [ 0.435492] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1195lb # [ 0.420545] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1196worker2 # [ 0.436484] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1197worker1 # [ 0.407182] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1198client # [ 0.436326] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1199worker1 # [ 0.407921] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1200worker2 # [ 0.439314] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1201lb # [ 0.423788] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1202worker1 # [ 0.408271] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1203client # [ 0.437152] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1204lb # [ 0.424556] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1205worker2 # [ 0.440358] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1206lb # [ 0.424928] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1207worker1 # [ 0.409799] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1208client # [ 0.438633] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1209worker2 # [ 0.442291] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1210lb # [ 0.426471] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1211client # [ 0.439473] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1212worker2 # [ 0.443514] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1213client # [ 0.440331] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1214worker1 # [ 0.412280] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1215worker1 # [ 0.413137] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1216worker2 # [ 0.444577] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1217client # [ 0.441171] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1218worker2 # [ 0.445531] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1219lb # [ 0.429260] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1220client # [ 0.442327] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1221worker1 # [ 0.413982] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1222lb # [ 0.430553] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1223lb # [ 0.430928] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1224worker1 # [ 0.414800] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1225worker2 # [ 0.447951] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1226client # [ 0.442801] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1227lb # [ 0.431913] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1228worker2 # [ 0.448292] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1229client # [ 0.445243] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1230lb # [ 0.433431] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1231worker1 # [ 0.416257] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1232worker2 # [ 0.449514] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1233worker1 # [ 0.417066] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1234worker2 # [ 0.451054] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1235worker1 # [ 0.417994] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1236lb # [ 0.434918] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1237lb # [ 0.435886] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1238worker1 # [ 0.418834] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1239worker2 # [ 0.454312] ACPI: PCI: Interrupt link LNKA configured for IRQ 101240lb # [ 0.437379] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1241worker2 # [ 0.455340] ACPI: PCI: Interrupt link LNKB configured for IRQ 101242worker1 # [ 0.421250] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1243worker2 # [ 0.456384] ACPI: PCI: Interrupt link LNKC configured for IRQ 111244worker1 # [ 0.422082] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1245lb # [ 0.439653] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1246worker2 # [ 0.457380] ACPI: PCI: Interrupt link LNKD configured for IRQ 111247lb # [ 0.439913] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1248worker1 # [ 0.422951] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1249worker2 # [ 0.458385] ACPI: PCI: Interrupt link LNKE configured for IRQ 101250lb # [ 0.440886] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1251worker2 # [ 0.459379] ACPI: PCI: Interrupt link LNKF configured for IRQ 101252worker1 # [ 0.423848] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1253client # [ 0.445800] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1254worker2 # [ 0.460383] ACPI: PCI: Interrupt link LNKG configured for IRQ 111255lb # [ 0.442341] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1256worker2 # [ 0.461386] ACPI: PCI: Interrupt link LNKH configured for IRQ 111257client # [ 0.448211] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1258worker2 # [ 0.462318] ACPI: PCI: Interrupt link GSIA configured for IRQ 161259worker1 # [ 0.425258] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1260worker2 # [ 0.463239] ACPI: PCI: Interrupt link GSIB configured for IRQ 171261client # [ 0.448835] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1262worker1 # [ 0.426084] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1263worker2 # [ 0.463985] ACPI: PCI: Interrupt link GSIC configured for IRQ 181264client # [ 0.450168] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1265lb # [ 0.444793] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1266worker2 # [ 0.464297] ACPI: PCI: Interrupt link GSID configured for IRQ 191267worker1 # [ 0.426949] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1268lb # [ 0.445913] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1269worker2 # [ 0.465297] ACPI: PCI: Interrupt link GSIE configured for IRQ 201270worker1 # [ 0.427257] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1271client # [ 0.450900] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1272lb # [ 0.446873] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1273worker2 # [ 0.466288] ACPI: PCI: Interrupt link GSIF configured for IRQ 211274worker2 # [ 0.467207] ACPI: PCI: Interrupt link GSIG configured for IRQ 221275worker1 # [ 0.428753] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1276client # [ 0.451914] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1277lb # [ 0.448303] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1278worker2 # [ 0.467976] ACPI: PCI: Interrupt link GSIH configured for IRQ 231279client # [ 0.452851] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1280worker1 # [ 0.430262] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1281lb # [ 0.449703] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1282worker2 # [ 0.469346] iommu: Default domain type: Translated1283worker1 # [ 0.431192] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1284worker2 # [ 0.470101] iommu: DMA domain TLB invalidation policy: lazy mode1285client # [ 0.454622] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1286lb # [ 0.450729] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1287worker2 # [ 0.471220] ACPI: bus type USB registered1288client # [ 0.455432] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1289worker1 # [ 0.432580] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1290lb # [ 0.452635] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1291worker2 # [ 0.471810] usbcore: registered new interface driver usbfs1292worker2 # [ 0.472297] usbcore: registered new interface driver hub1293client # [ 0.456660] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1294worker2 # [ 0.473119] usbcore: registered new device driver usb1295client # [ 0.458211] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1296worker2 # [ 0.474690] NetLabel: Initializing1297worker2 # [ 0.475272] NetLabel: domain hash size = 1281298worker1 # [ 0.435266] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1299worker2 # [ 0.475782] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1300worker1 # [ 0.436866] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1301worker2 # [ 0.476335] NetLabel: unlabeled traffic allowed by default1302worker1 # [ 0.437276] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1303worker2 # [ 0.477193] PCI: Using ACPI for IRQ routing1304client # [ 0.462110] ACPI: PCI: Interrupt link LNKA configured for IRQ 101305worker1 # [ 0.438809] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1306client # [ 0.462717] ACPI: PCI: Interrupt link LNKB configured for IRQ 101307client # [ 0.463706] ACPI: PCI: Interrupt link LNKC configured for IRQ 111308worker1 # [ 0.440304] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1309client # [ 0.464704] ACPI: PCI: Interrupt link LNKD configured for IRQ 111310worker1 # [ 0.441149] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1311client # [ 0.465700] ACPI: PCI: Interrupt link LNKE configured for IRQ 101312worker1 # [ 0.441965] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1313client # [ 0.466700] ACPI: PCI: Interrupt link LNKF configured for IRQ 101314lb # [ 0.453097] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1315client # [ 0.467704] ACPI: PCI: Interrupt link LNKG configured for IRQ 111316client # [ 0.468713] ACPI: PCI: Interrupt link LNKH configured for IRQ 111317worker1 # [ 0.442842] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1318client # [ 0.469651] ACPI: PCI: Interrupt link GSIA configured for IRQ 161319worker1 # [ 0.444042] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1320client # [ 0.470622] ACPI: PCI: Interrupt link GSIB configured for IRQ 171321client # [ 0.471620] ACPI: PCI: Interrupt link GSIC configured for IRQ 181322worker1 # [ 0.445121] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1323lb # [ 0.456499] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1324client # [ 0.472597] ACPI: PCI: Interrupt link GSID configured for IRQ 191325worker1 # [ 0.446866] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1326client # [ 0.473585] ACPI: PCI: Interrupt link GSIE configured for IRQ 201327lb # [ 0.457191] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1328client # [ 0.474346] ACPI: PCI: Interrupt link GSIF configured for IRQ 211329client # [ 0.474618] ACPI: PCI: Interrupt link GSIG configured for IRQ 221330lb # [ 0.458915] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1331client # [ 0.475617] ACPI: PCI: Interrupt link GSIH configured for IRQ 231332lb # [ 0.460126] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1333client # [ 0.477669] iommu: Default domain type: Translated1334client # [ 0.478504] iommu: DMA domain TLB invalidation policy: lazy mode1335lb # [ 0.461205] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1336client # [ 0.479623] ACPI: bus type USB registered1337lb # [ 0.462155] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1338client # [ 0.480397] usbcore: registered new interface driver usbfs1339client # [ 0.481275] usbcore: registered new interface driver hub1340client # [ 0.481617] usbcore: registered new device driver usb1341lb # [ 0.464936] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1342client # [ 0.483359] NetLabel: Initializing1343lb # [ 0.465849] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1344client # [ 0.483612] NetLabel: domain hash size = 1281345worker1 # [ 0.447457] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1346client # [ 0.484370] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1347lb # [ 0.467004] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1348client # [ 0.484675] NetLabel: unlabeled traffic allowed by default1349worker1 # [ 0.449989] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1350client # [ 0.485593] PCI: Using ACPI for IRQ routing1351lb # [ 0.468757] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1352worker1 # [ 0.450489] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1353worker1 # [ 0.451932] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1354lb # [ 0.472540] ACPI: PCI: Interrupt link LNKA configured for IRQ 101355worker1 # [ 0.452522] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1356lb # [ 0.473010] ACPI: PCI: Interrupt link LNKB configured for IRQ 101357worker1 # [ 0.453546] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1358lb # [ 0.474005] ACPI: PCI: Interrupt link LNKC configured for IRQ 111359worker1 # [ 0.454502] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1360lb # [ 0.475010] ACPI: PCI: Interrupt link LNKD configured for IRQ 111361lb # [ 0.476016] ACPI: PCI: Interrupt link LNKE configured for IRQ 101362worker1 # [ 0.456334] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1363lb # [ 0.477006] ACPI: PCI: Interrupt link LNKF configured for IRQ 101364worker1 # [ 0.457180] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1365lb # [ 0.478007] ACPI: PCI: Interrupt link LNKG configured for IRQ 111366lb # [ 0.479003] ACPI: PCI: Interrupt link LNKH configured for IRQ 111367worker1 # [ 0.458530] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1368lb # [ 0.479941] ACPI: PCI: Interrupt link GSIA configured for IRQ 161369lb # [ 0.480918] ACPI: PCI: Interrupt link GSIB configured for IRQ 171370worker1 # [ 0.460086] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1371worker2 # [ 0.519778] pci 0000:00:01.0: vgaarb: setting as boot VGA device1372lb # [ 0.481920] ACPI: PCI: Interrupt link GSIC configured for IRQ 181373lb # [ 0.482922] ACPI: PCI: Interrupt link GSID configured for IRQ 191374worker2 # [ 0.520281] pci 0000:00:01.0: vgaarb: bridge control possible1375lb # [ 0.483922] ACPI: PCI: Interrupt link GSIE configured for IRQ 201376worker1 # [ 0.463930] ACPI: PCI: Interrupt link LNKA configured for IRQ 101377worker2 # [ 0.520281] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1378lb # [ 0.484920] ACPI: PCI: Interrupt link GSIF configured for IRQ 211379worker2 # [ 0.520291] vgaarb: loaded1380worker1 # [ 0.464359] ACPI: PCI: Interrupt link LNKB configured for IRQ 101381lb # [ 0.485846] ACPI: PCI: Interrupt link GSIG configured for IRQ 221382worker2 # [ 0.521008] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01383lb # [ 0.486653] ACPI: PCI: Interrupt link GSIH configured for IRQ 231384worker1 # [ 0.465347] ACPI: PCI: Interrupt link LNKC configured for IRQ 111385worker2 # [ 0.521287] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1386worker1 # [ 0.466345] ACPI: PCI: Interrupt link LNKD configured for IRQ 111387lb # [ 0.487985] iommu: Default domain type: Translated1388worker1 # [ 0.467346] ACPI: PCI: Interrupt link LNKE configured for IRQ 101389lb # [ 0.488784] iommu: DMA domain TLB invalidation policy: lazy mode1390worker1 # [ 0.468356] ACPI: PCI: Interrupt link LNKF configured for IRQ 101391lb # [ 0.489955] ACPI: bus type USB registered1392worker1 # [ 0.469351] ACPI: PCI: Interrupt link LNKG configured for IRQ 111393lb # [ 0.490701] usbcore: registered new interface driver usbfs1394worker2 # [ 0.525373] clocksource: Switched to clocksource kvm-clock1395lb # [ 0.491608] usbcore: registered new interface driver hub1396lb # [ 0.491916] usbcore: registered new device driver usb1397worker1 # [ 0.470346] ACPI: PCI: Interrupt link LNKH configured for IRQ 111398lb # [ 0.494030] NetLabel: Initializing1399worker2 # [ 0.528772] VFS: Disk quotas dquot_6.6.01400worker1 # [ 0.471349] ACPI: PCI: Interrupt link GSIA configured for IRQ 161401lb # [ 0.494616] NetLabel: domain hash size = 1281402worker2 # [ 0.529432] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1403worker1 # [ 0.472265] ACPI: PCI: Interrupt link GSIB configured for IRQ 171404lb # [ 0.494906] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1405worker2 # [ 0.530705] pnp: PnP ACPI init1406worker1 # [ 0.473174] ACPI: PCI: Interrupt link GSIC configured for IRQ 181407worker2 # [ 0.531454] ACPI: IRQ 4 override to edge(!), high(!)1408lb # [ 0.495906] NetLabel: unlabeled traffic allowed by default1409worker2 # [ 0.532436] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1410lb # [ 0.497689] PCI: Using ACPI for IRQ routing1411worker2 # [ 0.533952] pnp: PnP ACPI: found 6 devices1412worker1 # [ 0.473959] ACPI: PCI: Interrupt link GSID configured for IRQ 191413worker1 # [ 0.474280] ACPI: PCI: Interrupt link GSIE configured for IRQ 201414worker1 # [ 0.475270] ACPI: PCI: Interrupt link GSIF configured for IRQ 211415worker1 # [ 0.476266] ACPI: PCI: Interrupt link GSIG configured for IRQ 221416worker1 # [ 0.477264] ACPI: PCI: Interrupt link GSIH configured for IRQ 231417worker1 # [ 0.479364] iommu: Default domain type: Translated1418worker1 # [ 0.480197] iommu: DMA domain TLB invalidation policy: lazy mode1419worker2 # [ 0.541511] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1420worker1 # [ 0.481283] ACPI: bus type USB registered1421worker2 # [ 0.542910] clocksource: Switched to clocksource acpi_pm1422worker1 # [ 0.482052] usbcore: registered new interface driver usbfs1423worker2 # [ 0.543831] NET: Registered PF_INET protocol family1424worker1 # [ 0.482946] usbcore: registered new interface driver hub1425worker2 # [ 0.544938] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1426worker1 # [ 0.483262] usbcore: registered new device driver usb1427client # [ 0.530980] pci 0000:00:01.0: vgaarb: setting as boot VGA device1428client # [ 0.531602] pci 0000:00:01.0: vgaarb: bridge control possible1429worker1 # [ 0.485225] NetLabel: Initializing1430worker1 # [ 0.485741] NetLabel: domain hash size = 1281431client # [ 0.531602] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1432client # [ 0.531613] vgaarb: loaded1433worker1 # [ 0.486250] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1434worker1 # [ 0.487289] NetLabel: unlabeled traffic allowed by default1435client # [ 0.532422] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01436worker1 # [ 0.488262] PCI: Using ACPI for IRQ routing1437client # [ 0.533222] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1438client # [ 0.537729] clocksource: Switched to clocksource kvm-clock1439client # [ 0.541621] VFS: Disk quotas dquot_6.6.01440client # [ 0.542385] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1441worker2 # [ 0.561405] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1442client # [ 0.543799] pnp: PnP ACPI init1443worker2 # [ 0.562912] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1444client # [ 0.544676] ACPI: IRQ 4 override to edge(!), high(!)1445worker2 # [ 0.564222] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1446client # [ 0.545690] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1447client # [ 0.547251] pnp: PnP ACPI: found 6 devices1448worker2 # [ 0.565558] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1449worker2 # [ 0.566800] TCP: Hash tables configured (established 16384 bind 16384)1450worker2 # [ 0.567932] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1451worker2 # [ 0.569220] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1452worker2 # [ 0.570324] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1453worker2 # [ 0.571585] NET: Registered PF_UNIX/PF_LOCAL protocol family1454worker2 # [ 0.572586] NET: Registered PF_XDP protocol family1455worker2 # [ 0.573419] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1456client # [ 0.555002] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1457worker2 # [ 0.574437] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1458client # [ 0.556465] clocksource: Switched to clocksource acpi_pm1459worker2 # [ 0.575409] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1460client # [ 0.557458] NET: Registered PF_INET protocol family1461worker2 # [ 0.576444] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1462lb # [ 0.540342] pci 0000:00:01.0: vgaarb: setting as boot VGA device1463worker2 # [ 0.577496] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1464client # [ 0.558628] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1465lb # [ 0.540902] pci 0000:00:01.0: vgaarb: bridge control possible1466worker2 # [ 0.578513] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1467lb # [ 0.540902] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1468lb # [ 0.540910] vgaarb: loaded1469worker2 # [ 0.580362] ACPI: \_SB_.GSIA: Enabled at IRQ 161470lb # [ 0.541660] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01471lb # [ 0.541906] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1472worker2 # [ 0.582869] ACPI: \_SB_.GSIB: Enabled at IRQ 171473worker2 # [ 0.584746] ACPI: \_SB_.GSIC: Enabled at IRQ 181474worker2 # [ 0.586635] ACPI: \_SB_.GSID: Enabled at IRQ 191475lb # [ 0.545997] clocksource: Switched to clocksource kvm-clock1476worker2 # [ 0.588221] PCI: CLS 0 bytes, default 641477lb # [ 0.547695] VFS: Disk quotas dquot_6.6.01478worker2 # [ 0.589063] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1479worker2 # [ 0.590689] Trying to unpack rootfs image as initramfs...1480lb # [ 0.550295] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1481lb # [ 0.551615] pnp: PnP ACPI init1482lb # [ 0.552418] ACPI: IRQ 4 override to edge(!), high(!)1483lb # [ 0.553410] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1484worker1 # [ 0.531571] pci 0000:00:01.0: vgaarb: setting as boot VGA device1485client # [ 0.575574] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1486worker1 # [ 0.532245] pci 0000:00:01.0: vgaarb: bridge control possible1487lb # [ 0.554918] pnp: PnP ACPI: found 6 devices1488client # [ 0.577113] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1489worker1 # [ 0.532245] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1490worker1 # [ 0.532255] vgaarb: loaded1491client # [ 0.578392] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1492worker1 # [ 0.533002] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01493client # [ 0.579720] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1494worker1 # [ 0.533251] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1495client # [ 0.580965] TCP: Hash tables configured (established 16384 bind 16384)1496client # [ 0.582106] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1497client # [ 0.583413] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1498client # [ 0.584540] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1499worker1 # [ 0.536341] clocksource: Switched to clocksource kvm-clock1500lb # [ 0.562486] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1501client # [ 0.585723] NET: Registered PF_UNIX/PF_LOCAL protocol family1502lb # [ 0.563866] clocksource: Switched to clocksource acpi_pm1503client # [ 0.586723] NET: Registered PF_XDP protocol family1504lb # [ 0.564793] NET: Registered PF_INET protocol family1505client # [ 0.587594] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1506worker1 # [ 0.539855] VFS: Disk quotas dquot_6.6.01507lb # [ 0.565952] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1508client # [ 0.588636] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1509worker1 # [ 0.540584] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1510client # [ 0.589622] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1511worker1 # [ 0.541951] pnp: PnP ACPI init1512client # [ 0.590733] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1513worker1 # [ 0.542777] ACPI: IRQ 4 override to edge(!), high(!)1514client # [ 0.591818] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1515worker1 # [ 0.543739] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1516client # [ 0.592933] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1517worker1 # [ 0.545286] pnp: PnP ACPI: found 6 devices1518client # [ 0.594703] ACPI: \_SB_.GSIA: Enabled at IRQ 161519client # [ 0.597561] ACPI: \_SB_.GSIB: Enabled at IRQ 171520client # [ 0.599592] ACPI: \_SB_.GSIC: Enabled at IRQ 181521client # [ 0.601534] ACPI: \_SB_.GSID: Enabled at IRQ 191522worker1 # [ 0.552878] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1523worker1 # [ 0.554378] clocksource: Switched to clocksource acpi_pm1524client # [ 0.603167] PCI: CLS 0 bytes, default 641525worker1 # [ 0.555349] NET: Registered PF_INET protocol family1526client # [ 0.604060] Trying to unpack rootfs image as initramfs...1527lb # [ 0.582580] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1528worker1 # [ 0.556554] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1529lb # [ 0.584105] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1530lb # [ 0.585394] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1531client # [ 0.608174] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1532lb # [ 0.586710] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1533lb # [ 0.587948] TCP: Hash tables configured (established 16384 bind 16384)1534lb # [ 0.589056] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1535lb # [ 0.590299] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1536lb # [ 0.591356] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1537lb # [ 0.592583] NET: Registered PF_UNIX/PF_LOCAL protocol family1538worker2 # [ 0.634394] Initialise system trusted keyrings1539lb # [ 0.593586] NET: Registered PF_XDP protocol family1540worker2 # [ 0.635297] workingset: timestamp_bits=40 max_order=19 bucket_order=01541lb # [ 0.594405] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1542lb # [ 0.595404] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1543lb # [ 0.596440] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1544lb # [ 0.597548] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1545lb # [ 0.598683] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1546lb # [ 0.599808] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1547worker1 # [ 0.573273] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1548worker1 # [ 0.574675] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1549lb # [ 0.601605] ACPI: \_SB_.GSIA: Enabled at IRQ 161550worker1 # [ 0.575995] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1551lb # [ 0.604047] ACPI: \_SB_.GSIB: Enabled at IRQ 171552worker1 # [ 0.577400] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1553worker1 # [ 0.578643] TCP: Hash tables configured (established 16384 bind 16384)1554lb # [ 0.606071] ACPI: \_SB_.GSIC: Enabled at IRQ 181555worker1 # [ 0.579856] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1556worker1 # [ 0.581234] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1557lb # [ 0.608165] ACPI: \_SB_.GSID: Enabled at IRQ 191558worker1 # [ 0.582424] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1559lb # [ 0.609857] PCI: CLS 0 bytes, default 641560worker1 # [ 0.583675] NET: Registered PF_UNIX/PF_LOCAL protocol family1561worker1 # [ 0.584652] NET: Registered PF_XDP protocol family1562lb # [ 0.610776] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1563worker1 # [ 0.585497] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1564lb # [ 0.612459] Trying to unpack rootfs image as initramfs...1565worker1 # [ 0.586502] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1566worker1 # [ 0.587522] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1567worker1 # [ 0.588644] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1568worker1 # [ 0.589779] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1569worker1 # [ 0.590903] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1570worker2 # [ 0.658741] Key type asymmetric registered1571worker2 # [ 0.659439] Asymmetric key parser 'x509' registered1572worker1 # [ 0.592624] ACPI: \_SB_.GSIA: Enabled at IRQ 161573worker2 # [ 0.660327] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1574worker1 # [ 0.594849] ACPI: \_SB_.GSIB: Enabled at IRQ 171575worker1 # [ 0.596815] ACPI: \_SB_.GSIC: Enabled at IRQ 181576worker2 # [ 0.665642] io scheduler mq-deadline registered1577worker1 # [ 0.598785] ACPI: \_SB_.GSID: Enabled at IRQ 191578worker2 # [ 0.666406] io scheduler kyber registered1579worker1 # [ 0.600568] PCI: CLS 0 bytes, default 641580client # [ 0.650408] Initialise system trusted keyrings1581worker1 # [ 0.601468] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1582worker2 # [ 0.670020] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1583client # [ 0.651374] workingset: timestamp_bits=40 max_order=19 bucket_order=01584worker1 # [ 0.603240] Trying to unpack rootfs image as initramfs...1585worker2 # [ 0.671279] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1586worker2 # [ 0.675268] Linux agpgart interface v0.1031587worker2 # [ 0.676047] ACPI: bus type drm_connector registered1588worker2 # [ 0.678903] usbcore: registered new interface driver usbserial_generic1589worker2 # [ 0.679996] usbserial: USB Serial support registered for generic1590worker2 # [ 0.683500] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1591worker2 # [ 0.683500] Please enable it if your BIOS has the CPPC option.1592worker2 # [ 0.685737] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1593worker2 # [ 0.688614] drop_monitor: Initializing network drop monitor service1594worker2 # [ 0.689797] NET: Registered PF_INET6 protocol family1595worker2 # [ 0.692918] Segment Routing with IPv61596client # [ 0.675187] Key type asymmetric registered1597client # [ 0.675943] Asymmetric key parser 'x509' registered1598worker2 # [ 0.695515] In-situ OAM (IOAM) with IPv61599client # [ 0.676819] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1600worker2 # [ 0.696548] IPI shorthand broadcast: enabled1601lb # [ 0.657038] Initialise system trusted keyrings1602client # [ 0.682080] io scheduler mq-deadline registered1603lb # [ 0.659954] workingset: timestamp_bits=40 max_order=19 bucket_order=01604client # [ 0.682847] io scheduler kyber registered1605client # [ 0.685575] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1606worker2 # [ 0.704738] sched_clock: Marking stable (571014238, 133469213)->(773143174, -68659723)1607client # [ 0.686896] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1608worker2 # [ 0.709669] registered taskstats version 11609worker2 # [ 0.710643] Loading compiled-in X.509 certificates1610client # [ 0.692604] Linux agpgart interface v0.1031611client # [ 0.693410] ACPI: bus type drm_connector registered1612worker1 # [ 0.647120] Initialise system trusted keyrings1613client # [ 0.696466] usbcore: registered new interface driver usbserial_generic1614client # [ 0.697577] usbserial: USB Serial support registered for generic1615worker1 # [ 0.650214] workingset: timestamp_bits=40 max_order=19 bucket_order=01616client # [ 0.700949] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1617client # [ 0.700949] Please enable it if your BIOS has the CPPC option.1618client # [ 0.703236] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1619lb # [ 0.683018] Key type asymmetric registered1620lb # [ 0.683759] Asymmetric key parser 'x509' registered1621lb # [ 0.684644] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1622client # [ 0.707161] drop_monitor: Initializing network drop monitor service1623client # [ 0.708353] NET: Registered PF_INET6 protocol family1624worker2 # [ 0.728505] Demotion targets for Node 0: null1625lb # [ 0.688075] io scheduler mq-deadline registered1626worker2 # [ 0.729392] Key type .fscrypt registered1627lb # [ 0.688829] io scheduler kyber registered1628client # [ 0.711439] Segment Routing with IPv61629worker2 # [ 0.730085] Key type fscrypt-provisioning registered1630worker2 # [ 0.730982] ima: No TPM chip found, activating TPM-bypass!1631client # [ 0.713961] In-situ OAM (IOAM) with IPv61632client # [ 0.715156] IPI shorthand broadcast: enabled1633lb # [ 0.693068] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1634worker2 # [ 0.735505] ima: Allocated hash algorithm: sha11635lb # [ 0.694371] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1636worker2 # [ 0.736303] ima: No architecture policies found1637worker2 # [ 0.737252] PM: Magic number: 10:931:4851638worker1 # [ 0.671394] Key type asymmetric registered1639worker1 # [ 0.672078] Asymmetric key parser 'x509' registered1640lb # [ 0.699697] Linux agpgart interface v0.1031641worker1 # [ 0.672992] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1642worker2 # [ 0.740887] RAS: Correctable Errors collector initialized.1643lb # [ 0.700453] ACPI: bus type drm_connector registered1644client # [ 0.723501] sched_clock: Marking stable (588014488, 134908535)->(800045400, -77122377)1645worker1 # [ 0.678292] io scheduler mq-deadline registered1646worker1 # [ 0.679066] io scheduler kyber registered1647lb # [ 0.705347] usbcore: registered new interface driver usbserial_generic1648lb # [ 0.706447] usbserial: USB Serial support registered for generic1649client # [ 0.729141] registered taskstats version 11650client # [ 0.730103] Loading compiled-in X.509 certificates1651worker1 # [ 0.681707] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1652worker2 # [ 0.749574] clk: Disabling unused clocks1653worker2 # [ 0.750247] PM: genpd: Disabling unused power domains1654worker1 # [ 0.683046] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1655lb # [ 0.709912] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1656lb # [ 0.709912] Please enable it if your BIOS has the CPPC option.1657lb # [ 0.712215] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1658worker1 # [ 0.688444] Linux agpgart interface v0.1031659worker1 # [ 0.689234] ACPI: bus type drm_connector registered1660lb # [ 0.716040] drop_monitor: Initializing network drop monitor service1661lb # [ 0.717269] NET: Registered PF_INET6 protocol family1662worker1 # [ 0.692492] usbcore: registered new interface driver usbserial_generic1663worker1 # [ 0.693580] usbserial: USB Serial support registered for generic1664lb # [ 0.720365] Segment Routing with IPv61665lb # [ 0.722917] In-situ OAM (IOAM) with IPv61666worker1 # [ 0.696181] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1667lb # [ 0.723965] IPI shorthand broadcast: enabled1668worker1 # [ 0.696181] Please enable it if your BIOS has the CPPC option.1669worker1 # [ 0.698403] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1670client # [ 0.748942] Demotion targets for Node 0: null1671client # [ 0.749863] Key type .fscrypt registered1672client # [ 0.750607] Key type fscrypt-provisioning registered1673worker1 # [ 0.702312] drop_monitor: Initializing network drop monitor service1674client # [ 0.751591] ima: No TPM chip found, activating TPM-bypass!1675worker1 # [ 0.703484] NET: Registered PF_INET6 protocol family1676worker1 # [ 0.706698] Segment Routing with IPv61677lb # [ 0.732372] sched_clock: Marking stable (598014519, 133877021)->(811651182, -79759642)1678client # [ 0.755943] ima: Allocated hash algorithm: sha11679client # [ 0.756787] ima: No architecture policies found1680worker1 # [ 0.709197] In-situ OAM (IOAM) with IPv61681worker1 # [ 0.710257] IPI shorthand broadcast: enabled1682client # [ 0.758953] PM: Magic number: 10:931:4851683lb # [ 0.736991] registered taskstats version 11684lb # [ 0.737935] Loading compiled-in X.509 certificates1685client # [ 0.760726] RAS: Correctable Errors collector initialized.1686worker1 # [ 0.718575] sched_clock: Marking stable (583013979, 135142765)->(799025005, -80868261)1687client # [ 0.769688] clk: Disabling unused clocks1688worker1 # [ 0.724340] registered taskstats version 11689worker1 # [ 0.725320] Loading compiled-in X.509 certificates1690client # [ 0.773944] PM: genpd: Disabling unused power domains1691lb # [ 0.756913] Demotion targets for Node 0: null1692lb # [ 0.757821] Key type .fscrypt registered1693lb # [ 0.758513] Key type fscrypt-provisioning registered1694lb # [ 0.759484] ima: No TPM chip found, activating TPM-bypass!1695lb # [ 0.762909] ima: Allocated hash algorithm: sha11696lb # [ 0.763717] ima: No architecture policies found1697lb # [ 0.765910] PM: Magic number: 10:931:4851698lb # [ 0.767600] RAS: Correctable Errors collector initialized.1699worker1 # [ 0.741750] Demotion targets for Node 0: null1700worker1 # [ 0.745211] Key type .fscrypt registered1701worker1 # [ 0.745896] Key type fscrypt-provisioning registered1702worker1 # [ 0.746866] ima: No TPM chip found, activating TPM-bypass!1703worker1 # [ 0.749172] ima: Allocated hash algorithm: sha11704worker1 # [ 0.749979] ima: No architecture policies found1705worker1 # [ 0.752170] PM: Magic number: 10:931:4851706lb # [ 0.778633] clk: Disabling unused clocks1707worker1 # [ 0.753794] RAS: Correctable Errors collector initialized.1708lb # [ 0.780904] PM: genpd: Disabling unused power domains1709worker1 # [ 0.764857] clk: Disabling unused clocks1710worker1 # [ 0.767178] PM: genpd: Disabling unused power domains1711worker2 # [ 0.911533] Freeing initrd memory: 29092K1712worker2 # [ 0.914642] Freeing unused decrypted memory: 2028K1713worker2 # [ 0.917216] Freeing unused kernel image (initmem) memory: 3644K1714worker2 # [ 0.918307] Write protecting the kernel read-only data: 32768k1715worker2 # [ 0.920175] Freeing unused kernel image (text/rodata gap) memory: 1208K1716worker2 # [ 0.921680] Freeing unused kernel image (rodata/data gap) memory: 752K1717client # [ 0.940481] Freeing initrd memory: 29096K1718client # [ 0.943954] Freeing unused decrypted memory: 2028K1719client # [ 0.946925] Freeing unused kernel image (initmem) memory: 3644K1720client # [ 0.948005] Write protecting the kernel read-only data: 32768k1721client # [ 0.949979] Freeing unused kernel image (text/rodata gap) memory: 1208K1722client # [ 0.951640] Freeing unused kernel image (rodata/data gap) memory: 752K1723worker2 # [ 0.973009] x86/mm: Checked W+X mappings: passed, no W+X pages found.1724worker2 # [ 0.974105] Run /init as init process1725worker2 # [ 0.984121] systemd[1]: Inserted module 'autofs4'1726lb # [ 0.947498] Freeing initrd memory: 29092K1727lb # [ 0.950722] Freeing unused decrypted memory: 2028K1728lb # [ 0.953415] Freeing unused kernel image (initmem) memory: 3644K1729lb # [ 0.954487] Write protecting the kernel read-only data: 32768k1730lb # [ 0.956389] Freeing unused kernel image (text/rodata gap) memory: 1208K1731lb # [ 0.957936] Freeing unused kernel image (rodata/data gap) memory: 752K1732worker2 # [ 1.000412] fuse: init (API version 7.45)1733worker1 # [ 0.933897] Freeing initrd memory: 29092K1734worker1 # [ 0.937136] Freeing unused decrypted memory: 2028K1735worker2 # [ 1.006199] ACPI: \_SB_.GSIG: Enabled at IRQ 221736worker1 # [ 0.939832] Freeing unused kernel image (initmem) memory: 3644K1737worker1 # [ 0.940965] Write protecting the kernel read-only data: 32768k1738worker2 # [ 1.008875] ACPI: \_SB_.GSIH: Enabled at IRQ 231739worker1 # [ 0.942900] Freeing unused kernel image (text/rodata gap) memory: 1208K1740worker1 # [ 0.944499] Freeing unused kernel image (rodata/data gap) memory: 752K1741worker2 # [ 1.012225] ACPI: \_SB_.GSIE: Enabled at IRQ 201742worker2 # [ 1.014677] ACPI: \_SB_.GSIF: Enabled at IRQ 211743client # [ 1.002899] x86/mm: Checked W+X mappings: passed, no W+X pages found.1744client # [ 1.004058] Run /init as init process1745client # [ 1.015041] systemd[1]: Inserted module 'autofs4'1746worker2 # [ 1.040121] systemd[1]: Successfully made /usr/ read-only.1747lb # [ 1.009190] x86/mm: Checked W+X mappings: passed, no W+X pages found.1748client # [ 1.031946] fuse: init (API version 7.45)1749lb # [ 1.010253] Run /init as init process1750client # [ 1.037978] ACPI: \_SB_.GSIG: Enabled at IRQ 221751client # [ 1.040668] ACPI: \_SB_.GSIH: Enabled at IRQ 231752lb # [ 1.021136] systemd[1]: Inserted module 'autofs4'1753client # [ 1.043899] ACPI: \_SB_.GSIE: Enabled at IRQ 201754worker1 # [ 0.995659] x86/mm: Checked W+X mappings: passed, no W+X pages found.1755worker1 # [ 0.996760] Run /init as init process1756client # [ 1.046374] ACPI: \_SB_.GSIF: Enabled at IRQ 211757worker1 # [ 1.007256] systemd[1]: Inserted module 'autofs4'1758lb # [ 1.037220] fuse: init (API version 7.45)1759lb # [ 1.042996] ACPI: \_SB_.GSIG: Enabled at IRQ 221760lb # [ 1.045374] ACPI: \_SB_.GSIH: Enabled at IRQ 231761lb # [ 1.048723] ACPI: \_SB_.GSIE: Enabled at IRQ 201762worker1 # [ 1.023545] fuse: init (API version 7.45)1763lb # [ 1.051089] ACPI: \_SB_.GSIF: Enabled at IRQ 211764client # [ 1.075988] systemd[1]: Successfully made /usr/ read-only.1765worker1 # [ 1.029312] ACPI: \_SB_.GSIG: Enabled at IRQ 221766worker1 # [ 1.032315] ACPI: \_SB_.GSIH: Enabled at IRQ 231767worker1 # [ 1.035515] ACPI: \_SB_.GSIE: Enabled at IRQ 201768worker1 # [ 1.037700] ACPI: \_SB_.GSIF: Enabled at IRQ 211769lb # [ 1.077678] systemd[1]: Successfully made /usr/ read-only.1770worker1 # [ 1.064986] systemd[1]: Successfully made /usr/ read-only.1771worker2 # [ 1.376263] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772worker2 # [ 1.385581] systemd[1]: Detected virtualization kvm.1773worker2 # [ 1.387212] systemd[1]: Detected architecture x86-64.1774worker2 # [ 1.388054] systemd[1]: Running in initrd.1775worker2 # [ 1.389086] systemd[1]: Initializing machine ID from random generator.1776worker2 # [ 1.390210] systemd[1]: Hostname set to <worker2>.1777client # [ 1.411750] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778client # [ 1.417476] systemd[1]: Detected virtualization kvm.1779client # [ 1.418518] systemd[1]: Detected architecture x86-64.1780client # [ 1.419528] systemd[1]: Running in initrd.1781client # [ 1.420953] systemd[1]: Initializing machine ID from random generator.1782client # [ 1.422380] systemd[1]: Hostname set to <client>.1783lb # [ 1.413206] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784lb # [ 1.418063] systemd[1]: Detected virtualization kvm.1785lb # [ 1.418864] systemd[1]: Detected architecture x86-64.1786lb # [ 1.419712] systemd[1]: Running in initrd.1787lb # [ 1.420768] systemd[1]: Initializing machine ID from random generator.1788lb # [ 1.421921] systemd[1]: Hostname set to <lb>.1789worker1 # [ 1.401549] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker1 # [ 1.413477] systemd[1]: Detected virtualization kvm.1791worker1 # [ 1.415562] systemd[1]: Detected architecture x86-64.1792worker1 # [ 1.417652] systemd[1]: Running in initrd.1793worker1 # [ 1.420229] systemd[1]: Initializing machine ID from random generator.1794worker1 # [ 1.423059] systemd[1]: Hostname set to <worker1>.1795worker2 # [ 1.579569] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker2 # [ 1.617669] systemd[1]: Queued start job for default target Initrd Default Target.1797worker2 # [ 1.620777] systemd[1]: Created slice Slice /system/modprobe.1798worker2 # [ 1.621924] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1799worker2 # [ 1.623279] systemd[1]: Expecting device /dev/disk/by-label/nixos...1800worker2 # [ 1.624335] systemd[1]: Reached target Path Units.1801worker2 # [ 1.625167] systemd[1]: Reached target Slice Units.1802worker2 # [ 1.626019] systemd[1]: Reached target Swaps.1803worker2 # [ 1.626795] systemd[1]: Reached target Timer Units.1804worker2 # [ 1.627724] systemd[1]: Listening on D-Bus System Message Bus Socket.1805worker2 # [ 1.628890] systemd[1]: Listening on Journal Socket (/dev/log).1806worker2 # [ 1.630011] systemd[1]: Listening on Journal Sockets.1807worker2 # [ 1.630994] systemd[1]: Listening on udev Control Socket.1808worker2 # [ 1.632010] systemd[1]: Listening on udev Kernel Socket.1809worker2 # [ 1.632933] systemd[1]: Reached target Socket Units.1810worker2 # [ 1.634689] systemd[1]: Starting Create List of Static Device Nodes...1811worker2 # [ 1.638280] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1812worker2 # [ 1.645037] systemd[1]: Starting Load Kernel Module configfs...1813lb # [ 1.609106] systemd[1]: bpf-restrict-fs: LSM BPF program attached1814client # [ 1.638682] systemd[1]: bpf-restrict-fs: LSM BPF program attached1815worker2 # [ 1.659549] systemd[1]: Starting Journal Service...1816worker2 # [ 1.667561] systemd[1]: Starting Load Kernel Modules...1817worker2 # [ 1.677195] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1818lb # [ 1.647610] systemd[1]: Queued start job for default target Initrd Default Target.1819lb # [ 1.652214] systemd[1]: Created slice Slice /system/modprobe.1820worker2 # [ 1.694871] netfs: FS-Cache loaded1821lb # [ 1.653456] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1822lb # [ 1.654858] systemd[1]: Expecting device /dev/disk/by-label/nixos...1823lb # [ 1.656010] systemd[1]: Reached target Path Units.1824lb # [ 1.656901] systemd[1]: Reached target Slice Units.1825lb # [ 1.657769] systemd[1]: Reached target Swaps.1826lb # [ 1.658589] systemd[1]: Reached target Timer Units.1827lb # [ 1.659582] systemd[1]: Listening on D-Bus System Message Bus Socket.1828lb # [ 1.660799] systemd[1]: Listening on Journal Socket (/dev/log).1829lb # [ 1.661979] systemd[1]: Listening on Journal Sockets.1830worker2 # [ 1.703114] 9pnet: Installing 9P2000 support1831lb # [ 1.662969] systemd[1]: Listening on udev Control Socket.1832lb # [ 1.664073] systemd[1]: Listening on udev Kernel Socket.1833lb # [ 1.665059] systemd[1]: Reached target Socket Units.1834lb # [ 1.666839] systemd[1]: Starting Create List of Static Device Nodes...1835client # [ 1.691516] systemd[1]: Queued start job for default target Initrd Default Target.1836lb # [ 1.670708] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1837client # [ 1.695272] systemd[1]: Created slice Slice /system/modprobe.1838client # [ 1.696394] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1839worker1 # [ 1.646927] systemd[1]: bpf-restrict-fs: LSM BPF program attached1840client # [ 1.697768] systemd[1]: Expecting device /dev/disk/by-label/nixos...1841client # [ 1.698815] systemd[1]: Reached target Path Units.1842worker2 # [ 1.717758] systemd[1]: Starting Coldplug All udev Devices...1843client # [ 1.699675] systemd[1]: Reached target Slice Units.1844lb # [ 1.677472] systemd[1]: Starting Load Kernel Module configfs...1845client # [ 1.700538] systemd[1]: Reached target Swaps.1846client # [ 1.701298] systemd[1]: Reached target Timer Units.1847client # [ 1.702223] systemd[1]: Listening on D-Bus System Message Bus Socket.1848client # [ 1.703411] systemd[1]: Listening on Journal Socket (/dev/log).1849client # [ 1.704538] systemd[1]: Listening on Journal Sockets.1850client # [ 1.705527] systemd[1]: Listening on udev Control Socket.1851client # [ 1.706507] systemd[1]: Listening on udev Kernel Socket.1852client # [ 1.707415] systemd[1]: Reached target Socket Units.1853client # [ 1.709160] systemd[1]: Starting Create List of Static Device Nodes...1854worker2 # [ 1.728846] systemd-journald[67]: Collecting audit messages is disabled.1855client # [ 1.712763] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1856lb # [ 1.691076] systemd[1]: Starting Journal Service...1857worker2 # [ 1.734554] systemd[1]: Finished Create List of Static Device Nodes.1858worker2 # [ 1.739094] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1859client # [ 1.723023] systemd[1]: Starting Load Kernel Module configfs...1860lb # [ 1.704991] systemd[1]: Starting Load Kernel Modules...1861worker2 # [ 1.745879] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1862worker2 # [ 1.750859] systemd[1]: modprobe@configfs.service: Deactivated successfully.1863client # [ 1.733036] systemd[1]: Starting Journal Service...1864worker2 # [ 1.757830] systemd[1]: Finished Load Kernel Module configfs.1865client # [ 1.743103] systemd[1]: Starting Load Kernel Modules...1866worker2 # [ 1.763905] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1867lb # [ 1.711972] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1868lb # [ 1.728229] netfs: FS-Cache loaded1869worker1 # [ 1.701858] systemd[1]: Queued start job for default target Initrd Default Target.1870worker2 # [ 1.769582] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1871lb # [ 1.732964] 9pnet: Installing 9P2000 support1872worker1 # [ 1.706471] systemd[1]: Created slice Slice /system/modprobe.1873worker2 # [ 1.774562] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1874worker1 # [ 1.707630] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1875worker1 # [ 1.708969] systemd[1]: Expecting device /dev/disk/by-label/nixos...1876worker1 # [ 1.710021] systemd[1]: Reached target Path Units.1877worker1 # [ 1.710852] systemd[1]: Reached target Slice Units.1878worker2 # [ 1.777615] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1879worker1 # [ 1.711703] systemd[1]: Reached target Swaps.1880worker1 # [ 1.712471] systemd[1]: Reached target Timer Units.1881client # [ 1.749001] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1882worker1 # [ 1.713391] systemd[1]: Listening on D-Bus System Message Bus Socket.1883worker1 # [ 1.714534] systemd[1]: Listening on Journal Socket (/dev/log).1884worker1 # [ 1.715631] systemd[1]: Listening on Journal Sockets.1885lb # [ 1.742080] systemd[1]: Starting Coldplug All udev Devices...1886worker1 # [ 1.716601] systemd[1]: Listening on udev Control Socket.1887worker1 # [ 1.717578] systemd[1]: Listening on udev Kernel Socket.1888worker1 # [ 1.718536] systemd[1]: Reached target Socket Units.1889worker1 # [ 1.720220] systemd[1]: Starting Create List of Static Device Nodes...1890client # [ 1.770960] netfs: FS-Cache loaded1891worker1 # [ 1.723964] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1892client # [ 1.776384] 9pnet: Installing 9P2000 support1893lb # [ 1.755990] systemd[1]: Finished Create List of Static Device Nodes.1894worker1 # [ 1.730736] systemd[1]: Starting Load Kernel Module configfs...1895lb # [ 1.762483] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1896worker2 # [ 1.804983] systemd[1]: Finished Load Kernel Modules.1897client # [ 1.788013] systemd[1]: Starting Coldplug All udev Devices...1898worker2 # [ 1.810677] systemd[1]: Starting Apply Kernel Variables...1899lb # [ 1.770391] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1900worker1 # [ 1.745206] systemd[1]: Starting Journal Service...1901lb # [ 1.775336] systemd[1]: modprobe@configfs.service: Deactivated successfully.1902lb # [ 1.779127] systemd-journald[67]: Collecting audit messages is disabled.1903worker2 # [ 1.821595] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1904client # [ 1.803608] systemd[1]: Finished Create List of Static Device Nodes.1905worker1 # [ 1.755241] systemd[1]: Starting Load Kernel Modules...1906lb # [ 1.782279] systemd[1]: Finished Load Kernel Module configfs.1907client # [ 1.810635] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1908lb # [ 1.787857] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1909worker2 # [ 1.830728] systemd[1]: Starting Create Static Device Nodes in /dev...1910lb # [ 1.791469] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1911client # [ 1.815666] systemd-journald[67]: Collecting audit messages is disabled.1912client # [ 1.820314] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1913lb # [ 1.800975] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1914worker1 # [ 1.763197] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1915client # [ 1.824337] systemd[1]: modprobe@configfs.service: Deactivated successfully.1916lb # [ 1.802447] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1917worker2 # [ 1.847805] systemd[1]: Finished Apply Kernel Variables.1918worker1 # [ 1.780879] netfs: FS-Cache loaded1919client # [ 1.829302] systemd[1]: Finished Load Kernel Module configfs.1920worker2 # [ 1.718170] systemd-modules-load[68]: Inserted module 'dm_mod'1921worker2 # [ 1.719355] systemd-modules-load[68]: Inserted module 'virtio_balloon'1922worker1 # [ 1.785607] 9pnet: Installing 9P2000 support1923client # [ 1.833362] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1924worker2 # [ 1.853797] systemd[1]: Started Journal Service.1925worker2 # [ 1.722299] systemd-modules-load[68]: Inserted module 'virtio_gpu'1926client # [ 1.837359] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1927client # [ 1.841945] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1928worker1 # [ 1.794920] systemd[1]: Starting Coldplug All udev Devices...1929client # [ 1.847000] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1930worker2 # [ 1.735195] systemd[1]: Finished Create Static Device Nodes in /dev.1931worker2 # [ 1.737129] systemd[1]: Reached target Preparation for Local File Systems.1932lb # [ 1.830406] systemd[1]: Finished Load Kernel Modules.1933worker2 # [ 1.738754] systemd[1]: Reached target Local File Systems.1934worker2 # [ 1.741858] systemd[1]: Starting Create System Files and Directories...1935worker1 # [ 1.808726] systemd[1]: Finished Create List of Static Device Nodes.1936lb # [ 1.839101] systemd[1]: Starting Apply Kernel Variables...1937worker1 # [ 1.813720] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1938worker2 # [ 1.749106] systemd[1]: Starting Rule-based Manager for Device Events and Files...1939worker1 # [ 1.821584] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1940client # [ 1.872466] systemd[1]: Finished Load Kernel Modules.1941lb # [ 1.851965] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1942worker1 # [ 1.826831] systemd-journald[67]: Collecting audit messages is disabled.1943worker1 # [ 1.828452] systemd[1]: modprobe@configfs.service: Deactivated successfully.1944client # [ 1.881140] systemd[1]: Starting Apply Kernel Variables...1945worker1 # [ 1.834506] systemd[1]: Finished Load Kernel Module configfs.1946lb # [ 1.861118] systemd[1]: Starting Create Static Device Nodes in /dev...1947worker2 # [ 1.772433] systemd[1]: Finished Create System Files and Directories.1948worker1 # [ 1.839794] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1949worker1 # [ 1.842352] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1950client # [ 1.897236] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1951lb # [ 1.877005] systemd[1]: Finished Apply Kernel Variables.1952worker1 # [ 1.851232] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1953worker1 # [ 1.854310] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1954client # [ 1.906121] systemd[1]: Starting Create Static Device Nodes in /dev...1955worker2 # [ 1.794347] systemd-udevd[83]: Using default interface naming scheme 'v261'.1956lb # [ 1.893677] systemd[1]: Started Journal Service.1957lb # [ 1.761212] systemd-modules-load[68]: Inserted module 'dm_mod'1958lb # [ 1.763443] systemd-modules-load[68]: Inserted module 'virtio_balloon'1959lb # [ 1.764495] systemd-modules-load[68]: Inserted module 'virtio_gpu'1960client # [ 1.921550] systemd[1]: Finished Apply Kernel Variables.1961lb # [ 1.768301] systemd[1]: Finished Create Static Device Nodes in /dev.1962lb # [ 1.769461] systemd[1]: Reached target Preparation for Local File Systems.1963lb # [ 1.770597] systemd[1]: Reached target Local File Systems.1964worker1 # [ 1.882744] systemd[1]: Finished Load Kernel Modules.1965lb # [ 1.776091] systemd[1]: Starting Create System Files and Directories...1966worker2 # [ 1.818316] systemd[1]: Started Rule-based Manager for Device Events and Files.1967lb # [ 1.779130] systemd[1]: Starting Rule-based Manager for Device Events and Files...1968worker1 # [ 1.889365] systemd[1]: Starting Apply Kernel Variables...1969client # [ 1.802216] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.938233] systemd[1]: Started Journal Service.1970client # 1971client # [ 1.805306] systemd-modules-load[68]: Inserted module 'virtio_balloon'1972client # [ 1.807754] systemd-modules-load[68]: Inserted module 'virtio_gpu'1973client # [ 1.813859] systemd[1]: Finished Create Static Device Nodes in /dev.1974worker1 # [ 1.903237] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1975client # [ 1.821102] systemd[1]: Reached target Preparation for Local File Systems.1976client # [ 1.822205] systemd[1]: Reached target Local File Systems.1977client # [ 1.825088] systemd[1]: Starting Create System Files and Directories...1978worker1 # [ 1.912337] systemd[1]: Starting Create Static Device Nodes in /dev...1979client # [ 1.830071] systemd[1]: Starting Rule-based Manager for Device Events and Files...1980lb # [ 1.810176] systemd[1]: Finished Create System Files and Directories.1981worker1 # [ 1.924325] systemd[1]: Finished Apply Kernel Variables.1982worker2 # [ 1.870470] systemd[1]: Finished Coldplug All udev Devices.1983worker2 # [ 1.872081] systemd[1]: Reached target System Initialization.1984worker2 # [ 1.874182] systemd[1]: Reached target Basic System.1985lb # [ 1.835248] systemd-udevd[78]: Using default interface naming scheme 'v261'.1986worker1 # [ 1.808529] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.944876] systemd[1]: Started Journal Service.1987worker1 # 1988worker1 # [ 1.812574] systemd-modules-load[68]: Inserted module 'virtio_balloon'1989client # [ 1.861474] systemd[1]: Finished Create System Files and Directories.1990worker1 # [ 1.813599] systemd-modules-load[68]: Inserted module 'virtio_gpu'1991worker1 # [ 1.817974] systemd[1]: Finished Create Static Device Nodes in /dev.1992worker1 # [ 1.825094] systemd[1]: Reached target Preparation for Local File Systems.1993worker1 # [ 1.826120] systemd[1]: Reached target Local File Systems.1994worker1 # [ 1.830068] systemd[1]: Starting Create System Files and Directories...1995client # [ 1.880902] systemd-udevd[83]: Using default interface naming scheme 'v261'.1996lb # [ 1.861073] systemd[1]: Started Rule-based Manager for Device Events and Files.1997worker1 # [ 1.835135] systemd[1]: Starting Rule-based Manager for Device Events and Files...1998client # [ 1.905073] systemd[1]: Started Rule-based Manager for Device Events and Files.1999worker1 # [ 1.858178] systemd[1]: Finished Create System Files and Directories.2000lb # [ 1.901926] systemd[1]: Finished Coldplug All udev Devices.
loading 1,926 hidden lines…
3927client # [ 12.997260] dhcpcd[566]: eth0: soliciting an IPv6 router3928client # [ 12.999563] dhcpcd[566]: eth0: Router Advertisement from fe80::23929client # [ 13.001688] dhcpcd[566]: eth0: adding address fec0::5054:ff:fe12:3456/643930client # [ 13.003838] dhcpcd[566]: eth0: adding route to fec0::/643931client # [ 13.005742] dhcpcd[566]: eth0: adding default route via fe80::23932lb # [ 13.011875] postgres[784]: [784] LOG: starting PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by clang version 21.1.8, 64-bit3933lb # [ 13.017342] postgres[784]: [784] LOG: listening on IPv6 address "::1", port 54323934lb # [ 13.018419] postgres[784]: [784] LOG: listening on IPv4 address "127.0.0.1", port 54323935lb # [ 13.020913] postgres[784]: [784] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"3936lb # [ 13.034156] postgres[793]: [793] LOG: database system was shut down at 2026-09-20 14:30:11 GMT3937lb # [ 13.039198] postgres[784]: [784] LOG: database system is ready to accept connections3938lb # [ 13.045209] systemd[1]: Started PostgreSQL Server.3939lb # [ 13.050134] systemd[1]: Starting PostgreSQL Setup Scripts...3940lb # [ 13.215801] postgresql-setup-start[804]: CREATE DATABASE3941lb # [ 13.260026] postgresql-setup-start[809]: CREATE ROLE3942lb # [ 13.286328] postgresql-setup-start[815]: ALTER DATABASE3943lb # [ 13.292884] systemd[1]: Finished PostgreSQL Setup Scripts.3944lb # [ 13.295452] systemd[1]: Reached target PostgreSQL.3945worker2 # [ 13.461301] dhcpcd[694]: eth0: soliciting an IPv6 router3946worker2 # [ 13.462532] dhcpcd[694]: eth0: Router Advertisement from fe80::23947worker2 # [ 13.463305] dhcpcd[694]: eth0: adding address fec0::5054:ff:fe12:3456/643948worker2 # [ 13.464124] dhcpcd[694]: eth0: adding route to fec0::/643949worker2 # [ 13.464836] dhcpcd[694]: eth0: adding default route via fe80::23950worker1 # [ 15.227687] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds3951worker1 # [ 15.230590] dhcpcd[692]: eth0: adding route to 10.0.2.0/243952worker1 # [ 15.232946] dhcpcd[692]: eth0: adding default route via 10.0.2.23953worker1 # [ 15.343541] systemd[1]: Started DHCP Client.3954worker1 # [ 15.345612] systemd[1]: Reached target Multi-User System.3955worker1 # [ 15.346531] systemd[1]: Startup finished in 865ms (kernel) + 3.388s (initrd) + 11.092s (userspace) = 15.345s.3956lb # [ 16.225641] dhcpcd[718]: eth0: leased 10.0.2.15 for 86400 seconds3957lb # [ 16.228431] dhcpcd[718]: eth0: adding route to 10.0.2.0/243958lb # [ 16.230558] dhcpcd[718]: eth0: adding default route via 10.0.2.23959lb # [ 16.369299] systemd[1]: Started DHCP Client.3960lb # [ 16.371145] systemd[1]: Reached target Network is Online.3961lb # [ 16.374667] systemd[1]: Started Envoy reverse proxy.3962lb # [ 16.378581] systemd[1]: Started mock-oidc.service.3963lb # [ 16.453839] mock-oidc-server[903]: Mock OIDC Server running3964lb # [ 16.455528] mock-oidc-server[903]: OIDC Address: 192.168.1.2:80803965lb # [ 16.456561] mock-oidc-server[903]: Issue Address: 0.0.0.0:80813966lb # [ 16.457600] mock-oidc-server[903]: Issuer: http://192.168.1.2:8080/oidc3967lb # [ 16.458711] mock-oidc-server[903]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3968lb # [ 16.459820] mock-oidc-server[903]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3969lb # [ 16.461167] mock-oidc-server[903]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3970worker2 # [ 16.546932] dhcpcd[694]: eth0: leased 10.0.2.15 for 86400 seconds3971worker2 # [ 16.547996] dhcpcd[694]: eth0: adding route to 10.0.2.0/243972worker2 # [ 16.548947] dhcpcd[694]: eth0: adding default route via 10.0.2.23973worker2 # [ 16.655656] systemd[1]: Started DHCP Client.3974worker2 # [ 16.657222] systemd[1]: Reached target Multi-User System.3975worker2 # [ 16.657964] systemd[1]: Startup finished in 844ms (kernel) + 5.054s (initrd) + 10.758s (userspace) = 16.656s.3976lb # [ 16.874354] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3977lb # [ 16.876141] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:469] statically linked extensions:3978lb # [ 16.877486] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3979lb # [ 16.879614] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13980lb # [ 16.881648] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3981lb # [ 16.894974] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3982lb # [ 16.896716] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3983lb # [ 16.899130] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3984lb # [ 16.902972] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3985lb # [ 16.907111] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3986lb # [ 16.908795] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3987lb # [ 16.910745] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3988lb # [ 16.913596] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3989lb # [ 16.915310] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3990lb # [ 16.917082] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3991lb # [ 16.919648] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3992lb # [ 16.921875] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3993lb # [ 16.924863] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3994lb # [ 16.926693] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3995lb # [ 16.928967] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3996lb # [ 16.930795] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3997lb # [ 16.932481] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3998lb # [ 16.936577] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3999lb # [ 16.940232] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4000lb # [ 16.942779] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4001lb # [ 16.946775] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4002lb # [ 16.948547] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4003lb # [ 16.951651] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4004lb # [ 16.953895] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4005lb # [ 16.956106] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4006lb # [ 16.957633] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4007lb # [ 16.959863] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4008lb # [ 16.964677] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4009lb # [ 16.966842] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4010lb # [ 16.971464] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4011lb # [ 16.974118] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4012lb # [ 16.975895] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4013lb # [ 16.978335] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4014lb # [ 16.983698] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4015lb # [ 16.985181] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4016lb # [ 16.987771] envoy[902]: [2026-09-20 14:30:16.054][902][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4017lb # [ 16.990266] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4018lb # [ 16.992639] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4019lb # [ 17.017904] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4020lb # [ 17.019570] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4021lb # [ 17.021367] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4022lb # [ 17.023788] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4023lb # [ 17.034933] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4024lb # [ 17.036853] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4025lb # [ 17.039949] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4026lb # [ 17.044068] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4027lb # [ 17.045853] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24028lb # [ 17.048165] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4029lb # [ 17.050070] envoy[902]: [2026-09-20 14:30:16.060][902][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4030lb # [ 17.051826] envoy[902]: [2026-09-20 14:30:16.062][902][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4031lb # [ 17.054574] envoy[902]: [2026-09-20 14:30:16.062][902][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4032lb # [ 17.056701] envoy[902]: [2026-09-20 14:30:16.062][902][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4033lb # [ 17.059151] envoy[902]: [2026-09-20 14:30:16.062][902][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4034lb # [ 17.060700] envoy[902]: [2026-09-20 14:30:16.062][902][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4035lb # [ 17.063505] envoy[902]: [2026-09-20 14:30:16.062][902][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4036lb # [ 17.065372] envoy[902]: [2026-09-20 14:30:16.062][902][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24037lb # [ 17.067038] envoy[902]: [2026-09-20 14:30:16.062][902][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4038lb # [ 17.068690] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4039lb # [ 17.070746] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4040lb # [ 17.073167] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24041lb # [ 17.074916] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4042lb # [ 17.077223] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84043lb # [ 17.079068] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4044lb # [ 17.082046] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4045lb # [ 17.084471] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4046lb # [ 17.090570] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4047lb # [ 17.092384] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4048lb # [ 17.094074] envoy[902]: [2026-09-20 14:30:16.068][902][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4049lb # [ 17.096955] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4050lb # [ 17.102949] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4051lb # [ 17.104982] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4052lb # [ 17.106744] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4053lb # [ 17.108909] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4054lb # [ 17.110665] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4055lb # [ 17.116111] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4056lb # [ 17.122413] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router4057lb # [ 17.124169] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4058lb # [ 17.126247] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4059lb # [ 17.128507] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4060lb # [ 17.130449] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4061lb # [ 17.133475] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4062lb # [ 17.142540] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4063lb # [ 17.144895] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4064lb # [ 17.146892] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4065lb # [ 17.148571] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4066lb # [ 17.150764] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4067lb # [ 17.152664] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4068lb # [ 17.155097] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4069lb # [ 17.159740] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4070lb # [ 17.161420] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4071lb # [ 17.163203] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4072lb # [ 17.165943] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4073lb # [ 17.172994] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4074lb # [ 17.175857] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4075lb # [ 17.177461] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4076lb # [ 17.179797] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4077lb # [ 17.182770] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4078lb # [ 17.184701] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4079lb # [ 17.187195] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4080lb # [ 17.189091] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4081lb # [ 17.190818] envoy[902]: [2026-09-20 14:30:16.069][902][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4082lb # [ 17.192692] envoy[902]: [2026-09-20 14:30:16.228][902][info][main] [source/server/server.cc:531] HTTP header map info:4083lb # [ 17.193921] envoy[902]: [2026-09-20 14:30:16.262][902][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4084lb # [ 17.206644] envoy[902]: [2026-09-20 14:30:16.262][902][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4085lb # [ 17.208451] envoy[902]: [2026-09-20 14:30:16.262][902][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4086lb # [ 17.215867] envoy[902]: [2026-09-20 14:30:16.262][902][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4087lb # [ 17.246663] envoy[902]: [2026-09-20 14:30:16.427][902][info][main] [source/server/server.cc:968] runtime: {}4088lb # [ 17.252081] envoy[902]: [2026-09-20 14:30:16.432][902][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4089lb # [ 17.254111] envoy[902]: [2026-09-20 14:30:16.434][902][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014090lb # [ 17.256519] envoy[902]: [2026-09-20 14:30:16.437][902][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014091lb # [ 17.259278] envoy[902]: [2026-09-20 14:30:16.440][902][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4092lb # [ 17.260929] envoy[902]: [2026-09-20 14:30:16.441][902][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4093lb # [ 17.262440] envoy[902]: [2026-09-20 14:30:16.441][902][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4094lb # [ 17.304457] envoy[902]: [2026-09-20 14:30:16.485][902][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4095worker2 # [ 17.347318] systemd[1]: Reached target Network is Online.4096worker2 # [ 17.350431] systemd[1]: Starting scheduler-token.service...4097worker1 # [ 17.281571] systemd[1]: Starting Nix worker-protocol over gRPC...4098lb # [ 17.324890] envoy[902]: [2026-09-20 14:30:16.505][902][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4099lb # [ 17.327269] envoy[902]: [2026-09-20 14:30:16.507][902][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4100lb # [ 17.330279] envoy[902]: [2026-09-20 14:30:16.508][902][info][main] [source/server/server.cc:1077] starting main dispatch loop4101client # [ 17.418762] dhcpcd[566]: eth0: leased 10.0.2.15 for 86400 seconds4102client # [ 17.420091] dhcpcd[566]: eth0: adding route to 10.0.2.0/244103client # [ 17.421164] dhcpcd[566]: eth0: adding default route via 10.0.2.24104worker2 # [ 17.458375] systemd[1]: scheduler-token.service: Deactivated successfully.4105worker2 # [ 17.459925] systemd[1]: Finished scheduler-token.service.4106worker2 # [ 17.461079] systemd[1]: scheduler-token.service: Consumed 32ms CPU time over 109ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4107worker2 # [ 17.466328] systemd[1]: Starting Nix worker-protocol over gRPC...4108client # [ 17.521704] systemd[1]: Started DHCP Client.4109client # [ 17.524089] systemd[1]: Reached target Multi-User System.4110client # [ 17.525211] systemd[1]: Startup finished in 872ms (kernel) + 4.988s (initrd) + 11.664s (userspace) = 17.525s.4111worker1 # [ 17.479195] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4112worker1 # [ 17.488564] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4113worker1 # [ 17.493440] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14114worker1 # [ 17.535715] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4115worker1 # [ 17.538710] systemd[1]: Started Nix worker-protocol over gRPC.4116lb # [ 17.616659] envoy[902]: [2026-09-20 14:30:16.797][902][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4117worker2 # [ 17.658663] nix-grpc-daemon[789]: ts=2026-09-20T14:30:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4118lb # [ 17.618748] envoy[902]: [2026-09-20 14:30:16.799][902][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4119lb # [ 17.620549] envoy[902]: [2026-09-20 14:30:16.799][902][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4120lb # [ 17.622398] envoy[902]: [2026-09-20 14:30:16.799][902][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4121worker2 # [ 17.669114] nix-grpc-daemon[789]: ts=2026-09-20T14:30:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4122worker2 # [ 17.673781] nix-grpc-daemon[789]: ts=2026-09-20T14:30:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14123worker1 # [ 17.611161] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=scheduler_connected addr=lb:500514124worker1 # [ 17.616741] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4125worker2 # [ 17.720281] nix-grpc-daemon[789]: ts=2026-09-20T14:30:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4126worker2 # [ 17.725565] systemd[1]: Started Nix worker-protocol over gRPC.4127worker2 # [ 17.801403] nix-grpc-daemon[789]: ts=2026-09-20T14:30:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4128worker1 # [ 17.817941] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=scheduler_connected addr=lb:500514129worker1 # [ 17.821836] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4130worker2 # [ 18.004488] nix-grpc-daemon[789]: ts=2026-09-20T14:30:16Z level=info event=scheduler_connected addr=lb:500514131worker2 # [ 18.008366] nix-grpc-daemon[789]: ts=2026-09-20T14:30:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4132worker1 # [ 18.022531] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=scheduler_connected addr=lb:500514133worker1 # [ 18.024535] nix-grpc-daemon[782]: ts=2026-09-20T14:30:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4134worker2 # [ 18.208821] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514135worker2 # [ 18.210515] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4136worker1 # [ 18.225686] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514137worker1 # [ 18.229617] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4138worker2 # [ 18.411852] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514139worker2 # [ 18.415682] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker1 # [ 18.430443] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514141worker1 # [ 18.434370] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker2 # [ 18.616836] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514143worker2 # [ 18.620606] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 18.635789] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514145worker1 # [ 18.639572] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4146worker2 # [ 18.823141] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514147worker2 # [ 18.826268] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4148worker1 # [ 18.840422] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514149worker1 # [ 18.844245] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4150worker2 # [ 19.024149] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514151worker2 # [ 19.028111] nix-grpc-daemon[789]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4152worker1 # [ 19.045089] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_connected addr=lb:500514153worker1 # [ 19.048898] nix-grpc-daemon[782]: ts=2026-09-20T14:30:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4154worker2 # [ 19.229515] nix-grpc-daemon[789]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514155worker2 # [ 19.233243] nix-grpc-daemon[789]: ts=2026-09-20T14:30:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4156worker1 # [ 19.249280] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514157worker1 # [ 19.251115] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4158worker2 # [ 19.434243] nix-grpc-daemon[789]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514159worker2 # [ 19.438227] nix-grpc-daemon[789]: ts=2026-09-20T14:30:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4160worker1 # [ 19.452297] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514161worker1 # [ 19.456652] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker2 # [ 19.639824] nix-grpc-daemon[789]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514163worker2 # [ 19.643599] nix-grpc-daemon[789]: ts=2026-09-20T14:30:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4164worker1 # [ 19.657878] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514165worker1 # [ 19.661496] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4166worker2 # [ 19.845732] nix-grpc-daemon[789]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514167worker2 # [ 19.848768] nix-grpc-daemon[789]: ts=2026-09-20T14:30:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4168worker1 # [ 19.859617] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514169worker1 # [ 19.863529] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4170worker2 # [ 20.046888] nix-grpc-daemon[789]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514171worker2 # [ 20.050724] nix-grpc-daemon[789]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4172worker1 # [ 20.064261] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_connected addr=lb:500514173worker1 # [ 20.068157] nix-grpc-daemon[782]: ts=2026-09-20T14:30:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4174worker2 # [ 20.252123] nix-grpc-daemon[789]: ts=2026-09-20T14:30:19Z level=info event=scheduler_connected addr=lb:500514175worker2 # [ 20.255892] nix-grpc-daemon[789]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker1 # [ 20.269096] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_connected addr=lb:500514177worker1 # [ 20.272989] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4178worker2 # [ 20.457191] nix-grpc-daemon[789]: ts=2026-09-20T14:30:19Z level=info event=scheduler_connected addr=lb:500514179worker2 # [ 20.461092] nix-grpc-daemon[789]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker1 # [ 20.473919] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_connected addr=lb:500514181worker1 # [ 20.477872] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4182worker2 # [ 20.661654] nix-grpc-daemon[789]: ts=2026-09-20T14:30:19Z level=info event=scheduler_connected addr=lb:500514183worker2 # [ 20.663582] nix-grpc-daemon[789]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker1 # [ 20.680193] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_connected addr=lb:500514185worker1 # [ 20.683332] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4186worker2 # [ 20.865987] nix-grpc-daemon[789]: ts=2026-09-20T14:30:19Z level=info event=scheduler_connected addr=lb:500514187worker2 # [ 20.869771] nix-grpc-daemon[789]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker1 # [ 20.880590] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_connected addr=lb:500514189worker1 # [ 20.882419] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4190worker2 # [ 21.071145] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514191worker2 # [ 21.074873] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192worker1 # [ 21.083405] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_connected addr=lb:500514193worker1 # [ 21.087566] nix-grpc-daemon[782]: ts=2026-09-20T14:30:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4194worker2 # [ 21.275366] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514195worker2 # [ 21.277119] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker1 # [ 21.288540] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514197worker1 # [ 21.292552] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4198worker2 # [ 21.477849] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514199worker2 # [ 21.479540] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4200worker1 # [ 21.493588] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514201worker1 # [ 21.497572] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4202worker2 # [ 21.680831] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514203worker2 # [ 21.684576] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204worker1 # [ 21.698534] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514205worker1 # [ 21.700542] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4206worker2 # [ 21.886332] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514207worker2 # [ 21.889917] nix-grpc-daemon[789]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4208worker1 # [ 21.901176] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514209worker1 # [ 21.902998] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4210worker2 # [ 22.091387] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_connected addr=lb:500514211worker2 # [ 22.095152] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4212worker1 # [ 22.103462] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_connected addr=lb:500514213worker1 # [ 22.105295] nix-grpc-daemon[782]: ts=2026-09-20T14:30:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4214worker2 # [ 22.296134] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_connected addr=lb:500514215worker2 # [ 22.299799] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4216worker1 # [ 22.306353] nix-grpc-daemon[782]: ts=2026-09-20T14:30:21Z level=info event=scheduler_connected addr=lb:500514217worker1 # [ 22.310228] nix-grpc-daemon[782]: ts=2026-09-20T14:30:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4218worker2 # [ 22.500948] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_connected addr=lb:500514219worker2 # [ 22.504631] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker1 # [ 22.510975] nix-grpc-daemon[782]: ts=2026-09-20T14:30:21Z level=info event=scheduler_connected addr=lb:500514221worker1 # [ 22.514961] nix-grpc-daemon[782]: ts=2026-09-20T14:30:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4222worker2 # [ 22.705781] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_connected addr=lb:500514223worker2 # [ 22.709460] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker1 # [ 22.717136] nix-grpc-daemon[782]: ts=2026-09-20T14:30:21Z level=info event=scheduler_connected addr=lb:500514225worker1 # [ 22.720635] nix-grpc-daemon[782]: ts=2026-09-20T14:30:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4226worker2 # [ 22.911779] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_connected addr=lb:500514227worker2 # [ 22.914973] nix-grpc-daemon[789]: ts=2026-09-20T14:30:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker1 # [ 22.917954] nix-grpc-daemon[782]: ts=2026-09-20T14:30:21Z level=info event=scheduler_connected addr=lb:500514229worker1 # [ 22.921883] nix-grpc-daemon[782]: ts=2026-09-20T14:30:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4230worker2 # [ 23.116223] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514231worker2 # [ 23.119865] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker1 # [ 23.122358] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514233worker1 # [ 23.124288] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4234worker2 # [ 23.321123] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514235worker2 # [ 23.324808] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker1 # [ 23.325157] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514237worker1 # [ 23.329174] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4238worker2 # [ 23.525675] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514239worker2 # [ 23.529358] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker1 # [ 23.529908] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514241worker1 # [ 23.533855] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4242worker2 # [ 23.730160] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514243worker2 # [ 23.733880] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker1 # [ 23.736170] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514245worker1 # [ 23.739509] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4246worker2 # [ 23.935793] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514247worker2 # [ 23.939526] nix-grpc-daemon[789]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248worker1 # [ 23.936840] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_connected addr=lb:500514249worker1 # [ 23.940722] nix-grpc-daemon[782]: ts=2026-09-20T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4250worker2 # [ 24.140896] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 24.145378] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252worker1 # [ 24.141751] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514253worker1 # [ 24.145694] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4254worker2 # [ 24.345780] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514255worker2 # [ 24.349527] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker1 # [ 24.346606] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514257worker1 # [ 24.350542] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4258worker2 # [ 24.550278] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514259worker2 # [ 24.551969] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4260worker1 # [ 24.551142] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514261worker1 # [ 24.554895] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4262worker2 # [ 24.752497] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514263worker2 # [ 24.754250] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264worker1 # [ 24.756969] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514265worker1 # [ 24.760671] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4266lb # [ 24.836290] rustfs-bucket-start[992]: mb s3://farm4267lb # [ 24.841777] systemd[1]: Finished rustfs-bucket.service.4268lb # [ 24.847732] systemd[1]: Starting niks3 server...4269worker2 # [ 24.955445] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514270worker2 # [ 24.956992] nix-grpc-daemon[789]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker1 # [ 24.958392] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_connected addr=lb:500514272worker1 # [ 24.962199] nix-grpc-daemon[782]: ts=2026-09-20T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273lb # [ 25.052516] postgres[1008]: [1008] ERROR: relation "goose_db_version" does not exist at character 364274lb # [ 25.053788] postgres[1008]: [1008] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4275lb # [ 25.092449] niks3-server[1001]: 2026/09/20 14:30:24 OK 20241026095416_initial_model.sql (20.35ms)4276lb # [ 25.095963] niks3-server[1001]: 2026/09/20 14:30:24 OK 20251210153512_drop_unused_gin_index.sql (2.37ms)4277lb # [ 25.100856] niks3-server[1001]: 2026/09/20 14:30:24 OK 20251218171726_add_pins.sql (3.35ms)4278lb # [ 25.106046] niks3-server[1001]: 2026/09/20 14:30:24 OK 20260628120000_add_object_size_and_stats.sql (5.39ms)4279lb # [ 25.110339] niks3-server[1001]: 2026/09/20 14:30:24 OK 20260905000000_add_claims.sql (2.91ms)4280lb # [ 25.114707] niks3-server[1001]: 2026/09/20 14:30:24 OK 20260920000000_drop_claims.sql (4.27ms)4281lb # [ 25.115870] niks3-server[1001]: 2026/09/20 14:30:24 goose: successfully migrated database to version: 202609200000004282worker2 # [ 25.157790] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_connected addr=lb:500514283worker2 # [ 25.159446] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4284lb # [ 25.122233] niks3-server[1001]: 2026/09/20 14:30:24 OK 1_commit_pending_closure.sql (7.48ms)4285lb # [ 25.124732] niks3-server[1001]: 2026/09/20 14:30:24 OK 2_object_stats_trigger.sql (2.51ms)4286lb # [ 25.125987] niks3-server[1001]: 2026/09/20 14:30:24 goose: up to current file version: 24287lb # [ 25.132609] niks3-server[1001]: 2026/09/20 14:30:24 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4288worker1 # [ 25.162758] nix-grpc-daemon[782]: ts=2026-09-20T14:30:24Z level=info event=scheduler_connected addr=lb:500514289worker1 # [ 25.166594] nix-grpc-daemon[782]: ts=2026-09-20T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4290lb # [ 25.215953] niks3-server[1001]: 2026/09/20 14:30:24 INFO Read proxy enabled — serving cache objects from S34291lb # [ 25.218306] niks3-server[1001]: 2026/09/20 14:30:24 INFO Using socket-activated listener address=0.0.0.0:57514292lb # [ 25.224794] niks3-server[1001]: 2026/09/20 14:30:24 INFO systemd watchdog enabled interval=15s4293lb # [ 25.227219] systemd[1]: Started niks3 server.4294lb # [ 25.228141] niks3-server[1001]: 2026/09/20 14:30:24 INFO Starting HTTP server address=0.0.0.0:57514295lb # [ 25.230667] systemd[1]: Reached target Multi-User System.4296lb # [ 25.232228] systemd[1]: Startup finished in 879ms (kernel) + 3.932s (initrd) + 20.419s (userspace) = 25.232s.4297lb # [ 25.236439] niks3-server[1001]: 2026/09/20 14:30:24 INFO lead: acquired remote=192.168.1.4:453344298worker2 # [ 25.282994] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_take_over4299worker2 # [ 25.360258] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_connected addr=lb:500514300worker2 # [ 25.361980] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker1 # [ 25.367210] nix-grpc-daemon[782]: ts=2026-09-20T14:30:24Z level=info event=scheduler_connected addr=lb:500514302worker1 # [ 25.370304] nix-grpc-daemon[782]: ts=2026-09-20T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker2 # [ 25.562755] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_connected addr=lb:500514304worker2 # [ 25.564372] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker1 # [ 25.571276] nix-grpc-daemon[782]: ts=2026-09-20T14:30:24Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 25.574998] nix-grpc-daemon[782]: ts=2026-09-20T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 25.765026] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 25.766917] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 25.776560] nix-grpc-daemon[782]: ts=2026-09-20T14:30:24Z level=info event=scheduler_connected addr=lb:500514310worker2 # [ 25.857525] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:376824311worker2 # [ 25.860094] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04312worker2 # [ 25.967860] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=scheduler_connected addr=lb:500514313worker2 # [ 25.984194] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=oidc_jwks provider=mock keys=14314worker2 # [ 25.986200] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:376824315worker2 # [ 25.987719] nix-grpc-daemon[789]: ts=2026-09-20T14:30:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04316lb: (finished: waiting for unit niks3.service, in 14.20 seconds)4317lb: waiting for TCP port 5751 on localhost4318lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4319lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4320lb: waiting for unit envoy.service4321lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4322lb: waiting for TCP port 50051 on localhost4323lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4324lb # [ 26.453061] envoy[902]: conn peer=[::1]:46028 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24325lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)4326worker2 # [ 26.580254] systemd[1]: Starting scheduler-token.service...4327worker2 # [ 26.635792] systemd[1]: scheduler-token.service: Deactivated successfully.4328worker2 # [ 26.637410] systemd[1]: Finished scheduler-token.service.4329worker2 # [ 26.638734] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 57ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4330lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4331lb: must succeed: curl -sf localhost:9901/clusters4332lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4333lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.08 seconds)4334subtest: one node schedules and both builders hold a WorkerSession on it4335lb: must succeed: curl -sf localhost:9901/clusters4336lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4337worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4338worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4339lb: must succeed: curl -sf localhost:9901/clusters4340lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4341lb: must succeed: curl -sf localhost:9901/clusters4342lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4343lb: must succeed: curl -sf localhost:9901/clusters4344lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4345lb: must succeed: curl -sf localhost:9901/clusters4346lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4347worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4348worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.02 seconds)4349worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4350worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4351lb: must succeed: curl -sf localhost:9901/clusters4352lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4353lb: must succeed: curl -sf localhost:9901/clusters4354lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4355worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4356worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4357worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4358worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4359worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4360worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4361worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4362worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4363(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.55 seconds)4364subtest: balancer auth: client cert, bearer token, nothing, unknown CN4365worker2 # [ 27.644398] nix-grpc-daemon[789]: ts=2026-09-20T14:30:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376984366worker2 # [ 27.651205] systemd[1]: Started Nix Daemon.4367worker2 # [ 27.842724] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)4368worker2 # [ 27.896672] nix-grpc-daemon[789]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 96 ms (attempt 1/5)4369worker2 # [ 27.995455] nix-grpc-daemon[789]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)4370worker2 # [ 28.144246] nix-grpc-daemon[789]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 3/5)4371worker2 # [ 28.167957] nix-grpc-daemon[789]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 4/5)4372worker2 # [ 28.231419] nix-grpc-daemon[789]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4373worker2 # [ 28.262816] nix-grpc-daemon[789]: ts=2026-09-20T14:30:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376984374worker2 # [ 28.280575] nix-grpc-daemon[789]: ts=2026-09-20T14:30:27Z level=debug event=path_info_miss path=2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test error="error: path '/nix/store/2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test' is required, but there is no substituter that can build it"4375worker1 # [ 28.236272] nix-grpc-daemon[782]: ts=2026-09-20T14:30:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:362024376worker1 # [ 28.246786] systemd[1]: Started Nix Daemon.4377worker1 # [ 28.460763] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4378worker1 # [ 28.496863] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 1/5)4379worker1 # [ 28.568412] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)4380worker1 # [ 28.769471] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 124 ms (attempt 3/5)4381worker1 # [ 28.899421] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 528 ms (attempt 4/5)4382worker1 # [ 29.439891] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4383worker1 # [ 29.458161] nix-grpc-daemon[782]: ts=2026-09-20T14:30:28Z level=debug event=path_info_miss path=2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test error="error: path '/nix/store/2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test' is required, but there is no substituter that can build it"4384client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4385client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds)4386client: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4387client: (finished: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.03 seconds)4388worker2 # [ 29.869640] nix-grpc-daemon[789]: ts=2026-09-20T14:30:28Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:376984389worker2 # [ 29.877428] nix-grpc-daemon[789]: ts=2026-09-20T14:30:28Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:376984390worker2 # [ 29.882196] nix-grpc-daemon[789]: ts=2026-09-20T14:30:28Z level=debug event=path_info_miss path=2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test error="error: path '/nix/store/2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test' is required, but there is no substituter that can build it"4391worker1 # [ 29.828913] nix-grpc-daemon[782]: ts=2026-09-20T14:30:28Z level=info event=oidc_jwks provider=mock keys=14392worker1 # [ 29.833647] nix-grpc-daemon[782]: ts=2026-09-20T14:30:28Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:362024393worker1 # [ 29.842688] nix-grpc-daemon[782]: ts=2026-09-20T14:30:28Z level=debug event=path_info_miss path=2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test error="error: path '/nix/store/2whqqd50ar8kfyxdzygqd5pm447skhkf-nixos-system-client-test' is required, but there is no substituter that can build it"4394client: must succeed: rm -r /var/lib/nix-grpc-store4395client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4396worker1 # [ 30.128819] nix-grpc-daemon[782]: ts=2026-09-20T14:30:29Z level=info event=denied method=QueryMissing cn=- role=none4397worker1 # [ 30.376154] nix-grpc-daemon[782]: ts=2026-09-20T14:30:29Z level=info event=denied method=QueryMissing cn=stranger role=none4398(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.20 seconds)4399subtest: DAG build is scheduled across workers and lands in the cache4400client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24401worker2 # [ 30.712291] nix-grpc-daemon[789]: ts=2026-09-20T14:30:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376984402client # don't know how to build these paths:4403client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4404worker2 # [ 30.722774] nix-grpc-daemon[789]: ts=2026-09-20T14:30:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376984405worker2 # [ 30.751375] nix-grpc-daemon[789]: ts=2026-09-20T14:30:29Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04406worker2 # [ 30.753639] nix-grpc-daemon[789]: ts=2026-09-20T14:30:29Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14407worker1 # [ 30.741393] nix-grpc-daemon[782]: ts=2026-09-20T14:30:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024408worker1 # [ 30.764362] nix-grpc-daemon[782]: ts=2026-09-20T14:30:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024409client # copying 1 paths to 'lb:50051'...4410worker1 # [ 30.777583] nix-grpc-daemon[782]: ts=2026-09-20T14:30:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024411client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4412worker1 # [ 30.806370] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4413worker1 # [ 30.834322] nix-grpc-daemon[782]: ts=2026-09-20T14:30:29Z level=info event=niks3_push_started pid=8524414worker1 # [ 31.092489] nix-grpc-daemon[852]: time=2026-09-20T14:30:29.974Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""4415worker1 # [ 31.258457] nix-daemon[842]: accepted connection from pid 865, user nix-grpc-daemon (trusted)4416worker1 # [ 31.276407] nix-daemon[842]: reaped child process 868, status = succeeded4417worker1 # [ 31.414547] nix-daemon[842]: accepted connection from pid 870, user nix-grpc-daemon (trusted)4418worker1 # [ 31.426604] nix-daemon[842]: reaped child process 873, status = succeeded4419lb # [ 31.468645] niks3-server[1001]: 2026/09/20 14:30:30 INFO Received uploads request method=POST path=/api/pending_closures4420worker1 # [ 31.465785] nix-grpc-daemon[852]: time=2026-09-20T14:30:30.350Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4421worker1 # [ 31.467927] nix-grpc-daemon[852]: time=2026-09-20T14:30:30.352Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4422lb # [ 31.535769] niks3-server[1001]: 2026/09/20 14:30:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4423lb # [ 31.546724] niks3-server[1001]: 2026/09/20 14:30:30 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4424lb # [ 31.559995] niks3-server[1001]: 2026/09/20 14:30:30 INFO Signed narinfos id=1 count=14425worker1 # [ 31.534364] nix-grpc-daemon[852]: time=2026-09-20T14:30:30.418Z level=INFO msg="Uploading 1 narinfos"4426lb # [ 31.569888] niks3-server[1001]: 2026/09/20 14:30:30 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4427lb # [ 31.578034] niks3-server[1001]: 2026/09/20 14:30:30 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4428lb # [ 31.587315] niks3-server[1001]: 2026/09/20 14:30:30 INFO Completed upload id=14429worker1 # [ 31.560148] nix-grpc-daemon[852]: time=2026-09-20T14:30:30.444Z level=INFO msg="Upload complete. (455ms)"4430lb # [ 31.593637] niks3-server[1001]: 2026/09/20 14:30:30 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4431worker1 # [ 31.564996] nix-grpc-daemon[782]: ts=2026-09-20T14:30:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=5044432worker1 # [ 31.571342] nix-grpc-daemon[782]: ts=2026-09-20T14:30:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024433worker1 # [ 31.583155] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4434worker1 # [ 31.590734] nix-daemon[842]: reaped child process 850, status = succeeded4435client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4436worker1 # [ 31.600489] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4437client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4438client # farm-a> LOG-farm-a-t14439worker1 # [ 33.229444] nix-daemon[842]: reaped child process 877, status = succeeded4440worker1 # [ 33.351300] nix-daemon[842]: accepted connection from pid 893, user nix-grpc-daemon (trusted)4441worker1 # [ 33.363277] nix-daemon[842]: reaped child process 896, status = succeeded4442lb # [ 33.404952] niks3-server[1001]: 2026/09/20 14:30:32 INFO Received uploads request method=POST path=/api/pending_closures4443worker1 # [ 33.389616] nix-grpc-daemon[852]: time=2026-09-20T14:30:32.273Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4444worker1 # [ 33.391192] nix-grpc-daemon[852]: time=2026-09-20T14:30:32.273Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4445lb # [ 33.446349] niks3-server[1001]: 2026/09/20 14:30:32 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4446lb # [ 33.456807] niks3-server[1001]: 2026/09/20 14:30:32 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4447lb # [ 33.467523] niks3-server[1001]: 2026/09/20 14:30:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4448worker1 # [ 33.445950] nix-grpc-daemon[852]: time=2026-09-20T14:30:32.330Z level=INFO msg="Uploading 1 narinfos"4449lb # [ 33.472967] niks3-server[1001]: 2026/09/20 14:30:32 INFO Signed narinfos id=2 count=14450lb # [ 33.476479] niks3-server[1001]: 2026/09/20 14:30:32 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4451lb # [ 33.486452] niks3-server[1001]: 2026/09/20 14:30:32 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4452lb # [ 33.492248] niks3-server[1001]: 2026/09/20 14:30:32 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4453lb # [ 33.494945] niks3-server[1001]: 2026/09/20 14:30:32 INFO Completed upload id=24454worker1 # [ 33.467610] nix-grpc-daemon[852]: time=2026-09-20T14:30:32.352Z level=INFO msg="Upload complete. (253ms)"4455worker1 # [ 33.473900] nix-grpc-daemon[782]: ts=2026-09-20T14:30:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14456worker1 # [ 33.487831] nix-daemon[842]: reaped child process 875, status = succeeded4457worker2 # [ 33.561591] nix-grpc-daemon[789]: ts=2026-09-20T14:30:32Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04458worker2 # [ 33.565483] nix-grpc-daemon[789]: ts=2026-09-20T14:30:32Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14459worker1 # [ 33.538305] nix-grpc-daemon[782]: ts=2026-09-20T14:30:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024460worker1 # [ 33.550724] nix-grpc-daemon[782]: ts=2026-09-20T14:30:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024461worker1 # [ 33.564130] nix-grpc-daemon[852]: time=2026-09-20T14:30:32.448Z level=INFO msg="All 1 paths already cached"4462client # copying 1 paths to 'lb:50051'...4463worker1 # [ 33.571926] nix-grpc-daemon[782]: ts=2026-09-20T14:30:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024464client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4465worker1 # [ 33.596713] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4466worker1 # [ 33.766233] nix-daemon[842]: accepted connection from pid 900, user nix-grpc-daemon (trusted)4467worker1 # [ 33.780744] nix-daemon[842]: reaped child process 903, status = succeeded4468worker1 # [ 33.927860] nix-daemon[842]: accepted connection from pid 905, user nix-grpc-daemon (trusted)4469worker1 # [ 33.939946] nix-daemon[842]: reaped child process 908, status = succeeded4470lb # [ 33.979764] niks3-server[1001]: 2026/09/20 14:30:33 INFO Received uploads request method=POST path=/api/pending_closures4471worker1 # [ 33.963413] nix-grpc-daemon[852]: time=2026-09-20T14:30:32.847Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4472worker1 # [ 33.964828] nix-grpc-daemon[852]: time=2026-09-20T14:30:32.847Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4473lb # [ 34.009807] niks3-server[1001]: 2026/09/20 14:30:33 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4474lb # [ 34.018976] niks3-server[1001]: 2026/09/20 14:30:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4475lb # [ 34.024503] niks3-server[1001]: 2026/09/20 14:30:33 INFO Signed narinfos id=3 count=14476worker1 # [ 33.999989] nix-grpc-daemon[852]: time=2026-09-20T14:30:32.884Z level=INFO msg="Uploading 1 narinfos"4477lb # [ 34.032556] niks3-server[1001]: 2026/09/20 14:30:33 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4478lb # [ 34.041694] niks3-server[1001]: 2026/09/20 14:30:33 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4479lb # [ 34.049601] niks3-server[1001]: 2026/09/20 14:30:33 INFO Completed upload id=34480worker1 # [ 34.022347] nix-grpc-daemon[852]: time=2026-09-20T14:30:32.906Z level=INFO msg="Upload complete. (410ms)"4481lb # [ 34.052090] niks3-server[1001]: 2026/09/20 14:30:33 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4482worker1 # [ 34.026254] nix-grpc-daemon[782]: ts=2026-09-20T14:30:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=6244483worker1 # [ 34.031198] nix-grpc-daemon[782]: ts=2026-09-20T14:30:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024484worker1 # [ 34.041104] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4485worker1 # [ 34.045260] nix-daemon[842]: reaped child process 898, status = succeeded4486worker1 # [ 34.050316] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4487client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4488client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4489client # farm-b> LOG-farm-b-t14490worker1 # [ 35.469859] nix-daemon[842]: reaped child process 912, status = succeeded4491worker1 # [ 35.618584] nix-daemon[842]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4492worker1 # [ 35.629962] nix-daemon[842]: reaped child process 931, status = succeeded4493lb # [ 35.667162] niks3-server[1001]: 2026/09/20 14:30:34 INFO Received uploads request method=POST path=/api/pending_closures4494worker1 # [ 35.650311] nix-grpc-daemon[852]: time=2026-09-20T14:30:34.534Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4495worker1 # [ 35.651788] nix-grpc-daemon[852]: time=2026-09-20T14:30:34.536Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4496lb # [ 35.700777] niks3-server[1001]: 2026/09/20 14:30:34 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4497lb # [ 35.709796] niks3-server[1001]: 2026/09/20 14:30:34 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4498lb # [ 35.717267] niks3-server[1001]: 2026/09/20 14:30:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4499lb # [ 35.722587] niks3-server[1001]: 2026/09/20 14:30:34 INFO Signed narinfos id=4 count=14500worker1 # [ 35.696677] nix-grpc-daemon[852]: time=2026-09-20T14:30:34.581Z level=INFO msg="Uploading 1 narinfos"4501lb # [ 35.725587] niks3-server[1001]: 2026/09/20 14:30:34 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4502lb # [ 35.729594] niks3-server[1001]: 2026/09/20 14:30:34 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4503lb # [ 35.733645] niks3-server[1001]: 2026/09/20 14:30:34 INFO Completed upload id=44504lb # [ 35.734755] niks3-server[1001]: 2026/09/20 14:30:34 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4505worker1 # [ 35.706262] nix-grpc-daemon[852]: time=2026-09-20T14:30:34.590Z level=INFO msg="Upload complete. (247ms)"4506worker1 # [ 35.710793] nix-grpc-daemon[782]: ts=2026-09-20T14:30:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14507worker2 # [ 35.786770] nix-grpc-daemon[789]: ts=2026-09-20T14:30:34Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04508worker1 # [ 35.722340] nix-daemon[842]: reaped child process 910, status = succeeded4509worker2 # [ 35.790901] nix-grpc-daemon[789]: ts=2026-09-20T14:30:34Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14510worker1 # [ 35.761427] nix-grpc-daemon[782]: ts=2026-09-20T14:30:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024511worker1 # [ 35.773729] nix-grpc-daemon[782]: ts=2026-09-20T14:30:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024512worker1 # [ 35.785888] nix-grpc-daemon[852]: time=2026-09-20T14:30:34.670Z level=INFO msg="All 2 paths already cached"4513client # copying 1 paths to 'lb:50051'...4514worker1 # [ 35.794627] nix-grpc-daemon[782]: ts=2026-09-20T14:30:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024515client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4516worker1 # [ 35.808126] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4517worker1 # [ 35.962063] nix-daemon[842]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4518worker1 # [ 35.975440] nix-daemon[842]: reaped child process 938, status = succeeded4519worker1 # [ 36.127406] nix-daemon[842]: accepted connection from pid 940, user nix-grpc-daemon (trusted)4520worker1 # [ 36.139265] nix-daemon[842]: reaped child process 943, status = succeeded4521lb # [ 36.182296] niks3-server[1001]: 2026/09/20 14:30:35 INFO Received uploads request method=POST path=/api/pending_closures4522worker1 # [ 36.163053] nix-grpc-daemon[852]: time=2026-09-20T14:30:35.047Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4523worker1 # [ 36.164626] nix-grpc-daemon[852]: time=2026-09-20T14:30:35.049Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4524lb # [ 36.210614] niks3-server[1001]: 2026/09/20 14:30:35 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4525lb # [ 36.223494] niks3-server[1001]: 2026/09/20 14:30:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4526lb # [ 36.228630] niks3-server[1001]: 2026/09/20 14:30:35 INFO Signed narinfos id=5 count=14527worker1 # [ 36.204092] nix-grpc-daemon[852]: time=2026-09-20T14:30:35.088Z level=INFO msg="Uploading 1 narinfos"4528lb # [ 36.234059] niks3-server[1001]: 2026/09/20 14:30:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4529lb # [ 36.246219] niks3-server[1001]: 2026/09/20 14:30:35 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4530lb # [ 36.251665] niks3-server[1001]: 2026/09/20 14:30:35 INFO Completed upload id=54531worker1 # [ 36.227174] nix-grpc-daemon[852]: time=2026-09-20T14:30:35.111Z level=INFO msg="Upload complete. (408ms)"4532lb # [ 36.255837] niks3-server[1001]: 2026/09/20 14:30:35 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4533worker1 # [ 36.230962] nix-grpc-daemon[782]: ts=2026-09-20T14:30:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=7524534worker1 # [ 36.235897] nix-grpc-daemon[782]: ts=2026-09-20T14:30:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024535worker1 # [ 36.246111] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4536worker1 # [ 36.250695] nix-daemon[842]: reaped child process 933, status = succeeded4537worker1 # [ 36.255895] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4538client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4539client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4540client # farm-top> LOG-farm-top-t14541worker1 # [ 37.281928] nix-daemon[842]: reaped child process 947, status = succeeded4542lb # [ 37.313611] envoy[902]: conn peer=[2001:db8:1::1]:53982 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=19344543lb # [ 37.317443] envoy[902]: conn peer=[2001:db8:1::1]:53986 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=362 ms=534544lb # [ 37.320728] envoy[902]: conn peer=[2001:db8:1::1]:53992 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=144545lb # [ 37.324092] envoy[902]: conn peer=[2001:db8:1::1]:54000 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=124546worker1 # [ 37.433801] nix-daemon[842]: accepted connection from pid 962, user nix-grpc-daemon (trusted)4547worker1 # [ 37.446150] nix-daemon[842]: reaped child process 965, status = succeeded4548lb # [ 37.489615] niks3-server[1001]: 2026/09/20 14:30:36 INFO Received uploads request method=POST path=/api/pending_closures4549worker1 # [ 37.470459] nix-grpc-daemon[852]: time=2026-09-20T14:30:36.354Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4550worker1 # [ 37.473056] nix-grpc-daemon[852]: time=2026-09-20T14:30:36.356Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4551lb # [ 37.526683] niks3-server[1001]: 2026/09/20 14:30:36 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4552lb # [ 37.533785] niks3-server[1001]: 2026/09/20 14:30:36 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4553lb # [ 37.546664] niks3-server[1001]: 2026/09/20 14:30:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4554lb # [ 37.553359] niks3-server[1001]: 2026/09/20 14:30:36 INFO Signed narinfos id=6 count=14555worker1 # [ 37.528500] nix-grpc-daemon[852]: time=2026-09-20T14:30:36.412Z level=INFO msg="Uploading 1 narinfos"4556lb # [ 37.557432] niks3-server[1001]: 2026/09/20 14:30:36 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4557lb # [ 37.566397] niks3-server[1001]: 2026/09/20 14:30:36 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4558lb # [ 37.573363] niks3-server[1001]: 2026/09/20 14:30:36 INFO Completed upload id=64559lb # [ 37.574968] niks3-server[1001]: 2026/09/20 14:30:36 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4560worker1 # [ 37.546066] nix-grpc-daemon[852]: time=2026-09-20T14:30:36.430Z level=INFO msg="Upload complete. (281ms)"4561worker1 # [ 37.552595] nix-grpc-daemon[782]: ts=2026-09-20T14:30:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14562worker1 # [ 37.564935] nix-daemon[842]: reaped child process 945, status = succeeded4563client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.18 seconds)4564client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4565client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4566client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4567client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4568client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4569client # copying 3 paths...4570client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4571client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4572client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4573client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4574(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.71 seconds)4575subtest: repeat is answered Cached by the scheduler without building4576worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4577worker1 # finding garbage collector roots...4578worker1 # deleting specified paths...4579worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4580worker1 # deleting unused links...4581worker1 # note: hard linking is currently saving 0.0 KiB4582worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.22 seconds)4583worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4584worker2 # finding garbage collector roots...4585worker2 # deleting specified paths...4586worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4587worker2 # deleting unused links...4588worker2 # note: hard linking is currently saving 0.0 KiB4589worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4590client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24591worker1 # [ 38.839751] nix-grpc-daemon[782]: ts=2026-09-20T14:30:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362024592client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4593client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4594worker1 # [ 38.859478] nix-grpc-daemon[782]: ts=2026-09-20T14:30:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024595worker1 # [ 38.872528] nix-grpc-daemon[852]: time=2026-09-20T14:30:37.756Z level=INFO msg="All 2 paths already cached"4596worker2 # [ 38.957437] nix-grpc-daemon[789]: ts=2026-09-20T14:30:37Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14597client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.32 seconds)4598client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4599client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4600lb: must succeed: curl -sf localhost:9901/clusters4601lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4602lb: must succeed: curl -sf localhost:9901/clusters4603lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4604worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4605worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4606(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.27 seconds)4607subtest: inputs already in the cache are not sent to the scheduler4608lb: must succeed: curl -sf localhost:9901/clusters4609lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4610lb: must succeed: curl -sf localhost:9901/clusters4611lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4612worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4613worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4614client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24615client # don't know how to build these paths:4616client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4617worker1 # [ 39.765094] nix-grpc-daemon[782]: ts=2026-09-20T14:30:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362024618worker1 # [ 39.770983] nix-grpc-daemon[782]: ts=2026-09-20T14:30:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024619worker1 # [ 39.776850] nix-grpc-daemon[852]: time=2026-09-20T14:30:38.659Z level=INFO msg="All 2 paths already cached"4620worker2 # [ 39.852327] nix-grpc-daemon[789]: ts=2026-09-20T14:30:38Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04621worker2 # [ 39.856846] nix-grpc-daemon[789]: ts=2026-09-20T14:30:38Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14622worker1 # [ 39.826925] nix-grpc-daemon[782]: ts=2026-09-20T14:30:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024623worker1 # [ 39.837384] nix-grpc-daemon[782]: ts=2026-09-20T14:30:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024624worker1 # [ 39.847900] nix-grpc-daemon[852]: time=2026-09-20T14:30:38.732Z level=INFO msg="All 2 paths already cached"4625client # copying 1 paths to 'lb:50051'...4626client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4627worker1 # [ 39.855663] nix-grpc-daemon[782]: ts=2026-09-20T14:30:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024628worker1 # [ 39.866807] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4629worker1 # [ 40.034345] nix-daemon[842]: accepted connection from pid 983, user nix-grpc-daemon (trusted)4630worker1 # [ 40.048530] nix-daemon[842]: reaped child process 986, status = succeeded4631worker1 # [ 40.195492] nix-daemon[842]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4632worker1 # [ 40.207828] nix-daemon[842]: reaped child process 991, status = succeeded4633lb # [ 40.250197] niks3-server[1001]: 2026/09/20 14:30:39 INFO Received uploads request method=POST path=/api/pending_closures4634worker1 # [ 40.227073] nix-grpc-daemon[852]: time=2026-09-20T14:30:39.111Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4635worker1 # [ 40.228950] nix-grpc-daemon[852]: time=2026-09-20T14:30:39.113Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4636lb # [ 40.264442] niks3-server[1001]: 2026/09/20 14:30:39 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4637lb # [ 40.268716] niks3-server[1001]: 2026/09/20 14:30:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4638lb # [ 40.270320] niks3-server[1001]: 2026/09/20 14:30:39 INFO Signed narinfos id=7 count=14639worker1 # [ 40.244389] nix-grpc-daemon[852]: time=2026-09-20T14:30:39.128Z level=INFO msg="Uploading 1 narinfos"4640lb # [ 40.275462] niks3-server[1001]: 2026/09/20 14:30:39 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4641lb # [ 40.281137] niks3-server[1001]: 2026/09/20 14:30:39 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4642lb # [ 40.283307] niks3-server[1001]: 2026/09/20 14:30:39 INFO Completed upload id=74643worker1 # [ 40.256924] nix-grpc-daemon[852]: time=2026-09-20T14:30:39.141Z level=INFO msg="Upload complete. (378ms)"4644lb # [ 40.285374] niks3-server[1001]: 2026/09/20 14:30:39 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4645worker1 # [ 40.258896] nix-grpc-daemon[782]: ts=2026-09-20T14:30:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=7524646worker1 # [ 40.262648] nix-grpc-daemon[782]: ts=2026-09-20T14:30:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024647worker1 # [ 40.269267] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4648worker1 # [ 40.271857] nix-daemon[842]: reaped child process 981, status = succeeded4649worker1 # [ 40.276978] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4650client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4651client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4652client # farm-top> LOG-farm-top-t1b4653worker1 # [ 41.503601] nix-daemon[842]: reaped child process 995, status = succeeded4654worker1 # [ 41.666955] nix-daemon[842]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4655worker1 # [ 41.678242] nix-daemon[842]: reaped child process 1014, status = succeeded4656lb # [ 41.721695] niks3-server[1001]: 2026/09/20 14:30:40 INFO Received uploads request method=POST path=/api/pending_closures4657worker1 # [ 41.702744] nix-grpc-daemon[852]: time=2026-09-20T14:30:40.587Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4658worker1 # [ 41.704261] nix-grpc-daemon[852]: time=2026-09-20T14:30:40.588Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4659lb # [ 41.753330] niks3-server[1001]: 2026/09/20 14:30:40 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4660lb # [ 41.769617] niks3-server[1001]: 2026/09/20 14:30:40 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4661lb # [ 41.779068] niks3-server[1001]: 2026/09/20 14:30:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4662lb # [ 41.784372] niks3-server[1001]: 2026/09/20 14:30:40 INFO Signed narinfos id=8 count=14663worker1 # [ 41.759651] nix-grpc-daemon[852]: time=2026-09-20T14:30:40.643Z level=INFO msg="Uploading 1 narinfos"4664lb # [ 41.793165] niks3-server[1001]: 2026/09/20 14:30:40 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4665lb # [ 41.799750] niks3-server[1001]: 2026/09/20 14:30:40 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4666lb # [ 41.807942] niks3-server[1001]: 2026/09/20 14:30:40 INFO Completed upload id=84667worker1 # [ 41.780657] nix-grpc-daemon[852]: time=2026-09-20T14:30:40.665Z level=INFO msg="Upload complete. (295ms)"4668lb # [ 41.809966] niks3-server[1001]: 2026/09/20 14:30:40 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4669worker1 # [ 41.786753] nix-grpc-daemon[782]: ts=2026-09-20T14:30:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14670worker1 # [ 41.800882] nix-daemon[842]: reaped child process 993, status = succeeded4671client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.32 seconds)4672client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4673client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds)4674lb: must succeed: curl -sf localhost:9901/clusters4675lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4676lb: must succeed: curl -sf localhost:9901/clusters4677lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4678worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4679worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4680(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.74 seconds)4681subtest: two clients wanting the same drv build it once4682client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4683client # [ 42.208471] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4684client # Running as unit: dedup1.service; invocation ID: 49c5c6ccee0f425b9b51c4634495910a4685client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.04 seconds)4686client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4687worker2 # [ 42.438576] nix-grpc-daemon[789]: ts=2026-09-20T14:30:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376984688client # [ 42.419937] nix[887]: don't know how to build these paths:4689client # [ 42.420873] nix[887]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4690worker2 # [ 42.445374] nix-grpc-daemon[789]: ts=2026-09-20T14:30:41Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04691worker2 # [ 42.447825] nix-grpc-daemon[789]: ts=2026-09-20T14:30:41Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14692worker2 # [ 42.450675] nix-grpc-daemon[789]: ts=2026-09-20T14:30:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376984693worker2 # [ 42.462046] nix-grpc-daemon[789]: ts=2026-09-20T14:30:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376984694client # [ 42.448092] nix[887]: copying 1 paths to 'lb:50051'...4695worker2 # [ 42.468564] nix-grpc-daemon[789]: ts=2026-09-20T14:30:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376984696client # [ 42.449166] nix[887]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4697worker2 # [ 42.475568] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)4698worker2 # [ 42.488901] nix-grpc-daemon[789]: ts=2026-09-20T14:30:41Z level=info event=niks3_push_started pid=9214699worker2 # [ 42.719242] nix-grpc-daemon[921]: time=2026-09-20T14:30:41.666Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""4700worker2 # [ 42.884238] nix-daemon[870]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4701worker2 # [ 42.899914] nix-daemon[870]: reaped child process 937, status = succeeded4702worker2 # [ 43.034600] nix-daemon[870]: accepted connection from pid 939, user nix-grpc-daemon (trusted)4703worker2 # [ 43.046993] nix-daemon[870]: reaped child process 942, status = succeeded4704lb # [ 43.023647] niks3-server[1001]: 2026/09/20 14:30:42 INFO Received uploads request method=POST path=/api/pending_closures4705worker2 # [ 43.074363] nix-grpc-daemon[921]: time=2026-09-20T14:30:42.024Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4706worker2 # [ 43.076569] nix-grpc-daemon[921]: time=2026-09-20T14:30:42.026Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4707lb # [ 43.056628] niks3-server[1001]: 2026/09/20 14:30:42 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4708lb # [ 43.065615] niks3-server[1001]: 2026/09/20 14:30:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4709lb # [ 43.072082] niks3-server[1001]: 2026/09/20 14:30:42 INFO Signed narinfos id=9 count=14710worker2 # [ 43.113499] nix-grpc-daemon[921]: time=2026-09-20T14:30:42.063Z level=INFO msg="Uploading 1 narinfos"4711lb # [ 43.075541] niks3-server[1001]: 2026/09/20 14:30:42 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4712lb # [ 43.079959] niks3-server[1001]: 2026/09/20 14:30:42 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4713lb # [ 43.082543] niks3-server[1001]: 2026/09/20 14:30:42 INFO Completed upload id=94714worker2 # [ 43.124090] nix-grpc-daemon[921]: time=2026-09-20T14:30:42.073Z level=INFO msg="Upload complete. (394ms)"4715worker2 # [ 43.125881] nix-grpc-daemon[789]: ts=2026-09-20T14:30:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=0 paths=1 nar_bytes_in=4324716lb # [ 43.085198] niks3-server[1001]: 2026/09/20 14:30:42 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4717worker2 # [ 43.132361] nix-grpc-daemon[789]: ts=2026-09-20T14:30:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376984718worker2 # [ 43.136070] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)4719worker2 # [ 43.138155] nix-daemon[870]: reaped child process 919, status = succeeded4720worker2 # [ 43.143550] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)4721client # [ 43.122513] nix[887]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4722client # [ 43.282279] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4723client # Running as unit: dedup2.service; invocation ID: 93ae8378716f4dc6a545a23a86fc99c04724client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4725worker1 # [ 43.521596] nix-grpc-daemon[782]: ts=2026-09-20T14:30:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362024726client # [ 43.571990] nix[904]: don't know how to build these paths:4727client # [ 43.572898] nix[904]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4728worker2 # [ 43.600980] nix-grpc-daemon[789]: ts=2026-09-20T14:30:42Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04729worker2 # [ 43.605316] nix-grpc-daemon[789]: ts=2026-09-20T14:30:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376984730client # [ 43.587797] nix[904]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4731client: must succeed: sleep 24732client: (finished: must succeed: sleep 2, in 2.05 seconds)4733client: waiting for success: ! systemctl is-active dedup1 dedup24734worker2 # [ 46.761380] nix-daemon[870]: reaped child process 946, status = succeeded4735worker2 # [ 46.908091] nix-daemon[870]: accepted connection from pid 978, user nix-grpc-daemon (trusted)4736worker2 # [ 46.919606] nix-daemon[870]: reaped child process 981, status = succeeded4737lb # [ 46.889414] niks3-server[1001]: 2026/09/20 14:30:46 INFO Received uploads request method=POST path=/api/pending_closures4738worker2 # [ 46.941518] nix-grpc-daemon[921]: time=2026-09-20T14:30:45.890Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4739worker2 # [ 46.943335] nix-grpc-daemon[921]: time=2026-09-20T14:30:45.892Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4740lb # [ 46.940753] niks3-server[1001]: 2026/09/20 14:30:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4741lb # [ 46.945204] niks3-server[1001]: 2026/09/20 14:30:46 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4742lb # [ 46.950346] niks3-server[1001]: 2026/09/20 14:30:46 INFO Signed narinfos id=10 count=14743worker2 # [ 46.994721] nix-grpc-daemon[921]: time=2026-09-20T14:30:45.943Z level=INFO msg="Uploading 1 narinfos"4744lb # [ 46.954479] niks3-server[1001]: 2026/09/20 14:30:46 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4745lb # [ 46.957474] niks3-server[1001]: 2026/09/20 14:30:46 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4746lb # [ 46.966112] niks3-server[1001]: 2026/09/20 14:30:46 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4747lb # [ 46.972442] niks3-server[1001]: 2026/09/20 14:30:46 INFO Completed upload id=104748worker2 # [ 47.014229] nix-grpc-daemon[921]: time=2026-09-20T14:30:45.963Z level=INFO msg="Upload complete. (257ms)"4749lb # [ 46.975143] niks3-server[1001]: 2026/09/20 14:30:46 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4750worker2 # [ 47.020647] nix-grpc-daemon[789]: ts=2026-09-20T14:30:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14751worker2 # [ 47.027984] nix-grpc-daemon[789]: ts=2026-09-20T14:30:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14752worker2 # [ 47.037336] nix-daemon[870]: reaped child process 944, status = succeeded4753client # [ 47.058292] systemd[1]: dedup2.service: Deactivated successfully.4754client # [ 47.061696] systemd[1]: dedup2.service: Consumed 162ms CPU time over 3.780s wall clock time, 35.1M memory peak, 5.8K incoming IP traffic, 5.6K outgoing IP traffic.4755client # [ 47.069987] systemd[1]: dedup1.service: Deactivated successfully.4756client # [ 47.072901] systemd[1]: dedup1.service: Consumed 128ms CPU time over 4.861s wall clock time, 41.5M memory peak, 6.9K incoming IP traffic, 8.2K outgoing IP traffic.4757lb # [ 47.314237] envoy[902]: conn peer=[2001:db8:1::1]:54012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9537 tx=3857 ms=69494758lb # [ 47.317853] envoy[902]: conn peer=[2001:db8:1::1]:45890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=684759lb # [ 47.321245] envoy[902]: conn peer=[2001:db8:1::1]:45906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1724 ms=20364760lb # [ 47.324711] envoy[902]: conn peer=[2001:db8:1::1]:35450 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1085 ms=34514761lb # [ 47.328245] envoy[902]: conn peer=[2001:db8:1::1]:45912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2968 tx=1351 ms=46084762client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4763client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4764client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4765worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4766worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4767worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4768worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4769worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4770worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4771worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4772worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4773(finished: subtest: two clients wanting the same drv build it once, in 5.87 seconds)4774subtest: build hook: nix-daemon with builders = grpc://lb4775client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4776client # [ 48.260144] systemd[1]: Started Nix Daemon.4777client # [ 48.438938] nix-daemon[946]: accepted connection from pid 943, user root (trusted)4778worker1 # [ 48.574594] nix-grpc-daemon[782]: ts=2026-09-20T14:30:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:362024779worker2 # [ 48.648930] nix-grpc-daemon[789]: ts=2026-09-20T14:30:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376984780worker2 # [ 48.660862] nix-grpc-daemon[789]: ts=2026-09-20T14:30:47Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04781worker2 # [ 48.664819] nix-grpc-daemon[789]: ts=2026-09-20T14:30:47Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14782worker1 # [ 48.638395] nix-grpc-daemon[782]: ts=2026-09-20T14:30:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024783worker1 # [ 48.650489] nix-grpc-daemon[782]: ts=2026-09-20T14:30:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024784worker1 # [ 48.661710] nix-grpc-daemon[782]: ts=2026-09-20T14:30:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024785worker1 # [ 48.675823] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4786worker1 # [ 48.847207] nix-daemon[842]: accepted connection from pid 1039, user nix-grpc-daemon (trusted)4787worker1 # [ 48.860792] nix-daemon[842]: reaped child process 1042, status = succeeded4788worker1 # [ 49.011312] nix-daemon[842]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)4789worker1 # [ 49.022038] nix-daemon[842]: reaped child process 1047, status = succeeded4790lb # [ 49.064265] niks3-server[1001]: 2026/09/20 14:30:48 INFO Received uploads request method=POST path=/api/pending_closures4791worker1 # [ 49.045353] nix-grpc-daemon[852]: time=2026-09-20T14:30:47.929Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4792worker1 # [ 49.047051] nix-grpc-daemon[852]: time=2026-09-20T14:30:47.931Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4793lb # [ 49.095133] niks3-server[1001]: 2026/09/20 14:30:48 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4794lb # [ 49.106247] niks3-server[1001]: 2026/09/20 14:30:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4795worker1 # [ 49.084676] nix-grpc-daemon[852]: time=2026-09-20T14:30:47.969Z level=INFO msg="Uploading 1 narinfos"4796lb # [ 49.112754] niks3-server[1001]: 2026/09/20 14:30:48 INFO Signed narinfos id=11 count=14797lb # [ 49.118616] niks3-server[1001]: 2026/09/20 14:30:48 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4798lb # [ 49.129432] niks3-server[1001]: 2026/09/20 14:30:48 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4799lb # [ 49.135286] niks3-server[1001]: 2026/09/20 14:30:48 INFO Completed upload id=114800worker1 # [ 49.107562] nix-grpc-daemon[852]: time=2026-09-20T14:30:47.991Z level=INFO msg="Upload complete. (420ms)"4801lb # [ 49.140123] niks3-server[1001]: 2026/09/20 14:30:48 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4802worker1 # [ 49.112150] nix-grpc-daemon[782]: ts=2026-09-20T14:30:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=5124803worker1 # [ 49.117816] nix-grpc-daemon[782]: ts=2026-09-20T14:30:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024804worker1 # [ 49.129299] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4805worker1 # [ 49.131400] nix-daemon[842]: reaped child process 1037, status = succeeded4806worker1 # [ 49.136601] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4807worker1 # [ 50.165509] nix-daemon[842]: reaped child process 1051, status = succeeded4808worker1 # [ 50.310557] nix-daemon[842]: accepted connection from pid 1066, user nix-grpc-daemon (trusted)4809worker1 # [ 50.322468] nix-daemon[842]: reaped child process 1069, status = succeeded4810lb # [ 50.363652] niks3-server[1001]: 2026/09/20 14:30:49 INFO Received uploads request method=POST path=/api/pending_closures4811worker1 # [ 50.343959] nix-grpc-daemon[852]: time=2026-09-20T14:30:49.228Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4812worker1 # [ 50.347060] nix-grpc-daemon[852]: time=2026-09-20T14:30:49.230Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4813lb # [ 50.403776] niks3-server[1001]: 2026/09/20 14:30:49 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4814lb # [ 50.411488] niks3-server[1001]: 2026/09/20 14:30:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4815lb # [ 50.417862] niks3-server[1001]: 2026/09/20 14:30:49 INFO Signed narinfos id=12 count=14816worker1 # [ 50.393627] nix-grpc-daemon[852]: time=2026-09-20T14:30:49.277Z level=INFO msg="Uploading 1 narinfos"4817lb # [ 50.425346] niks3-server[1001]: 2026/09/20 14:30:49 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4818lb # [ 50.433659] niks3-server[1001]: 2026/09/20 14:30:49 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4819lb # [ 50.440498] niks3-server[1001]: 2026/09/20 14:30:49 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4820lb # [ 50.446165] niks3-server[1001]: 2026/09/20 14:30:49 INFO Completed upload id=124821worker1 # [ 50.420185] nix-grpc-daemon[852]: time=2026-09-20T14:30:49.304Z level=INFO msg="Upload complete. (276ms)"4822lb # [ 50.447956] niks3-server[1001]: 2026/09/20 14:30:49 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4823worker1 # [ 50.424435] nix-grpc-daemon[782]: ts=2026-09-20T14:30:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14824worker2 # [ 50.497596] nix-grpc-daemon[789]: ts=2026-09-20T14:30:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376984825worker1 # [ 50.432524] nix-daemon[842]: reaped child process 1049, status = succeeded4826worker2 # [ 50.504857] nix-grpc-daemon[789]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4827worker1 # [ 50.469088] nix-grpc-daemon[782]: ts=2026-09-20T14:30:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:362024828worker1 # [ 50.474235] nix-grpc-daemon[782]: ts=2026-09-20T14:30:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_out=1284829worker2 # [ 50.769467] nix-grpc-daemon[789]: ts=2026-09-20T14:30:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376984830worker1 # [ 50.705653] nix-grpc-daemon[782]: ts=2026-09-20T14:30:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024831worker1 # [ 50.713811] nix-grpc-daemon[852]: time=2026-09-20T14:30:49.598Z level=INFO msg="All 1 paths already cached"4832worker2 # [ 50.793784] nix-grpc-daemon[789]: ts=2026-09-20T14:30:49Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04833worker2 # [ 50.796072] nix-grpc-daemon[789]: ts=2026-09-20T14:30:49Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14834worker1 # [ 50.728826] nix-grpc-daemon[782]: ts=2026-09-20T14:30:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024835worker1 # [ 50.738521] nix-grpc-daemon[782]: ts=2026-09-20T14:30:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024836worker1 # [ 50.748177] nix-grpc-daemon[852]: time=2026-09-20T14:30:49.632Z level=INFO msg="All 1 paths already cached"4837worker1 # [ 50.754834] nix-grpc-daemon[782]: ts=2026-09-20T14:30:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024838worker1 # [ 50.764163] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4839worker1 # [ 50.902641] nix-daemon[842]: accepted connection from pid 1073, user nix-grpc-daemon (trusted)4840worker1 # [ 50.915847] nix-daemon[842]: reaped child process 1076, status = succeeded4841worker1 # [ 51.034399] nix-daemon[842]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4842worker1 # [ 51.047281] nix-daemon[842]: reaped child process 1081, status = succeeded4843lb # [ 51.086866] niks3-server[1001]: 2026/09/20 14:30:50 INFO Received uploads request method=POST path=/api/pending_closures4844worker1 # [ 51.067930] nix-grpc-daemon[852]: time=2026-09-20T14:30:49.952Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4845worker1 # [ 51.069740] nix-grpc-daemon[852]: time=2026-09-20T14:30:49.953Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4846lb # [ 51.114557] niks3-server[1001]: 2026/09/20 14:30:50 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4847lb # [ 51.124354] niks3-server[1001]: 2026/09/20 14:30:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4848lb # [ 51.128371] niks3-server[1001]: 2026/09/20 14:30:50 INFO Signed narinfos id=13 count=14849worker1 # [ 51.104233] nix-grpc-daemon[852]: time=2026-09-20T14:30:49.988Z level=INFO msg="Uploading 1 narinfos"4850lb # [ 51.135663] niks3-server[1001]: 2026/09/20 14:30:50 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4851lb # [ 51.147127] niks3-server[1001]: 2026/09/20 14:30:50 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4852lb # [ 51.151390] niks3-server[1001]: 2026/09/20 14:30:50 INFO Completed upload id=134853worker1 # [ 51.126402] nix-grpc-daemon[852]: time=2026-09-20T14:30:50.010Z level=INFO msg="Upload complete. (349ms)"4854lb # [ 51.154929] niks3-server[1001]: 2026/09/20 14:30:50 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4855worker1 # [ 51.128560] nix-grpc-daemon[782]: ts=2026-09-20T14:30:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=6244856worker1 # [ 51.134328] nix-grpc-daemon[782]: ts=2026-09-20T14:30:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024857worker1 # [ 51.143472] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4858worker1 # [ 51.146987] nix-daemon[842]: reaped child process 1071, status = succeeded4859worker1 # [ 51.152092] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4860worker1 # [ 51.980971] nix-daemon[842]: reaped child process 1085, status = succeeded4861worker1 # [ 52.104319] nix-daemon[842]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)4862worker1 # [ 52.118572] nix-daemon[842]: reaped child process 1103, status = succeeded4863lb # [ 52.156255] niks3-server[1001]: 2026/09/20 14:30:51 INFO Received uploads request method=POST path=/api/pending_closures4864worker1 # [ 52.132909] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.017Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4865worker1 # [ 52.134366] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.018Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4866lb # [ 52.180619] niks3-server[1001]: 2026/09/20 14:30:51 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4867lb # [ 52.195937] niks3-server[1001]: 2026/09/20 14:30:51 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4868lb # [ 52.205731] niks3-server[1001]: 2026/09/20 14:30:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4869worker1 # [ 52.183399] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.067Z level=INFO msg="Uploading 1 narinfos"4870lb # [ 52.211149] niks3-server[1001]: 2026/09/20 14:30:51 INFO Signed narinfos id=14 count=14871lb # [ 52.214818] niks3-server[1001]: 2026/09/20 14:30:51 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4872lb # [ 52.225536] niks3-server[1001]: 2026/09/20 14:30:51 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4873lb # [ 52.228189] niks3-server[1001]: 2026/09/20 14:30:51 INFO Completed upload id=144874lb # [ 52.229751] niks3-server[1001]: 2026/09/20 14:30:51 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4875worker1 # [ 52.201987] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.086Z level=INFO msg="Upload complete. (242ms)"4876worker1 # [ 52.206641] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14877worker1 # [ 52.213292] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:362024878worker1 # [ 52.222932] nix-daemon[842]: reaped child process 1083, status = succeeded4879worker1 # [ 52.226215] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:362024880worker1 # [ 52.227784] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_out=1764881worker2 # [ 52.558606] nix-grpc-daemon[789]: ts=2026-09-20T14:30:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376984882worker1 # [ 52.494123] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024883worker1 # [ 52.503447] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.387Z level=INFO msg="All 2 paths already cached"4884worker2 # [ 52.586723] nix-grpc-daemon[789]: ts=2026-09-20T14:30:51Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04885worker1 # [ 52.520156] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024886worker2 # [ 52.590244] nix-grpc-daemon[789]: ts=2026-09-20T14:30:51Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14887worker1 # [ 52.532310] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024888worker1 # [ 52.543551] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.428Z level=INFO msg="All 2 paths already cached"4889worker1 # [ 52.550792] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362024890worker1 # [ 52.559656] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4891worker1 # [ 52.739298] nix-daemon[842]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4892worker1 # [ 52.753837] nix-daemon[842]: reaped child process 1110, status = succeeded4893worker1 # [ 52.909273] nix-daemon[842]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)4894worker1 # [ 52.923621] nix-daemon[842]: reaped child process 1115, status = succeeded4895lb # [ 52.962787] niks3-server[1001]: 2026/09/20 14:30:52 INFO Received uploads request method=POST path=/api/pending_closures4896worker1 # [ 52.939640] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.824Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4897worker1 # [ 52.941575] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.825Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4898lb # [ 52.977875] niks3-server[1001]: 2026/09/20 14:30:52 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4899lb # [ 52.982559] niks3-server[1001]: 2026/09/20 14:30:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4900lb # [ 52.984882] niks3-server[1001]: 2026/09/20 14:30:52 INFO Signed narinfos id=15 count=14901worker1 # [ 52.957312] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.841Z level=INFO msg="Uploading 1 narinfos"4902lb # [ 52.987715] niks3-server[1001]: 2026/09/20 14:30:52 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4903lb # [ 52.992688] niks3-server[1001]: 2026/09/20 14:30:52 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4904lb # [ 52.996430] niks3-server[1001]: 2026/09/20 14:30:52 INFO Completed upload id=154905worker1 # [ 52.968999] nix-grpc-daemon[852]: time=2026-09-20T14:30:51.853Z level=INFO msg="Upload complete. (394ms)"4906lb # [ 52.997787] niks3-server[1001]: 2026/09/20 14:30:52 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4907worker1 # [ 52.971138] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 paths=1 nar_bytes_in=7524908worker1 # [ 52.975780] nix-grpc-daemon[782]: ts=2026-09-20T14:30:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362024909worker1 # [ 52.982128] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4910worker1 # [ 52.984725] nix-daemon[842]: reaped child process 1105, status = succeeded4911worker1 # [ 52.990067] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4912worker1 # [ 53.617295] nix-daemon[842]: reaped child process 1119, status = succeeded4913worker1 # [ 53.748324] nix-daemon[842]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)4914worker1 # [ 53.762459] nix-daemon[842]: reaped child process 1137, status = succeeded4915lb # [ 53.802604] niks3-server[1001]: 2026/09/20 14:30:52 INFO Received uploads request method=POST path=/api/pending_closures4916worker1 # [ 53.783333] nix-grpc-daemon[852]: time=2026-09-20T14:30:52.667Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4917worker1 # [ 53.784845] nix-grpc-daemon[852]: time=2026-09-20T14:30:52.669Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4918lb # [ 53.847245] niks3-server[1001]: 2026/09/20 14:30:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4919lb # [ 53.852525] niks3-server[1001]: 2026/09/20 14:30:53 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4920lb # [ 53.857875] niks3-server[1001]: 2026/09/20 14:30:53 INFO Signed narinfos id=16 count=14921worker1 # [ 53.833411] nix-grpc-daemon[852]: time=2026-09-20T14:30:52.717Z level=INFO msg="Uploading 1 narinfos"4922lb # [ 53.863625] niks3-server[1001]: 2026/09/20 14:30:53 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4923lb # [ 53.875152] niks3-server[1001]: 2026/09/20 14:30:53 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4924lb # [ 53.882220] niks3-server[1001]: 2026/09/20 14:30:53 INFO Completed upload id=164925worker1 # [ 53.854868] nix-grpc-daemon[852]: time=2026-09-20T14:30:52.739Z level=INFO msg="Upload complete. (257ms)"4926lb # [ 53.885671] niks3-server[1001]: 2026/09/20 14:30:53 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4927lb # [ 53.889479] niks3-server[1001]: 2026/09/20 14:30:53 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4928worker1 # [ 53.861120] nix-grpc-daemon[782]: ts=2026-09-20T14:30:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14929worker1 # [ 53.872721] nix-daemon[842]: reaped child process 1117, status = succeeded4930worker2 # [ 53.940296] nix-grpc-daemon[789]: ts=2026-09-20T14:30:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376984931worker2 # [ 53.955449] nix-grpc-daemon[789]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4932worker2 # [ 53.966961] nix-grpc-daemon[789]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4933worker2 # [ 53.994954] nix-grpc-daemon[789]: ts=2026-09-20T14:30:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376984934worker2 # [ 54.000684] nix-grpc-daemon[789]: ts=2026-09-20T14:30:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=0 paths=1 nar_bytes_out=2324935client # [ 54.093878] nix-daemon[946]: reaped child process 948, status = succeeded4936client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.05 seconds)4937client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4938client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4939client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4940client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4941client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4942client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4943(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.15 seconds)4944subtest: an input only one worker has still reaches the builder4945client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4946client # this derivation will be built:4947client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4948client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4949client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.41 seconds)4950client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4951client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.18 seconds)4952worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4953worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.21 seconds)4954worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4955worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)4956client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24957client # [ 55.220459] nix-daemon[946]: accepted connection from pid 1029, user root (trusted)4958client # this derivation will be built:4959client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4960worker2 # [ 55.430369] nix-grpc-daemon[789]: ts=2026-09-20T14:30:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376984961client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...4962worker1 # [ 55.365524] nix-grpc-daemon[782]: ts=2026-09-20T14:30:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362024963worker1 # [ 55.540893] nix-daemon[842]: accepted connection from pid 1144, user nix-grpc-daemon (trusted)4964worker1 # [ 55.551599] nix-daemon[842]: reaped child process 1147, status = succeeded4965lb # [ 55.595802] niks3-server[1001]: 2026/09/20 14:30:54 INFO Received uploads request method=POST path=/api/pending_closures4966worker1 # [ 55.576488] nix-grpc-daemon[852]: time=2026-09-20T14:30:54.460Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4967worker1 # [ 55.577844] nix-grpc-daemon[852]: time=2026-09-20T14:30:54.460Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4968lb # [ 55.623792] niks3-server[1001]: 2026/09/20 14:30:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4969lb # [ 55.632630] niks3-server[1001]: 2026/09/20 14:30:54 INFO Signed narinfos id=17 count=14970worker1 # [ 55.608806] nix-grpc-daemon[852]: time=2026-09-20T14:30:54.493Z level=INFO msg="Uploading 1 narinfos"4971lb # [ 55.638181] niks3-server[1001]: 2026/09/20 14:30:54 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4972lb # [ 55.646841] niks3-server[1001]: 2026/09/20 14:30:54 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4973lb # [ 55.654980] niks3-server[1001]: 2026/09/20 14:30:54 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4974lb # [ 55.661989] niks3-server[1001]: 2026/09/20 14:30:54 INFO Completed upload id=174975worker1 # [ 55.636944] nix-grpc-daemon[852]: time=2026-09-20T14:30:54.521Z level=INFO msg="Upload complete. (266ms)"4976lb # [ 55.664560] niks3-server[1001]: 2026/09/20 14:30:54 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo4977worker2 # [ 55.725144] nix-grpc-daemon[789]: ts=2026-09-20T14:30:54Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=04978worker2 # [ 55.729284] nix-grpc-daemon[789]: ts=2026-09-20T14:30:54Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14979worker2 # [ 55.778138] nix-grpc-daemon[789]: ts=2026-09-20T14:30:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376984980worker2 # [ 55.790670] nix-grpc-daemon[789]: ts=2026-09-20T14:30:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376984981client # copying 2 paths to 'lb:50051'...4982client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...4983client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...4984worker2 # [ 55.805383] nix-grpc-daemon[789]: ts=2026-09-20T14:30:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376984985worker2 # [ 55.816152] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)4986worker2 # [ 55.994875] nix-daemon[870]: accepted connection from pid 1008, user nix-grpc-daemon (trusted)4987worker2 # [ 56.007972] nix-daemon[870]: reaped child process 1011, status = succeeded4988worker2 # [ 56.162634] nix-daemon[870]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)4989worker2 # [ 56.174163] nix-daemon[870]: reaped child process 1016, status = succeeded4990lb # [ 56.147945] niks3-server[1001]: 2026/09/20 14:30:55 INFO Received uploads request method=POST path=/api/pending_closures4991lb # [ 56.155642] niks3-server[1001]: 2026/09/20 14:30:55 INFO Received uploads request method=POST path=/api/pending_closures4992worker2 # [ 56.205279] nix-grpc-daemon[921]: time=2026-09-20T14:30:55.154Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4993worker2 # [ 56.206712] nix-grpc-daemon[921]: time=2026-09-20T14:30:55.154Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"4994worker2 # [ 56.208424] nix-grpc-daemon[921]: time=2026-09-20T14:30:55.154Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"4995lb # [ 56.185120] niks3-server[1001]: 2026/09/20 14:30:55 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst4996lb # [ 56.199811] niks3-server[1001]: 2026/09/20 14:30:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4997lb # [ 56.203101] niks3-server[1001]: 2026/09/20 14:30:55 INFO Signed narinfos id=18 count=14998lb # [ 56.204786] niks3-server[1001]: 2026/09/20 14:30:55 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls4999lb # [ 56.206642] niks3-server[1001]: 2026/09/20 14:30:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5000lb # [ 56.208618] niks3-server[1001]: 2026/09/20 14:30:55 INFO Signed narinfos id=19 count=25001worker2 # [ 56.250777] nix-grpc-daemon[921]: time=2026-09-20T14:30:55.199Z level=INFO msg="Uploading 3 narinfos"5002lb # [ 56.210759] niks3-server[1001]: 2026/09/20 14:30:55 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5003lb # [ 56.213774] niks3-server[1001]: 2026/09/20 14:30:55 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5004lb # [ 56.222927] niks3-server[1001]: 2026/09/20 14:30:55 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5005lb # [ 56.228758] niks3-server[1001]: 2026/09/20 14:30:55 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5006lb # [ 56.231663] niks3-server[1001]: 2026/09/20 14:30:55 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5007lb # [ 56.233986] niks3-server[1001]: 2026/09/20 14:30:55 INFO Completed upload id=185008lb # [ 56.235827] niks3-server[1001]: 2026/09/20 14:30:55 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5009lb # [ 56.237249] niks3-server[1001]: 2026/09/20 14:30:55 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5010lb # [ 56.239764] niks3-server[1001]: 2026/09/20 14:30:55 INFO Completed upload id=195011worker2 # [ 56.282568] nix-grpc-daemon[921]: time=2026-09-20T14:30:55.232Z level=INFO msg="Upload complete. (451ms)"5012worker2 # [ 56.287178] nix-grpc-daemon[789]: ts=2026-09-20T14:30:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=0 paths=2 nar_bytes_in=9765013worker2 # [ 56.292530] nix-grpc-daemon[789]: ts=2026-09-20T14:30:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376985014worker2 # [ 56.303182] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)5015worker2 # [ 56.307123] nix-daemon[870]: reaped child process 1006, status = succeeded5016worker2 # [ 56.313423] nix-grpc-daemon[789]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5017worker2 # [ 56.320266] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)5018client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5019client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5020worker2 # [ 57.345308] nix-daemon[870]: reaped child process 1021, status = succeeded5021lb # [ 57.314511] envoy[902]: conn peer=[2001:db8:1::1]:35474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=295022lb # [ 57.318345] envoy[902]: conn peer=[2001:db8:1::1]:35464 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3415 tx=1773 ms=19235023lb # [ 57.321999] envoy[902]: conn peer=[2001:db8:1::1]:35484 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3880 tx=1877 ms=15785024lb # [ 57.326458] envoy[902]: conn peer=[2001:db8:1::1]:35490 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=515025lb # [ 57.329967] envoy[902]: conn peer=[2001:db8:1::1]:40428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=585026lb # [ 57.333470] envoy[902]: conn peer=[2001:db8:1::1]:35506 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4302 tx=2221 ms=14955027worker2 # [ 57.505714] nix-daemon[870]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)5028worker2 # [ 57.517264] nix-daemon[870]: reaped child process 1039, status = succeeded5029lb # [ 57.492228] niks3-server[1001]: 2026/09/20 14:30:56 INFO Received uploads request method=POST path=/api/pending_closures5030worker2 # [ 57.542187] nix-grpc-daemon[921]: time=2026-09-20T14:30:56.491Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5031lb # [ 57.524080] niks3-server[1001]: 2026/09/20 14:30:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5032lb # [ 57.531243] niks3-server[1001]: 2026/09/20 14:30:56 INFO Signed narinfos id=20 count=15033worker2 # [ 57.575943] nix-grpc-daemon[921]: time=2026-09-20T14:30:56.525Z level=INFO msg="Uploading 1 narinfos"5034lb # [ 57.537074] niks3-server[1001]: 2026/09/20 14:30:56 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5035lb # [ 57.542384] niks3-server[1001]: 2026/09/20 14:30:56 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5036lb # [ 57.554248] niks3-server[1001]: 2026/09/20 14:30:56 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5037lb # [ 57.560504] niks3-server[1001]: 2026/09/20 14:30:56 INFO Completed upload id=205038worker2 # [ 57.602125] nix-grpc-daemon[921]: time=2026-09-20T14:30:56.551Z level=INFO msg="Upload complete. (273ms)"5039lb # [ 57.563295] niks3-server[1001]: 2026/09/20 14:30:56 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5040worker2 # [ 57.607363] nix-grpc-daemon[789]: ts=2026-09-20T14:30:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15041worker2 # [ 57.618264] nix-daemon[870]: reaped child process 1018, status = succeeded5042client # copying 1 paths...5043worker1 # [ 57.585852] nix-grpc-daemon[782]: ts=2026-09-20T14:30:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:362025044worker1 # [ 57.596422] nix-grpc-daemon[782]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5045client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5046worker2 # [ 57.691255] nix-grpc-daemon[789]: ts=2026-09-20T14:30:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376985047worker2 # [ 57.696066] nix-grpc-daemon[789]: ts=2026-09-20T14:30:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=0 paths=1 nar_bytes_out=1205048client # [ 57.722697] nix-daemon[946]: reaped child process 1032, status = succeeded5049client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.71 seconds)5050client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25051client # [ 57.903037] nix-daemon[946]: accepted connection from pid 1051, user root (trusted)5052client # this derivation will be built:5053client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5054worker1 # [ 58.119457] nix-grpc-daemon[782]: ts=2026-09-20T14:30:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:362025055worker1 # [ 58.121919] nix-grpc-daemon[782]: ts=2026-09-20T14:30:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362025056client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5057worker1 # [ 58.128994] nix-grpc-daemon[852]: time=2026-09-20T14:30:57.013Z level=INFO msg="All 1 paths already cached"5058worker2 # [ 58.210244] nix-grpc-daemon[789]: ts=2026-09-20T14:30:57Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05059worker2 # [ 58.214967] nix-grpc-daemon[789]: ts=2026-09-20T14:30:57Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15060worker1 # [ 58.189948] nix-grpc-daemon[782]: ts=2026-09-20T14:30:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362025061worker1 # [ 58.202391] nix-grpc-daemon[782]: ts=2026-09-20T14:30:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362025062client # copying 1 paths to 'lb:50051'...5063client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5064worker1 # [ 58.221136] nix-grpc-daemon[782]: ts=2026-09-20T14:30:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362025065worker1 # [ 58.232278] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5066worker1 # [ 58.248178] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)5067worker1 # [ 58.353789] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 150 ms (attempt 2/5)5068worker1 # [ 58.506525] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 226 ms (attempt 3/5)5069worker1 # [ 58.739213] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 4/5)5070worker1 # [ 58.915241] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5071worker1 # [ 58.920899] nix-grpc-daemon[782]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5072worker1 # [ 59.088149] nix-daemon[842]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)5073worker1 # [ 59.101214] nix-daemon[842]: reaped child process 1159, status = succeeded5074worker1 # [ 59.247040] nix-daemon[842]: accepted connection from pid 1161, user nix-grpc-daemon (trusted)5075worker1 # [ 59.259128] nix-daemon[842]: reaped child process 1164, status = succeeded5076lb # [ 59.301514] niks3-server[1001]: 2026/09/20 14:30:58 INFO Received uploads request method=POST path=/api/pending_closures5077worker1 # [ 59.411465] nix-grpc-daemon[852]: time=2026-09-20T14:30:58.294Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5078worker1 # [ 59.414988] nix-grpc-daemon[852]: time=2026-09-20T14:30:58.294Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5079lb # [ 59.459903] niks3-server[1001]: 2026/09/20 14:30:58 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5080lb # [ 59.465346] niks3-server[1001]: 2026/09/20 14:30:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5081lb # [ 59.472571] niks3-server[1001]: 2026/09/20 14:30:58 INFO Signed narinfos id=21 count=15082worker1 # [ 59.445683] nix-grpc-daemon[852]: time=2026-09-20T14:30:58.329Z level=INFO msg="Uploading 1 narinfos"5083lb # [ 59.484219] niks3-server[1001]: 2026/09/20 14:30:58 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5084lb # [ 59.488956] niks3-server[1001]: 2026/09/20 14:30:58 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5085lb # [ 59.498551] niks3-server[1001]: 2026/09/20 14:30:58 INFO Completed upload id=215086worker1 # [ 59.471210] nix-grpc-daemon[852]: time=2026-09-20T14:30:58.355Z level=INFO msg="Upload complete. (530ms)"5087lb # [ 59.501971] niks3-server[1001]: 2026/09/20 14:30:58 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5088worker1 # [ 59.476589] nix-grpc-daemon[782]: ts=2026-09-20T14:30:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 paths=1 nar_bytes_in=5605089worker1 # [ 59.482724] nix-grpc-daemon[782]: ts=2026-09-20T14:30:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362025090worker1 # [ 59.496145] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5091worker1 # [ 59.510360] nix-daemon[842]: reaped child process 1150, status = succeeded5092client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5093worker1 # [ 59.517517] nix-daemon[842]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5094client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5095worker1 # [ 60.749710] nix-daemon[842]: reaped child process 1168, status = succeeded5096worker1 # [ 60.881210] nix-daemon[842]: accepted connection from pid 1183, user nix-grpc-daemon (trusted)5097worker1 # [ 60.893058] nix-daemon[842]: reaped child process 1186, status = succeeded5098lb # [ 60.932293] niks3-server[1001]: 2026/09/20 14:31:00 INFO Received uploads request method=POST path=/api/pending_closures5099worker1 # [ 60.911188] nix-grpc-daemon[852]: time=2026-09-20T14:30:59.795Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5100lb # [ 60.959573] niks3-server[1001]: 2026/09/20 14:31:00 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5101lb # [ 60.965178] niks3-server[1001]: 2026/09/20 14:31:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5102lb # [ 60.969390] niks3-server[1001]: 2026/09/20 14:31:00 INFO Signed narinfos id=22 count=15103worker1 # [ 60.944490] nix-grpc-daemon[852]: time=2026-09-20T14:30:59.829Z level=INFO msg="Uploading 1 narinfos"5104lb # [ 60.980231] niks3-server[1001]: 2026/09/20 14:31:00 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5105lb # [ 60.991565] niks3-server[1001]: 2026/09/20 14:31:00 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5106lb # [ 60.996051] niks3-server[1001]: 2026/09/20 14:31:00 INFO Completed upload id=225107worker1 # [ 60.971050] nix-grpc-daemon[852]: time=2026-09-20T14:30:59.855Z level=INFO msg="Upload complete. (242ms)"5108lb # [ 60.999597] niks3-server[1001]: 2026/09/20 14:31:00 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5109worker1 # [ 60.974964] nix-grpc-daemon[782]: ts=2026-09-20T14:30:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36202 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15110client # copying 1 paths...5111worker2 # [ 61.054170] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376985112worker1 # [ 60.987187] nix-daemon[842]: reaped child process 1166, status = succeeded5113worker2 # [ 61.062726] nix-grpc-daemon[789]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5114client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5115worker2 # [ 61.087374] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376985116worker2 # [ 61.089358] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=0 paths=1 nar_bytes_out=1205117client # [ 61.134743] nix-daemon[946]: reaped child process 1054, status = succeeded5118client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.41 seconds)5119client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25120client # [ 61.300063] nix-daemon[946]: accepted connection from pid 1073, user root (trusted)5121client # this derivation will be built:5122client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5123worker2 # [ 61.501159] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376985124client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5125worker2 # [ 61.508919] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376985126worker2 # [ 61.776467] nix-grpc-daemon[921]: time=2026-09-20T14:31:00.725Z level=INFO msg="All 1 paths already cached"5127worker2 # [ 61.796267] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05128worker2 # [ 61.800690] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15129worker2 # [ 61.851090] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376985130worker2 # [ 61.866649] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376985131worker2 # [ 61.880915] nix-grpc-daemon[921]: time=2026-09-20T14:31:00.830Z level=INFO msg="All 1 paths already cached"5132client # copying 1 paths to 'lb:50051'...5133worker2 # [ 61.890550] nix-grpc-daemon[789]: ts=2026-09-20T14:31:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376985134client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5135worker2 # [ 61.896715] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)5136worker2 # [ 62.030135] nix-daemon[870]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)5137worker2 # [ 62.043126] nix-daemon[870]: reaped child process 1051, status = succeeded5138worker2 # [ 62.192969] nix-daemon[870]: accepted connection from pid 1053, user nix-grpc-daemon (trusted)5139worker2 # [ 62.204951] nix-daemon[870]: reaped child process 1056, status = succeeded5140lb # [ 62.178644] niks3-server[1001]: 2026/09/20 14:31:01 INFO Received uploads request method=POST path=/api/pending_closures5141worker2 # [ 62.228597] nix-grpc-daemon[921]: time=2026-09-20T14:31:01.178Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5142worker2 # [ 62.229943] nix-grpc-daemon[921]: time=2026-09-20T14:31:01.178Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5143lb # [ 62.209142] niks3-server[1001]: 2026/09/20 14:31:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5144worker2 # [ 62.259849] nix-grpc-daemon[921]: time=2026-09-20T14:31:01.209Z level=INFO msg="Uploading 1 narinfos"5145lb # [ 62.219247] niks3-server[1001]: 2026/09/20 14:31:01 INFO Signed narinfos id=23 count=15146lb # [ 62.223494] niks3-server[1001]: 2026/09/20 14:31:01 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5147lb # [ 62.231471] niks3-server[1001]: 2026/09/20 14:31:01 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5148lb # [ 62.242135] niks3-server[1001]: 2026/09/20 14:31:01 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5149lb # [ 62.247175] niks3-server[1001]: 2026/09/20 14:31:01 INFO Completed upload id=235150worker2 # [ 62.288570] nix-grpc-daemon[921]: time=2026-09-20T14:31:01.238Z level=INFO msg="Upload complete. (385ms)"5151lb # [ 62.251540] niks3-server[1001]: 2026/09/20 14:31:01 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5152worker2 # [ 62.293589] nix-grpc-daemon[789]: ts=2026-09-20T14:31:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=0 paths=1 nar_bytes_in=5605153worker2 # [ 62.298099] nix-grpc-daemon[789]: ts=2026-09-20T14:31:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376985154worker2 # [ 62.308074] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)5155worker2 # [ 62.311489] nix-daemon[870]: reaped child process 1046, status = succeeded5156worker2 # [ 62.316493] nix-daemon[870]: accepted connection from pid 789, user nix-grpc-daemon (trusted)5157client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5158client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5159worker2 # [ 63.336334] nix-daemon[870]: reaped child process 1060, status = succeeded5160worker2 # [ 63.489087] nix-daemon[870]: accepted connection from pid 1075, user nix-grpc-daemon (trusted)5161worker2 # [ 63.500933] nix-daemon[870]: reaped child process 1078, status = succeeded5162lb # [ 63.474215] niks3-server[1001]: 2026/09/20 14:31:02 INFO Received uploads request method=POST path=/api/pending_closures5163worker2 # [ 63.523638] nix-grpc-daemon[921]: time=2026-09-20T14:31:02.473Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5164lb # [ 63.503658] niks3-server[1001]: 2026/09/20 14:31:02 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5165lb # [ 63.512448] niks3-server[1001]: 2026/09/20 14:31:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5166worker2 # [ 63.559547] nix-grpc-daemon[921]: time=2026-09-20T14:31:02.509Z level=INFO msg="Uploading 1 narinfos"5167lb # [ 63.518564] niks3-server[1001]: 2026/09/20 14:31:02 INFO Signed narinfos id=24 count=15168lb # [ 63.522988] niks3-server[1001]: 2026/09/20 14:31:02 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5169lb # [ 63.535779] niks3-server[1001]: 2026/09/20 14:31:02 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5170lb # [ 63.540512] niks3-server[1001]: 2026/09/20 14:31:02 INFO Completed upload id=245171worker2 # [ 63.581850] nix-grpc-daemon[921]: time=2026-09-20T14:31:02.531Z level=INFO msg="Upload complete. (256ms)"5172lb # [ 63.543369] niks3-server[1001]: 2026/09/20 14:31:02 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5173worker2 # [ 63.587397] nix-grpc-daemon[789]: ts=2026-09-20T14:31:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15174worker2 # [ 63.598510] nix-daemon[870]: reaped child process 1058, status = succeeded5175client # copying 1 paths...5176client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5177worker2 # [ 63.633591] nix-grpc-daemon[789]: ts=2026-09-20T14:31:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376985178worker2 # [ 63.646131] nix-grpc-daemon[789]: ts=2026-09-20T14:31:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376985179worker2 # [ 63.649887] nix-grpc-daemon[789]: ts=2026-09-20T14:31:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37698 duration_s=0 paths=1 nar_bytes_out=1205180client # [ 63.717879] nix-daemon[946]: reaped child process 1076, status = succeeded5181client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.58 seconds)5182(finished: subtest: an input only one worker has still reaches the builder, in 9.53 seconds)5183subtest: upload whose reference the worker lacks is completed from the cache5184client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5185client # this derivation will be built:5186client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5187client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5188client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.36 seconds)5189client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5190client # this derivation will be built:5191client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5192client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5193client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.38 seconds)5194client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5195client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5196worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5197worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.18 seconds)5198worker2 # [ 64.884280] systemd[1]: Stopping Nix worker-protocol over gRPC...5199worker2 # [ 65.742517] nix-grpc-daemon[789]: ts=2026-09-20T14:31:04Z level=info event=signal_exit5200worker2 # [ 65.988142] nix-grpc-daemon[789]: ts=2026-09-20T14:31:04Z level=info event=scheduler_yield5201worker2 # [ 65.990129] nix-grpc-daemon[789]: ts=2026-09-20T14:31:04Z level=info event=scheduler_restarting addr=lb:500515202lb # [ 65.950198] niks3-server[1001]: 2026/09/20 14:31:05 INFO lead: released remote=192.168.1.4:453345203worker2 # [ 65.995758] nix-grpc-daemon[789]: ts=2026-09-20T14:31:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:376825204worker2 # [ 65.997888] nix-grpc-daemon[789]: ts=2026-09-20T14:31:04Z level=info event=worker_gone addr=192.168.1.4:500515205worker2 # [ 65.999121] nix-grpc-daemon[789]: ts=2026-09-20T14:31:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:376825206worker1 # [ 65.928911] nix-grpc-daemon[782]: ts=2026-09-20T14:31:04Z level=info event=scheduler_restarting addr=lb:500515207worker2 # [ 66.000421] nix-grpc-daemon[789]: ts=2026-09-20T14:31:04Z level=info event=worker_gone addr=192.168.1.3:500515208worker2 # [ 66.191230] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=scheduler_connected addr=lb:500515209worker2 # [ 66.193122] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5210worker1 # [ 66.129262] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_connected addr=lb:500515211worker1 # [ 66.132931] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5212worker2 # [ 66.393880] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=scheduler_connected addr=lb:500515213worker2 # [ 66.395760] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5214worker1 # [ 66.333904] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_connected addr=lb:500515215worker1 # [ 66.337673] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5216worker2 # [ 66.596460] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=scheduler_connected addr=lb:500515217worker2 # [ 66.598117] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5218worker1 # [ 66.538458] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_connected addr=lb:500515219worker1 # [ 66.542306] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5220worker2 # [ 66.799107] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=scheduler_connected addr=lb:500515221worker2 # [ 66.800833] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5222worker1 # [ 66.742889] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_connected addr=lb:500515223worker1 # [ 66.746648] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5224worker2 # [ 66.988708] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=scheduler_restarting5225worker2 # [ 67.009634] nix-grpc-daemon[789]: ts=2026-09-20T14:31:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5226worker2 # [ 67.011929] nix-daemon[870]: reaped child process 872, status = succeeded5227worker1 # [ 66.948485] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_connected addr=lb:500515228worker1 # [ 66.951935] nix-grpc-daemon[782]: ts=2026-09-20T14:31:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5229worker1 # [ 67.152921] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_connected addr=lb:500515230worker1 # [ 67.156948] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5231lb # [ 67.315626] envoy[902]: conn peer=[2001:db8:1::1]:40440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3854 tx=1879 ms=22735232lb # [ 67.319367] envoy[902]: conn peer=[2001:db8:1::1]:40446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=473 tx=307 ms=105233lb # [ 67.322751] envoy[902]: conn peer=[2001:db8:1::1]:40472 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=135234lb # [ 67.326160] envoy[902]: conn peer=[2001:db8:1::1]:40456 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3700 tx=1916 ms=29135235lb # [ 67.329656] envoy[902]: conn peer=[2001:db8:1::1]:40486 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3703 tx=1806 ms=21955236lb # [ 67.333179] envoy[902]: conn peer=[2001:db8:1::1]:47794 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=505237lb # [ 67.336693] envoy[902]: conn peer=[2001:db8:1::4]:55584 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=37929 tx=1950 ms=492135238worker1 # [ 67.357722] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_connected addr=lb:500515239worker1 # [ 67.361722] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5240worker1 # [ 67.562341] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_connected addr=lb:500515241worker1 # [ 67.566206] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5242worker1 # [ 67.766959] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_connected addr=lb:500515243worker1 # [ 67.770883] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5244worker2 # [ 67.869671] nix-grpc-daemon[789]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5245worker2 # [ 67.873169] nix-grpc-daemon[789]: E0000 00:00:1789914666.817969 832 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5246worker2 # [ 67.877810] nix-grpc-daemon[789]: E0000 00:00:1789914666.818596 832 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5247worker1 # [ 67.971665] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_connected addr=lb:500515248worker1 # [ 67.973602] nix-grpc-daemon[782]: ts=2026-09-20T14:31:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5249worker2 # [ 68.128953] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5250worker2 # [ 68.132591] systemd[1]: Stopped Nix worker-protocol over gRPC.5251worker2 # [ 68.135377] systemd[1]: nix-grpc-daemon.service: Consumed 1.319s CPU time over 50.659s wall clock time, 52.7M memory peak, 54.5K incoming IP traffic, 157.9K outgoing IP traffic.5252worker2 # [ 68.146194] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5253worker2 # [ 68.150160] systemd[1]: Closed Nix worker-protocol over gRPC.5254worker2 # [ 68.153053] systemd[1]: nix-grpc-daemon.socket: Consumed 390us CPU time over 57.553s wall clock time, 256K memory peak, 71K incoming IP traffic, 58.6K outgoing IP traffic.5255lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5256lb: must succeed: curl -sf localhost:9901/clusters5257lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5258lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5259client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25260worker1 # [ 68.174531] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_connected addr=lb:500515261worker1 # [ 68.178601] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5262worker1 # [ 68.379605] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_connected addr=lb:500515263worker1 # [ 68.385755] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5264client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5265worker1 # [ 68.390813] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362025266worker1 # [ 68.394755] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:362025267client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.25 seconds)5268worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5269worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5270worker2 # [ 68.567329] systemd[1]: Listening on Nix worker-protocol over gRPC.5271worker1 # [ 68.558977] systemd[1]: Stopping Nix worker-protocol over gRPC...5272worker1 # [ 68.563256] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=signal_exit5273worker1 # [ 68.584406] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_connected addr=lb:500515274worker1 # [ 68.587388] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5275worker1 # [ 68.791371] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_connected addr=lb:500515276worker1 # [ 68.795251] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5277worker1 # [ 68.996759] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_connected addr=lb:500515278worker1 # [ 69.000540] nix-grpc-daemon[782]: ts=2026-09-20T14:31:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5279worker2 # [ 69.113347] systemd[1]: Starting scheduler-token.service...5280worker2 # [ 69.183928] systemd[1]: scheduler-token.service: Deactivated successfully.5281worker2 # [ 69.186091] systemd[1]: Finished scheduler-token.service.5282worker2 # [ 69.187856] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 72ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5283worker2 # [ 69.194090] systemd[1]: Starting Nix worker-protocol over gRPC...5284worker1 # [ 69.200955] nix-grpc-daemon[782]: ts=2026-09-20T14:31:08Z level=info event=scheduler_connected addr=lb:500515285worker1 # [ 69.202655] nix-grpc-daemon[782]: ts=2026-09-20T14:31:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5286worker2 # [ 69.340844] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5287worker2 # [ 69.342530] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5288worker2 # [ 69.345777] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15289worker2 # [ 69.353664] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5290worker2 # [ 69.357140] systemd[1]: Started Nix worker-protocol over gRPC.5291worker2 # [ 69.367103] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=scheduler_take_over5292lb # [ 69.324566] niks3-server[1001]: 2026/09/20 14:31:08 INFO lead: acquired remote=192.168.1.4:395845293worker2 # [ 69.386177] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=scheduler_connected addr=lb:500515294worker2 # [ 69.392785] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=oidc_jwks provider=mock keys=15295worker2 # [ 69.394181] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:453005296worker2 # [ 69.395733] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05297worker1 # [ 69.403214] nix-grpc-daemon[782]: ts=2026-09-20T14:31:08Z level=info event=scheduler_connected addr=lb:500515298worker2 # [ 69.473864] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:453005299worker2 # [ 69.476420] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05300worker1 # [ 69.811162] nix-grpc-daemon[782]: ts=2026-09-20T14:31:08Z level=info event=scheduler_restarting5301worker2 # [ 69.888896] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453005302worker2 # [ 69.892399] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:08Z level=info event=worker_gone addr=192.168.1.3:500515303worker1 # [ 69.835371] nix-grpc-daemon[782]: ts=2026-09-20T14:31:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5304worker1 # [ 69.848688] nix-daemon[842]: reaped child process 844, status = succeeded5305worker1 # [ 70.002425] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5306worker1 # [ 70.006479] systemd[1]: Stopped Nix worker-protocol over gRPC.5307worker1 # [ 70.009648] systemd[1]: nix-grpc-daemon.service: Consumed 2.635s CPU time over 52.721s wall clock time, 61.5M memory peak, 89.8K incoming IP traffic, 166.7K outgoing IP traffic.5308worker1 # [ 70.020599] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5309worker1 # [ 70.024879] systemd[1]: Closed Nix worker-protocol over gRPC.5310worker1 # [ 70.028230] systemd[1]: nix-grpc-daemon.socket: Consumed 306us CPU time over 1min 1.459s wall clock time, 256K memory peak, 71.5K incoming IP traffic, 59.6K outgoing IP traffic.5311lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5312lb: must succeed: curl -sf localhost:9901/clusters5313lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5314lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5315client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25316worker2 # [ 70.395553] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453025317worker2 # [ 70.407375] nix-daemon[870]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5318worker2 # [ 70.425366] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 1/5)5319worker2 # [ 70.479321] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 103 ms (attempt 2/5)5320worker2 # [ 70.588330] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 244 ms (attempt 3/5)5321worker2 # [ 70.838201] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 642 ms (attempt 4/5)5322worker2 # [ 71.487362] nix-grpc-daemon[1102]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5323client # copying 2 paths...5324client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5325worker2 # [ 71.510094] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453025326client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5327worker2 # [ 71.523338] nix-daemon[870]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)5328worker2 # [ 71.545400] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:10Z level=info event=niks3_push_started pid=11585329worker2 # [ 71.751316] nix-grpc-daemon[1158]: time=2026-09-20T14:31:10.700Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5330worker2 # [ 71.916099] nix-daemon[870]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5331worker2 # [ 71.926893] nix-daemon[870]: reaped child process 1174, status = succeeded5332lb # [ 71.904164] niks3-server[1001]: 2026/09/20 14:31:11 INFO Received uploads request method=POST path=/api/pending_closures5333lb # [ 71.914150] niks3-server[1001]: 2026/09/20 14:31:11 INFO Received uploads request method=POST path=/api/pending_closures5334worker2 # [ 71.962757] nix-grpc-daemon[1158]: time=2026-09-20T14:31:10.912Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5335worker2 # [ 71.964288] nix-grpc-daemon[1158]: time=2026-09-20T14:31:10.913Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5336worker2 # [ 71.966131] nix-grpc-daemon[1158]: time=2026-09-20T14:31:10.915Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5337lb # [ 71.955120] niks3-server[1001]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5338lb # [ 71.975913] niks3-server[1001]: 2026/09/20 14:31:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5339lb # [ 71.985762] niks3-server[1001]: 2026/09/20 14:31:11 INFO Signed narinfos id=26 count=25340lb # [ 71.989993] niks3-server[1001]: 2026/09/20 14:31:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5341lb # [ 71.995963] niks3-server[1001]: 2026/09/20 14:31:11 INFO Signed narinfos id=25 count=15342worker2 # [ 72.037128] nix-grpc-daemon[1158]: time=2026-09-20T14:31:10.986Z level=INFO msg="Uploading 3 narinfos"5343lb # [ 71.996920] niks3-server[1001]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5344lb # [ 72.002391] niks3-server[1001]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5345lb # [ 72.012484] niks3-server[1001]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5346lb # [ 72.017673] niks3-server[1001]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5347lb # [ 72.021373] niks3-server[1001]: 2026/09/20 14:31:11 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5348lb # [ 72.024541] niks3-server[1001]: 2026/09/20 14:31:11 INFO Completed upload id=255349lb # [ 72.026380] niks3-server[1001]: 2026/09/20 14:31:11 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5350lb # [ 72.028150] niks3-server[1001]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5351lb # [ 72.030529] niks3-server[1001]: 2026/09/20 14:31:11 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5352lb # [ 72.032700] niks3-server[1001]: 2026/09/20 14:31:11 INFO Completed upload id=265353worker2 # [ 72.074460] nix-grpc-daemon[1158]: time=2026-09-20T14:31:11.023Z level=INFO msg="Upload complete. (318ms)"5354worker2 # [ 72.080149] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45302 duration_s=0 paths=2 nar_bytes_in=3125355worker2 # [ 72.091500] nix-daemon[870]: reaped child process 1156, status = succeeded5356client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.94 seconds)5357worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5358worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.01 seconds)5359worker1 # [ 72.120580] systemd[1]: Listening on Nix worker-protocol over gRPC.5360worker1 # [ 72.127152] systemd[1]: Starting Nix worker-protocol over gRPC...5361worker1 # [ 72.283773] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:11Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5362worker1 # [ 72.285556] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5363worker1 # [ 72.289373] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15364worker1 # [ 72.296422] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5365worker1 # [ 72.300952] systemd[1]: Started Nix worker-protocol over gRPC.5366lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5367lb: must succeed: curl -sf localhost:9901/clusters5368worker1 # [ 72.327720] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:11Z level=info event=scheduler_connected addr=lb:500515369worker2 # [ 72.397917] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:11Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:453005370worker2 # [ 72.399843] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05371lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5372lb: must succeed: curl -sf localhost:9901/clusters5373lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5374lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)5375(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.75 seconds)5376subtest: scheduler down: the other node takes the lock and keeps it5377lb: must succeed: curl -sf localhost:9901/clusters5378lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5379lb: must succeed: curl -sf localhost:9901/clusters5380lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5381lb: must succeed: curl -sf localhost:9901/clusters5382lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5383lb: must succeed: curl -sf localhost:9901/clusters5384lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5385worker2 # [ 73.701680] systemd[1]: Stopping Nix worker-protocol over gRPC...5386worker2 # [ 74.358566] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=signal_exit5387worker2 # [ 74.368357] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=scheduler_yield5388lb # [ 74.328820] niks3-server[1001]: 2026/09/20 14:31:13 INFO lead: released remote=192.168.1.4:395845389worker1 # [ 74.305442] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:13Z level=info event=scheduler_restarting addr=lb:500515390worker2 # [ 74.375662] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453005391worker2 # [ 74.382300] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=worker_gone addr=192.168.1.3:500515392worker2 # [ 74.385579] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=scheduler_restarting addr=lb:500515393worker2 # [ 74.390732] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453005394worker2 # [ 74.394170] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=worker_gone addr=192.168.1.4:500515395worker1 # [ 74.505762] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:13Z level=info event=scheduler_connected addr=lb:500515396worker1 # [ 74.507662] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5397worker2 # [ 74.581171] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=scheduler_connected addr=lb:500515398worker2 # [ 74.582889] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5399worker1 # [ 74.708034] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:13Z level=info event=scheduler_connected addr=lb:500515400worker1 # [ 74.709956] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5401worker2 # [ 74.784060] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=scheduler_connected addr=lb:500515402worker2 # [ 74.788223] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5403worker1 # [ 74.910484] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:13Z level=info event=scheduler_connected addr=lb:500515404worker1 # [ 74.912379] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5405worker2 # [ 74.989071] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=scheduler_connected addr=lb:500515406worker2 # [ 74.992849] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker1 # [ 75.113100] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:13Z level=info event=scheduler_connected addr=lb:500515408worker1 # [ 75.114943] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5409worker2 # [ 75.194371] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:14Z level=info event=scheduler_connected addr=lb:500515410worker2 # [ 75.198945] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5411worker2 # [ 75.372842] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:14Z level=info event=scheduler_restarting5412worker2 # [ 75.383168] nix-grpc-daemon[1102]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5413worker1 # [ 75.316689] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:14Z level=info event=scheduler_connected addr=lb:500515414worker2 # [ 75.386652] nix-grpc-daemon[1102]: E0000 00:00:1789914674.332679 1145 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5415worker1 # [ 75.320331] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5416worker2 # [ 75.391993] nix-grpc-daemon[1102]: E0000 00:00:1789914674.341271 1145 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5417worker2 # [ 75.409238] nix-grpc-daemon[1102]: ts=2026-09-20T14:31:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5418worker2 # [ 75.418818] nix-daemon[870]: reaped child process 1151, status = succeeded5419worker1 # [ 75.520982] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:14Z level=info event=scheduler_connected addr=lb:500515420worker1 # [ 75.524935] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5421worker1 # [ 75.725601] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:14Z level=info event=scheduler_connected addr=lb:500515422worker1 # [ 75.729414] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5423worker1 # [ 75.930284] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:14Z level=info event=scheduler_connected addr=lb:500515424worker1 # [ 75.934251] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5425worker1 # [ 76.134943] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_connected addr=lb:500515426worker1 # [ 76.139107] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5427worker1 # [ 76.340574] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_connected addr=lb:500515428worker1 # [ 76.344242] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5429worker1 # [ 76.544958] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_connected addr=lb:500515430worker1 # [ 76.549085] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5431worker1 # [ 76.749494] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_connected addr=lb:500515432worker1 # [ 76.753258] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker1 # [ 76.953592] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_connected addr=lb:500515434worker1 # [ 76.955657] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker1 # [ 77.156132] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=scheduler_connected addr=lb:500515436worker1 # [ 77.157911] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437lb # [ 77.315626] envoy[902]: conn peer=[2001:db8:1::1]:47800 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=155438lb # [ 77.320692] envoy[902]: conn peer=[2001:db8:1::3]:33726 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13472 tx=2755 ms=522225439lb # [ 77.325170] envoy[902]: conn peer=[2001:db8:1::1]:47814 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1008 tx=305 ms=16965440lb # [ 77.331855] envoy[902]: conn peer=[2001:db8:1::4]:47226 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4478 tx=349 ms=60175441lb # [ 77.339131] niks3-server[1001]: 2026/09/20 14:31:16 INFO lead: acquired remote=192.168.1.3:490325442worker1 # [ 77.314674] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=scheduler_take_over5443worker2 # [ 77.382410] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5444worker2 # [ 77.385651] systemd[1]: Stopped Nix worker-protocol over gRPC.5445worker2 # [ 77.389309] systemd[1]: nix-grpc-daemon.service: Consumed 338ms CPU time over 8.186s wall clock time, 32.8M memory peak, 19.4K incoming IP traffic, 26.1K outgoing IP traffic.5446worker2 # [ 77.400316] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5447worker2 # [ 77.403774] systemd[1]: Closed Nix worker-protocol over gRPC.5448worker2 # [ 77.407255] systemd[1]: nix-grpc-daemon.socket: Consumed 500us CPU time over 8.841s wall clock time, 256K memory peak, 19.5K incoming IP traffic, 13.9K outgoing IP traffic.5449worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5450worker1 # [ 77.358819] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=scheduler_connected addr=lb:500515451worker1 # [ 77.360533] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5452worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)5453worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5454worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5455worker1 # [ 77.561101] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=scheduler_connected addr=lb:500515456worker1 # [ 77.563202] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5457worker1 # [ 77.763805] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=scheduler_connected addr=lb:500515458worker1 # [ 77.769831] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:472305459worker1 # [ 77.773345] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05460worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5461worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5462client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25463worker1 # [ 78.742218] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:472445464worker1 # [ 78.747258] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5465client # don't know how to build these paths:5466client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5467worker1 # [ 78.755335] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472445468worker1 # [ 78.760694] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 1/5)5469worker1 # [ 78.833153] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 2/5)5470worker1 # [ 78.957700] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 291 ms (attempt 3/5)5471worker1 # [ 79.251242] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 4/5)5472worker1 # [ 79.263836] nix-grpc-daemon[1209]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5473worker1 # [ 79.284669] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:18Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05474worker1 # [ 79.286897] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:18Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15475worker1 # [ 79.330992] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472445476worker1 # [ 79.339411] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472445477client # copying 1 paths to 'lb:50051'...5478worker1 # [ 79.347822] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472445479client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5480worker1 # [ 79.354489] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5481worker1 # [ 79.365496] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:18Z level=info event=niks3_push_started pid=12795482worker1 # [ 79.542161] nix-grpc-daemon[1279]: time=2026-09-20T14:31:18.426Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5483worker1 # [ 79.711490] nix-daemon[842]: accepted connection from pid 1292, user nix-grpc-daemon (trusted)5484worker1 # [ 79.724210] nix-daemon[842]: reaped child process 1295, status = succeeded5485worker1 # [ 79.857819] nix-daemon[842]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5486worker1 # [ 79.869917] nix-daemon[842]: reaped child process 1300, status = succeeded5487lb # [ 79.910438] niks3-server[1001]: 2026/09/20 14:31:19 INFO Received uploads request method=POST path=/api/pending_closures5488worker1 # [ 79.891599] nix-grpc-daemon[1279]: time=2026-09-20T14:31:18.775Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5489worker1 # [ 79.892974] nix-grpc-daemon[1279]: time=2026-09-20T14:31:18.775Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5490lb # [ 79.942070] niks3-server[1001]: 2026/09/20 14:31:19 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5491lb # [ 79.952815] niks3-server[1001]: 2026/09/20 14:31:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5492lb # [ 79.958484] niks3-server[1001]: 2026/09/20 14:31:19 INFO Signed narinfos id=27 count=15493worker1 # [ 79.933669] nix-grpc-daemon[1279]: time=2026-09-20T14:31:18.817Z level=INFO msg="Uploading 1 narinfos"5494lb # [ 79.971493] niks3-server[1001]: 2026/09/20 14:31:19 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5495lb # [ 79.977133] niks3-server[1001]: 2026/09/20 14:31:19 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5496lb # [ 79.982715] niks3-server[1001]: 2026/09/20 14:31:19 INFO Completed upload id=275497worker1 # [ 79.955333] nix-grpc-daemon[1279]: time=2026-09-20T14:31:18.839Z level=INFO msg="Upload complete. (409ms)"5498worker1 # [ 79.959422] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47244 duration_s=0 paths=1 nar_bytes_in=5205499worker1 # [ 79.965276] nix-grpc-daemon[1279]: time=2026-09-20T14:31:18.843Z level=WARN msg="Failed to register uploaded object" key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo error="server returned 404: object is not pending upload\n"5500worker1 # [ 79.968179] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472445501worker1 # [ 79.974508] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5502worker1 # [ 79.977621] nix-daemon[842]: reaped child process 1277, status = succeeded5503worker1 # [ 79.982789] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5504client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5505client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5506client # farm-a> LOG-farm-a-on-standby5507worker1 # [ 80.607632] nix-daemon[842]: reaped child process 1304, status = succeeded5508worker1 # [ 80.775114] nix-daemon[842]: accepted connection from pid 1319, user nix-grpc-daemon (trusted)5509worker1 # [ 80.787274] nix-daemon[842]: reaped child process 1322, status = succeeded5510lb # [ 80.828914] niks3-server[1001]: 2026/09/20 14:31:20 INFO Received uploads request method=POST path=/api/pending_closures5511worker1 # [ 80.813312] nix-grpc-daemon[1279]: time=2026-09-20T14:31:19.696Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5512worker1 # [ 80.814689] nix-grpc-daemon[1279]: time=2026-09-20T14:31:19.696Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5513lb # [ 80.873246] niks3-server[1001]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5514lb # [ 80.877793] niks3-server[1001]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5515lb # [ 80.883325] niks3-server[1001]: 2026/09/20 14:31:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5516lb # [ 80.888853] niks3-server[1001]: 2026/09/20 14:31:20 INFO Signed narinfos id=28 count=15517worker1 # [ 80.862092] nix-grpc-daemon[1279]: time=2026-09-20T14:31:19.746Z level=INFO msg="Uploading 1 narinfos"5518lb # [ 80.894930] niks3-server[1001]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5519lb # [ 80.903171] niks3-server[1001]: 2026/09/20 14:31:20 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5520lb # [ 80.908180] niks3-server[1001]: 2026/09/20 14:31:20 INFO Completed upload id=285521worker1 # [ 80.880342] nix-grpc-daemon[1279]: time=2026-09-20T14:31:19.764Z level=INFO msg="Upload complete. (290ms)"5522lb # [ 80.911200] niks3-server[1001]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5523worker1 # [ 80.884912] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47244 duration_s=0 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15524worker1 # [ 80.904148] nix-daemon[842]: reaped child process 1302, status = succeeded5525worker1 # [ 80.906268] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:19Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05526worker1 # [ 80.908421] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:19Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15527worker1 # [ 80.950584] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472445528worker1 # [ 80.963941] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472445529worker1 # [ 80.976336] nix-grpc-daemon[1279]: time=2026-09-20T14:31:19.860Z level=INFO msg="All 1 paths already cached"5530client # copying 1 paths to 'lb:50051'...5531client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5532worker1 # [ 80.983959] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472445533worker1 # [ 80.992552] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5534worker1 # [ 81.162782] nix-daemon[842]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5535worker1 # [ 81.175249] nix-daemon[842]: reaped child process 1331, status = succeeded5536worker1 # [ 81.332307] nix-daemon[842]: accepted connection from pid 1333, user nix-grpc-daemon (trusted)5537worker1 # [ 81.344257] nix-daemon[842]: reaped child process 1336, status = succeeded5538lb # [ 81.386520] niks3-server[1001]: 2026/09/20 14:31:20 INFO Received uploads request method=POST path=/api/pending_closures5539worker1 # [ 81.366743] nix-grpc-daemon[1279]: time=2026-09-20T14:31:20.251Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5540worker1 # [ 81.368753] nix-grpc-daemon[1279]: time=2026-09-20T14:31:20.252Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5541lb # [ 81.413426] niks3-server[1001]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5542lb # [ 81.421076] niks3-server[1001]: 2026/09/20 14:31:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5543worker1 # [ 81.399820] nix-grpc-daemon[1279]: time=2026-09-20T14:31:20.284Z level=INFO msg="Uploading 1 narinfos"5544lb # [ 81.427103] niks3-server[1001]: 2026/09/20 14:31:20 INFO Signed narinfos id=29 count=15545lb # [ 81.434215] niks3-server[1001]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5546lb # [ 81.445116] niks3-server[1001]: 2026/09/20 14:31:20 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5547worker1 # [ 81.423524] nix-grpc-daemon[1279]: time=2026-09-20T14:31:20.308Z level=INFO msg="Upload complete. (418ms)"5548lb # [ 81.451246] niks3-server[1001]: 2026/09/20 14:31:20 INFO Completed upload id=295549worker1 # [ 81.425666] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47244 duration_s=0 paths=1 nar_bytes_in=6405550lb # [ 81.454566] niks3-server[1001]: 2026/09/20 14:31:20 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5551worker1 # [ 81.431409] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472445552worker1 # [ 81.435920] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5553worker1 # [ 81.438211] nix-daemon[842]: reaped child process 1326, status = succeeded5554worker1 # [ 81.443514] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5555client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5556client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5557client # farm-b> LOG-farm-b-on-standby5558worker1 # [ 83.070471] nix-daemon[842]: reaped child process 1340, status = succeeded5559worker1 # [ 83.205955] nix-daemon[842]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)5560worker1 # [ 83.218283] nix-daemon[842]: reaped child process 1358, status = succeeded5561lb # [ 83.255924] niks3-server[1001]: 2026/09/20 14:31:22 INFO Received uploads request method=POST path=/api/pending_closures5562worker1 # [ 83.236539] nix-grpc-daemon[1279]: time=2026-09-20T14:31:22.120Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5563worker1 # [ 83.238101] nix-grpc-daemon[1279]: time=2026-09-20T14:31:22.122Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5564lb # [ 83.291861] niks3-server[1001]: 2026/09/20 14:31:22 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5565lb # [ 83.300745] niks3-server[1001]: 2026/09/20 14:31:22 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5566lb # [ 83.309771] niks3-server[1001]: 2026/09/20 14:31:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5567lb # [ 83.315858] niks3-server[1001]: 2026/09/20 14:31:22 INFO Signed narinfos id=30 count=15568worker1 # [ 83.291562] nix-grpc-daemon[1279]: time=2026-09-20T14:31:22.175Z level=INFO msg="Uploading 1 narinfos"5569lb # [ 83.320249] niks3-server[1001]: 2026/09/20 14:31:22 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5570lb # [ 83.330200] niks3-server[1001]: 2026/09/20 14:31:22 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5571lb # [ 83.338259] niks3-server[1001]: 2026/09/20 14:31:22 INFO Completed upload id=305572lb # [ 83.340097] niks3-server[1001]: 2026/09/20 14:31:22 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5573worker1 # [ 83.310983] nix-grpc-daemon[1279]: time=2026-09-20T14:31:22.195Z level=INFO msg="Upload complete. (259ms)"5574worker1 # [ 83.317723] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47244 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15575worker1 # [ 83.329330] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:22Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05576worker1 # [ 83.333156] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:22Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15577worker1 # [ 83.342109] nix-daemon[842]: reaped child process 1338, status = succeeded5578worker1 # [ 83.373397] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472445579worker1 # [ 83.385905] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472445580worker1 # [ 83.397882] nix-grpc-daemon[1279]: time=2026-09-20T14:31:22.282Z level=INFO msg="All 2 paths already cached"5581client # copying 1 paths to 'lb:50051'...5582client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5583worker1 # [ 83.406157] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472445584worker1 # [ 83.414681] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5585worker1 # [ 83.576465] nix-daemon[842]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)5586worker1 # [ 83.588754] nix-daemon[842]: reaped child process 1367, status = succeeded5587worker1 # [ 83.743271] nix-daemon[842]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5588worker1 # [ 83.755296] nix-daemon[842]: reaped child process 1372, status = succeeded5589lb # [ 83.797273] niks3-server[1001]: 2026/09/20 14:31:22 INFO Received uploads request method=POST path=/api/pending_closures5590worker1 # [ 83.777300] nix-grpc-daemon[1279]: time=2026-09-20T14:31:22.661Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5591worker1 # [ 83.778865] nix-grpc-daemon[1279]: time=2026-09-20T14:31:22.663Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5592lb # [ 83.827690] niks3-server[1001]: 2026/09/20 14:31:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5593lb # [ 83.834628] niks3-server[1001]: 2026/09/20 14:31:23 INFO Signed narinfos id=31 count=15594worker1 # [ 83.809861] nix-grpc-daemon[1279]: time=2026-09-20T14:31:22.694Z level=INFO msg="Uploading 1 narinfos"5595lb # [ 83.843099] niks3-server[1001]: 2026/09/20 14:31:23 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5596lb # [ 83.848595] niks3-server[1001]: 2026/09/20 14:31:23 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5597lb # [ 83.858911] niks3-server[1001]: 2026/09/20 14:31:23 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5598lb # [ 83.864443] niks3-server[1001]: 2026/09/20 14:31:23 INFO Completed upload id=315599worker1 # [ 83.837106] nix-grpc-daemon[1279]: time=2026-09-20T14:31:22.721Z level=INFO msg="Upload complete. (409ms)"5600worker1 # [ 83.838905] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47244 duration_s=0 paths=1 nar_bytes_in=7685601lb # [ 83.868764] niks3-server[1001]: 2026/09/20 14:31:23 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5602worker1 # [ 83.845783] nix-daemon[842]: reaped child process 1362, status = succeeded5603worker1 # [ 83.847362] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472445604worker1 # [ 83.851109] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5605worker1 # [ 83.858098] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5606client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5607client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5608client # farm-top> LOG-farm-top-on-standby5609worker1 # [ 85.082673] nix-daemon[842]: reaped child process 1376, status = succeeded5610worker1 # [ 85.229089] nix-daemon[842]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5611worker1 # [ 85.239942] nix-daemon[842]: reaped child process 1394, status = succeeded5612lb # [ 85.281058] niks3-server[1001]: 2026/09/20 14:31:24 INFO Received uploads request method=POST path=/api/pending_closures5613worker1 # [ 85.261711] nix-grpc-daemon[1279]: time=2026-09-20T14:31:24.146Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5614worker1 # [ 85.263285] nix-grpc-daemon[1279]: time=2026-09-20T14:31:24.147Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5615lb # [ 85.316964] niks3-server[1001]: 2026/09/20 14:31:24 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5616lb # [ 85.327141] niks3-server[1001]: 2026/09/20 14:31:24 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5617lb # [ 85.333274] niks3-server[1001]: 2026/09/20 14:31:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5618lb # [ 85.336791] niks3-server[1001]: 2026/09/20 14:31:24 INFO Signed narinfos id=32 count=15619worker1 # [ 85.311748] nix-grpc-daemon[1279]: time=2026-09-20T14:31:24.196Z level=INFO msg="Uploading 1 narinfos"5620lb # [ 85.343857] niks3-server[1001]: 2026/09/20 14:31:24 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5621lb # [ 85.352343] niks3-server[1001]: 2026/09/20 14:31:24 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5622lb # [ 85.357222] niks3-server[1001]: 2026/09/20 14:31:24 INFO Completed upload id=325623worker1 # [ 85.329902] nix-grpc-daemon[1279]: time=2026-09-20T14:31:24.214Z level=INFO msg="Upload complete. (261ms)"5624lb # [ 85.360417] niks3-server[1001]: 2026/09/20 14:31:24 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5625worker1 # [ 85.336438] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47244 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15626worker1 # [ 85.348871] nix-daemon[842]: reaped child process 1374, status = succeeded5627client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.87 seconds)5628client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5629client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.19 seconds)5630worker2 # [ 85.702747] systemd[1]: Listening on Nix worker-protocol over gRPC.5631worker2 # [ 85.710716] systemd[1]: Starting scheduler-token.service...5632worker2 # [ 85.768748] systemd[1]: scheduler-token.service: Deactivated successfully.5633worker2 # [ 85.770325] systemd[1]: Finished scheduler-token.service.5634worker2 # [ 85.771614] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 60ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5635worker2 # [ 85.776719] systemd[1]: Starting Nix worker-protocol over gRPC...5636worker2 # [ 85.906697] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5637worker2 # [ 85.908887] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5638worker2 # [ 85.912509] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15639worker2 # [ 85.918812] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5640worker2 # [ 85.921312] systemd[1]: Started Nix worker-protocol over gRPC.5641lb: must succeed: curl -sf localhost:9901/clusters5642worker2 # [ 85.951517] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:24Z level=info event=scheduler_connected addr=lb:500515643worker1 # [ 85.892161] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:24Z level=info event=oidc_jwks provider=mock keys=15644worker1 # [ 85.896488] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:472305645worker1 # [ 85.900359] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05646lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5647worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5648worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5649lb: must succeed: curl -sf localhost:9901/clusters5650lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5651lb: must succeed: curl -sf localhost:9901/clusters5652lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5653worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5654worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5655(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.75 seconds)5656subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5657client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5658client # [ 86.277759] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5659client # Running as unit: rs.service; invocation ID: 82263864ca9640d786af02ca2d6a54a65660client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5661worker2 # [ 86.526678] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:473445662worker2 # [ 86.537717] nix-daemon[870]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)5663client # [ 86.529445] nix[1177]: don't know how to build these paths:5664client # [ 86.530469] nix[1177]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5665worker1 # [ 86.486826] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:25Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05666worker1 # [ 86.488964] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:25Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15667worker1 # [ 86.531918] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472445668worker1 # [ 86.545317] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472445669client # [ 86.604410] nix[1177]: copying 1 paths to 'lb:50051'...5670worker1 # [ 86.556912] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472445671client # [ 86.607273] nix[1177]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5672worker1 # [ 86.570364] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5673worker1 # [ 86.745394] nix-daemon[842]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5674worker1 # [ 86.757540] nix-daemon[842]: reaped child process 1410, status = succeeded5675worker1 # [ 86.921226] nix-daemon[842]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5676worker1 # [ 86.933066] nix-daemon[842]: reaped child process 1415, status = succeeded5677lb # [ 86.977292] niks3-server[1001]: 2026/09/20 14:31:26 INFO Received uploads request method=POST path=/api/pending_closures5678worker1 # [ 86.958112] nix-grpc-daemon[1279]: time=2026-09-20T14:31:25.842Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5679worker1 # [ 86.959580] nix-grpc-daemon[1279]: time=2026-09-20T14:31:25.842Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5680lb # [ 87.004376] niks3-server[1001]: 2026/09/20 14:31:26 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5681lb # [ 87.016582] niks3-server[1001]: 2026/09/20 14:31:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5682lb # [ 87.020835] niks3-server[1001]: 2026/09/20 14:31:26 INFO Signed narinfos id=33 count=15683worker1 # [ 86.996400] nix-grpc-daemon[1279]: time=2026-09-20T14:31:25.880Z level=INFO msg="Uploading 1 narinfos"5684lb # [ 87.027584] niks3-server[1001]: 2026/09/20 14:31:26 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5685lb # [ 87.040700] niks3-server[1001]: 2026/09/20 14:31:26 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5686lb # [ 87.045500] niks3-server[1001]: 2026/09/20 14:31:26 INFO Completed upload id=335687worker1 # [ 87.020518] nix-grpc-daemon[1279]: time=2026-09-20T14:31:25.905Z level=INFO msg="Upload complete. (438ms)"5688lb # [ 87.048990] niks3-server[1001]: 2026/09/20 14:31:26 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5689worker1 # [ 87.023891] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47244 duration_s=0 paths=1 nar_bytes_in=4165690worker1 # [ 87.029272] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472445691worker1 # [ 87.039106] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5692worker1 # [ 87.046571] nix-daemon[842]: reaped child process 1405, status = succeeded5693client # [ 87.098818] nix[1177]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5694worker1 # [ 87.050544] nix-daemon[842]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5695lb # [ 87.317189] envoy[902]: conn peer=[2001:db8:1::1]:52412 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9620 tx=3883 ms=66145696lb: must succeed: curl -sf localhost:9901/clusters5697lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5698lb: must succeed: curl -sf localhost:9901/clusters5699lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5700lb: must succeed: curl -sf localhost:9901/clusters5701lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5702lb: must succeed: curl -sf localhost:9901/clusters5703lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5704worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5705worker2 # [ 89.179951] systemd-journald[287]: Received client request to rotate journal, rotating.5706worker2 # [ 89.199736] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/f0ebc66aebb64967ac759f4c0f7ffae9.5707worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5708worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5709worker1 # [ 89.047488] systemd[1]: Stopping Nix worker-protocol over gRPC...5710worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5711client: must succeed: sleep 5.05712worker1 # [ 89.305782] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:28Z level=info event=draining5713client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5714worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5715worker1 # [ 94.482402] nix-daemon[842]: reaped child process 1419, status = succeeded5716worker1 # [ 94.638072] nix-daemon[842]: accepted connection from pid 1459, user nix-grpc-daemon (trusted)5717worker1 # [ 94.649889] nix-daemon[842]: reaped child process 1462, status = succeeded5718lb # [ 94.690679] niks3-server[1001]: 2026/09/20 14:31:33 INFO Received uploads request method=POST path=/api/pending_closures5719worker1 # [ 94.668331] nix-grpc-daemon[1279]: time=2026-09-20T14:31:33.552Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5720worker1 # [ 94.669842] nix-grpc-daemon[1279]: time=2026-09-20T14:31:33.554Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5721lb # [ 94.715466] niks3-server[1001]: 2026/09/20 14:31:33 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5722lb # [ 94.719243] niks3-server[1001]: 2026/09/20 14:31:33 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5723lb # [ 94.721936] niks3-server[1001]: 2026/09/20 14:31:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5724lb # [ 94.724599] niks3-server[1001]: 2026/09/20 14:31:33 INFO Signed narinfos id=34 count=15725worker1 # [ 94.697283] nix-grpc-daemon[1279]: time=2026-09-20T14:31:33.581Z level=INFO msg="Uploading 1 narinfos"5726lb # [ 94.730055] niks3-server[1001]: 2026/09/20 14:31:33 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5727lb # [ 94.733805] niks3-server[1001]: 2026/09/20 14:31:33 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5728lb # [ 94.737073] niks3-server[1001]: 2026/09/20 14:31:33 INFO Completed upload id=345729worker1 # [ 94.709472] nix-grpc-daemon[1279]: time=2026-09-20T14:31:33.594Z level=INFO msg="Upload complete. (245ms)"5730worker1 # [ 94.712775] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47244 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15731lb # [ 94.746269] niks3-server[1001]: 2026/09/20 14:31:33 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5732worker1 # [ 94.720930] nix-daemon[842]: reaped child process 1417, status = succeeded5733client # [ 94.792287] systemd[1]: rs.service: Deactivated successfully.5734client # [ 94.793876] systemd[1]: rs.service: Consumed 125ms CPU time over 8.516s wall clock time, 41.4M memory peak, 6.9K incoming IP traffic, 8.3K outgoing IP traffic.5735worker1 # [ 95.306482] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=signal_exit5736worker1 # [ 95.530174] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=scheduler_yield5737lb # [ 95.560238] niks3-server[1001]: 2026/09/20 14:31:34 INFO lead: released remote=192.168.1.3:490325738worker1 # [ 95.537934] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=scheduler_restarting addr=lb:500515739worker2 # [ 95.607925] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:34Z level=info event=scheduler_restarting addr=lb:500515740worker1 # [ 95.542623] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:472305741worker1 # [ 95.545947] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=worker_gone addr=192.168.1.4:500515742worker1 # [ 95.549221] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:472305743worker1 # [ 95.552441] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=worker_gone addr=192.168.1.3:500515744worker1 # [ 95.739655] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=scheduler_connected addr=lb:500515745worker2 # [ 95.808519] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:34Z level=info event=scheduler_connected addr=lb:500515746worker1 # [ 95.743278] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5747worker2 # [ 95.812471] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5748lb # [ 95.888378] niks3-server[1001]: 2026/09/20 14:31:35 INFO lead: acquired remote=192.168.1.4:342685749worker2 # [ 95.933622] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:34Z level=info event=scheduler_take_over5750worker1 # [ 95.944078] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=scheduler_connected addr=lb:500515751worker2 # [ 96.013408] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:34Z level=info event=scheduler_connected addr=lb:500515752worker1 # [ 95.948124] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5753worker2 # [ 96.017198] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5754worker2 # [ 96.218382] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_connected addr=lb:500515755worker1 # [ 96.149299] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:35Z level=info event=scheduler_connected addr=lb:500515756worker2 # [ 96.222243] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5757worker1 # [ 96.153360] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5758worker2 # [ 96.422696] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_connected addr=lb:500515759worker1 # [ 96.354505] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:35Z level=info event=scheduler_connected addr=lb:500515760worker2 # [ 96.425813] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5761worker1 # [ 96.358387] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5762worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.15 seconds)5763client: waiting for success: ! systemctl is-active rs5764client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5765client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5766worker1 # [ 96.534665] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:35Z level=info event=scheduler_restarting5767client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5768client: must fail: journalctl -u rs | grep -q 'reconnecting'5769worker2 # [ 96.628227] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_connected addr=lb:500515770worker2 # [ 96.631424] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5771worker1 # [ 96.569915] nix-daemon[842]: reaped child process 1270, status = succeeded5772worker1 # [ 96.572373] nix-grpc-daemon[1209]: ts=2026-09-20T14:31:35Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5773client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5774lb: must succeed: curl -sf localhost:9901/clusters5775lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5776worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5777worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5778worker1 # [ 96.731856] nix-grpc-daemon[1209]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5779worker1 # [ 96.735261] nix-grpc-daemon[1209]: E0000 00:00:1789914695.615813 1252 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5780worker1 # [ 96.740537] nix-grpc-daemon[1209]: E0000 00:00:1789914695.616339 1252 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5781worker2 # [ 96.829157] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_connected addr=lb:500515782worker2 # [ 96.832933] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5783worker2 # [ 97.033787] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_connected addr=lb:500515784worker2 # [ 97.037564] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5785worker2 # [ 97.238779] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:36Z level=info event=scheduler_connected addr=lb:500515786worker2 # [ 97.242577] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5787lb # [ 97.317112] envoy[902]: conn peer=[2001:db8:1::1]:49700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2947 tx=1349 ms=82745788lb # [ 97.320895] envoy[902]: conn peer=[2001:db8:1::3]:56814 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5568 tx=1292 ms=242375789worker2 # [ 97.443473] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:36Z level=info event=scheduler_connected addr=lb:500515790worker2 # [ 97.456137] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:36Z level=info event=oidc_jwks provider=mock keys=15791worker2 # [ 97.459855] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:359725792worker2 # [ 97.463786] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05793lb: must succeed: curl -sf localhost:9901/clusters5794lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5795worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5796worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5797worker1 # [ 98.043550] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5798worker1 # [ 98.047690] systemd[1]: Stopped Nix worker-protocol over gRPC.5799worker1 # [ 98.049791] systemd[1]: nix-grpc-daemon.service: Consumed 1.294s CPU time over 25.916s wall clock time, 35.6M memory peak, 48.8K incoming IP traffic, 96.2K outgoing IP traffic.5800worker1 # [ 98.065340] systemd[1]: Starting Nix worker-protocol over gRPC...5801worker1 # [ 98.209856] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5802worker1 # [ 98.211957] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5803worker1 # [ 98.215673] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15804worker1 # [ 98.222057] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5805worker1 # [ 98.225288] systemd[1]: Started Nix worker-protocol over gRPC.5806worker1 # [ 98.248307] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:37Z level=info event=scheduler_connected addr=lb:500515807worker2 # [ 98.321089] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:359725808worker2 # [ 98.324783] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05809lb: must succeed: curl -sf localhost:9901/clusters5810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5811worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5812worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5813(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.79 seconds)5814subtest: scheduler restart mid-build: build finishes, no double build5815client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5816client # [ 99.069061] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5817client # Running as unit: mid.service; invocation ID: 4f286f86d97f4c5eb148a910c8ff85255818client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5819worker2 # [ 99.311963] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:473445820client # [ 99.298110] nix[1213]: don't know how to build these paths:5821client # [ 99.299041] nix[1213]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5822worker2 # [ 99.328318] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:38Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05823worker2 # [ 99.332431] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:38Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15824worker2 # [ 99.370494] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473445825worker2 # [ 99.381839] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5)5826worker2 # [ 99.427597] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)5827worker2 # [ 99.548448] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 373 ms (attempt 3/5)5828worker2 # [ 99.927862] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 4/5)5829worker2 # [ 100.132141] nix-grpc-daemon[1198]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5830worker2 # [ 100.147310] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473445831client # [ 100.135181] nix[1213]: copying 1 paths to 'lb:50051'...5832client # [ 100.139192] nix[1213]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5833worker2 # [ 100.159373] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473445834worker2 # [ 100.170085] nix-daemon[870]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)5835worker2 # [ 100.189171] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:39Z level=info event=niks3_push_started pid=13195836worker2 # [ 100.375947] nix-grpc-daemon[1319]: time=2026-09-20T14:31:39.325Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5837worker2 # [ 100.544834] nix-daemon[870]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)5838worker2 # [ 100.555439] nix-daemon[870]: reaped child process 1339, status = succeeded5839worker2 # [ 100.724413] nix-daemon[870]: accepted connection from pid 1341, user nix-grpc-daemon (trusted)5840worker2 # [ 100.735886] nix-daemon[870]: reaped child process 1344, status = succeeded5841lb # [ 100.712603] niks3-server[1001]: 2026/09/20 14:31:39 INFO Received uploads request method=POST path=/api/pending_closures5842worker2 # [ 100.765640] nix-grpc-daemon[1319]: time=2026-09-20T14:31:39.715Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5843worker2 # [ 100.768033] nix-grpc-daemon[1319]: time=2026-09-20T14:31:39.716Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5844lb # [ 100.749549] niks3-server[1001]: 2026/09/20 14:31:39 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5845lb # [ 100.760445] niks3-server[1001]: 2026/09/20 14:31:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5846lb # [ 100.765911] niks3-server[1001]: 2026/09/20 14:31:39 INFO Signed narinfos id=35 count=15847worker2 # [ 100.810216] nix-grpc-daemon[1319]: time=2026-09-20T14:31:39.759Z level=INFO msg="Uploading 1 narinfos"5848lb # [ 100.776144] niks3-server[1001]: 2026/09/20 14:31:39 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5849lb # [ 100.785104] niks3-server[1001]: 2026/09/20 14:31:39 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5850lb # [ 100.788964] niks3-server[1001]: 2026/09/20 14:31:39 INFO Completed upload id=355851lb # [ 100.791158] niks3-server[1001]: 2026/09/20 14:31:39 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo5852worker2 # [ 100.831851] nix-grpc-daemon[1319]: time=2026-09-20T14:31:39.781Z level=INFO msg="Upload complete. (452ms)"5853worker2 # [ 100.837990] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47344 duration_s=0 paths=1 nar_bytes_in=4245854worker2 # [ 100.841789] nix-grpc-daemon[1198]: ts=2026-09-20T14:31:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473445855worker2 # [ 100.851186] nix-daemon[870]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)5856worker2 # [ 100.855119] nix-daemon[870]: reaped child process 1317, status = succeeded5857client # [ 100.838293] nix[1213]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5858worker2 # [ 100.858891] nix-daemon[870]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)5859lb: must succeed: curl -sf localhost:9901/clusters5860lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5861lb: must succeed: curl -sf localhost:9901/clusters5862lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5863worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5864worker2 # [ 102.816885] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1198 (nix-grpc-daemon) on client request.5865worker2 # [ 102.820288] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5866lb # [ 102.786605] niks3-server[1001]: 2026/09/20 14:31:41 INFO lead: released remote=192.168.1.4:342685867worker1 # [ 102.765612] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:41Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5868client # [ 102.815617] nix[1213]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5869worker2 # [ 102.840144] systemd[1]: Starting scheduler-token.service...5870client # [ 102.820466] nix[1213]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5871worker2 # [ 102.852695] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5872worker2 # [ 102.854338] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5873worker2 # [ 102.855587] systemd[1]: nix-grpc-daemon.service: Consumed 435ms CPU time over 17.075s wall clock time, 33.5M memory peak, 16.8K incoming IP traffic, 38K outgoing IP traffic.5874worker2 # [ 102.861827] nix-daemon[870]: reaped child process 1254, status = succeeded5875worker2 # [ 102.864479] nix-daemon[870]: reaped child process 1346, status = succeeded5876worker2 # [ 102.873236] nix-daemon[1348]: unexpected Nix daemon error: error: interrupted by the user5877worker2 # [ 102.878899] nix-daemon[870]: reaped child process 1348, status = failed with exit code 15878worker2 # [ 102.920615] systemd[1]: scheduler-token.service: Deactivated successfully.5879worker2 # [ 102.922193] systemd[1]: Finished scheduler-token.service.5880worker2 # [ 102.923704] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 81ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5881worker2 # [ 102.926199] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5882worker2 # [ 102.930678] systemd[1]: Starting scheduler-token.service...5883worker2 # [ 102.998846] systemd[1]: scheduler-token.service: Deactivated successfully.5884worker2 # [ 102.999880] systemd[1]: Finished scheduler-token.service.5885worker2 # [ 103.001265] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 68ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5886worker2 # [ 103.006676] systemd[1]: Starting Nix worker-protocol over gRPC...5887worker1 # [ 102.965868] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:41Z level=info event=scheduler_connected addr=lb:500515888worker2 # [ 103.166874] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5889worker2 # [ 103.169283] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5890worker2 # [ 103.172884] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15891worker2 # [ 103.180058] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5892worker2 # [ 103.181672] systemd[1]: Started Nix worker-protocol over gRPC.5893worker2 # [ 103.193085] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=scheduler_take_over5894worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.41 seconds)5895lb # [ 103.150382] niks3-server[1001]: 2026/09/20 14:31:42 INFO lead: acquired remote=192.168.1.4:334985896client: must succeed: sleep 5.05897worker2 # [ 103.197697] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:591025898worker2 # [ 103.200292] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05899worker2 # [ 103.215968] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=scheduler_connected addr=lb:500515900worker2 # [ 103.221062] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=oidc_jwks provider=mock keys=15901worker2 # [ 103.222554] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:591025902worker2 # [ 103.224144] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05903worker2 # [ 103.349445] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05904worker2 # [ 103.353821] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:42Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=15905worker1 # [ 103.330156] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402805906worker1 # [ 103.341920] nix-daemon[842]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)5907worker1 # [ 103.358952] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5)5908worker1 # [ 103.368514] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 2/5)5909worker1 # [ 103.433302] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 158 ms (attempt 3/5)5910worker1 # [ 103.591554] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 443 ms (attempt 4/5)5911worker1 # [ 104.037826] nix-grpc-daemon[1469]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5912worker1 # [ 104.055033] nix-grpc-daemon[1469]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...5913worker1 # [ 104.069302] nix-daemon[842]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)5914worker1 # [ 104.074878] nix-daemon[842]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)5915client # [ 104.124941] nix[1213]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5916lb # [ 107.317505] envoy[902]: conn peer=[2001:db8:1::4]:50102 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8646 tx=618 ms=168905917client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5918client: waiting for success: ! systemctl is-active mid5919worker1 # [ 108.486672] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:47Z level=info event=niks3_push_started pid=15705920worker1 # [ 108.517711] nix-daemon[842]: reaped child process 1546, status = succeeded5921worker1 # [ 108.689206] nix-grpc-daemon[1570]: time=2026-09-20T14:31:47.573Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5922worker1 # [ 108.832335] nix-daemon[842]: accepted connection from pid 1583, user nix-grpc-daemon (trusted)5923worker1 # [ 108.844926] nix-daemon[842]: reaped child process 1586, status = succeeded5924lb # [ 108.882553] niks3-server[1001]: 2026/09/20 14:31:48 INFO Received uploads request method=POST path=/api/pending_closures5925worker1 # [ 108.865079] nix-grpc-daemon[1570]: time=2026-09-20T14:31:47.749Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5926worker1 # [ 108.866594] nix-grpc-daemon[1570]: time=2026-09-20T14:31:47.751Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5927lb # [ 108.927857] niks3-server[1001]: 2026/09/20 14:31:48 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5928lb # [ 108.934682] niks3-server[1001]: 2026/09/20 14:31:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5929lb # [ 108.942307] niks3-server[1001]: 2026/09/20 14:31:48 INFO Signed narinfos id=36 count=15930worker1 # [ 108.917636] nix-grpc-daemon[1570]: time=2026-09-20T14:31:47.802Z level=INFO msg="Uploading 1 narinfos"5931lb # [ 108.950325] niks3-server[1001]: 2026/09/20 14:31:48 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5932lb # [ 108.966576] niks3-server[1001]: 2026/09/20 14:31:48 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5933lb # [ 108.973760] niks3-server[1001]: 2026/09/20 14:31:48 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5934lb # [ 108.976040] niks3-server[1001]: 2026/09/20 14:31:48 INFO Completed upload id=365935worker1 # [ 108.948726] nix-grpc-daemon[1570]: time=2026-09-20T14:31:47.833Z level=INFO msg="Upload complete. (256ms)"5936worker1 # [ 108.956196] nix-grpc-daemon[1570]: time=2026-09-20T14:31:47.837Z level=WARN msg="Failed to register uploaded object" key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo error="server returned 404: object is not pending upload\n"5937worker1 # [ 108.962343] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40280 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15938worker1 # [ 108.972974] nix-daemon[842]: reaped child process 1544, status = succeeded5939client # [ 109.034304] systemd[1]: mid.service: Deactivated successfully.5940client # [ 109.035897] systemd[1]: mid.service: Consumed 135ms CPU time over 9.966s wall clock time, 41.4M memory peak, 8.1K incoming IP traffic, 9.8K outgoing IP traffic.5941client: (finished: waiting for success: ! systemctl is-active mid, in 1.11 seconds)5942client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5943client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5944(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.57 seconds)5945subtest: interrupting the client stops the build on the worker5946client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5947client # [ 109.613429] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5948client # Running as unit: intr.service; invocation ID: 97f031df6d4e421883f351f73b4820d45949client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5950worker2 # [ 109.844400] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:591385951worker2 # [ 109.856119] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5952client # [ 109.848262] nix[1249]: don't know how to build these paths:5953client # [ 109.849182] nix[1249]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5954worker2 # [ 109.879311] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:48Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05955worker2 # [ 109.885173] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:48Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=15956worker2 # [ 109.929617] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591385957worker2 # [ 109.942133] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 1/5)5958worker2 # [ 109.953822] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 2/5)5959worker2 # [ 110.043868] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 386 ms (attempt 3/5)5960worker2 # [ 110.439232] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 381 ms (attempt 4/5)5961worker2 # [ 110.827097] nix-grpc-daemon[1390]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5962worker2 # [ 110.853443] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591385963client # [ 110.842259] nix[1249]: copying 1 paths to 'lb:50051'...5964client # [ 110.846352] nix[1249]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5965worker2 # [ 110.868296] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591385966worker2 # [ 110.887191] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5967worker2 # [ 110.909712] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:49Z level=info event=niks3_push_started pid=14665968worker2 # [ 111.102861] nix-grpc-daemon[1466]: time=2026-09-20T14:31:50.051Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5969worker2 # [ 111.262683] nix-daemon[870]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)5970worker2 # [ 111.277385] nix-daemon[870]: reaped child process 1482, status = succeeded5971worker2 # [ 111.435308] nix-daemon[870]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)5972worker2 # [ 111.452106] nix-daemon[870]: reaped child process 1487, status = succeeded5973lb # [ 111.422626] niks3-server[1001]: 2026/09/20 14:31:50 INFO Received uploads request method=POST path=/api/pending_closures5974worker2 # [ 111.470189] nix-grpc-daemon[1466]: time=2026-09-20T14:31:50.419Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5975worker2 # [ 111.471723] nix-grpc-daemon[1466]: time=2026-09-20T14:31:50.421Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5976lb # [ 111.451807] niks3-server[1001]: 2026/09/20 14:31:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5977lb # [ 111.455146] niks3-server[1001]: 2026/09/20 14:31:50 INFO Signed narinfos id=37 count=15978worker2 # [ 111.496786] nix-grpc-daemon[1466]: time=2026-09-20T14:31:50.446Z level=INFO msg="Uploading 1 narinfos"5979lb # [ 111.457024] niks3-server[1001]: 2026/09/20 14:31:50 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5980lb # [ 111.461313] niks3-server[1001]: 2026/09/20 14:31:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5981lb # [ 111.466050] niks3-server[1001]: 2026/09/20 14:31:50 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5982lb # [ 111.472042] niks3-server[1001]: 2026/09/20 14:31:50 INFO Completed upload id=375983worker2 # [ 111.514920] nix-grpc-daemon[1466]: time=2026-09-20T14:31:50.464Z level=INFO msg="Upload complete. (407ms)"5984lb # [ 111.473406] niks3-server[1001]: 2026/09/20 14:31:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo5985worker2 # [ 111.517571] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=0 paths=1 nar_bytes_in=4245986worker2 # [ 111.520855] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591385987worker2 # [ 111.530854] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5988worker2 # [ 111.533183] nix-daemon[870]: reaped child process 1463, status = succeeded5989worker2 # [ 111.538765] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5990client # [ 111.518499] nix[1249]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5991client: must succeed: systemctl kill -s INT intr5992client # [ 113.207533] systemd[1]: intr.service: Sent signal SIGINT to main process 1249 (nix) on client request.5993worker2 # [ 113.233718] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:52Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5994client # [ 113.214944] nix[1249]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5995client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)5996client # [ 113.222694] nix[1249]: error: interrupted by the user5997client # [ 113.252394] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5998client # [ 113.254209] systemd[1]: intr.service: Failed with result 'exit-code'.5999client # [ 113.256071] systemd[1]: intr.service: Consumed 124ms CPU time over 3.640s wall clock time, 43.4M memory peak, 6.1K incoming IP traffic, 8K outgoing IP traffic.6000(finished: subtest: interrupting the client stops the build on the worker, in 3.75 seconds)6001subtest: a deploy mid-build drains: the build finishes, the new generation takes over6002client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6003worker2 # [ 113.360287] nix-daemon[1491]: unexpected Nix daemon error: error: interrupted by the user6004worker2 # [ 113.362208] nix-daemon[870]: reaped child process 1489, status = succeeded6005worker2 # [ 113.365508] nix-daemon[870]: reaped child process 1491, status = failed with exit code 16006client # [ 113.384812] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6007client # Running as unit: sw.service; invocation ID: 8dc1f9b724fa4556b313df5cd5c339246008client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.06 seconds)6009worker1 # [ 113.523811] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:402806010client # [ 113.578865] nix[1270]: don't know how to build these paths:6011client # [ 113.580330] nix[1270]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6012worker2 # [ 113.608910] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:52Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06013worker2 # [ 113.612956] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:52Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16014worker1 # [ 113.584155] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402806015worker1 # [ 113.595856] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:402806016client # [ 113.653614] nix[1270]: copying 1 paths to 'lb:50051'...6017client # [ 113.657147] nix[1270]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6018worker1 # [ 113.608746] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:402806019worker1 # [ 113.619904] nix-daemon[842]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)6020worker1 # [ 113.778337] nix-daemon[842]: accepted connection from pid 1614, user nix-grpc-daemon (trusted)6021worker1 # [ 113.791503] nix-daemon[842]: reaped child process 1617, status = succeeded6022worker1 # [ 113.916043] nix-daemon[842]: accepted connection from pid 1619, user nix-grpc-daemon (trusted)6023worker1 # [ 113.927774] nix-daemon[842]: reaped child process 1622, status = succeeded6024lb # [ 113.965564] niks3-server[1001]: 2026/09/20 14:31:53 INFO Received uploads request method=POST path=/api/pending_closures6025worker1 # [ 113.949315] nix-grpc-daemon[1570]: time=2026-09-20T14:31:52.833Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6026worker1 # [ 113.950734] nix-grpc-daemon[1570]: time=2026-09-20T14:31:52.833Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6027lb # [ 113.995497] niks3-server[1001]: 2026/09/20 14:31:53 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6028lb # [ 114.005379] niks3-server[1001]: 2026/09/20 14:31:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6029lb # [ 114.010693] niks3-server[1001]: 2026/09/20 14:31:53 INFO Signed narinfos id=38 count=16030worker1 # [ 113.986383] nix-grpc-daemon[1570]: time=2026-09-20T14:31:52.870Z level=INFO msg="Uploading 1 narinfos"6031lb # [ 114.017096] niks3-server[1001]: 2026/09/20 14:31:53 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6032lb # [ 114.027136] niks3-server[1001]: 2026/09/20 14:31:53 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6033lb # [ 114.033529] niks3-server[1001]: 2026/09/20 14:31:53 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6034lb # [ 114.037874] niks3-server[1001]: 2026/09/20 14:31:53 INFO Completed upload id=386035worker1 # [ 114.010556] nix-grpc-daemon[1570]: time=2026-09-20T14:31:52.895Z level=INFO msg="Upload complete. (379ms)"6036worker1 # [ 114.013915] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40280 duration_s=0 paths=1 nar_bytes_in=4166037worker1 # [ 114.019272] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402806038worker1 # [ 114.027488] nix-daemon[842]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)6039worker1 # [ 114.034405] nix-daemon[842]: reaped child process 1612, status = succeeded6040worker1 # [ 114.039258] nix-daemon[842]: accepted connection from pid 1469, user nix-grpc-daemon (trusted)6041client # [ 114.087713] nix[1270]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6042worker1: must succeed: systemctl reload nix-grpc-daemon.service6043worker1 # [ 115.826262] systemd[1]: Reloading Nix worker-protocol over gRPC...6044worker1 # [ 115.862581] systemd[1]: Reloaded Nix worker-protocol over gRPC.6045worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.07 seconds)6046worker1: must succeed: pgrep -f 'read -t [9]0 x'6047worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6048client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26049worker2 # [ 116.224383] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:591386050client # don't know how to build these paths:6051client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6052worker2 # [ 116.232540] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591386053worker2 # [ 116.254254] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:55Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06054worker2 # [ 116.258125] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:55Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16055worker1 # [ 116.228416] nix-grpc-daemon[1469]: ts=2026-09-20T14:31:55Z level=info event=draining6056worker2 # [ 116.306682] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386057worker2 # [ 116.318567] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591386058client # copying 1 paths to 'lb:50051'...6059worker2 # [ 116.330768] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591386060client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6061worker2 # [ 116.358073] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6062worker2 # [ 116.511983] nix-daemon[870]: accepted connection from pid 1541, user nix-grpc-daemon (trusted)6063worker2 # [ 116.523994] nix-daemon[870]: reaped child process 1544, status = succeeded6064worker2 # [ 116.653747] nix-daemon[870]: accepted connection from pid 1546, user nix-grpc-daemon (trusted)6065worker2 # [ 116.665630] nix-daemon[870]: reaped child process 1549, status = succeeded6066lb # [ 116.641673] niks3-server[1001]: 2026/09/20 14:31:55 INFO Received uploads request method=POST path=/api/pending_closures6067worker2 # [ 116.690381] nix-grpc-daemon[1466]: time=2026-09-20T14:31:55.639Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6068worker2 # [ 116.691820] nix-grpc-daemon[1466]: time=2026-09-20T14:31:55.639Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6069lb # [ 116.665345] niks3-server[1001]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6070lb # [ 116.677565] niks3-server[1001]: 2026/09/20 14:31:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6071worker2 # [ 116.726381] nix-grpc-daemon[1466]: time=2026-09-20T14:31:55.675Z level=INFO msg="Uploading 1 narinfos"6072lb # [ 116.684768] niks3-server[1001]: 2026/09/20 14:31:55 INFO Signed narinfos id=39 count=16073lb # [ 116.689095] niks3-server[1001]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6074lb # [ 116.700668] niks3-server[1001]: 2026/09/20 14:31:55 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6075worker2 # [ 116.750512] nix-grpc-daemon[1466]: time=2026-09-20T14:31:55.700Z level=INFO msg="Upload complete. (381ms)"6076lb # [ 116.709065] niks3-server[1001]: 2026/09/20 14:31:55 INFO Completed upload id=396077worker2 # [ 116.753075] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=0 paths=1 nar_bytes_in=5286078lb # [ 116.711957] niks3-server[1001]: 2026/09/20 14:31:55 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6079worker2 # [ 116.758560] nix-daemon[870]: reaped child process 1539, status = succeeded6080worker2 # [ 116.759984] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386081worker2 # [ 116.763528] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6082worker2 # [ 116.769726] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6083client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6084client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6085lb # [ 117.318657] envoy[902]: conn peer=[2001:db8:1::1]:44860 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3848 tx=1660 ms=97246086lb # [ 117.323059] envoy[902]: conn peer=[2001:db8:1::1]:42762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2924 tx=758 ms=34146087client # farm-a> LOG-farm-a-during-drain6088worker2 # [ 117.989184] nix-daemon[870]: reaped child process 1553, status = succeeded6089worker2 # [ 118.147428] nix-daemon[870]: accepted connection from pid 1568, user nix-grpc-daemon (trusted)6090worker2 # [ 118.157857] nix-daemon[870]: reaped child process 1571, status = succeeded6091lb # [ 118.132961] niks3-server[1001]: 2026/09/20 14:31:57 INFO Received uploads request method=POST path=/api/pending_closures6092worker2 # [ 118.184551] nix-grpc-daemon[1466]: time=2026-09-20T14:31:57.133Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6093worker2 # [ 118.186192] nix-grpc-daemon[1466]: time=2026-09-20T14:31:57.135Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6094lb # [ 118.173317] niks3-server[1001]: 2026/09/20 14:31:57 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6095lb # [ 118.181271] niks3-server[1001]: 2026/09/20 14:31:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6096lb # [ 118.184537] niks3-server[1001]: 2026/09/20 14:31:57 INFO Signed narinfos id=40 count=16097worker2 # [ 118.227705] nix-grpc-daemon[1466]: time=2026-09-20T14:31:57.176Z level=INFO msg="Uploading 1 narinfos"6098lb # [ 118.186516] niks3-server[1001]: 2026/09/20 14:31:57 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6099lb # [ 118.193992] niks3-server[1001]: 2026/09/20 14:31:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6100lb # [ 118.199104] niks3-server[1001]: 2026/09/20 14:31:57 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6101lb # [ 118.204688] niks3-server[1001]: 2026/09/20 14:31:57 INFO Completed upload id=406102worker2 # [ 118.246156] nix-grpc-daemon[1466]: time=2026-09-20T14:31:57.195Z level=INFO msg="Upload complete. (267ms)"6103lb # [ 118.206031] niks3-server[1001]: 2026/09/20 14:31:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6104worker2 # [ 118.249758] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16105worker2 # [ 118.260257] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:57Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06106worker2 # [ 118.262238] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:57Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16107worker2 # [ 118.267163] nix-daemon[870]: reaped child process 1551, status = succeeded6108worker2 # [ 118.303185] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386109worker2 # [ 118.314177] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591386110worker2 # [ 118.325689] nix-grpc-daemon[1466]: time=2026-09-20T14:31:57.275Z level=INFO msg="All 1 paths already cached"6111client # copying 1 paths to 'lb:50051'...6112client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6113worker2 # [ 118.332500] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591386114worker2 # [ 118.340720] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6115worker2 # [ 118.498977] nix-daemon[870]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6116worker2 # [ 118.511280] nix-daemon[870]: reaped child process 1580, status = succeeded6117worker2 # [ 118.668559] nix-daemon[870]: accepted connection from pid 1582, user nix-grpc-daemon (trusted)6118worker2 # [ 118.680163] nix-daemon[870]: reaped child process 1585, status = succeeded6119lb # [ 118.654166] niks3-server[1001]: 2026/09/20 14:31:57 INFO Received uploads request method=POST path=/api/pending_closures6120worker2 # [ 118.699976] nix-grpc-daemon[1466]: time=2026-09-20T14:31:57.649Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6121worker2 # [ 118.702134] nix-grpc-daemon[1466]: time=2026-09-20T14:31:57.651Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6122lb # [ 118.667756] niks3-server[1001]: 2026/09/20 14:31:57 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6123lb # [ 118.671581] niks3-server[1001]: 2026/09/20 14:31:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6124lb # [ 118.673229] niks3-server[1001]: 2026/09/20 14:31:57 INFO Signed narinfos id=41 count=16125worker2 # [ 118.716073] nix-grpc-daemon[1466]: time=2026-09-20T14:31:57.665Z level=INFO msg="Uploading 1 narinfos"6126lb # [ 118.677068] niks3-server[1001]: 2026/09/20 14:31:57 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6127lb # [ 118.681520] niks3-server[1001]: 2026/09/20 14:31:57 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6128lb # [ 118.684154] niks3-server[1001]: 2026/09/20 14:31:57 INFO Completed upload id=416129worker2 # [ 118.725710] nix-grpc-daemon[1466]: time=2026-09-20T14:31:57.675Z level=INFO msg="Upload complete. (373ms)"6130worker2 # [ 118.727636] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=0 paths=1 nar_bytes_in=6406131lb # [ 118.686859] niks3-server[1001]: 2026/09/20 14:31:57 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6132worker2 # [ 118.730650] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386133worker2 # [ 118.737142] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6134worker2 # [ 118.739367] nix-daemon[870]: reaped child process 1575, status = succeeded6135worker2 # [ 118.744458] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6136client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6137client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6138client # farm-b> LOG-farm-b-during-drain6139worker2 # [ 120.359215] nix-daemon[870]: reaped child process 1589, status = succeeded6140worker2 # [ 120.478791] nix-daemon[870]: accepted connection from pid 1604, user nix-grpc-daemon (trusted)6141worker2 # [ 120.490812] nix-daemon[870]: reaped child process 1607, status = succeeded6142lb # [ 120.462128] niks3-server[1001]: 2026/09/20 14:31:59 INFO Received uploads request method=POST path=/api/pending_closures6143worker2 # [ 120.511109] nix-grpc-daemon[1466]: time=2026-09-20T14:31:59.460Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6144worker2 # [ 120.512591] nix-grpc-daemon[1466]: time=2026-09-20T14:31:59.462Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6145lb # [ 120.497249] niks3-server[1001]: 2026/09/20 14:31:59 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6146lb # [ 120.504301] niks3-server[1001]: 2026/09/20 14:31:59 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6147lb # [ 120.515395] niks3-server[1001]: 2026/09/20 14:31:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6148lb # [ 120.522955] niks3-server[1001]: 2026/09/20 14:31:59 INFO Signed narinfos id=42 count=16149worker2 # [ 120.564580] nix-grpc-daemon[1466]: time=2026-09-20T14:31:59.514Z level=INFO msg="Uploading 1 narinfos"6150lb # [ 120.527354] niks3-server[1001]: 2026/09/20 14:31:59 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6151lb # [ 120.536694] niks3-server[1001]: 2026/09/20 14:31:59 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6152lb # [ 120.543052] niks3-server[1001]: 2026/09/20 14:31:59 INFO Completed upload id=426153worker2 # [ 120.584609] nix-grpc-daemon[1466]: time=2026-09-20T14:31:59.534Z level=INFO msg="Upload complete. (232ms)"6154lb # [ 120.545315] niks3-server[1001]: 2026/09/20 14:31:59 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6155worker2 # [ 120.589779] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16156worker2 # [ 120.601627] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:59Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06157worker2 # [ 120.605102] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:59Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16158worker2 # [ 120.613509] nix-daemon[870]: reaped child process 1587, status = succeeded6159worker2 # [ 120.644723] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386160worker2 # [ 120.654937] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591386161worker2 # [ 120.664794] nix-grpc-daemon[1466]: time=2026-09-20T14:31:59.614Z level=INFO msg="All 2 paths already cached"6162client # copying 1 paths to 'lb:50051'...6163client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6164worker2 # [ 120.670986] nix-grpc-daemon[1390]: ts=2026-09-20T14:31:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591386165worker2 # [ 120.678978] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6166worker2 # [ 120.837514] nix-daemon[870]: accepted connection from pid 1613, user nix-grpc-daemon (trusted)6167worker2 # [ 120.849492] nix-daemon[870]: reaped child process 1616, status = succeeded6168worker2 # [ 121.007718] nix-daemon[870]: accepted connection from pid 1618, user nix-grpc-daemon (trusted)6169worker2 # [ 121.019407] nix-daemon[870]: reaped child process 1621, status = succeeded6170lb # [ 120.991492] niks3-server[1001]: 2026/09/20 14:32:00 INFO Received uploads request method=POST path=/api/pending_closures6171worker2 # [ 121.037212] nix-grpc-daemon[1466]: time=2026-09-20T14:31:59.986Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6172worker2 # [ 121.038778] nix-grpc-daemon[1466]: time=2026-09-20T14:31:59.988Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6173lb # [ 121.006846] niks3-server[1001]: 2026/09/20 14:32:00 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6174lb # [ 121.010055] niks3-server[1001]: 2026/09/20 14:32:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6175lb # [ 121.012763] niks3-server[1001]: 2026/09/20 14:32:00 INFO Signed narinfos id=43 count=16176worker2 # [ 121.054333] nix-grpc-daemon[1466]: time=2026-09-20T14:32:00.003Z level=INFO msg="Uploading 1 narinfos"6177lb # [ 121.015996] niks3-server[1001]: 2026/09/20 14:32:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6178lb # [ 121.019460] niks3-server[1001]: 2026/09/20 14:32:00 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6179lb # [ 121.023183] niks3-server[1001]: 2026/09/20 14:32:00 INFO Completed upload id=436180worker2 # [ 121.065796] nix-grpc-daemon[1466]: time=2026-09-20T14:32:00.015Z level=INFO msg="Upload complete. (375ms)"6181lb # [ 121.024698] niks3-server[1001]: 2026/09/20 14:32:00 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6182worker2 # [ 121.067827] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=0 paths=1 nar_bytes_in=7686183worker2 # [ 121.070499] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386184worker2 # [ 121.076715] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6185worker2 # [ 121.079662] nix-daemon[870]: reaped child process 1611, status = succeeded6186client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6187client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6188worker2 # [ 121.084574] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6189client # farm-top> LOG-farm-top-during-drain6190worker2 # [ 122.098122] nix-daemon[870]: reaped child process 1625, status = succeeded6191worker2 # [ 122.214891] nix-daemon[870]: accepted connection from pid 1640, user nix-grpc-daemon (trusted)6192worker2 # [ 122.226561] nix-daemon[870]: reaped child process 1643, status = succeeded6193lb # [ 122.195954] niks3-server[1001]: 2026/09/20 14:32:01 INFO Received uploads request method=POST path=/api/pending_closures6194worker2 # [ 122.241936] nix-grpc-daemon[1466]: time=2026-09-20T14:32:01.191Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6195worker2 # [ 122.243429] nix-grpc-daemon[1466]: time=2026-09-20T14:32:01.193Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6196lb # [ 122.229739] niks3-server[1001]: 2026/09/20 14:32:01 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6197lb # [ 122.237775] niks3-server[1001]: 2026/09/20 14:32:01 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6198lb # [ 122.248857] niks3-server[1001]: 2026/09/20 14:32:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6199worker2 # [ 122.295802] nix-grpc-daemon[1466]: time=2026-09-20T14:32:01.245Z level=INFO msg="Uploading 1 narinfos"6200lb # [ 122.254177] niks3-server[1001]: 2026/09/20 14:32:01 INFO Signed narinfos id=44 count=16201lb # [ 122.259671] niks3-server[1001]: 2026/09/20 14:32:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6202lb # [ 122.268603] niks3-server[1001]: 2026/09/20 14:32:01 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6203lb # [ 122.274287] niks3-server[1001]: 2026/09/20 14:32:01 INFO Completed upload id=446204worker2 # [ 122.315949] nix-grpc-daemon[1466]: time=2026-09-20T14:32:01.265Z level=INFO msg="Upload complete. (225ms)"6205lb # [ 122.276255] niks3-server[1001]: 2026/09/20 14:32:01 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6206worker2 # [ 122.321635] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16207worker2 # [ 122.333401] nix-daemon[870]: reaped child process 1623, status = succeeded6208client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.39 seconds)6209client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6210client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6211worker1: must succeed: pkill -f 'read -t [9]0 x'6212worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6213client: waiting for success: ! systemctl is-active sw6214worker1 # [ 122.671876] nix-daemon[842]: reaped child process 1626, status = succeeded6215worker1 # [ 122.805292] nix-daemon[842]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6216worker1 # [ 122.817141] nix-daemon[842]: reaped child process 1669, status = succeeded6217lb # [ 122.854896] niks3-server[1001]: 2026/09/20 14:32:02 INFO Received uploads request method=POST path=/api/pending_closures6218worker1 # [ 122.835432] nix-grpc-daemon[1570]: time=2026-09-20T14:32:01.719Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6219worker1 # [ 122.836930] nix-grpc-daemon[1570]: time=2026-09-20T14:32:01.721Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6220lb # [ 122.891302] niks3-server[1001]: 2026/09/20 14:32:02 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6221lb # [ 122.901252] niks3-server[1001]: 2026/09/20 14:32:02 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6222lb # [ 122.910377] niks3-server[1001]: 2026/09/20 14:32:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6223lb # [ 122.915126] niks3-server[1001]: 2026/09/20 14:32:02 INFO Signed narinfos id=45 count=16224worker1 # [ 122.891106] nix-grpc-daemon[1570]: time=2026-09-20T14:32:01.775Z level=INFO msg="Uploading 1 narinfos"6225lb # [ 122.920680] niks3-server[1001]: 2026/09/20 14:32:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6226lb # [ 122.930180] niks3-server[1001]: 2026/09/20 14:32:02 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6227lb # [ 122.937115] niks3-server[1001]: 2026/09/20 14:32:02 INFO Completed upload id=456228worker1 # [ 122.909495] nix-grpc-daemon[1570]: time=2026-09-20T14:32:01.794Z level=INFO msg="Upload complete. (255ms)"6229lb # [ 122.939787] niks3-server[1001]: 2026/09/20 14:32:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6230worker1 # [ 122.915464] nix-grpc-daemon[1469]: ts=2026-09-20T14:32:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40280 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16231worker1 # [ 122.927518] nix-daemon[842]: reaped child process 1624, status = succeeded6232client # [ 123.004269] systemd[1]: sw.service: Deactivated successfully.6233client # [ 123.008243] systemd[1]: sw.service: Consumed 128ms CPU time over 9.621s wall clock time, 43.4M memory peak, 6.7K incoming IP traffic, 8.1K outgoing IP traffic.6234worker1 # [ 123.230614] nix-grpc-daemon[1469]: ts=2026-09-20T14:32:02Z level=info event=signal_exit6235client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6236client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6237client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6238worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6239worker2 # [ 124.307192] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:591026240worker1 # [ 124.241536] nix-grpc-daemon[1469]: ts=2026-09-20T14:32:03Z level=info event=scheduler_restarting6241worker1 # [ 124.243248] nix-grpc-daemon[1469]: ts=2026-09-20T14:32:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6242worker2 # [ 124.311316] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:03Z level=info event=worker_gone addr=192.168.1.3:500516243worker1 # [ 124.252655] nix-daemon[842]: reaped child process 1538, status = succeeded6244worker1 # [ 125.088309] nix-grpc-daemon[1469]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6245worker1 # [ 125.089857] nix-grpc-daemon[1469]: E0000 00:00:1789914723.972745 1512 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6246worker1 # [ 125.091814] nix-grpc-daemon[1469]: E0000 00:00:1789914723.972861 1512 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6247worker1 # [ 126.257727] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6248worker1 # [ 126.259466] systemd[1]: nix-grpc-daemon.service: Consumed 617ms CPU time over 28.192s wall clock time, 32.6M memory peak, 20.9K incoming IP traffic, 27.3K outgoing IP traffic.6249worker1 # [ 126.266429] systemd[1]: Starting Nix worker-protocol over gRPC...6250worker1 # [ 126.413856] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6251worker1 # [ 126.415641] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6252worker1 # [ 126.419402] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16253worker1 # [ 126.425894] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6254worker1 # [ 126.428482] systemd[1]: Started Nix worker-protocol over gRPC.6255worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.79 seconds)6256lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6257lb: must succeed: curl -sf localhost:9901/clusters6258worker1 # [ 126.452846] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:05Z level=info event=scheduler_connected addr=lb:500516259worker2 # [ 126.524390] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:591026260worker2 # [ 126.528633] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06261lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6262lb # [ 127.318886] envoy[902]: conn peer=[2001:db8:1::1]:60764 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9542 tx=3801 ms=61226263lb # [ 127.322682] envoy[902]: conn peer=[2001:db8:1::1]:60756 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2919 tx=1331 ms=94046264lb # [ 127.326309] envoy[902]: conn peer=[2001:db8:1::3]:53956 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1114 tx=448 ms=259946265lb: must succeed: curl -sf localhost:9901/clusters6266lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6267lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.12 seconds)6268lb: must succeed: curl -sf localhost:9901/clusters6269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6270worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6271worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6272(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.38 seconds)6273subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6274client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16275worker1 # [ 127.886797] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:435786276worker1 # [ 127.898122] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6277worker1 # [ 127.910739] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786278worker1 # [ 127.921214] nix-grpc-daemon[1683]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 1/5)6279worker1 # [ 128.025698] nix-grpc-daemon[1683]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 110 ms (attempt 2/5)6280worker1 # [ 128.141967] nix-grpc-daemon[1683]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 243 ms (attempt 3/5)6281worker1 # [ 128.385290] nix-grpc-daemon[1683]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 377 ms (attempt 4/5)6282worker1 # [ 128.768272] nix-grpc-daemon[1683]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6283worker2 # [ 128.884889] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:07Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06284worker2 # [ 128.889575] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:07Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16285worker2 # [ 128.894087] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:07Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06286worker2 # [ 128.897831] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:07Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06287worker1 # [ 128.865518] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786288worker1 # [ 128.879134] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786289worker1 # [ 128.892401] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435786290worker1 # [ 128.899657] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6291worker1 # [ 128.920940] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:07Z level=info event=niks3_push_started pid=17396292worker1 # [ 129.092536] nix-grpc-daemon[1739]: time=2026-09-20T14:32:07.976Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6293worker1 # [ 129.238308] nix-daemon[842]: accepted connection from pid 1752, user nix-grpc-daemon (trusted)6294worker1 # [ 129.248855] nix-daemon[842]: reaped child process 1755, status = succeeded6295worker1 # [ 129.399487] nix-daemon[842]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6296worker1 # [ 129.411577] nix-daemon[842]: reaped child process 1760, status = succeeded6297lb # [ 129.451759] niks3-server[1001]: 2026/09/20 14:32:08 INFO Received uploads request method=POST path=/api/pending_closures6298worker1 # [ 129.432025] nix-grpc-daemon[1739]: time=2026-09-20T14:32:08.316Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6299worker1 # [ 129.434520] nix-grpc-daemon[1739]: time=2026-09-20T14:32:08.318Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6300lb # [ 129.483149] niks3-server[1001]: 2026/09/20 14:32:08 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6301lb # [ 129.494301] niks3-server[1001]: 2026/09/20 14:32:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6302lb # [ 129.500243] niks3-server[1001]: 2026/09/20 14:32:08 INFO Signed narinfos id=46 count=16303worker1 # [ 129.475914] nix-grpc-daemon[1739]: time=2026-09-20T14:32:08.360Z level=INFO msg="Uploading 1 narinfos"6304lb # [ 129.514776] niks3-server[1001]: 2026/09/20 14:32:08 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6305lb # [ 129.518886] niks3-server[1001]: 2026/09/20 14:32:08 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6306worker1 # [ 129.496915] nix-grpc-daemon[1739]: time=2026-09-20T14:32:08.381Z level=INFO msg="Upload complete. (401ms)"6307lb # [ 129.524697] niks3-server[1001]: 2026/09/20 14:32:08 INFO Completed upload id=466308worker1 # [ 129.498914] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=0 paths=1 nar_bytes_in=5046309lb # [ 129.529670] niks3-server[1001]: 2026/09/20 14:32:08 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6310worker1 # [ 129.504517] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786311worker1 # [ 129.509283] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6312worker1 # [ 129.511525] nix-daemon[842]: reaped child process 1737, status = succeeded6313worker1 # [ 129.516530] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6314worker1 # [ 130.342798] nix-daemon[842]: reaped child process 1764, status = succeeded6315worker1 # [ 130.469503] nix-daemon[842]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6316worker1 # [ 130.481424] nix-daemon[842]: reaped child process 1782, status = succeeded6317lb # [ 130.518363] niks3-server[1001]: 2026/09/20 14:32:09 INFO Received uploads request method=POST path=/api/pending_closures6318worker1 # [ 130.495189] nix-grpc-daemon[1739]: time=2026-09-20T14:32:09.379Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6319worker1 # [ 130.496656] nix-grpc-daemon[1739]: time=2026-09-20T14:32:09.381Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6320lb # [ 130.535875] niks3-server[1001]: 2026/09/20 14:32:09 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6321lb # [ 130.542473] niks3-server[1001]: 2026/09/20 14:32:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6322lb # [ 130.544692] niks3-server[1001]: 2026/09/20 14:32:09 INFO Signed narinfos id=47 count=16323worker1 # [ 130.518738] nix-grpc-daemon[1739]: time=2026-09-20T14:32:09.402Z level=INFO msg="Uploading 1 narinfos"6324lb # [ 130.547641] niks3-server[1001]: 2026/09/20 14:32:09 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6325lb # [ 130.550472] niks3-server[1001]: 2026/09/20 14:32:09 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6326lb # [ 130.554054] niks3-server[1001]: 2026/09/20 14:32:09 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6327lb # [ 130.557676] niks3-server[1001]: 2026/09/20 14:32:09 INFO Completed upload id=476328worker1 # [ 130.530206] nix-grpc-daemon[1739]: time=2026-09-20T14:32:09.414Z level=INFO msg="Upload complete. (207ms)"6329lb # [ 130.558972] niks3-server[1001]: 2026/09/20 14:32:09 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6330worker1 # [ 130.532554] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16331worker1 # [ 130.539204] nix-daemon[842]: reaped child process 1762, status = succeeded6332worker1 # [ 130.540687] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786333worker2 # [ 130.604963] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:09Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16334worker1 # [ 130.546561] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786335worker2 # [ 130.615079] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:09Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06336worker1 # [ 130.551802] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435786337worker1 # [ 130.556293] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6338worker1 # [ 130.682238] nix-daemon[842]: accepted connection from pid 1786, user nix-grpc-daemon (trusted)6339worker1 # [ 130.694676] nix-daemon[842]: reaped child process 1789, status = succeeded6340worker1 # [ 130.819760] nix-daemon[842]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)6341worker1 # [ 130.831375] nix-daemon[842]: reaped child process 1794, status = succeeded6342lb # [ 130.868710] niks3-server[1001]: 2026/09/20 14:32:10 INFO Received uploads request method=POST path=/api/pending_closures6343worker1 # [ 130.845390] nix-grpc-daemon[1739]: time=2026-09-20T14:32:09.729Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6344worker1 # [ 130.846860] nix-grpc-daemon[1739]: time=2026-09-20T14:32:09.731Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6345lb # [ 130.883474] niks3-server[1001]: 2026/09/20 14:32:10 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6346lb # [ 130.887797] niks3-server[1001]: 2026/09/20 14:32:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6347lb # [ 130.889609] niks3-server[1001]: 2026/09/20 14:32:10 INFO Signed narinfos id=48 count=16348worker1 # [ 130.861989] nix-grpc-daemon[1739]: time=2026-09-20T14:32:09.746Z level=INFO msg="Uploading 1 narinfos"6349lb # [ 130.893263] niks3-server[1001]: 2026/09/20 14:32:10 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6350lb # [ 130.897249] niks3-server[1001]: 2026/09/20 14:32:10 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6351lb # [ 130.900389] niks3-server[1001]: 2026/09/20 14:32:10 INFO Completed upload id=486352worker1 # [ 130.874031] nix-grpc-daemon[1739]: time=2026-09-20T14:32:09.758Z level=INFO msg="Upload complete. (313ms)"6353lb # [ 130.902516] niks3-server[1001]: 2026/09/20 14:32:10 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6354worker1 # [ 130.875876] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=0 paths=1 nar_bytes_in=5046355worker1 # [ 130.879464] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786356worker1 # [ 130.886398] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6357worker1 # [ 130.887652] nix-daemon[842]: reaped child process 1784, status = succeeded6358worker1 # [ 130.892730] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6359worker1 # [ 131.906552] nix-daemon[842]: reaped child process 1798, status = succeeded6360worker1 # [ 132.026431] nix-daemon[842]: accepted connection from pid 1813, user nix-grpc-daemon (trusted)6361worker1 # [ 132.038916] nix-daemon[842]: reaped child process 1816, status = succeeded6362lb # [ 132.077955] niks3-server[1001]: 2026/09/20 14:32:11 INFO Received uploads request method=POST path=/api/pending_closures6363worker1 # [ 132.056756] nix-grpc-daemon[1739]: time=2026-09-20T14:32:10.941Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6364worker1 # [ 132.058223] nix-grpc-daemon[1739]: time=2026-09-20T14:32:10.942Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6365lb # [ 132.117774] niks3-server[1001]: 2026/09/20 14:32:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6366lb # [ 132.122119] niks3-server[1001]: 2026/09/20 14:32:11 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6367lb # [ 132.129765] niks3-server[1001]: 2026/09/20 14:32:11 INFO Signed narinfos id=49 count=16368worker1 # [ 132.105683] nix-grpc-daemon[1739]: time=2026-09-20T14:32:10.990Z level=INFO msg="Uploading 1 narinfos"6369lb # [ 132.135524] niks3-server[1001]: 2026/09/20 14:32:11 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6370lb # [ 132.148670] niks3-server[1001]: 2026/09/20 14:32:11 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6371lb # [ 132.154797] niks3-server[1001]: 2026/09/20 14:32:11 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6372lb # [ 132.159886] niks3-server[1001]: 2026/09/20 14:32:11 INFO Completed upload id=496373lb # [ 132.161239] niks3-server[1001]: 2026/09/20 14:32:11 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6374worker1 # [ 132.132602] nix-grpc-daemon[1739]: time=2026-09-20T14:32:11.017Z level=INFO msg="Upload complete. (233ms)"6375worker1 # [ 132.139886] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16376worker2 # [ 132.211955] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:11Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16377worker1 # [ 132.148754] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786378worker2 # [ 132.219763] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:11Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06379worker1 # [ 132.158573] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786380worker1 # [ 132.164910] nix-daemon[842]: reaped child process 1796, status = succeeded6381worker1 # [ 132.169258] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435786382worker1 # [ 132.173061] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6383worker1 # [ 132.317215] nix-daemon[842]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6384worker1 # [ 132.329240] nix-daemon[842]: reaped child process 1823, status = succeeded6385worker1 # [ 132.474661] nix-daemon[842]: accepted connection from pid 1825, user nix-grpc-daemon (trusted)6386worker1 # [ 132.486557] nix-daemon[842]: reaped child process 1828, status = succeeded6387lb # [ 132.527602] niks3-server[1001]: 2026/09/20 14:32:11 INFO Received uploads request method=POST path=/api/pending_closures6388worker1 # [ 132.507596] nix-grpc-daemon[1739]: time=2026-09-20T14:32:11.391Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6389worker1 # [ 132.508942] nix-grpc-daemon[1739]: time=2026-09-20T14:32:11.391Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6390lb # [ 132.553661] niks3-server[1001]: 2026/09/20 14:32:11 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6391lb # [ 132.563290] niks3-server[1001]: 2026/09/20 14:32:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6392worker1 # [ 132.540808] nix-grpc-daemon[1739]: time=2026-09-20T14:32:11.425Z level=INFO msg="Uploading 1 narinfos"6393lb # [ 132.568238] niks3-server[1001]: 2026/09/20 14:32:11 INFO Signed narinfos id=50 count=16394lb # [ 132.575301] niks3-server[1001]: 2026/09/20 14:32:11 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6395lb # [ 132.586507] niks3-server[1001]: 2026/09/20 14:32:11 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6396worker1 # [ 132.564199] nix-grpc-daemon[1739]: time=2026-09-20T14:32:11.448Z level=INFO msg="Upload complete. (385ms)"6397lb # [ 132.591898] niks3-server[1001]: 2026/09/20 14:32:11 INFO Completed upload id=506398worker1 # [ 132.566135] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=0 paths=1 nar_bytes_in=5046399lb # [ 132.594984] niks3-server[1001]: 2026/09/20 14:32:11 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6400worker1 # [ 132.571394] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786401worker1 # [ 132.576114] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6402worker1 # [ 132.578363] nix-daemon[842]: reaped child process 1818, status = succeeded6403worker1 # [ 132.583545] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6404worker1 # [ 134.199548] nix-daemon[842]: reaped child process 1832, status = succeeded6405worker1 # [ 134.317895] nix-daemon[842]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)6406worker1 # [ 134.330155] nix-daemon[842]: reaped child process 1850, status = succeeded6407lb # [ 134.368184] niks3-server[1001]: 2026/09/20 14:32:13 INFO Received uploads request method=POST path=/api/pending_closures6408worker1 # [ 134.352279] nix-grpc-daemon[1739]: time=2026-09-20T14:32:13.235Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6409worker1 # [ 134.353671] nix-grpc-daemon[1739]: time=2026-09-20T14:32:13.235Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6410lb # [ 134.403663] niks3-server[1001]: 2026/09/20 14:32:13 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6411lb # [ 134.410294] niks3-server[1001]: 2026/09/20 14:32:13 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6412lb # [ 134.422112] niks3-server[1001]: 2026/09/20 14:32:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6413lb # [ 134.426858] niks3-server[1001]: 2026/09/20 14:32:13 INFO Signed narinfos id=51 count=16414worker1 # [ 134.402108] nix-grpc-daemon[1739]: time=2026-09-20T14:32:13.286Z level=INFO msg="Uploading 1 narinfos"6415lb # [ 134.432569] niks3-server[1001]: 2026/09/20 14:32:13 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6416lb # [ 134.442989] niks3-server[1001]: 2026/09/20 14:32:13 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6417lb # [ 134.445814] niks3-server[1001]: 2026/09/20 14:32:13 INFO Completed upload id=516418worker1 # [ 134.418411] nix-grpc-daemon[1739]: time=2026-09-20T14:32:13.302Z level=INFO msg="Upload complete. (226ms)"6419lb # [ 134.447531] niks3-server[1001]: 2026/09/20 14:32:13 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6420worker1 # [ 134.422285] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16421worker1 # [ 134.426041] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786422worker2 # [ 134.493348] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:13Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16423worker1 # [ 134.432315] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786424worker1 # [ 134.437164] nix-daemon[842]: reaped child process 1830, status = succeeded6425worker2 # [ 134.503311] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:13Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06426worker1 # [ 134.438690] nix-grpc-daemon[1739]: time=2026-09-20T14:32:13.322Z level=INFO msg="All 1 paths already cached"6427worker1 # [ 134.440544] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435786428worker1 # [ 134.445486] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6429worker1 # [ 134.563330] nix-daemon[842]: accepted connection from pid 1854, user nix-grpc-daemon (trusted)6430worker1 # [ 134.575240] nix-daemon[842]: reaped child process 1857, status = succeeded6431worker1 # [ 134.704465] nix-daemon[842]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)6432worker1 # [ 134.716350] nix-daemon[842]: reaped child process 1862, status = succeeded6433lb # [ 134.753333] niks3-server[1001]: 2026/09/20 14:32:13 INFO Received uploads request method=POST path=/api/pending_closures6434worker1 # [ 134.733395] nix-grpc-daemon[1739]: time=2026-09-20T14:32:13.617Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6435worker1 # [ 134.734875] nix-grpc-daemon[1739]: time=2026-09-20T14:32:13.619Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6436lb # [ 134.781213] niks3-server[1001]: 2026/09/20 14:32:13 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6437lb # [ 134.792913] niks3-server[1001]: 2026/09/20 14:32:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6438lb # [ 134.798645] niks3-server[1001]: 2026/09/20 14:32:13 INFO Signed narinfos id=52 count=16439worker1 # [ 134.774115] nix-grpc-daemon[1739]: time=2026-09-20T14:32:13.658Z level=INFO msg="Uploading 1 narinfos"6440lb # [ 134.802105] niks3-server[1001]: 2026/09/20 14:32:13 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6441lb # [ 134.814099] niks3-server[1001]: 2026/09/20 14:32:13 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6442worker1 # [ 134.792098] nix-grpc-daemon[1739]: time=2026-09-20T14:32:13.676Z level=INFO msg="Upload complete. (341ms)"6443lb # [ 134.819603] niks3-server[1001]: 2026/09/20 14:32:14 INFO Completed upload id=526444worker1 # [ 134.794155] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=0 paths=1 nar_bytes_in=6246445lb # [ 134.824529] niks3-server[1001]: 2026/09/20 14:32:14 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6446worker1 # [ 134.799251] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786447worker1 # [ 134.808064] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6448worker1 # [ 134.812111] nix-daemon[842]: reaped child process 1852, status = succeeded6449worker1 # [ 134.817100] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6450worker1 # [ 135.643454] nix-daemon[842]: reaped child process 1866, status = succeeded6451worker1 # [ 135.791195] nix-daemon[842]: accepted connection from pid 1881, user nix-grpc-daemon (trusted)6452worker1 # [ 135.803101] nix-daemon[842]: reaped child process 1884, status = succeeded6453lb # [ 135.844948] niks3-server[1001]: 2026/09/20 14:32:15 INFO Received uploads request method=POST path=/api/pending_closures6454worker1 # [ 135.826083] nix-grpc-daemon[1739]: time=2026-09-20T14:32:14.710Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6455worker1 # [ 135.828145] nix-grpc-daemon[1739]: time=2026-09-20T14:32:14.712Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6456lb # [ 135.886739] niks3-server[1001]: 2026/09/20 14:32:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6457lb # [ 135.892240] niks3-server[1001]: 2026/09/20 14:32:15 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6458lb # [ 135.895959] niks3-server[1001]: 2026/09/20 14:32:15 INFO Signed narinfos id=53 count=16459worker1 # [ 135.871225] nix-grpc-daemon[1739]: time=2026-09-20T14:32:14.755Z level=INFO msg="Uploading 1 narinfos"6460lb # [ 135.902710] niks3-server[1001]: 2026/09/20 14:32:15 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6461lb # [ 135.906570] niks3-server[1001]: 2026/09/20 14:32:15 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6462lb # [ 135.914344] niks3-server[1001]: 2026/09/20 14:32:15 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6463lb # [ 135.917565] niks3-server[1001]: 2026/09/20 14:32:15 INFO Completed upload id=536464lb # [ 135.919107] niks3-server[1001]: 2026/09/20 14:32:15 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6465worker1 # [ 135.890265] nix-grpc-daemon[1739]: time=2026-09-20T14:32:14.774Z level=INFO msg="Upload complete. (266ms)"6466worker2 # [ 135.964693] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:14Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16467worker1 # [ 135.896942] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16468worker2 # [ 135.970388] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:14Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06469worker1 # [ 135.902900] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786470worker1 # [ 135.913307] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786471worker1 # [ 135.915441] nix-daemon[842]: reaped child process 1864, status = succeeded6472worker1 # [ 135.919198] nix-grpc-daemon[1739]: time=2026-09-20T14:32:14.803Z level=INFO msg="All 1 paths already cached"6473worker1 # [ 135.922735] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435786474worker1 # [ 135.927037] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6475worker1 # [ 136.077093] nix-daemon[842]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6476worker1 # [ 136.087737] nix-daemon[842]: reaped child process 1891, status = succeeded6477worker1 # [ 136.236902] nix-daemon[842]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)6478worker1 # [ 136.248646] nix-daemon[842]: reaped child process 1896, status = succeeded6479lb # [ 136.289909] niks3-server[1001]: 2026/09/20 14:32:15 INFO Received uploads request method=POST path=/api/pending_closures6480worker1 # [ 136.270318] nix-grpc-daemon[1739]: time=2026-09-20T14:32:15.154Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6481worker1 # [ 136.272159] nix-grpc-daemon[1739]: time=2026-09-20T14:32:15.156Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6482lb # [ 136.319578] niks3-server[1001]: 2026/09/20 14:32:15 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6483lb # [ 136.330134] niks3-server[1001]: 2026/09/20 14:32:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6484lb # [ 136.339519] niks3-server[1001]: 2026/09/20 14:32:15 INFO Signed narinfos id=54 count=16485worker1 # [ 136.314401] nix-grpc-daemon[1739]: time=2026-09-20T14:32:15.198Z level=INFO msg="Uploading 1 narinfos"6486lb # [ 136.347430] niks3-server[1001]: 2026/09/20 14:32:15 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6487lb # [ 136.357242] niks3-server[1001]: 2026/09/20 14:32:15 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6488lb # [ 136.359625] niks3-server[1001]: 2026/09/20 14:32:15 INFO Completed upload id=546489lb # [ 136.361816] niks3-server[1001]: 2026/09/20 14:32:15 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6490worker1 # [ 136.333431] nix-grpc-daemon[1739]: time=2026-09-20T14:32:15.217Z level=INFO msg="Upload complete. (400ms)"6491worker1 # [ 136.336941] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=0 paths=1 nar_bytes_in=6246492worker1 # [ 136.342273] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786493worker1 # [ 136.349867] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6494worker1 # [ 136.357447] nix-daemon[842]: reaped child process 1886, status = succeeded6495worker1 # [ 136.362452] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6496worker1 # [ 137.789472] nix-daemon[842]: reaped child process 1900, status = succeeded6497worker1 # [ 137.939541] nix-daemon[842]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6498worker1 # [ 137.951459] nix-daemon[842]: reaped child process 1918, status = succeeded6499lb # [ 137.993667] niks3-server[1001]: 2026/09/20 14:32:17 INFO Received uploads request method=POST path=/api/pending_closures6500worker1 # [ 137.974412] nix-grpc-daemon[1739]: time=2026-09-20T14:32:16.858Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6501worker1 # [ 137.975870] nix-grpc-daemon[1739]: time=2026-09-20T14:32:16.860Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6502lb # [ 138.028631] niks3-server[1001]: 2026/09/20 14:32:17 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6503lb # [ 138.041727] niks3-server[1001]: 2026/09/20 14:32:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6504lb # [ 138.048348] niks3-server[1001]: 2026/09/20 14:32:17 INFO Signed narinfos id=55 count=16505worker1 # [ 138.023973] nix-grpc-daemon[1739]: time=2026-09-20T14:32:16.908Z level=INFO msg="Uploading 1 narinfos"6506lb # [ 138.053660] niks3-server[1001]: 2026/09/20 14:32:17 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6507lb # [ 138.060823] niks3-server[1001]: 2026/09/20 14:32:17 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6508lb # [ 138.068796] niks3-server[1001]: 2026/09/20 14:32:17 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6509lb # [ 138.073878] niks3-server[1001]: 2026/09/20 14:32:17 INFO Completed upload id=556510worker1 # [ 138.048697] nix-grpc-daemon[1739]: time=2026-09-20T14:32:16.933Z level=INFO msg="Upload complete. (278ms)"6511lb # [ 138.077852] niks3-server[1001]: 2026/09/20 14:32:17 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6512worker1 # [ 138.053882] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16513worker2 # [ 138.123990] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:17Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16514worker1 # [ 138.062564] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786515worker2 # [ 138.133935] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:17Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06516worker1 # [ 138.072578] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786517worker1 # [ 138.079889] nix-daemon[842]: reaped child process 1898, status = succeeded6518worker1 # [ 138.085995] nix-grpc-daemon[1739]: time=2026-09-20T14:32:16.970Z level=INFO msg="All 1 paths already cached"6519worker1 # [ 138.091685] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435786520worker1 # [ 138.098327] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6521worker1 # [ 138.247456] nix-daemon[842]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6522worker1 # [ 138.259556] nix-daemon[842]: reaped child process 1925, status = succeeded6523worker1 # [ 138.410704] nix-daemon[842]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6524worker1 # [ 138.421230] nix-daemon[842]: reaped child process 1930, status = succeeded6525lb # [ 138.461611] niks3-server[1001]: 2026/09/20 14:32:17 INFO Received uploads request method=POST path=/api/pending_closures6526worker1 # [ 138.442200] nix-grpc-daemon[1739]: time=2026-09-20T14:32:17.326Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6527worker1 # [ 138.443647] nix-grpc-daemon[1739]: time=2026-09-20T14:32:17.328Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6528lb # [ 138.488591] niks3-server[1001]: 2026/09/20 14:32:17 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6529lb # [ 138.501290] niks3-server[1001]: 2026/09/20 14:32:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6530lb # [ 138.507111] niks3-server[1001]: 2026/09/20 14:32:17 INFO Signed narinfos id=56 count=16531worker1 # [ 138.482862] nix-grpc-daemon[1739]: time=2026-09-20T14:32:17.367Z level=INFO msg="Uploading 1 narinfos"6532lb # [ 138.513596] niks3-server[1001]: 2026/09/20 14:32:17 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6533lb # [ 138.526806] niks3-server[1001]: 2026/09/20 14:32:17 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6534worker1 # [ 138.504138] nix-grpc-daemon[1739]: time=2026-09-20T14:32:17.388Z level=INFO msg="Upload complete. (399ms)"6535lb # [ 138.531869] niks3-server[1001]: 2026/09/20 14:32:17 INFO Completed upload id=566536worker1 # [ 138.506123] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=0 paths=1 nar_bytes_in=6246537lb # [ 138.535488] niks3-server[1001]: 2026/09/20 14:32:17 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6538worker1 # [ 138.512571] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786539worker1 # [ 138.520341] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6540worker1 # [ 138.524821] nix-daemon[842]: reaped child process 1920, status = succeeded6541worker1 # [ 138.530611] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6542worker1 # [ 139.559679] nix-daemon[842]: reaped child process 1934, status = succeeded6543worker1 # [ 139.689312] nix-daemon[842]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6544worker1 # [ 139.701520] nix-daemon[842]: reaped child process 1952, status = succeeded6545lb # [ 139.739152] niks3-server[1001]: 2026/09/20 14:32:18 INFO Received uploads request method=POST path=/api/pending_closures6546worker1 # [ 139.720733] nix-grpc-daemon[1739]: time=2026-09-20T14:32:18.605Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6547worker1 # [ 139.722190] nix-grpc-daemon[1739]: time=2026-09-20T14:32:18.606Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6548lb # [ 139.769945] niks3-server[1001]: 2026/09/20 14:32:18 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6549lb # [ 139.785510] niks3-server[1001]: 2026/09/20 14:32:18 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6550lb # [ 139.795918] niks3-server[1001]: 2026/09/20 14:32:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6551lb # [ 139.801600] niks3-server[1001]: 2026/09/20 14:32:18 INFO Signed narinfos id=57 count=16552worker1 # [ 139.777319] nix-grpc-daemon[1739]: time=2026-09-20T14:32:18.661Z level=INFO msg="Uploading 1 narinfos"6553lb # [ 139.804684] niks3-server[1001]: 2026/09/20 14:32:18 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6554lb # [ 139.815884] niks3-server[1001]: 2026/09/20 14:32:18 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6555lb # [ 139.818839] niks3-server[1001]: 2026/09/20 14:32:18 INFO Completed upload id=576556worker1 # [ 139.791601] nix-grpc-daemon[1739]: time=2026-09-20T14:32:18.676Z level=INFO msg="Upload complete. (253ms)"6557lb # [ 139.820833] niks3-server[1001]: 2026/09/20 14:32:19 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6558worker1 # [ 139.798439] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16559worker2 # [ 139.867934] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:18Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16560worker2 # [ 139.877733] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:18Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06561worker1 # [ 139.811390] nix-daemon[842]: reaped child process 1932, status = succeeded6562worker1 # [ 139.842285] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786563worker1 # [ 139.853937] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786564worker1 # [ 139.866363] nix-grpc-daemon[1739]: time=2026-09-20T14:32:18.750Z level=INFO msg="All 2 paths already cached"6565worker1 # [ 139.873128] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435786566worker1 # [ 139.881247] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6567worker1 # [ 140.022303] nix-daemon[842]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)6568worker1 # [ 140.034516] nix-daemon[842]: reaped child process 1959, status = succeeded6569worker1 # [ 140.185542] nix-daemon[842]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)6570worker1 # [ 140.197396] nix-daemon[842]: reaped child process 1964, status = succeeded6571lb # [ 140.239070] niks3-server[1001]: 2026/09/20 14:32:19 INFO Received uploads request method=POST path=/api/pending_closures6572worker1 # [ 140.220175] nix-grpc-daemon[1739]: time=2026-09-20T14:32:19.104Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6573worker1 # [ 140.221572] nix-grpc-daemon[1739]: time=2026-09-20T14:32:19.104Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6574lb # [ 140.263961] niks3-server[1001]: 2026/09/20 14:32:19 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6575lb # [ 140.276215] niks3-server[1001]: 2026/09/20 14:32:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6576worker1 # [ 140.254963] nix-grpc-daemon[1739]: time=2026-09-20T14:32:19.139Z level=INFO msg="Uploading 1 narinfos"6577lb # [ 140.282343] niks3-server[1001]: 2026/09/20 14:32:19 INFO Signed narinfos id=58 count=16578lb # [ 140.286977] niks3-server[1001]: 2026/09/20 14:32:19 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6579lb # [ 140.300060] niks3-server[1001]: 2026/09/20 14:32:19 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6580worker1 # [ 140.277226] nix-grpc-daemon[1739]: time=2026-09-20T14:32:19.161Z level=INFO msg="Upload complete. (383ms)"6581lb # [ 140.304987] niks3-server[1001]: 2026/09/20 14:32:19 INFO Completed upload id=586582worker1 # [ 140.279673] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=0 paths=1 nar_bytes_in=7526583lb # [ 140.308837] niks3-server[1001]: 2026/09/20 14:32:19 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6584worker1 # [ 140.285909] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786585worker1 # [ 140.296188] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6586worker1 # [ 140.298494] nix-daemon[842]: reaped child process 1954, status = succeeded6587worker1 # [ 140.303833] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6588worker1 # [ 141.117118] nix-daemon[842]: reaped child process 1968, status = succeeded6589worker1 # [ 141.230343] nix-daemon[842]: accepted connection from pid 1983, user nix-grpc-daemon (trusted)6590worker1 # [ 141.241605] nix-daemon[842]: reaped child process 1986, status = succeeded6591lb # [ 141.278352] niks3-server[1001]: 2026/09/20 14:32:20 INFO Received uploads request method=POST path=/api/pending_closures6592worker1 # [ 141.258839] nix-grpc-daemon[1739]: time=2026-09-20T14:32:20.143Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6593worker1 # [ 141.260281] nix-grpc-daemon[1739]: time=2026-09-20T14:32:20.144Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6594lb # [ 141.316611] niks3-server[1001]: 2026/09/20 14:32:20 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6595lb # [ 141.326206] niks3-server[1001]: 2026/09/20 14:32:20 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6596lb # [ 141.335702] niks3-server[1001]: 2026/09/20 14:32:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6597lb # [ 141.341916] niks3-server[1001]: 2026/09/20 14:32:20 INFO Signed narinfos id=59 count=16598worker1 # [ 141.316698] nix-grpc-daemon[1739]: time=2026-09-20T14:32:20.201Z level=INFO msg="Uploading 1 narinfos"6599lb # [ 141.344476] niks3-server[1001]: 2026/09/20 14:32:20 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6600lb # [ 141.354810] niks3-server[1001]: 2026/09/20 14:32:20 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6601lb # [ 141.359214] niks3-server[1001]: 2026/09/20 14:32:20 INFO Completed upload id=596602worker1 # [ 141.331791] nix-grpc-daemon[1739]: time=2026-09-20T14:32:20.216Z level=INFO msg="Upload complete. (221ms)"6603lb # [ 141.363080] niks3-server[1001]: 2026/09/20 14:32:20 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6604worker1 # [ 141.337398] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=16605worker2 # [ 141.406837] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:20Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16606worker1 # [ 141.342827] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786607worker1 # [ 141.353530] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786608worker1 # [ 141.355566] nix-daemon[842]: reaped child process 1966, status = succeeded6609worker1 # [ 141.361976] nix-grpc-daemon[1739]: time=2026-09-20T14:32:20.246Z level=INFO msg="All 2 paths already cached"6610worker1 # [ 141.366337] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435786611worker1 # [ 141.375657] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6612worker1 # [ 141.521696] nix-daemon[842]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)6613worker1 # [ 141.533961] nix-daemon[842]: reaped child process 1993, status = succeeded6614worker1 # [ 141.678109] nix-daemon[842]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)6615worker1 # [ 141.689984] nix-daemon[842]: reaped child process 1998, status = succeeded6616lb # [ 141.726875] niks3-server[1001]: 2026/09/20 14:32:20 INFO Received uploads request method=POST path=/api/pending_closures6617worker1 # [ 141.703385] nix-grpc-daemon[1739]: time=2026-09-20T14:32:20.587Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6618worker1 # [ 141.704889] nix-grpc-daemon[1739]: time=2026-09-20T14:32:20.589Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6619lb # [ 141.753821] niks3-server[1001]: 2026/09/20 14:32:20 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6620lb # [ 141.762390] niks3-server[1001]: 2026/09/20 14:32:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6621lb # [ 141.766488] niks3-server[1001]: 2026/09/20 14:32:20 INFO Signed narinfos id=60 count=16622worker1 # [ 141.741959] nix-grpc-daemon[1739]: time=2026-09-20T14:32:20.626Z level=INFO msg="Uploading 1 narinfos"6623lb # [ 141.772577] niks3-server[1001]: 2026/09/20 14:32:20 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6624lb # [ 141.783110] niks3-server[1001]: 2026/09/20 14:32:20 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6625lb # [ 141.787964] niks3-server[1001]: 2026/09/20 14:32:20 INFO Completed upload id=606626worker1 # [ 141.763177] nix-grpc-daemon[1739]: time=2026-09-20T14:32:20.647Z level=INFO msg="Upload complete. (382ms)"6627lb # [ 141.792836] niks3-server[1001]: 2026/09/20 14:32:20 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6628worker1 # [ 141.766716] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=0 paths=1 nar_bytes_in=7526629worker1 # [ 141.771818] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786630worker1 # [ 141.780491] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6631worker1 # [ 141.784137] nix-daemon[842]: reaped child process 1988, status = succeeded6632worker1 # [ 141.788787] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6633worker1 # [ 143.000814] nix-daemon[842]: reaped child process 2002, status = succeeded6634worker1 # [ 143.135539] nix-daemon[842]: accepted connection from pid 2017, user nix-grpc-daemon (trusted)6635worker1 # [ 143.147785] nix-daemon[842]: reaped child process 2020, status = succeeded6636lb # [ 143.188797] niks3-server[1001]: 2026/09/20 14:32:22 INFO Received uploads request method=POST path=/api/pending_closures6637worker1 # [ 143.169141] nix-grpc-daemon[1739]: time=2026-09-20T14:32:22.053Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6638worker1 # [ 143.170631] nix-grpc-daemon[1739]: time=2026-09-20T14:32:22.055Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6639lb # [ 143.229367] niks3-server[1001]: 2026/09/20 14:32:22 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6640lb # [ 143.234734] niks3-server[1001]: 2026/09/20 14:32:22 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6641lb # [ 143.240503] niks3-server[1001]: 2026/09/20 14:32:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6642lb # [ 143.245695] niks3-server[1001]: 2026/09/20 14:32:22 INFO Signed narinfos id=61 count=16643worker1 # [ 143.221298] nix-grpc-daemon[1739]: time=2026-09-20T14:32:22.105Z level=INFO msg="Uploading 1 narinfos"6644lb # [ 143.251151] niks3-server[1001]: 2026/09/20 14:32:22 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6645lb # [ 143.260339] niks3-server[1001]: 2026/09/20 14:32:22 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6646lb # [ 143.265434] niks3-server[1001]: 2026/09/20 14:32:22 INFO Completed upload id=616647worker1 # [ 143.237651] nix-grpc-daemon[1739]: time=2026-09-20T14:32:22.122Z level=INFO msg="Upload complete. (240ms)"6648lb # [ 143.270417] niks3-server[1001]: 2026/09/20 14:32:22 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6649worker1 # [ 143.243272] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=16650worker2 # [ 143.312941] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:22Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16651worker1 # [ 143.255864] nix-daemon[842]: reaped child process 2000, status = succeeded6652worker1 # [ 143.293566] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786653worker1 # [ 143.306672] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786654worker1 # [ 143.319123] nix-grpc-daemon[1739]: time=2026-09-20T14:32:22.203Z level=INFO msg="All 2 paths already cached"6655worker1 # [ 143.326159] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:435786656worker1 # [ 143.336211] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6657worker1 # [ 143.496680] nix-daemon[842]: accepted connection from pid 2024, user nix-grpc-daemon (trusted)6658worker1 # [ 143.509401] nix-daemon[842]: reaped child process 2027, status = succeeded6659worker1 # [ 143.657542] nix-daemon[842]: accepted connection from pid 2029, user nix-grpc-daemon (trusted)6660worker1 # [ 143.669444] nix-daemon[842]: reaped child process 2032, status = succeeded6661lb # [ 143.710896] niks3-server[1001]: 2026/09/20 14:32:22 INFO Received uploads request method=POST path=/api/pending_closures6662worker1 # [ 143.691470] nix-grpc-daemon[1739]: time=2026-09-20T14:32:22.576Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6663worker1 # [ 143.692990] nix-grpc-daemon[1739]: time=2026-09-20T14:32:22.577Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6664lb # [ 143.740145] niks3-server[1001]: 2026/09/20 14:32:22 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6665lb # [ 143.750531] niks3-server[1001]: 2026/09/20 14:32:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6666lb # [ 143.756379] niks3-server[1001]: 2026/09/20 14:32:22 INFO Signed narinfos id=62 count=16667worker1 # [ 143.733202] nix-grpc-daemon[1739]: time=2026-09-20T14:32:22.617Z level=INFO msg="Uploading 1 narinfos"6668lb # [ 143.762454] niks3-server[1001]: 2026/09/20 14:32:22 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6669lb # [ 143.773274] niks3-server[1001]: 2026/09/20 14:32:22 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6670worker1 # [ 143.750991] nix-grpc-daemon[1739]: time=2026-09-20T14:32:22.635Z level=INFO msg="Upload complete. (403ms)"6671lb # [ 143.779045] niks3-server[1001]: 2026/09/20 14:32:22 INFO Completed upload id=626672worker1 # [ 143.752968] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=0 paths=1 nar_bytes_in=7526673lb # [ 143.784719] niks3-server[1001]: 2026/09/20 14:32:22 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6674worker1 # [ 143.757644] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:435786675worker1 # [ 143.767096] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6676worker1 # [ 143.769567] nix-daemon[842]: reaped child process 2022, status = succeeded6677worker1 # [ 143.774543] nix-daemon[842]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6678worker1 # [ 144.800673] nix-daemon[842]: reaped child process 2036, status = succeeded6679worker1 # [ 144.951899] nix-daemon[842]: accepted connection from pid 2051, user nix-grpc-daemon (trusted)6680worker1 # [ 144.963287] nix-daemon[842]: reaped child process 2054, status = succeeded6681lb # [ 145.003338] niks3-server[1001]: 2026/09/20 14:32:24 INFO Received uploads request method=POST path=/api/pending_closures6682worker1 # [ 144.985278] nix-grpc-daemon[1739]: time=2026-09-20T14:32:23.869Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6683worker1 # [ 144.986759] nix-grpc-daemon[1739]: time=2026-09-20T14:32:23.871Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6684lb # [ 145.037360] niks3-server[1001]: 2026/09/20 14:32:24 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6685lb # [ 145.051749] niks3-server[1001]: 2026/09/20 14:32:24 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6686lb # [ 145.057108] niks3-server[1001]: 2026/09/20 14:32:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6687worker1 # [ 145.034086] nix-grpc-daemon[1739]: time=2026-09-20T14:32:23.918Z level=INFO msg="Uploading 1 narinfos"6688lb # [ 145.061476] niks3-server[1001]: 2026/09/20 14:32:24 INFO Signed narinfos id=63 count=16689lb # [ 145.072831] niks3-server[1001]: 2026/09/20 14:32:24 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6690lb # [ 145.075505] niks3-server[1001]: 2026/09/20 14:32:24 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6691lb # [ 145.080744] niks3-server[1001]: 2026/09/20 14:32:24 INFO Completed upload id=636692worker1 # [ 145.053370] nix-grpc-daemon[1739]: time=2026-09-20T14:32:23.937Z level=INFO msg="Upload complete. (271ms)"6693lb # [ 145.083819] niks3-server[1001]: 2026/09/20 14:32:24 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6694worker1 # [ 145.057142] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43578 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=16695worker1 # [ 145.064350] nix-daemon[842]: reaped child process 2034, status = succeeded6696client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.45 seconds)6697worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6698worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6699client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'6700client # [ 145.256972] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6701client # Running as unit: gpu.service; invocation ID: 38d46cceeac24c27bcb29ecd96c6b8246702client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.07 seconds)6703lb: must succeed: curl -sf localhost:9901/clusters6704lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6705lb: must succeed: curl -sf localhost:9901/clusters6706lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6707worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6708worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6709client # [ 145.479952] nix[1350]: don't know how to build these paths:6710client # [ 145.481227] nix[1350]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv6711worker2 # [ 145.499261] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:591386712worker1 # [ 145.432909] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:435786713client # [ 145.503467] nix[1350]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker6714worker2 # [ 145.523148] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:24Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06715lb: must succeed: curl -sf localhost:9901/clusters6716lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6717lb: must succeed: curl -sf localhost:9901/clusters6718lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6719worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6720worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6721client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6722lb # [ 147.320916] envoy[902]: conn peer=[2001:db8:1::1]:34564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27025 tx=10562 ms=171946723client # [ 175.504727] nix[1350]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}6724client # [ 175.514237] nix[1350]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed6725client # [ 175.553125] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6726client # [ 175.555935] systemd[1]: gpu.service: Failed with result 'exit-code'.6727client # [ 175.560480] systemd[1]: gpu.service: Consumed 138ms CPU time over 30.302s wall clock time, 37.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6728client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.65 seconds)6729client: must succeed: journalctl -u gpu -o cat6730client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)6731lb: must succeed: curl -sf localhost:9901/clusters6732lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6733lb: must succeed: curl -sf localhost:9901/clusters6734lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6735worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6736worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6737(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.79 seconds)6738subtest: low disk drains a worker and builds go to the other6739worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6740lb # [ 177.323264] envoy[902]: conn peer=[2001:db8:1::1]:49798 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300476741worker1 # [ 177.450125] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:56Z level=info event=unhealthy reason=min_free6742worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)6743worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6744worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.04 seconds)6745client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26746worker1 # [ 177.723932] nix-grpc-daemon[1683]: ts=2026-09-20T14:32:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:435786747client # don't know how to build these paths:6748client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6749worker2 # [ 177.803948] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591386750worker2 # [ 177.829233] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:56Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06751worker2 # [ 177.833297] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:56Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16752worker2 # [ 177.872418] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386753worker2 # [ 177.884944] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591386754client # copying 1 paths to 'lb:50051'...6755client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6756worker2 # [ 177.897464] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591386757worker2 # [ 177.909263] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6758worker2 # [ 178.067983] nix-daemon[870]: accepted connection from pid 1677, user nix-grpc-daemon (trusted)6759worker2 # [ 178.084267] nix-daemon[870]: reaped child process 1680, status = succeeded6760worker2 # [ 178.220581] nix-daemon[870]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6761worker2 # [ 178.235622] nix-daemon[870]: reaped child process 1685, status = succeeded6762lb # [ 178.204769] niks3-server[1001]: 2026/09/20 14:32:57 INFO Received uploads request method=POST path=/api/pending_closures6763worker2 # [ 178.256152] nix-grpc-daemon[1466]: time=2026-09-20T14:32:57.205Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6764worker2 # [ 178.258175] nix-grpc-daemon[1466]: time=2026-09-20T14:32:57.207Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6765lb # [ 178.238553] niks3-server[1001]: 2026/09/20 14:32:57 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6766lb # [ 178.249631] niks3-server[1001]: 2026/09/20 14:32:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6767worker2 # [ 178.297482] nix-grpc-daemon[1466]: time=2026-09-20T14:32:57.246Z level=INFO msg="Uploading 1 narinfos"6768lb # [ 178.255654] niks3-server[1001]: 2026/09/20 14:32:57 INFO Signed narinfos id=64 count=16769lb # [ 178.267326] niks3-server[1001]: 2026/09/20 14:32:57 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6770lb # [ 178.268964] niks3-server[1001]: 2026/09/20 14:32:57 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6771lb # [ 178.275192] niks3-server[1001]: 2026/09/20 14:32:57 INFO Completed upload id=646772worker2 # [ 178.318385] nix-grpc-daemon[1466]: time=2026-09-20T14:32:57.267Z level=INFO msg="Upload complete. (395ms)"6773lb # [ 178.277218] niks3-server[1001]: 2026/09/20 14:32:57 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6774worker2 # [ 178.322057] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=0 paths=1 nar_bytes_in=5126775worker2 # [ 178.324645] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386776worker2 # [ 178.332947] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6777worker2 # [ 178.339422] nix-daemon[870]: reaped child process 1675, status = succeeded6778client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6779worker2 # [ 178.344992] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6780client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6781client # farm-a> LOG-farm-a-drain6782worker2 # [ 179.562148] nix-daemon[870]: reaped child process 1689, status = succeeded6783worker2 # [ 179.688861] nix-daemon[870]: accepted connection from pid 1704, user nix-grpc-daemon (trusted)6784worker2 # [ 179.702793] nix-daemon[870]: reaped child process 1707, status = succeeded6785lb # [ 179.673276] niks3-server[1001]: 2026/09/20 14:32:58 INFO Received uploads request method=POST path=/api/pending_closures6786worker2 # [ 179.722649] nix-grpc-daemon[1466]: time=2026-09-20T14:32:58.672Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6787worker2 # [ 179.724207] nix-grpc-daemon[1466]: time=2026-09-20T14:32:58.673Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6788lb # [ 179.706095] niks3-server[1001]: 2026/09/20 14:32:58 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6789lb # [ 179.709371] niks3-server[1001]: 2026/09/20 14:32:58 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6790lb # [ 179.712393] niks3-server[1001]: 2026/09/20 14:32:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6791lb # [ 179.714944] niks3-server[1001]: 2026/09/20 14:32:58 INFO Signed narinfos id=65 count=16792worker2 # [ 179.757779] nix-grpc-daemon[1466]: time=2026-09-20T14:32:58.707Z level=INFO msg="Uploading 1 narinfos"6793lb # [ 179.718124] niks3-server[1001]: 2026/09/20 14:32:58 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6794lb # [ 179.722504] niks3-server[1001]: 2026/09/20 14:32:58 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6795lb # [ 179.725397] niks3-server[1001]: 2026/09/20 14:32:58 INFO Completed upload id=656796worker2 # [ 179.768048] nix-grpc-daemon[1466]: time=2026-09-20T14:32:58.717Z level=INFO msg="Upload complete. (216ms)"6797lb # [ 179.727156] niks3-server[1001]: 2026/09/20 14:32:58 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6798worker2 # [ 179.771567] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=16799worker2 # [ 179.777564] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:58Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06800worker2 # [ 179.779747] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:58Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16801worker2 # [ 179.785569] nix-daemon[870]: reaped child process 1687, status = succeeded6802worker2 # [ 179.821828] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386803worker2 # [ 179.830929] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591386804worker2 # [ 179.839923] nix-grpc-daemon[1466]: time=2026-09-20T14:32:58.789Z level=INFO msg="All 1 paths already cached"6805client # copying 1 paths to 'lb:50051'...6806client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6807worker2 # [ 179.846524] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591386808worker2 # [ 179.855136] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6809worker2 # [ 180.008229] nix-daemon[870]: accepted connection from pid 1713, user nix-grpc-daemon (trusted)6810worker2 # [ 180.022936] nix-daemon[870]: reaped child process 1716, status = succeeded6811worker2 # [ 180.177787] nix-daemon[870]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)6812worker2 # [ 180.191987] nix-daemon[870]: reaped child process 1721, status = succeeded6813lb # [ 180.163645] niks3-server[1001]: 2026/09/20 14:32:59 INFO Received uploads request method=POST path=/api/pending_closures6814worker2 # [ 180.214461] nix-grpc-daemon[1466]: time=2026-09-20T14:32:59.164Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6815worker2 # [ 180.216205] nix-grpc-daemon[1466]: time=2026-09-20T14:32:59.165Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6816lb # [ 180.196930] niks3-server[1001]: 2026/09/20 14:32:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6817lb # [ 180.202715] niks3-server[1001]: 2026/09/20 14:32:59 INFO Signed narinfos id=66 count=16818worker2 # [ 180.247117] nix-grpc-daemon[1466]: time=2026-09-20T14:32:59.196Z level=INFO msg="Uploading 1 narinfos"6819lb # [ 180.210068] niks3-server[1001]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6820lb # [ 180.217892] niks3-server[1001]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6821lb # [ 180.226453] niks3-server[1001]: 2026/09/20 14:32:59 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6822lb # [ 180.232754] niks3-server[1001]: 2026/09/20 14:32:59 INFO Completed upload id=666823worker2 # [ 180.274364] nix-grpc-daemon[1466]: time=2026-09-20T14:32:59.223Z level=INFO msg="Upload complete. (410ms)"6824lb # [ 180.237356] niks3-server[1001]: 2026/09/20 14:32:59 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6825worker2 # [ 180.278264] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=0 paths=1 nar_bytes_in=6246826worker2 # [ 180.283474] nix-grpc-daemon[1390]: ts=2026-09-20T14:32:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386827worker2 # [ 180.293150] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6828worker2 # [ 180.301164] nix-daemon[870]: reaped child process 1711, status = succeeded6829worker2 # [ 180.304451] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6830client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6831client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6832client # farm-b> LOG-farm-b-drain6833worker2 # [ 181.320660] nix-daemon[870]: reaped child process 1725, status = succeeded6834worker2 # [ 181.462377] nix-daemon[870]: accepted connection from pid 1740, user nix-grpc-daemon (trusted)6835worker2 # [ 181.477092] nix-daemon[870]: reaped child process 1743, status = succeeded6836lb # [ 181.449578] niks3-server[1001]: 2026/09/20 14:33:00 INFO Received uploads request method=POST path=/api/pending_closures6837worker2 # [ 181.499685] nix-grpc-daemon[1466]: time=2026-09-20T14:33:00.449Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6838worker2 # [ 181.501346] nix-grpc-daemon[1466]: time=2026-09-20T14:33:00.451Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6839lb # [ 181.485211] niks3-server[1001]: 2026/09/20 14:33:00 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6840lb # [ 181.497757] niks3-server[1001]: 2026/09/20 14:33:00 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6841lb # [ 181.505572] niks3-server[1001]: 2026/09/20 14:33:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6842worker2 # [ 181.552217] nix-grpc-daemon[1466]: time=2026-09-20T14:33:00.501Z level=INFO msg="Uploading 1 narinfos"6843lb # [ 181.510627] niks3-server[1001]: 2026/09/20 14:33:00 INFO Signed narinfos id=67 count=16844lb # [ 181.513966] niks3-server[1001]: 2026/09/20 14:33:00 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6845lb # [ 181.524098] niks3-server[1001]: 2026/09/20 14:33:00 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6846lb # [ 181.530066] niks3-server[1001]: 2026/09/20 14:33:00 INFO Completed upload id=676847lb # [ 181.531585] niks3-server[1001]: 2026/09/20 14:33:00 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6848worker2 # [ 181.571554] nix-grpc-daemon[1466]: time=2026-09-20T14:33:00.521Z level=INFO msg="Upload complete. (258ms)"6849worker2 # [ 181.576350] nix-grpc-daemon[1390]: ts=2026-09-20T14:33:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=16850worker2 # [ 181.583214] nix-grpc-daemon[1390]: ts=2026-09-20T14:33:00Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06851worker2 # [ 181.585887] nix-grpc-daemon[1390]: ts=2026-09-20T14:33:00Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16852worker2 # [ 181.587791] nix-grpc-daemon[1390]: ts=2026-09-20T14:33:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386853worker2 # [ 181.593296] nix-daemon[870]: reaped child process 1723, status = succeeded6854worker2 # [ 181.597397] nix-grpc-daemon[1390]: ts=2026-09-20T14:33:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591386855worker2 # [ 181.604732] nix-grpc-daemon[1466]: time=2026-09-20T14:33:00.554Z level=INFO msg="All 2 paths already cached"6856client # copying 1 paths to 'lb:50051'...6857worker2 # [ 181.610393] nix-grpc-daemon[1390]: ts=2026-09-20T14:33:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591386858client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6859worker2 # [ 181.615849] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6860worker2 # [ 181.767451] nix-daemon[870]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6861worker2 # [ 181.783563] nix-daemon[870]: reaped child process 1752, status = succeeded6862worker2 # [ 181.934323] nix-daemon[870]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)6863worker2 # [ 181.948996] nix-daemon[870]: reaped child process 1757, status = succeeded6864lb # [ 181.920802] niks3-server[1001]: 2026/09/20 14:33:01 INFO Received uploads request method=POST path=/api/pending_closures6865worker2 # [ 181.970527] nix-grpc-daemon[1466]: time=2026-09-20T14:33:00.919Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6866worker2 # [ 181.971989] nix-grpc-daemon[1466]: time=2026-09-20T14:33:00.919Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6867lb # [ 181.946580] niks3-server[1001]: 2026/09/20 14:33:01 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6868lb # [ 181.956636] niks3-server[1001]: 2026/09/20 14:33:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6869lb # [ 181.960791] niks3-server[1001]: 2026/09/20 14:33:01 INFO Signed narinfos id=68 count=16870worker2 # [ 182.007002] nix-grpc-daemon[1466]: time=2026-09-20T14:33:00.956Z level=INFO msg="Uploading 1 narinfos"6871lb # [ 181.969515] niks3-server[1001]: 2026/09/20 14:33:01 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6872lb # [ 181.980555] niks3-server[1001]: 2026/09/20 14:33:01 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6873lb # [ 181.986533] niks3-server[1001]: 2026/09/20 14:33:01 INFO Completed upload id=686874worker2 # [ 182.030237] nix-grpc-daemon[1466]: time=2026-09-20T14:33:00.979Z level=INFO msg="Upload complete. (407ms)"6875lb # [ 181.991946] niks3-server[1001]: 2026/09/20 14:33:01 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6876worker2 # [ 182.032665] nix-grpc-daemon[1390]: ts=2026-09-20T14:33:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=0 paths=1 nar_bytes_in=7526877worker2 # [ 182.035726] nix-grpc-daemon[1390]: ts=2026-09-20T14:33:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591386878worker2 # [ 182.045831] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6879worker2 # [ 182.047799] nix-daemon[870]: reaped child process 1747, status = succeeded6880worker2 # [ 182.053349] nix-daemon[870]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6881client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6882client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6883client # farm-top> LOG-farm-top-drain6884worker2 # [ 183.081850] nix-daemon[870]: reaped child process 1761, status = succeeded6885worker2 # [ 183.219276] nix-daemon[870]: accepted connection from pid 1776, user nix-grpc-daemon (trusted)6886worker2 # [ 183.232849] nix-daemon[870]: reaped child process 1779, status = succeeded6887lb # [ 183.204594] niks3-server[1001]: 2026/09/20 14:33:02 INFO Received uploads request method=POST path=/api/pending_closures6888worker2 # [ 183.254466] nix-grpc-daemon[1466]: time=2026-09-20T14:33:02.203Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6889worker2 # [ 183.257429] nix-grpc-daemon[1466]: time=2026-09-20T14:33:02.205Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6890lb # [ 183.238439] niks3-server[1001]: 2026/09/20 14:33:02 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6891lb # [ 183.251246] niks3-server[1001]: 2026/09/20 14:33:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6892lb # [ 183.255312] niks3-server[1001]: 2026/09/20 14:33:02 INFO Signed narinfos id=69 count=16893worker2 # [ 183.299452] nix-grpc-daemon[1466]: time=2026-09-20T14:33:02.248Z level=INFO msg="Uploading 1 narinfos"6894lb # [ 183.262737] niks3-server[1001]: 2026/09/20 14:33:02 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6895lb # [ 183.269505] niks3-server[1001]: 2026/09/20 14:33:02 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6896lb # [ 183.277684] niks3-server[1001]: 2026/09/20 14:33:02 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6897lb # [ 183.283311] niks3-server[1001]: 2026/09/20 14:33:02 INFO Completed upload id=696898lb # [ 183.284858] niks3-server[1001]: 2026/09/20 14:33:02 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6899worker2 # [ 183.326327] nix-grpc-daemon[1466]: time=2026-09-20T14:33:02.275Z level=INFO msg="Upload complete. (265ms)"6900worker2 # [ 183.334570] nix-grpc-daemon[1390]: ts=2026-09-20T14:33:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59138 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=16901worker2 # [ 183.347219] nix-daemon[870]: reaped child process 1759, status = succeeded6902client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.80 seconds)6903client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6904client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.21 seconds)6905worker1: must succeed: rm /nix/.rw-store/fill6906worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)6907worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6908worker1 # [ 184.452228] nix-grpc-daemon[1683]: ts=2026-09-20T14:33:03Z level=info event=healthy reason=min_free6909worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)6910(finished: subtest: low disk drains a worker and builds go to the other, in 8.35 seconds)6911(finished: run the VM test script, in 185.75 seconds)6912test script finished in 185.95s6913cleanup6914kill QemuMachine (pid 49)6915client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6916kill QemuMachine (pid 51)6917lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6918kill QemuMachine (pid 50)6919worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6920kill QemuMachine (pid 52)6921worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6922(finished: cleanup, in 0.99 seconds)6923additionally exposed symbols:6924 client, lb, worker1, worker2,6925 vlan1,6926 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh