vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #492
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1: QEMU running (pid 51)13worker2 # Disk image does not exist, creating the virtualisation disk image...14worker1 # Disk image does not exist, creating the virtualisation disk image...15worker2 # Formatting '/build/vm-state-worker2/tmp.rtV8ihFIpA', fmt=raw size=107374182416client # Disk image does not exist, creating the virtualisation disk image...17lb: QEMU running (pid 50)18worker2: QEMU running (pid 52)19worker1 # Formatting '/build/vm-state-worker1/tmp.fnBxMZaO4x', fmt=raw size=107374182420client # Formatting '/build/vm-state-client/tmp.fn5PFK4hMW', fmt=raw size=107374182421worker2 # mke2fs 1.47.4 (6-Mar-2025)22lb # Disk image does not exist, creating the virtualisation disk image...23client: QEMU running (pid 49)24worker1 # mke2fs 1.47.4 (6-Mar-2025)25client # mke2fs 1.47.4 (6-Mar-2025)26worker1 # Discarding device blocks: 0/262144 done27lb # Formatting '/build/vm-state-lb/tmp.046rrZl6ST', fmt=raw size=107374182428client # Discarding device blocks: 0/262144 done29worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes30worker1 # Filesystem UUID: b728a6a6-870e-4ca3-9302-5100bf1d1e3431worker1 # Superblock backups stored on blocks:32client # Creating filesystem with 262144 4k blocks and 65536 inodes33worker2 # Discarding device blocks: 0/262144 done34(finished: start all VMs, in 0.18 seconds)35lb # mke2fs 1.47.4 (6-Mar-2025)36worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes37client: waiting for unit network-addresses-eth1.service38lb # Discarding device blocks: 0/262144 done39worker2 # Filesystem UUID: cf42d618-ac73-42e0-9c68-9105aa78023240client: waiting for the VM to finish booting41lb # Creating filesystem with 262144 4k blocks and 65536 inodes42worker2 # Superblock backups stored on blocks:43lb # Filesystem UUID: 9b80635b-9b68-4eea-b35c-10f25ea019f144worker2 # 32768, 98304, 163840, 22937645client # Filesystem UUID: 4714f011-4d7e-4ce8-a38d-3fb3339dcec746worker1 # 32768, 98304, 163840, 22937647lb # Superblock backups stored on blocks:48worker2 # 49client # Superblock backups stored on blocks:50worker1 # 51lb # 32768, 98304, 163840, 22937652worker2 # Allocating group tables: 0/8 done53client # 32768, 98304, 163840, 22937654worker1 # Allocating group tables: 0/8 done55lb # 56worker2 # Writing inode tables: 0/8 done57client # 58worker1 # Writing inode tables: 0/8 done59lb # Allocating group tables: 0/8 done60worker2 # Creating journal (8192 blocks): done61client # Allocating group tables: 0/8 done62worker1 # Creating journal (8192 blocks): done63lb # Writing inode tables: 0/8 done64worker2 # Writing superblocks and filesystem accounting information: 0/8 done65client # Writing inode tables: 0/8 done66worker1 # Writing superblocks and filesystem accounting information: 0/8 done67lb # Creating journal (8192 blocks): done68worker2 # 69client # Creating journal (8192 blocks): done70worker1 # 71lb # Writing superblocks and filesystem accounting information: 0/8 done72worker2 # Virtualisation disk image created.73client # Writing superblocks and filesystem accounting information: 0/8 done74worker1 # Virtualisation disk image created.75client # 76client # Virtualisation disk image created.77lb # 78lb # Virtualisation disk image created.79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...87worker2 # 88worker2 # 89worker2 # 90worker2 # 91worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0092lb # 93lb # 94lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0095client # 96client # 97worker1 # 98worker1 # 99worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...100worker2 # 101worker2 # 102client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00103lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...104lb # 105lb # 106lb # 107lb # 108worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00109lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00110client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...111client # 112client # 113client # 114client # 115worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...116worker1 # 117worker1 # 118worker1 # 119worker1 # 120client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00121worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00122lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...123lb # 124lb # 125client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...126client # 127client # 128worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker1 # 130worker1 # 131worker2 # Booting from ROM...132client # Booting from ROM...133lb # Booting from ROM...134worker1 # Booting from ROM...135worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3lnqm3j8dnpdr5a15fghvr8w83n98qvj-nixos-system-worker2-test/init regInfo=/nix/store/lblpg9rp3wa28rg32d5cpxhb7fpqc5v1-closure-info/registration console=ttyS0,115200n8 console=tty0137worker2 # [ 0.000000] BIOS-provided physical RAM map:138worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved147worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved148worker2 # [ 0.000000] NX (Execute Disable) protection: active149worker2 # [ 0.000000] APIC: Static calls initialized150worker2 # [ 0.000000] SMBIOS 2.8 present.151worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014152client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026153worker2 # [ 0.000000] DMI: Memory slots populated: 1/1154worker2 # [ 0.000000] Hypervisor detected: KVM155worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000156worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00157worker2 # [ 0.000001] kvm-clock: using sched offset of 466927036 cycles158client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wnk38lrfrycmn4za5rgns10spvrp4vd9-nixos-system-client-test/init regInfo=/nix/store/q628mnwhcv6wlxd97gj9pi4dydd3xdvg-closure-info/registration console=ttyS0,115200n8 console=tty0159worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns160client # [ 0.000000] BIOS-provided physical RAM map:161worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor162client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable163worker2 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000164client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved165worker2 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs166client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved167worker2 # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT168client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable169worker2 # [ 0.002746] found SMP MP-table at [mem 0x000f5450-0x000f545f]170worker2 # [ 0.002757] Using GB pages for direct mapping171client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved172worker2 # [ 0.002826] RAMDISK: [mem 0x5e367000-0x5ffcffff]173client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved174worker2 # [ 0.002835] ACPI: Early table checksum verification disabled175client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved176worker2 # [ 0.002838] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )177client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved178worker2 # [ 0.002842] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)179client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved180worker2 # [ 0.002846] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)181client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved182lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026183client # [ 0.000000] NX (Execute Disable) protection: active184worker2 # [ 0.002852] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)185client # [ 0.000000] APIC: Static calls initialized186worker2 # [ 0.002855] ACPI: FACS 0x000000005FFE0000 000040187client # [ 0.000000] SMBIOS 2.8 present.188worker2 # [ 0.002856] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)189client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014190client # [ 0.000000] DMI: Memory slots populated: 1/1191worker2 # [ 0.002858] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)192client # [ 0.000000] Hypervisor detected: KVM193lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/j1nn9bbhrn5xz154akfhw453f8i07i4d-nixos-system-lb-test/init regInfo=/nix/store/hq8v7nws787a9kwd3h009z1n9dqal4fl-closure-info/registration console=ttyS0,115200n8 console=tty0194worker2 # [ 0.002859] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)195client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000196lb # [ 0.000000] BIOS-provided physical RAM map:197client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00198lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable199worker2 # [ 0.002861] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)200client # [ 0.000000] kvm-clock: using sched offset of 480694846 cycles201lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved202worker2 # [ 0.002862] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]203client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns204lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved205worker2 # [ 0.002863] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]206client # [ 0.000004] tsc: Detected 2400.012 MHz processor207lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable208worker2 # [ 0.002864] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]209client # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000210lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved211worker2 # [ 0.002864] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]212client # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs213lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved214worker2 # [ 0.002865] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]215client # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT216worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026217lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved218worker2 # [ 0.002865] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]219client # [ 0.002745] found SMP MP-table at [mem 0x000f5450-0x000f545f]220worker2 # [ 0.002865] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]221client # [ 0.002756] Using GB pages for direct mapping222lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved223client # [ 0.002868] RAMDISK: [mem 0x5e366000-0x5ffcffff]224worker2 # [ 0.003097] No NUMA configuration found225lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved226worker2 # [ 0.003098] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]227client # [ 0.002876] ACPI: Early table checksum verification disabled228lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved229worker2 # [ 0.003100] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]230client # [ 0.002878] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )231worker2 # [ 0.005304] Zone ranges:232lb # [ 0.000000] NX (Execute Disable) protection: active233worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n017fyfr0aiz5grv6h3p0iai3qmjfgm3-nixos-system-worker1-test/init regInfo=/nix/store/qccrij3danxi8vdb92x8j7h5510rbg58-closure-info/registration console=ttyS0,115200n8 console=tty0234lb # [ 0.000000] APIC: Static calls initialized235worker2 # [ 0.005305] DMA [mem 0x0000000000001000-0x0000000000ffffff]236lb # [ 0.000000] SMBIOS 2.8 present.237worker1 # [ 0.000000] BIOS-provided physical RAM map:238client # [ 0.002882] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)239worker2 # [ 0.005307] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]240client # [ 0.002886] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)241worker2 # [ 0.005308] Normal empty242worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable243lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014244worker2 # [ 0.005308] Device empty245worker2 # [ 0.005309] Movable zone start for each node246worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved247client # [ 0.002893] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)248lb # [ 0.000000] DMI: Memory slots populated: 1/1249worker2 # [ 0.005310] Early memory node ranges250lb # [ 0.000000] Hypervisor detected: KVM251client # [ 0.002895] ACPI: FACS 0x000000005FFE0000 000040252worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved253lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000254worker2 # [ 0.005310] node 0: [mem 0x0000000000001000-0x000000000009efff]255client # [ 0.002896] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)256worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable257lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00258worker2 # [ 0.005311] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]259worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved260lb # [ 0.000000] kvm-clock: using sched offset of 499007115 cycles261client # [ 0.002898] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)262worker2 # [ 0.005312] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]263worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved264worker2 # [ 0.005332] On node 0, zone DMA: 1 pages in unavailable ranges265client # [ 0.002899] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)266lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns267worker2 # [ 0.005589] On node 0, zone DMA: 97 pages in unavailable ranges268worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved269lb # [ 0.000005] tsc: Detected 2400.012 MHz processor270client # [ 0.002901] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)271worker2 # [ 0.031420] On node 0, zone DMA32: 40 pages in unavailable ranges272lb # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000273worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved274worker2 # [ 0.031869] ACPI: PM-Timer IO Port: 0x608275client # [ 0.002902] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]276worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved277lb # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs278worker2 # [ 0.031879] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])279client # [ 0.002903] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]280worker2 # [ 0.031904] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23281lb # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT282worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved283client # [ 0.002904] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]284worker1 # [ 0.000000] NX (Execute Disable) protection: active285lb # [ 0.002734] found SMP MP-table at [mem 0x000f5450-0x000f545f]286worker2 # [ 0.031907] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)287worker1 # [ 0.000000] APIC: Static calls initialized288client # [ 0.002904] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]289lb # [ 0.002745] Using GB pages for direct mapping290worker2 # [ 0.031908] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)291worker1 # [ 0.000000] SMBIOS 2.8 present.292lb # [ 0.002827] RAMDISK: [mem 0x5e367000-0x5ffcffff]293client # [ 0.002905] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]294worker2 # [ 0.031909] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)295lb # [ 0.002837] ACPI: Early table checksum verification disabled296worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014297client # [ 0.002905] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]298worker2 # [ 0.031910] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)299lb # [ 0.002840] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )300worker1 # [ 0.000000] DMI: Memory slots populated: 1/1301client # [ 0.002906] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]302worker1 # [ 0.000000] Hypervisor detected: KVM303worker2 # [ 0.031911] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)304client # [ 0.003127] No NUMA configuration found305lb # [ 0.002844] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)306worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000307worker2 # [ 0.031913] ACPI: Using ACPI (MADT) for SMP configuration information308client # [ 0.003127] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]309worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00310lb # [ 0.002848] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)311worker2 # [ 0.031914] ACPI: HPET id: 0x8086a201 base: 0xfed00000312client # [ 0.003130] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]313worker1 # [ 0.000000] kvm-clock: using sched offset of 497239224 cycles314worker2 # [ 0.031918] TSC deadline timer available315client # [ 0.005329] Zone ranges:316lb # [ 0.002856] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)317worker2 # [ 0.031922] CPU topo: Max. logical packages: 1318client # [ 0.005330] DMA [mem 0x0000000000001000-0x0000000000ffffff]319lb # [ 0.002858] ACPI: FACS 0x000000005FFE0000 000040320worker2 # [ 0.031923] CPU topo: Max. logical dies: 1321worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns322worker2 # [ 0.031923] CPU topo: Max. dies per package: 1323client # [ 0.005332] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]324worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor325client # [ 0.005332] Normal empty326lb # [ 0.002859] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)327worker2 # [ 0.031927] CPU topo: Max. threads per core: 1328client # [ 0.005333] Device empty329worker1 # [ 0.000818] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000330worker2 # [ 0.031927] CPU topo: Num. cores per package: 1331client # [ 0.005334] Movable zone start for each node332lb # [ 0.002861] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)333worker2 # [ 0.031928] CPU topo: Num. threads per package: 1334client # [ 0.005334] Early memory node ranges335worker1 # [ 0.000845] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs336worker2 # [ 0.031928] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs337client # [ 0.005335] node 0: [mem 0x0000000000001000-0x000000000009efff]338lb # [ 0.002862] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)339worker1 # [ 0.000848] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT340client # [ 0.005336] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]341worker2 # [ 0.031944] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()342lb # [ 0.002864] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)343worker1 # [ 0.002743] found SMP MP-table at [mem 0x000f5450-0x000f545f]344client # [ 0.005337] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]345worker1 # [ 0.002755] Using GB pages for direct mapping346worker2 # [ 0.031975] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]347lb # [ 0.002865] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]348worker1 # [ 0.002851] RAMDISK: [mem 0x5e367000-0x5ffcffff]349client # [ 0.005355] On node 0, zone DMA: 1 pages in unavailable ranges350worker2 # [ 0.031977] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]351lb # [ 0.002866] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]352worker1 # [ 0.002860] ACPI: Early table checksum verification disabled353client # [ 0.005611] On node 0, zone DMA: 97 pages in unavailable ranges354worker2 # [ 0.031978] [mem 0x60000000-0xafffffff] available for PCI devices355worker1 # [ 0.002863] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )356lb # [ 0.002867] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]357client # [ 0.031378] On node 0, zone DMA32: 40 pages in unavailable ranges358worker2 # [ 0.031979] Booting paravirtualized kernel on KVM359client # [ 0.031829] ACPI: PM-Timer IO Port: 0x608360lb # [ 0.002867] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]361worker1 # [ 0.002867] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)362client # [ 0.031840] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])363worker2 # [ 0.031982] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns364lb # [ 0.002868] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]365worker1 # [ 0.002871] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)366client # [ 0.031866] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23367worker2 # [ 0.036435] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1368lb # [ 0.002868] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]369client # [ 0.031868] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)370worker1 # [ 0.002878] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)371worker2 # [ 0.038722] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152372lb # [ 0.002869] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]373client # [ 0.031870] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)374worker1 # [ 0.002880] ACPI: FACS 0x000000005FFE0000 000040375worker2 # [ 0.038766] kvm-guest: PV spinlocks disabled, single CPU376lb # [ 0.003106] No NUMA configuration found377client # [ 0.031871] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)378lb # [ 0.003107] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]379worker1 # [ 0.002881] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)380client # [ 0.031872] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)381lb # [ 0.003109] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]382worker1 # [ 0.002883] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)383lb # [ 0.005832] Zone ranges:384client # [ 0.031872] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)385lb # [ 0.005833] DMA [mem 0x0000000000001000-0x0000000000ffffff]386client # [ 0.031875] ACPI: Using ACPI (MADT) for SMP configuration information387worker1 # [ 0.002885] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)388lb # [ 0.005835] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]389client # [ 0.031876] ACPI: HPET id: 0x8086a201 base: 0xfed00000390worker2 # [ 0.038767] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3lnqm3j8dnpdr5a15fghvr8w83n98qvj-nixos-system-worker2-test/init regInfo=/nix/store/lblpg9rp3wa28rg32d5cpxhb7fpqc5v1-closure-info/registration console=ttyS0,115200n8 console=tty0391lb # [ 0.005836] Normal empty392client # [ 0.031880] TSC deadline timer available393worker1 # [ 0.002886] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)394lb # [ 0.005837] Device empty395client # [ 0.031885] CPU topo: Max. logical packages: 1396lb # [ 0.005837] Movable zone start for each node397worker1 # [ 0.002887] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]398client # [ 0.031885] CPU topo: Max. logical dies: 1399lb # [ 0.005838] Early memory node ranges400worker2 # [ 0.038858] Unknown kernel command line parameters "regInfo=/nix/store/lblpg9rp3wa28rg32d5cpxhb7fpqc5v1-closure-info/registration", will be passed to user space.401client # [ 0.031886] CPU topo: Max. dies per package: 1402worker2 # [ 0.038871] random: crng init done403worker1 # [ 0.002888] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]404lb # [ 0.005838] node 0: [mem 0x0000000000001000-0x000000000009efff]405client # [ 0.031889] CPU topo: Max. threads per core: 1406worker2 # [ 0.038871] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes407lb # [ 0.005839] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]408worker1 # [ 0.002889] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]409client # [ 0.031890] CPU topo: Num. cores per package: 1410client # [ 0.031890] CPU topo: Num. threads per package: 1411lb # [ 0.005840] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]412worker2 # [ 0.040949] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)413worker1 # [ 0.002889] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]414client # [ 0.031890] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs415lb # [ 0.005860] On node 0, zone DMA: 1 pages in unavailable ranges416worker1 # [ 0.002890] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]417worker2 # [ 0.041995] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)418lb # [ 0.006141] On node 0, zone DMA: 97 pages in unavailable ranges419worker2 # [ 0.042025] Fallback order for Node 0: 0420client # [ 0.031907] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()421worker1 # [ 0.002890] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]422lb # [ 0.034246] On node 0, zone DMA32: 40 pages in unavailable ranges423worker2 # [ 0.042028] Built 1 zonelists, mobility grouping on. Total pages: 393078424client # [ 0.031936] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]425worker2 # [ 0.042029] Policy zone: DMA32426worker1 # [ 0.002891] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]427lb # [ 0.034713] ACPI: PM-Timer IO Port: 0x608428worker1 # [ 0.003147] No NUMA configuration found429client # [ 0.031937] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]430lb # [ 0.034724] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])431worker2 # [ 0.044466] mem auto-init: stack:all(zero), heap alloc:on, heap free:off432worker1 # [ 0.003148] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]433client # [ 0.031939] [mem 0x60000000-0xafffffff] available for PCI devices434worker2 # [ 0.047418] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1435lb # [ 0.034753] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23436client # [ 0.031940] Booting paravirtualized kernel on KVM437worker1 # [ 0.003150] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]438worker2 # [ 0.050871] allocated 3145728 bytes of page_ext439lb # [ 0.034756] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)440worker1 # [ 0.005926] Zone ranges:441worker2 # [ 0.060467] ftrace: allocating 48731 entries in 192 pages442lb # [ 0.034757] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)443worker1 # [ 0.005927] DMA [mem 0x0000000000001000-0x0000000000ffffff]444worker2 # [ 0.060469] ftrace: allocated 192 pages with 2 groups445client # [ 0.031943] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns446worker2 # [ 0.061299] Dynamic Preempt: lazy447worker1 # [ 0.005928] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]448lb # [ 0.034758] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)449client # [ 0.036386] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1450worker1 # [ 0.005929] Normal empty451worker2 # [ 0.061476] rcu: Preemptible hierarchical RCU implementation.452worker1 # [ 0.005930] Device empty453lb # [ 0.034760] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)454worker2 # [ 0.061476] rcu: RCU event tracing is enabled.455client # [ 0.038490] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152456worker1 # [ 0.005930] Movable zone start for each node457worker1 # [ 0.005931] Early memory node ranges458client # [ 0.038533] kvm-guest: PV spinlocks disabled, single CPU459worker2 # [ 0.061477] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.460lb # [ 0.034760] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)461worker2 # [ 0.061478] Trampoline variant of Tasks RCU enabled.462worker1 # [ 0.005932] node 0: [mem 0x0000000000001000-0x000000000009efff]463lb # [ 0.034763] ACPI: Using ACPI (MADT) for SMP configuration information464worker2 # [ 0.061479] Rude variant of Tasks RCU enabled.465lb # [ 0.034764] ACPI: HPET id: 0x8086a201 base: 0xfed00000466worker1 # [ 0.005933] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]467lb # [ 0.034767] TSC deadline timer available468worker2 # [ 0.061479] Tracing variant of Tasks RCU enabled.469worker1 # [ 0.005934] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]470lb # [ 0.034771] CPU topo: Max. logical packages: 1471worker2 # [ 0.061480] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.472lb # [ 0.034772] CPU topo: Max. logical dies: 1473worker1 # [ 0.005955] On node 0, zone DMA: 1 pages in unavailable ranges474worker2 # [ 0.061480] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1475lb # [ 0.034772] CPU topo: Max. dies per package: 1476client # [ 0.038535] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wnk38lrfrycmn4za5rgns10spvrp4vd9-nixos-system-client-test/init regInfo=/nix/store/q628mnwhcv6wlxd97gj9pi4dydd3xdvg-closure-info/registration console=ttyS0,115200n8 console=tty0477worker1 # [ 0.006241] On node 0, zone DMA: 97 pages in unavailable ranges478lb # [ 0.034775] CPU topo: Max. threads per core: 1479worker2 # [ 0.061501] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.480worker1 # [ 0.034553] On node 0, zone DMA32: 40 pages in unavailable ranges481lb # [ 0.034776] CPU topo: Num. cores per package: 1482worker1 # [ 0.035054] ACPI: PM-Timer IO Port: 0x608483lb # [ 0.034776] CPU topo: Num. threads per package: 1484client # [ 0.038624] Unknown kernel command line parameters "regInfo=/nix/store/q628mnwhcv6wlxd97gj9pi4dydd3xdvg-closure-info/registration", will be passed to user space.485worker2 # [ 0.061503] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.486worker1 # [ 0.035066] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])487client # [ 0.038637] random: crng init done488lb # [ 0.034777] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs489worker1 # [ 0.035097] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23490worker2 # [ 0.061504] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.491client # [ 0.038638] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes492lb # [ 0.034796] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()493worker2 # [ 0.065816] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16494worker1 # [ 0.035099] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)495client # [ 0.040703] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)496lb # [ 0.034828] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]497worker2 # [ 0.066076] rcu: srcu_init: Setting srcu_struct sizes based on contention.498worker1 # [ 0.035101] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)499client # [ 0.042170] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)500lb # [ 0.034829] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]501client # [ 0.042200] Fallback order for Node 0: 0502worker1 # [ 0.035102] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)503worker2 # [ 0.066083] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns504lb # [ 0.034831] [mem 0x60000000-0xafffffff] available for PCI devices505client # [ 0.042202] Built 1 zonelists, mobility grouping on. Total pages: 393078506worker1 # [ 0.035103] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)507lb # [ 0.034832] Booting paravirtualized kernel on KVM508client # [ 0.042203] Policy zone: DMA32509worker2 # [ 0.066188] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)510worker1 # [ 0.035104] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)511worker2 # [ 0.069639] Console: colour VGA+ 80x25512client # [ 0.045075] mem auto-init: stack:all(zero), heap alloc:on, heap free:off513lb # [ 0.034836] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns514worker1 # [ 0.035106] ACPI: Using ACPI (MADT) for SMP configuration information515worker2 # [ 0.069642] printk: legacy console [tty0] enabled516client # [ 0.048085] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1517worker2 # [ 0.098812] printk: legacy console [ttyS0] enabled518worker1 # [ 0.035107] ACPI: HPET id: 0x8086a201 base: 0xfed00000519lb # [ 0.039302] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1520client # [ 0.051452] allocated 3145728 bytes of page_ext521worker1 # [ 0.035110] TSC deadline timer available522worker2 # [ 0.196203] ACPI: Core revision 20250807523client # [ 0.060861] ftrace: allocating 48731 entries in 192 pages524lb # [ 0.041578] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152525worker1 # [ 0.035114] CPU topo: Max. logical packages: 1526client # [ 0.060863] ftrace: allocated 192 pages with 2 groups527lb # [ 0.041620] kvm-guest: PV spinlocks disabled, single CPU528worker1 # [ 0.035115] CPU topo: Max. logical dies: 1529client # [ 0.061698] Dynamic Preempt: lazy530worker1 # [ 0.035116] CPU topo: Max. dies per package: 1531worker2 # [ 0.197060] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns532client # [ 0.061833] rcu: Preemptible hierarchical RCU implementation.533worker1 # [ 0.035119] CPU topo: Max. threads per core: 1534client # [ 0.061834] rcu: RCU event tracing is enabled.535worker2 # [ 0.198557] APIC: Switch to symmetric I/O mode setup536worker1 # [ 0.035120] CPU topo: Num. cores per package: 1537client # [ 0.061834] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.538worker2 # [ 0.199473] x2apic enabled539worker1 # [ 0.035120] CPU topo: Num. threads per package: 1540client # [ 0.061836] Trampoline variant of Tasks RCU enabled.541worker1 # [ 0.035120] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs542worker2 # [ 0.200193] APIC: Switched APIC routing to: physical x2apic543client # [ 0.061836] Rude variant of Tasks RCU enabled.544lb # [ 0.041621] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/j1nn9bbhrn5xz154akfhw453f8i07i4d-nixos-system-lb-test/init regInfo=/nix/store/hq8v7nws787a9kwd3h009z1n9dqal4fl-closure-info/registration console=ttyS0,115200n8 console=tty0545worker1 # [ 0.035141] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()546client # [ 0.061836] Tracing variant of Tasks RCU enabled.547client # [ 0.061837] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.548worker1 # [ 0.035178] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]549worker2 # [ 0.201943] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1550lb # [ 0.041717] Unknown kernel command line parameters "regInfo=/nix/store/hq8v7nws787a9kwd3h009z1n9dqal4fl-closure-info/registration", will be passed to user space.551client # [ 0.061838] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1552lb # [ 0.041730] random: crng init done553worker1 # [ 0.035180] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]554worker1 # [ 0.035181] [mem 0x60000000-0xafffffff] available for PCI devices555lb # [ 0.041731] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes556client # [ 0.061891] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.557worker2 # [ 0.202886] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns558worker1 # [ 0.035183] Booting paravirtualized kernel on KVM559lb # [ 0.044000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)560client # [ 0.061893] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.561worker2 # [ 0.204474] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)562worker1 # [ 0.035186] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns563lb # [ 0.045123] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)564client # [ 0.061894] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.565lb # [ 0.045157] Fallback order for Node 0: 0566worker2 # [ 0.206556] x86/cpu: User Mode Instruction Prevention (UMIP) activated567worker1 # [ 0.039662] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1568client # [ 0.066406] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16569lb # [ 0.045160] Built 1 zonelists, mobility grouping on. Total pages: 393078570worker1 # [ 0.041930] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152571lb # [ 0.045161] Policy zone: DMA32572worker2 # [ 0.207607] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127573client # [ 0.066692] rcu: srcu_init: Setting srcu_struct sizes based on contention.574worker1 # [ 0.041977] kvm-guest: PV spinlocks disabled, single CPU575lb # [ 0.048093] mem auto-init: stack:all(zero), heap alloc:on, heap free:off576worker2 # [ 0.208471] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0577client # [ 0.066699] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns578lb # [ 0.051439] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1579lb # [ 0.054928] allocated 3145728 bytes of page_ext580client # [ 0.066800] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)581lb # [ 0.064991] ftrace: allocating 48731 entries in 192 pages582worker2 # [ 0.209475] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto583client # [ 0.070335] Console: colour VGA+ 80x25584client # [ 0.070337] printk: legacy console [tty0] enabled585lb # [ 0.064992] ftrace: allocated 192 pages with 2 groups586lb # [ 0.065865] Dynamic Preempt: lazy587client # [ 0.100032] printk: legacy console [ttyS0] enabled588worker2 # [ 0.210471] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl589worker1 # [ 0.041979] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n017fyfr0aiz5grv6h3p0iai3qmjfgm3-nixos-system-worker1-test/init regInfo=/nix/store/qccrij3danxi8vdb92x8j7h5510rbg58-closure-info/registration console=ttyS0,115200n8 console=tty0590lb # [ 0.066000] rcu: Preemptible hierarchical RCU implementation.591client # [ 0.196766] ACPI: Core revision 20250807592lb # [ 0.066001] rcu: RCU event tracing is enabled.593worker2 # [ 0.212471] Transient Scheduler Attacks: Vulnerable: No microcode594lb # [ 0.066001] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.595worker2 # [ 0.213471] Spectre V2 : Mitigation: Enhanced / Automatic IBRS596worker1 # [ 0.042083] Unknown kernel command line parameters "regInfo=/nix/store/qccrij3danxi8vdb92x8j7h5510rbg58-closure-info/registration", will be passed to user space.597lb # [ 0.066003] Trampoline variant of Tasks RCU enabled.598client # [ 0.197625] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns599worker1 # [ 0.042095] random: crng init done600lb # [ 0.066004] Rude variant of Tasks RCU enabled.601worker2 # [ 0.214471] Speculative Return Stack Overflow: Mitigation: Safe RET602client # [ 0.199105] APIC: Switch to symmetric I/O mode setup603worker1 # [ 0.042096] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes604lb # [ 0.066004] Tracing variant of Tasks RCU enabled.605client # [ 0.200034] x2apic enabled606worker1 # [ 0.044338] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)607lb # [ 0.066005] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.608client # [ 0.200766] APIC: Switched APIC routing to: physical x2apic609lb # [ 0.066006] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1610worker1 # [ 0.045468] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)611worker1 # [ 0.045499] Fallback order for Node 0: 0612lb # [ 0.066068] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.613worker1 # [ 0.045502] Built 1 zonelists, mobility grouping on. Total pages: 393078614worker2 # [ 0.215471] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization615client # [ 0.202548] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1616worker1 # [ 0.045504] Policy zone: DMA32617lb # [ 0.066070] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.618worker2 # [ 0.217480] Spectre V2 : Enabling IBPB for BPF619worker1 # [ 0.048438] mem auto-init: stack:all(zero), heap alloc:on, heap free:off620client # [ 0.203484] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns621lb # [ 0.066071] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.622worker1 # [ 0.051745] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1623worker2 # [ 0.218983] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier624worker1 # [ 0.055291] allocated 3145728 bytes of page_ext625lb # [ 0.070483] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16626client # [ 0.205043] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)627worker2 # [ 0.219471] active return thunk: srso_alias_return_thunk628worker1 # [ 0.065251] ftrace: allocating 48731 entries in 192 pages629lb # [ 0.070779] rcu: srcu_init: Setting srcu_struct sizes based on contention.630worker1 # [ 0.065254] ftrace: allocated 192 pages with 2 groups631client # [ 0.207126] x86/cpu: User Mode Instruction Prevention (UMIP) activated632worker1 # [ 0.066120] Dynamic Preempt: lazy633worker2 # [ 0.221099] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'634lb # [ 0.070786] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns635worker1 # [ 0.066261] rcu: Preemptible hierarchical RCU implementation.636worker2 # [ 0.222329] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'637client # [ 0.208206] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127638worker1 # [ 0.066262] rcu: RCU event tracing is enabled.639lb # [ 0.070897] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)640worker2 # [ 0.223183] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'641client # [ 0.209041] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0642worker1 # [ 0.066263] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.643lb # [ 0.074379] Console: colour VGA+ 80x25644lb # [ 0.074383] printk: legacy console [tty0] enabled645worker1 # [ 0.066264] Trampoline variant of Tasks RCU enabled.646worker2 # [ 0.224189] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'647lb # [ 0.104468] printk: legacy console [ttyS0] enabled648worker1 # [ 0.066265] Rude variant of Tasks RCU enabled.649client # [ 0.210044] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto650worker1 # [ 0.066265] Tracing variant of Tasks RCU enabled.651worker2 # [ 0.225206] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'652lb # [ 0.201607] ACPI: Core revision 20250807653worker1 # [ 0.066266] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.654client # [ 0.211041] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl655worker2 # [ 0.225471] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'656worker1 # [ 0.066266] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1657lb # [ 0.202500] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns658client # [ 0.213041] Transient Scheduler Attacks: Vulnerable: No microcode659worker2 # [ 0.226471] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'660lb # [ 0.204035] APIC: Switch to symmetric I/O mode setup661worker1 # [ 0.066289] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.662client # [ 0.214040] Spectre V2 : Mitigation: Enhanced / Automatic IBRS663worker2 # [ 0.228360] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256664lb # [ 0.204979] x2apic enabled665client # [ 0.215040] Speculative Return Stack Overflow: Mitigation: Safe RET666worker1 # [ 0.066290] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.667worker2 # [ 0.229184] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64668lb # [ 0.205771] APIC: Switched APIC routing to: physical x2apic669worker1 # [ 0.066291] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.670worker2 # [ 0.230177] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512671worker1 # [ 0.070708] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16672worker2 # [ 0.231208] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024673lb # [ 0.207610] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1674worker1 # [ 0.070974] rcu: srcu_init: Setting srcu_struct sizes based on contention.675worker2 # [ 0.232204] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8676client # [ 0.215973] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization677worker1 # [ 0.070983] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns678lb # [ 0.208658] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns679client # [ 0.217048] Spectre V2 : Enabling IBPB for BPF680worker2 # [ 0.232471] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.681worker1 # [ 0.071100] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)682lb # [ 0.210307] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)683worker1 # [ 0.074798] Console: colour VGA+ 80x25684client # [ 0.218552] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier685worker1 # [ 0.074801] printk: legacy console [tty0] enabled686client # [ 0.219041] active return thunk: srso_alias_return_thunk687lb # [ 0.211398] x86/cpu: User Mode Instruction Prevention (UMIP) activated688worker1 # [ 0.104989] printk: legacy console [ttyS0] enabled689worker1 # [ 0.203925] ACPI: Core revision 20250807690lb # [ 0.212431] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127691client # [ 0.220674] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'692client # [ 0.221040] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'693lb # [ 0.213998] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0694worker1 # [ 0.204900] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns695client # [ 0.222751] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'696worker1 # [ 0.206483] APIC: Switch to symmetric I/O mode setup697lb # [ 0.215072] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto698client # [ 0.223768] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'699worker1 # [ 0.207493] x2apic enabled700client # [ 0.224775] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'701lb # [ 0.216304] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl702worker1 # [ 0.208289] APIC: Switched APIC routing to: physical x2apic703client # [ 0.225040] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'704lb # [ 0.217305] Transient Scheduler Attacks: Vulnerable: No microcode705lb # [ 0.219063] Spectre V2 : Mitigation: Enhanced / Automatic IBRS706client # [ 0.226040] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'707worker1 # [ 0.210207] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1708client # [ 0.227931] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256709lb # [ 0.220016] Speculative Return Stack Overflow: Mitigation: Safe RET710client # [ 0.228738] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64711worker1 # [ 0.211220] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns712client # [ 0.229747] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512713worker1 # [ 0.212985] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)714client # [ 0.230040] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024715client # [ 0.231040] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8716worker1 # [ 0.216040] x86/cpu: User Mode Instruction Prevention (UMIP) activated717lb # [ 0.221080] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization718lb # [ 0.222312] Spectre V2 : Enabling IBPB for BPF719worker1 # [ 0.217122] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127720client # [ 0.232040] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.721worker1 # [ 0.217983] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0722lb # [ 0.223305] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier723lb # [ 0.224305] active return thunk: srso_alias_return_thunk724worker1 # [ 0.218986] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto725lb # [ 0.225999] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'726lb # [ 0.227304] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'727worker1 # [ 0.219983] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl728lb # [ 0.228304] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'729worker1 # [ 0.220981] Transient Scheduler Attacks: Vulnerable: No microcode730lb # [ 0.229304] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'731worker1 # [ 0.221982] Spectre V2 : Mitigation: Enhanced / Automatic IBRS732lb # [ 0.230305] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'733worker1 # [ 0.222982] Speculative Return Stack Overflow: Mitigation: Safe RET734lb # [ 0.231304] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'735lb # [ 0.232304] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'736lb # [ 0.233305] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256737worker1 # [ 0.223982] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization738lb # [ 0.234304] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64739worker1 # [ 0.225991] Spectre V2 : Enabling IBPB for BPF740lb # [ 0.235304] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512741worker1 # [ 0.227546] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier742lb # [ 0.236304] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024743worker1 # [ 0.228983] active return thunk: srso_alias_return_thunk744lb # [ 0.237304] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8745worker1 # [ 0.230004] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'746lb # [ 0.238304] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.747worker1 # [ 0.230981] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748worker1 # [ 0.231982] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749worker2 # [ 0.267333] Freeing SMP alternatives memory: 44K750worker2 # [ 0.268043] pid_max: default: 32768 minimum: 301751worker1 # [ 0.232982] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'752worker2 # [ 0.268557] LSM: initializing lsm=capability,landlock,yama,bpf,ima753worker1 # [ 0.233982] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'754worker2 # [ 0.269584] landlock: Up and running.755worker1 # [ 0.234982] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'756worker2 # [ 0.270472] Yama: becoming mindful.757worker2 # [ 0.271598] LSM support for eBPF active758worker1 # [ 0.235982] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'759worker1 # [ 0.237983] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256760worker2 # [ 0.272360] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)761worker1 # [ 0.238982] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64762worker2 # [ 0.273513] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)763worker1 # [ 0.239952] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512764worker1 # [ 0.240738] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024765worker2 # [ 0.275913] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)766worker1 # [ 0.241983] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8767worker2 # [ 0.277008] Performance Events: Fam17h+ core perfctr, AMD PMU driver.768worker1 # [ 0.242983] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.769worker2 # [ 0.277486] ... version: 2770worker2 # [ 0.278221] ... bit width: 48771worker2 # [ 0.278473] ... generic counters: 6772worker2 # [ 0.279205] ... generic bitmap: 000000000000003f773worker2 # [ 0.279473] ... fixed-purpose counters: 0774client # [ 0.265437] Freeing SMP alternatives memory: 44K775worker2 # [ 0.280196] ... fixed-purpose bitmap: 0000000000000000776client # [ 0.266043] pid_max: default: 32768 minimum: 301777worker2 # [ 0.280473] ... value mask: 0000ffffffffffff778client # [ 0.266846] LSM: initializing lsm=capability,landlock,yama,bpf,ima779worker2 # [ 0.281347] ... max period: 00007fffffffffff780client # [ 0.268074] landlock: Up and running.781client # [ 0.268678] Yama: becoming mindful.782worker2 # [ 0.282181] ... global_ctrl mask: 000000000000003f783client # [ 0.269251] LSM support for eBPF active784worker2 # [ 0.282581] signal: max sigframe size: 3376785worker2 # [ 0.283425] rcu: Hierarchical SRCU implementation.786client # [ 0.270163] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)787worker2 # [ 0.284067] rcu: Max phase no-delay instances is 400.788client # [ 0.271077] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)789client # [ 0.273604] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)790client # [ 0.274573] Performance Events: Fam17h+ core perfctr, AMD PMU driver.791client # [ 0.275045] ... version: 2792worker2 # [ 0.288878] smp: Bringing up secondary CPUs ...793client # [ 0.275731] ... bit width: 48794worker2 # [ 0.289487] smp: Brought up 1 node, 1 CPU795client # [ 0.276043] ... generic counters: 6796worker2 # [ 0.290177] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)797client # [ 0.276723] ... generic bitmap: 000000000000003f798client # [ 0.277052] ... fixed-purpose counters: 0799client # [ 0.277742] ... fixed-purpose bitmap: 0000000000000000800worker2 # [ 0.290619] Memory: 1454600K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)801client # [ 0.278043] ... value mask: 0000ffffffffffff802worker2 # [ 0.291737] devtmpfs: initialized803client # [ 0.278891] ... max period: 00007fffffffffff804worker2 # [ 0.292531] x86/mm: Memory block size: 128MB805client # [ 0.279678] ... global_ctrl mask: 000000000000003f806lb # [ 0.274411] Freeing SMP alternatives memory: 44K807client # [ 0.280164] signal: max sigframe size: 3376808lb # [ 0.275222] pid_max: default: 32768 minimum: 301809client # [ 0.280969] rcu: Hierarchical SRCU implementation.810worker2 # [ 0.294293] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)811lb # [ 0.275957] LSM: initializing lsm=capability,landlock,yama,bpf,ima812client # [ 0.281608] rcu: Max phase no-delay instances is 400.813lb # [ 0.277163] landlock: Up and running.814worker2 # [ 0.295344] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).815lb # [ 0.277754] Yama: becoming mindful.816worker2 # [ 0.296564] pinctrl core: initialized pinctrl subsystem817lb # [ 0.278516] LSM support for eBPF active818worker2 # [ 0.297726] PM: RTC time: 10:55:31, date: 2026-09-22819lb # [ 0.279438] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)820lb # [ 0.280345] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)821client # [ 0.286497] smp: Bringing up secondary CPUs ...822client # [ 0.287057] smp: Brought up 1 node, 1 CPU823worker2 # [ 0.301175] NET: Registered PF_NETLINK/PF_ROUTE protocol family824client # [ 0.287711] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)825lb # [ 0.282942] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)826worker2 # [ 0.301951] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations827lb # [ 0.283867] Performance Events: Fam17h+ core perfctr, AMD PMU driver.828client # [ 0.288270] Memory: 1454568K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)829worker2 # [ 0.302494] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations830lb # [ 0.284309] ... version: 2831client # [ 0.289261] devtmpfs: initialized832lb # [ 0.285008] ... bit width: 48833worker1 # [ 0.277952] Freeing SMP alternatives memory: 44K834client # [ 0.290071] x86/mm: Memory block size: 128MB835lb # [ 0.285307] ... generic counters: 6836worker1 # [ 0.278566] pid_max: default: 32768 minimum: 301837worker2 # [ 0.303941] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations838lb # [ 0.286001] ... generic bitmap: 000000000000003f839worker2 # [ 0.304483] audit: initializing netlink subsys (disabled)840worker1 # [ 0.279096] LSM: initializing lsm=capability,landlock,yama,bpf,ima841lb # [ 0.286317] ... fixed-purpose counters: 0842worker1 # [ 0.280091] landlock: Up and running.843client # [ 0.291787] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)844worker2 # [ 0.305615] thermal_sys: Registered thermal governor 'fair_share'845lb # [ 0.287027] ... fixed-purpose bitmap: 0000000000000000846worker1 # [ 0.280983] Yama: becoming mindful.847worker2 # [ 0.305617] thermal_sys: Registered thermal governor 'bang_bang'848worker1 # [ 0.282193] LSM support for eBPF active849lb # [ 0.287307] ... value mask: 0000ffffffffffff850client # [ 0.292904] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).851worker2 # [ 0.306475] thermal_sys: Registered thermal governor 'step_wise'852lb # [ 0.288233] ... max period: 00007fffffffffff853client # [ 0.294111] pinctrl core: initialized pinctrl subsystem854worker1 # [ 0.282927] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)855worker2 # [ 0.307392] thermal_sys: Registered thermal governor 'user_space'856lb # [ 0.288987] ... global_ctrl mask: 000000000000003f857client # [ 0.295181] PM: RTC time: 10:55:31, date: 2026-09-22858worker1 # [ 0.284038] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)859lb # [ 0.289421] signal: max sigframe size: 3376860worker2 # [ 0.308219] audit: type=2000 audit(1790074532.560:1): state=initialized audit_enabled=0 res=1861lb # [ 0.290230] rcu: Hierarchical SRCU implementation.862worker2 # [ 0.309477] thermal_sys: Registered thermal governor 'power_allocator'863lb # [ 0.290899] rcu: Max phase no-delay instances is 400.864worker2 # [ 0.309491] cpuidle: using governor menu865worker1 # [ 0.286583] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)866client # [ 0.298650] NET: Registered PF_NETLINK/PF_ROUTE protocol family867worker2 # [ 0.311695] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5868worker1 # [ 0.287561] Performance Events: Fam17h+ core perfctr, AMD PMU driver.869client # [ 0.299530] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations870worker1 # [ 0.287987] ... version: 2871worker2 # [ 0.312703] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]872worker1 # [ 0.288692] ... bit width: 48873client # [ 0.300063] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations874worker1 # [ 0.288985] ... generic counters: 6875worker2 # [ 0.313477] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry876worker1 # [ 0.289695] ... generic bitmap: 000000000000003f877lb # [ 0.295977] smp: Bringing up secondary CPUs ...878worker2 # [ 0.314486] PCI: Using configuration type 1 for base access879client # [ 0.301325] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations880lb # [ 0.296321] smp: Brought up 1 node, 1 CPU881worker1 # [ 0.289998] ... fixed-purpose counters: 0882client # [ 0.302053] audit: initializing netlink subsys (disabled)883worker1 # [ 0.290703] ... fixed-purpose bitmap: 0000000000000000884lb # [ 0.296973] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)885worker2 # [ 0.315540] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.886client # [ 0.303171] thermal_sys: Registered thermal governor 'fair_share'887worker1 # [ 0.290985] ... value mask: 0000ffffffffffff888client # [ 0.303173] thermal_sys: Registered thermal governor 'bang_bang'889worker1 # [ 0.291889] ... max period: 00007fffffffffff890client # [ 0.304044] thermal_sys: Registered thermal governor 'step_wise'891worker1 # [ 0.292677] ... global_ctrl mask: 000000000000003f892lb # [ 0.297894] Memory: 1454584K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)893client # [ 0.304974] thermal_sys: Registered thermal governor 'user_space'894worker1 # [ 0.293099] signal: max sigframe size: 3376895lb # [ 0.299562] devtmpfs: initialized896worker1 # [ 0.293912] rcu: Hierarchical SRCU implementation.897lb # [ 0.300390] x86/mm: Memory block size: 128MB898client # [ 0.305751] audit: type=2000 audit(1790074532.573:1): state=initialized audit_enabled=0 res=1899worker1 # [ 0.294578] rcu: Max phase no-delay instances is 400.900worker2 # [ 0.320606] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages901client # [ 0.307046] thermal_sys: Registered thermal governor 'power_allocator'902worker2 # [ 0.321475] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page903client # [ 0.307059] cpuidle: using governor menu904lb # [ 0.302200] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)905client # [ 0.309249] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5906lb # [ 0.303208] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).907lb # [ 0.304404] pinctrl core: initialized pinctrl subsystem908client # [ 0.310276] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]909lb # [ 0.305526] PM: RTC time: 10:55:31, date: 2026-09-22910worker1 # [ 0.299598] smp: Bringing up secondary CPUs ...911client # [ 0.311045] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry912worker1 # [ 0.299999] smp: Brought up 1 node, 1 CPU913worker2 # [ 0.326474] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages914client # [ 0.312054] PCI: Using configuration type 1 for base access915worker2 # [ 0.327473] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page916worker1 # [ 0.300670] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)917client # [ 0.313125] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.918lb # [ 0.309178] NET: Registered PF_NETLINK/PF_ROUTE protocol family919worker1 # [ 0.301205] Memory: 1454616K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)920worker1 # [ 0.302306] devtmpfs: initialized921lb # [ 0.310592] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations922worker1 # [ 0.303148] x86/mm: Memory block size: 128MB923lb # [ 0.311331] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations924lb # [ 0.312598] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations925worker1 # [ 0.305021] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)926lb # [ 0.313317] audit: initializing netlink subsys (disabled)927client # [ 0.320287] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages928worker1 # [ 0.306015] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).929lb # [ 0.314576] thermal_sys: Registered thermal governor 'fair_share'930client # [ 0.321044] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page931worker1 # [ 0.307081] pinctrl core: initialized pinctrl subsystem932lb # [ 0.314578] thermal_sys: Registered thermal governor 'bang_bang'933worker2 # [ 0.337856] ACPI: Added _OSI(Module Device)934worker1 # [ 0.308252] PM: RTC time: 10:55:31, date: 2026-09-22935lb # [ 0.315308] thermal_sys: Registered thermal governor 'step_wise'936worker2 # [ 0.338475] ACPI: Added _OSI(Processor Device)937lb # [ 0.316245] thermal_sys: Registered thermal governor 'user_space'938worker2 # [ 0.339183] ACPI: Added _OSI(Processor Aggregator Device)939lb # [ 0.317030] audit: type=2000 audit(1790074532.594:1): state=initialized audit_enabled=0 res=1940lb # [ 0.318310] thermal_sys: Registered thermal governor 'power_allocator'941client # [ 0.326044] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages942lb # [ 0.318324] cpuidle: using governor menu943worker1 # [ 0.312056] NET: Registered PF_NETLINK/PF_ROUTE protocol family944client # [ 0.327044] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page945worker1 # [ 0.313503] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations946lb # [ 0.320554] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5947worker1 # [ 0.314010] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations948lb # [ 0.321576] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]949lb # [ 0.322310] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry950worker1 # [ 0.315286] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations951lb # [ 0.323319] PCI: Using configuration type 1 for base access952worker1 # [ 0.315995] audit: initializing netlink subsys (disabled)953worker2 # [ 0.347688] ACPI: 1 ACPI AML tables successfully acquired and loaded954worker1 # [ 0.317180] thermal_sys: Registered thermal governor 'fair_share'955lb # [ 0.324455] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.956worker1 # [ 0.317183] thermal_sys: Registered thermal governor 'bang_bang'957worker1 # [ 0.317986] thermal_sys: Registered thermal governor 'step_wise'958client # [ 0.335646] ACPI: Added _OSI(Module Device)959client # [ 0.336044] ACPI: Added _OSI(Processor Device)960worker1 # [ 0.318957] thermal_sys: Registered thermal governor 'user_space'961worker2 # [ 0.351875] ACPI: Interpreter enabled962client # [ 0.336779] ACPI: Added _OSI(Processor Aggregator Device)963worker2 # [ 0.352490] ACPI: PM: (supports S0 S3 S4 S5)964worker1 # [ 0.319738] audit: type=2000 audit(1790074532.601:1): state=initialized audit_enabled=0 res=1965worker2 # [ 0.353232] ACPI: Using IOAPIC for interrupt routing966worker1 # [ 0.320989] thermal_sys: Registered thermal governor 'power_allocator'967worker1 # [ 0.321012] cpuidle: using governor menu968lb # [ 0.331574] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages969worker1 # [ 0.323228] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5970worker2 # [ 0.355522] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug971lb # [ 0.332308] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page972worker1 # [ 0.324254] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]973worker2 # [ 0.356474] PCI: Using E820 reservations for host bridge windows974worker1 # [ 0.324989] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry975client # [ 0.343575] ACPI: 1 ACPI AML tables successfully acquired and loaded976worker1 # [ 0.325998] PCI: Using configuration type 1 for base access977worker2 # [ 0.359584] ACPI: Enabled 2 GPEs in block 00 to 3F978worker1 # [ 0.327137] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.979lb # [ 0.337308] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages980lb # [ 0.338308] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page981client # [ 0.346964] ACPI: Interpreter enabled982client # [ 0.347519] ACPI: PM: (supports S0 S3 S4 S5)983client # [ 0.348043] ACPI: Using IOAPIC for interrupt routing984client # [ 0.348920] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug985worker2 # [ 0.367078] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])986worker1 # [ 0.334301] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages987worker1 # [ 0.334987] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page988worker2 # [ 0.367480] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]989worker1 # [ 0.335986] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages990worker2 # [ 0.368552] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]991client # [ 0.354044] PCI: Using E820 reservations for host bridge windows992lb # [ 0.346924] ACPI: Added _OSI(Module Device)993lb # [ 0.347308] ACPI: Added _OSI(Processor Device)994client # [ 0.355092] ACPI: Enabled 2 GPEs in block 00 to 3F995worker2 # [ 0.369596] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]996lb # [ 0.348017] ACPI: Added _OSI(Processor Aggregator Device)997worker2 # [ 0.370885] PCI host bridge to bus 0000:00998worker2 # [ 0.371478] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]999worker2 # [ 0.372474] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1000worker2 # [ 0.373474] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1001worker1 # [ 0.342986] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1002worker2 # [ 0.374474] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1003worker2 # [ 0.375474] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1004lb # [ 0.354878] ACPI: 1 ACPI AML tables successfully acquired and loaded1005worker2 # [ 0.376474] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1006client # [ 0.363695] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1007worker2 # [ 0.377474] pci_bus 0000:00: root bus resource [bus 00-ff]1008client # [ 0.364644] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1009worker2 # [ 0.378446] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1010worker1 # [ 0.349424] ACPI: Added _OSI(Module Device)1011lb # [ 0.358219] ACPI: Interpreter enabled1012client # [ 0.365122] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1013lb # [ 0.358755] ACPI: PM: (supports S0 S3 S4 S5)1014worker1 # [ 0.349987] ACPI: Added _OSI(Processor Device)1015worker2 # [ 0.379925] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1016client # [ 0.366164] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1017worker1 # [ 0.350704] ACPI: Added _OSI(Processor Aggregator Device)1018lb # [ 0.359308] ACPI: Using IOAPIC for interrupt routing1019client # [ 0.367374] PCI host bridge to bus 0000:001020lb # [ 0.360155] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1021client # [ 0.368047] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1022client # [ 0.369043] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1023client # [ 0.370043] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1024client # [ 0.371043] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1025lb # [ 0.365307] PCI: Using E820 reservations for host bridge windows1026client # [ 0.372043] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1027worker2 # [ 0.382525] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1028lb # [ 0.366391] ACPI: Enabled 2 GPEs in block 00 to 3F1029worker1 # [ 0.357587] ACPI: 1 ACPI AML tables successfully acquired and loaded1030client # [ 0.373043] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1031worker2 # [ 0.383489] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1032client # [ 0.374043] pci_bus 0000:00: root bus resource [bus 00-ff]1033worker2 # [ 0.384464] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1034client # [ 0.375054] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1035worker2 # [ 0.385498] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1036worker1 # [ 0.361411] ACPI: Interpreter enabled1037client # [ 0.376496] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1038worker1 # [ 0.362002] ACPI: PM: (supports S0 S3 S4 S5)1039worker2 # [ 0.387374] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1040worker1 # [ 0.362727] ACPI: Using IOAPIC for interrupt routing1041worker2 # [ 0.389501] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1042lb # [ 0.375031] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1043worker1 # [ 0.365047] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1044worker2 # [ 0.390336] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1045worker1 # [ 0.365986] PCI: Using E820 reservations for host bridge windows1046worker2 # [ 0.391234] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1047lb # [ 0.375891] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1048worker2 # [ 0.391479] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1049lb # [ 0.376385] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1050client # [ 0.379108] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1051lb # [ 0.377430] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1052worker1 # [ 0.369100] ACPI: Enabled 2 GPEs in block 00 to 3F1053worker2 # [ 0.393011] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1054client # [ 0.380054] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1055lb # [ 0.378708] PCI host bridge to bus 0000:001056client # [ 0.381031] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1057lb # [ 0.379312] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1058worker2 # [ 0.394481] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1059lb # [ 0.380307] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1060client # [ 0.382011] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1061worker2 # [ 0.395350] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1062lb # [ 0.381307] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1063worker2 # [ 0.396198] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1064client # [ 0.383828] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1065lb # [ 0.382307] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1066worker2 # [ 0.397024] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1067lb # [ 0.383307] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1068lb # [ 0.384307] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1069worker1 # [ 0.377829] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1070client # [ 0.387058] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1071lb # [ 0.385308] pci_bus 0000:00: root bus resource [bus 00-ff]1072client # [ 0.387881] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1073worker2 # [ 0.399483] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1074worker1 # [ 0.378800] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1075lb # [ 0.386364] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1076worker2 # [ 0.400465] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1077client # [ 0.388765] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1078worker1 # [ 0.380400] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1079client # [ 0.389049] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1080worker2 # [ 0.401206] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1081lb # [ 0.387780] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1082worker1 # [ 0.381110] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1083worker2 # [ 0.402049] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1084client # [ 0.390675] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1085worker1 # [ 0.382355] PCI host bridge to bus 0000:001086worker1 # [ 0.382991] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1087worker2 # [ 0.403481] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1088worker1 # [ 0.383986] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1089worker2 # [ 0.404327] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1090worker1 # [ 0.384985] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1091worker2 # [ 0.405229] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1092client # [ 0.393058] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1093worker1 # [ 0.385985] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1094client # [ 0.394049] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1095worker1 # [ 0.386985] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1096worker2 # [ 0.406016] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1097client # [ 0.395065] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1098lb # [ 0.390412] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1099worker1 # [ 0.387985] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1100client # [ 0.396593] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1101worker2 # [ 0.407502] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1102worker1 # [ 0.388986] pci_bus 0000:00: root bus resource [bus 00-ff]1103lb # [ 0.391319] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1104worker2 # [ 0.408324] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1105lb # [ 0.392231] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1106worker1 # [ 0.390022] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1107worker2 # [ 0.409250] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1108client # [ 0.398050] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1109lb # [ 0.393649] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1110client # [ 0.398898] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1111worker1 # [ 0.391468] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1112worker2 # [ 0.410881] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1113client # [ 0.399746] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1114lb # [ 0.395033] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1115client # [ 0.400594] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1116worker2 # [ 0.413474] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1117worker2 # [ 0.414325] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1118lb # [ 0.398339] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1119client # [ 0.403051] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1120lb # [ 0.399205] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1121worker2 # [ 0.415495] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1122client # [ 0.403938] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1123lb # [ 0.400062] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1124client # [ 0.404762] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1125worker2 # [ 0.417072] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1126lb # [ 0.400313] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1127client # [ 0.405596] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1128worker1 # [ 0.394111] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1129worker2 # [ 0.418481] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1130lb # [ 0.401873] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1131worker1 # [ 0.394998] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1132worker2 # [ 0.419345] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1133worker1 # [ 0.396351] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1134worker2 # [ 0.420319] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1135client # [ 0.407059] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1136lb # [ 0.403325] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1137worker2 # [ 0.421293] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1138client # [ 0.408049] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1139worker1 # [ 0.397319] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1140lb # [ 0.404168] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1141client # [ 0.408971] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1142lb # [ 0.405046] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1143worker2 # [ 0.422755] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1144worker1 # [ 0.398755] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1145client # [ 0.410471] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1146worker2 # [ 0.424288] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1147lb # [ 0.405878] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1148worker2 # [ 0.425208] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1149client # [ 0.412050] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1150worker1 # [ 0.402022] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1151client # [ 0.412909] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1152worker2 # [ 0.426023] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1153worker1 # [ 0.402924] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1154lb # [ 0.408314] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1155client # [ 0.413796] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1156lb # [ 0.409159] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1157worker1 # [ 0.403774] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1158worker2 # [ 0.427483] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1159client # [ 0.414629] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1160worker1 # [ 0.404992] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1161lb # [ 0.410027] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1162worker2 # [ 0.428322] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1163worker2 # [ 0.429173] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1164client # [ 0.416050] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1165lb # [ 0.410949] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1166worker1 # [ 0.406574] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1167client # [ 0.416865] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1168worker2 # [ 0.430034] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1169client # [ 0.417819] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1170lb # [ 0.412317] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1171worker1 # [ 0.408018] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1172client # [ 0.418941] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1173worker2 # [ 0.432474] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1174lb # [ 0.413146] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1175worker1 # [ 0.408893] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1176worker2 # [ 0.433286] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1177client # [ 0.420312] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1178worker1 # [ 0.409758] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1179lb # [ 0.414027] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1180worker2 # [ 0.434191] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1181client # [ 0.422055] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1182worker1 # [ 0.411391] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1183lb # [ 0.414900] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1184worker2 # [ 0.435043] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1185client # [ 0.422969] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1186worker2 # [ 0.436100] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1187lb # [ 0.416323] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1188client # [ 0.424707] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1189worker2 # [ 0.436698] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1190lb # [ 0.417153] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1191worker1 # [ 0.414021] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1192lb # [ 0.418025] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1193worker1 # [ 0.414922] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1194worker2 # [ 0.438395] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1195client # [ 0.426074] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1196worker1 # [ 0.415739] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1197client # [ 0.426879] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1198lb # [ 0.418899] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1199client # [ 0.427756] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1200worker1 # [ 0.416621] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1201client # [ 0.428623] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1202lb # [ 0.421547] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1203worker1 # [ 0.418853] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1204lb # [ 0.422314] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1205worker1 # [ 0.419689] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1206client # [ 0.430071] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1207lb # [ 0.423263] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1208client # [ 0.431049] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1209worker1 # [ 0.420034] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1210client # [ 0.432042] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1211lb # [ 0.424709] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1212worker1 # [ 0.421582] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1213client # [ 0.433627] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1214lb # [ 0.426323] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1215client # [ 0.434622] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1216worker2 # [ 0.439279] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1217lb # [ 0.427161] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1218worker1 # [ 0.423021] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1219lb # [ 0.428025] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1220client # [ 0.435243] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1221worker2 # [ 0.441005] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1222worker1 # [ 0.423894] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1223lb # [ 0.428314] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1224client # [ 0.436599] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1225worker1 # [ 0.424747] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1226worker2 # [ 0.441700] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1227worker2 # [ 0.442967] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1228lb # [ 0.430076] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1229worker1 # [ 0.426408] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1230worker2 # [ 0.443727] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1231lb # [ 0.432291] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1232lb # [ 0.433010] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1233worker2 # [ 0.444758] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1234worker1 # [ 0.429017] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1235worker2 # [ 0.445743] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1236lb # [ 0.433877] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1237worker1 # [ 0.429939] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1238worker1 # [ 0.430753] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1239worker2 # [ 0.447284] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1240worker2 # [ 0.448111] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1241lb # [ 0.435332] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1242worker1 # [ 0.432538] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1243worker2 # [ 0.448793] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1244lb # [ 0.436185] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1245lb # [ 0.437329] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1246worker2 # [ 0.450044] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1247worker1 # [ 0.433993] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1248client # [ 0.437265] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1249worker1 # [ 0.434847] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1250lb # [ 0.438884] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1251worker1 # [ 0.435784] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1252client # [ 0.440594] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1253lb # [ 0.440315] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1254worker2 # [ 0.453631] ACPI: PCI: Interrupt link LNKA configured for IRQ 101255worker1 # [ 0.436993] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1256lb # [ 0.441145] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1257worker2 # [ 0.454578] ACPI: PCI: Interrupt link LNKB configured for IRQ 101258client # [ 0.441288] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1259lb # [ 0.442010] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1260worker2 # [ 0.455575] ACPI: PCI: Interrupt link LNKC configured for IRQ 111261worker1 # [ 0.438572] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1262client # [ 0.442989] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1263worker2 # [ 0.456570] ACPI: PCI: Interrupt link LNKD configured for IRQ 111264lb # [ 0.442899] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1265worker2 # [ 0.457570] ACPI: PCI: Interrupt link LNKE configured for IRQ 101266worker1 # [ 0.439999] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1267client # [ 0.444044] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1268worker2 # [ 0.458573] ACPI: PCI: Interrupt link LNKF configured for IRQ 101269worker1 # [ 0.440964] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1270lb # [ 0.444006] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1271client # [ 0.445326] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1272worker2 # [ 0.459571] ACPI: PCI: Interrupt link LNKG configured for IRQ 111273worker1 # [ 0.442443] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1274worker2 # [ 0.460567] ACPI: PCI: Interrupt link LNKH configured for IRQ 111275lb # [ 0.445124] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1276client # [ 0.446290] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1277worker2 # [ 0.461497] ACPI: PCI: Interrupt link GSIA configured for IRQ 161278lb # [ 0.446923] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1279worker2 # [ 0.462441] ACPI: PCI: Interrupt link GSIB configured for IRQ 171280client # [ 0.448571] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1281worker2 # [ 0.463204] ACPI: PCI: Interrupt link GSIC configured for IRQ 181282worker1 # [ 0.444045] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1283worker2 # [ 0.463488] ACPI: PCI: Interrupt link GSID configured for IRQ 191284client # [ 0.449050] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1285worker1 # [ 0.445663] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1286worker2 # [ 0.464435] ACPI: PCI: Interrupt link GSIE configured for IRQ 201287client # [ 0.450308] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1288worker1 # [ 0.446009] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1289worker2 # [ 0.465237] ACPI: PCI: Interrupt link GSIF configured for IRQ 211290worker2 # [ 0.465487] ACPI: PCI: Interrupt link GSIG configured for IRQ 221291client # [ 0.451819] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1292worker1 # [ 0.447563] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1293worker2 # [ 0.466429] ACPI: PCI: Interrupt link GSIH configured for IRQ 231294worker2 # [ 0.468259] iommu: Default domain type: Translated1295worker1 # [ 0.448993] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1296worker2 # [ 0.468481] iommu: DMA domain TLB invalidation policy: lazy mode1297worker1 # [ 0.449884] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1298client # [ 0.454993] ACPI: PCI: Interrupt link LNKA configured for IRQ 101299worker2 # [ 0.469686] ACPI: bus type USB registered1300worker1 # [ 0.450754] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1301client # [ 0.455901] ACPI: PCI: Interrupt link LNKB configured for IRQ 101302worker2 # [ 0.470408] usbcore: registered new interface driver usbfs1303worker2 # [ 0.471142] usbcore: registered new interface driver hub1304client # [ 0.457019] ACPI: PCI: Interrupt link LNKC configured for IRQ 111305worker2 # [ 0.471483] usbcore: registered new device driver usb1306lb # [ 0.447512] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1307client # [ 0.457867] ACPI: PCI: Interrupt link LNKD configured for IRQ 111308worker1 # [ 0.453111] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1309worker2 # [ 0.473076] NetLabel: Initializing1310client # [ 0.458824] ACPI: PCI: Interrupt link LNKE configured for IRQ 101311lb # [ 0.449947] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1312worker2 # [ 0.473475] NetLabel: domain hash size = 1281313worker1 # [ 0.454694] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1314client # [ 0.459856] ACPI: PCI: Interrupt link LNKF configured for IRQ 101315worker2 # [ 0.474170] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1316lb # [ 0.450560] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1317client # [ 0.460827] ACPI: PCI: Interrupt link LNKG configured for IRQ 111318worker2 # [ 0.474509] NetLabel: unlabeled traffic allowed by default1319worker1 # [ 0.455222] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1320client # [ 0.461818] ACPI: PCI: Interrupt link LNKH configured for IRQ 111321worker2 # [ 0.475364] PCI: Using ACPI for IRQ routing1322lb # [ 0.451922] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1323worker1 # [ 0.456633] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1324client # [ 0.462079] ACPI: PCI: Interrupt link GSIA configured for IRQ 161325client # [ 0.463033] ACPI: PCI: Interrupt link GSIB configured for IRQ 171326lb # [ 0.452577] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1327client # [ 0.463775] ACPI: PCI: Interrupt link GSIC configured for IRQ 181328lb # [ 0.453615] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1329client # [ 0.464057] ACPI: PCI: Interrupt link GSID configured for IRQ 191330client # [ 0.465056] ACPI: PCI: Interrupt link GSIE configured for IRQ 201331lb # [ 0.454556] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1332client # [ 0.465995] ACPI: PCI: Interrupt link GSIF configured for IRQ 211333client # [ 0.466823] ACPI: PCI: Interrupt link GSIG configured for IRQ 221334lb # [ 0.456292] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1335client # [ 0.467771] ACPI: PCI: Interrupt link GSIH configured for IRQ 231336lb # [ 0.456925] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1337lb # [ 0.457649] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1338client # [ 0.469052] iommu: Default domain type: Translated1339client # [ 0.469858] iommu: DMA domain TLB invalidation policy: lazy mode1340lb # [ 0.459240] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1341client # [ 0.471116] ACPI: bus type USB registered1342client # [ 0.471912] usbcore: registered new interface driver usbfs1343client # [ 0.472786] usbcore: registered new interface driver hub1344worker1 # [ 0.457187] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1345client # [ 0.473053] usbcore: registered new device driver usb1346lb # [ 0.463041] ACPI: PCI: Interrupt link LNKA configured for IRQ 101347worker1 # [ 0.459670] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1348client # [ 0.474900] NetLabel: Initializing1349lb # [ 0.463414] ACPI: PCI: Interrupt link LNKB configured for IRQ 101350client # [ 0.475494] NetLabel: domain hash size = 1281351worker1 # [ 0.460241] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1352lb # [ 0.464409] ACPI: PCI: Interrupt link LNKC configured for IRQ 111353client # [ 0.476043] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1354lb # [ 0.465403] ACPI: PCI: Interrupt link LNKD configured for IRQ 111355client # [ 0.477038] NetLabel: unlabeled traffic allowed by default1356worker1 # [ 0.461615] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1357client # [ 0.477975] PCI: Using ACPI for IRQ routing1358lb # [ 0.466409] ACPI: PCI: Interrupt link LNKE configured for IRQ 101359worker1 # [ 0.462264] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1360lb # [ 0.467406] ACPI: PCI: Interrupt link LNKF configured for IRQ 101361lb # [ 0.468409] ACPI: PCI: Interrupt link LNKG configured for IRQ 111362worker1 # [ 0.463288] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1363lb # [ 0.469407] ACPI: PCI: Interrupt link LNKH configured for IRQ 111364worker1 # [ 0.464267] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1365lb # [ 0.470343] ACPI: PCI: Interrupt link GSIA configured for IRQ 161366lb # [ 0.472067] ACPI: PCI: Interrupt link GSIB configured for IRQ 171367lb # [ 0.472320] ACPI: PCI: Interrupt link GSIC configured for IRQ 181368lb # [ 0.473259] ACPI: PCI: Interrupt link GSID configured for IRQ 191369worker1 # [ 0.466015] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1370lb # [ 0.474034] ACPI: PCI: Interrupt link GSIE configured for IRQ 201371worker1 # [ 0.467040] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1372lb # [ 0.474320] ACPI: PCI: Interrupt link GSIF configured for IRQ 211373lb # [ 0.475273] ACPI: PCI: Interrupt link GSIG configured for IRQ 221374worker1 # [ 0.468365] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1375lb # [ 0.476032] ACPI: PCI: Interrupt link GSIH configured for IRQ 231376lb # [ 0.477356] iommu: Default domain type: Translated1377worker1 # [ 0.470535] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1378lb # [ 0.478155] iommu: DMA domain TLB invalidation policy: lazy mode1379lb # [ 0.479312] ACPI: bus type USB registered1380lb # [ 0.480068] usbcore: registered new interface driver usbfs1381lb # [ 0.480323] usbcore: registered new interface driver hub1382lb # [ 0.481187] usbcore: registered new device driver usb1383worker1 # [ 0.474427] ACPI: PCI: Interrupt link LNKA configured for IRQ 101384lb # [ 0.482873] NetLabel: Initializing1385worker1 # [ 0.475098] ACPI: PCI: Interrupt link LNKB configured for IRQ 101386lb # [ 0.483308] NetLabel: domain hash size = 1281387worker1 # [ 0.476085] ACPI: PCI: Interrupt link LNKC configured for IRQ 111388lb # [ 0.484116] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1389worker1 # [ 0.477081] ACPI: PCI: Interrupt link LNKD configured for IRQ 111390lb # [ 0.485089] NetLabel: unlabeled traffic allowed by default1391worker1 # [ 0.478103] ACPI: PCI: Interrupt link LNKE configured for IRQ 101392lb # [ 0.486022] PCI: Using ACPI for IRQ routing1393worker1 # [ 0.479083] ACPI: PCI: Interrupt link LNKF configured for IRQ 101394worker2 # [ 0.519123] pci 0000:00:01.0: vgaarb: setting as boot VGA device1395worker2 # [ 0.519470] pci 0000:00:01.0: vgaarb: bridge control possible1396worker1 # [ 0.480090] ACPI: PCI: Interrupt link LNKG configured for IRQ 111397worker2 # [ 0.519470] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1398worker1 # [ 0.481088] ACPI: PCI: Interrupt link LNKH configured for IRQ 111399worker2 # [ 0.519478] vgaarb: loaded1400worker1 # [ 0.482025] ACPI: PCI: Interrupt link GSIA configured for IRQ 161401worker2 # [ 0.520188] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01402worker1 # [ 0.482997] ACPI: PCI: Interrupt link GSIB configured for IRQ 171403worker2 # [ 0.520475] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1404worker1 # [ 0.483998] ACPI: PCI: Interrupt link GSIC configured for IRQ 181405worker1 # [ 0.484997] ACPI: PCI: Interrupt link GSID configured for IRQ 191406worker1 # [ 0.485996] ACPI: PCI: Interrupt link GSIE configured for IRQ 201407worker1 # [ 0.486996] ACPI: PCI: Interrupt link GSIF configured for IRQ 211408worker1 # [ 0.488000] ACPI: PCI: Interrupt link GSIG configured for IRQ 221409worker1 # [ 0.489000] ACPI: PCI: Interrupt link GSIH configured for IRQ 231410worker2 # [ 0.525562] clocksource: Switched to clocksource kvm-clock1411worker2 # [ 0.527175] VFS: Disk quotas dquot_6.6.01412worker1 # [ 0.491141] iommu: Default domain type: Translated1413worker1 # [ 0.491993] iommu: DMA domain TLB invalidation policy: lazy mode1414worker1 # [ 0.493264] ACPI: bus type USB registered1415worker2 # [ 0.529681] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1416worker2 # [ 0.530978] pnp: PnP ACPI init1417worker1 # [ 0.494057] usbcore: registered new interface driver usbfs1418worker1 # [ 0.495006] usbcore: registered new interface driver hub1419worker2 # [ 0.531771] ACPI: IRQ 4 override to edge(!), high(!)1420worker1 # [ 0.495964] usbcore: registered new device driver usb1421worker2 # [ 0.532705] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1422worker2 # [ 0.534180] pnp: PnP ACPI: found 6 devices1423worker1 # [ 0.497625] NetLabel: Initializing1424client # [ 0.522409] pci 0000:00:01.0: vgaarb: setting as boot VGA device1425worker1 # [ 0.497986] NetLabel: domain hash size = 1281426client # [ 0.523039] pci 0000:00:01.0: vgaarb: bridge control possible1427worker1 # [ 0.498735] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1428worker1 # [ 0.499055] NetLabel: unlabeled traffic allowed by default1429client # [ 0.523039] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1430client # [ 0.523047] vgaarb: loaded1431worker1 # [ 0.499998] PCI: Using ACPI for IRQ routing1432client # [ 0.523764] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01433client # [ 0.524044] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1434worker2 # [ 0.541738] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1435worker2 # [ 0.543127] clocksource: Switched to clocksource acpi_pm1436worker2 # [ 0.544044] NET: Registered PF_INET protocol family1437client # [ 0.529130] clocksource: Switched to clocksource kvm-clock1438worker2 # [ 0.545170] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1439client # [ 0.532816] VFS: Disk quotas dquot_6.6.01440client # [ 0.533673] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1441client # [ 0.535061] pnp: PnP ACPI init1442client # [ 0.535876] ACPI: IRQ 4 override to edge(!), high(!)1443client # [ 0.536887] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1444client # [ 0.538449] pnp: PnP ACPI: found 6 devices1445lb # [ 0.531030] pci 0000:00:01.0: vgaarb: setting as boot VGA device1446lb # [ 0.531303] pci 0000:00:01.0: vgaarb: bridge control possible1447lb # [ 0.531303] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1448worker2 # [ 0.561616] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1449client # [ 0.546157] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1450lb # [ 0.531311] vgaarb: loaded1451client # [ 0.547582] clocksource: Switched to clocksource acpi_pm1452worker2 # [ 0.563065] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1453lb # [ 0.532074] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01454client # [ 0.548548] NET: Registered PF_INET protocol family1455lb # [ 0.532308] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1456worker2 # [ 0.564307] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1457client # [ 0.549731] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1458worker2 # [ 0.565597] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1459worker2 # [ 0.566797] TCP: Hash tables configured (established 16384 bind 16384)1460worker2 # [ 0.567869] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1461lb # [ 0.535401] clocksource: Switched to clocksource kvm-clock1462worker2 # [ 0.569078] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1463worker2 # [ 0.570176] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1464worker2 # [ 0.571503] NET: Registered PF_UNIX/PF_LOCAL protocol family1465lb # [ 0.539125] VFS: Disk quotas dquot_6.6.01466worker2 # [ 0.572513] NET: Registered PF_XDP protocol family1467lb # [ 0.539872] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1468worker2 # [ 0.573370] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1469lb # [ 0.541215] pnp: PnP ACPI init1470worker2 # [ 0.574419] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1471lb # [ 0.542045] ACPI: IRQ 4 override to edge(!), high(!)1472worker2 # [ 0.575481] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1473lb # [ 0.543084] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1474worker2 # [ 0.576607] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1475lb # [ 0.544617] pnp: PnP ACPI: found 6 devices1476worker2 # [ 0.577731] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1477worker2 # [ 0.578867] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1478worker2 # [ 0.580661] ACPI: \_SB_.GSIA: Enabled at IRQ 161479client # [ 0.565969] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1480worker2 # [ 0.582853] ACPI: \_SB_.GSIB: Enabled at IRQ 171481client # [ 0.567566] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1482worker1 # [ 0.545660] pci 0000:00:01.0: vgaarb: setting as boot VGA device1483worker1 # [ 0.545981] pci 0000:00:01.0: vgaarb: bridge control possible1484worker2 # [ 0.584823] ACPI: \_SB_.GSIC: Enabled at IRQ 181485client # [ 0.568919] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1486lb # [ 0.552301] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1487worker1 # [ 0.545981] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1488client # [ 0.570248] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1489worker2 # [ 0.586717] ACPI: \_SB_.GSID: Enabled at IRQ 191490worker1 # [ 0.545990] vgaarb: loaded1491lb # [ 0.553729] clocksource: Switched to clocksource acpi_pm1492client # [ 0.571501] TCP: Hash tables configured (established 16384 bind 16384)1493worker1 # [ 0.546730] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01494lb # [ 0.554643] NET: Registered PF_INET protocol family1495worker2 # [ 0.588300] PCI: CLS 0 bytes, default 641496client # [ 0.572604] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1497worker1 # [ 0.546987] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1498lb # [ 0.556127] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1499client # [ 0.573854] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1500worker2 # [ 0.589174] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1501client # [ 0.574897] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1502worker2 # [ 0.590863] Trying to unpack rootfs image as initramfs...1503client # [ 0.576130] NET: Registered PF_UNIX/PF_LOCAL protocol family1504client # [ 0.577124] NET: Registered PF_XDP protocol family1505worker1 # [ 0.550074] clocksource: Switched to clocksource kvm-clock1506client # [ 0.577955] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1507client # [ 0.578984] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1508client # [ 0.580024] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1509worker1 # [ 0.554020] VFS: Disk quotas dquot_6.6.01510client # [ 0.581155] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1511worker1 # [ 0.554765] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1512client # [ 0.582250] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1513worker1 # [ 0.556173] pnp: PnP ACPI init1514client # [ 0.583414] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1515worker1 # [ 0.556973] ACPI: IRQ 4 override to edge(!), high(!)1516client # [ 0.585098] ACPI: \_SB_.GSIA: Enabled at IRQ 161517worker1 # [ 0.558065] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1518worker1 # [ 0.559651] pnp: PnP ACPI: found 6 devices1519client # [ 0.587269] ACPI: \_SB_.GSIB: Enabled at IRQ 171520client # [ 0.589222] ACPI: \_SB_.GSIC: Enabled at IRQ 181521lb # [ 0.572876] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1522client # [ 0.591311] ACPI: \_SB_.GSID: Enabled at IRQ 191523lb # [ 0.574357] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1524client # [ 0.592966] PCI: CLS 0 bytes, default 641525lb # [ 0.575675] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1526client # [ 0.593867] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1527lb # [ 0.576957] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1528worker1 # [ 0.567298] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1529client # [ 0.595616] Trying to unpack rootfs image as initramfs...1530worker1 # [ 0.568748] clocksource: Switched to clocksource acpi_pm1531lb # [ 0.578159] TCP: Hash tables configured (established 16384 bind 16384)1532worker1 # [ 0.569708] NET: Registered PF_INET protocol family1533lb # [ 0.579259] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1534lb # [ 0.580496] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1535worker1 # [ 0.570922] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1536lb # [ 0.581555] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1537lb # [ 0.582757] NET: Registered PF_UNIX/PF_LOCAL protocol family1538lb # [ 0.583703] NET: Registered PF_XDP protocol family1539lb # [ 0.584490] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1540lb # [ 0.585461] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1541lb # [ 0.586430] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1542lb # [ 0.587491] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1543lb # [ 0.588573] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1544lb # [ 0.589650] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1545lb # [ 0.591278] ACPI: \_SB_.GSIA: Enabled at IRQ 161546lb # [ 0.593439] ACPI: \_SB_.GSIB: Enabled at IRQ 171547lb # [ 0.595431] ACPI: \_SB_.GSIC: Enabled at IRQ 181548lb # [ 0.597421] ACPI: \_SB_.GSID: Enabled at IRQ 191549worker1 # [ 0.587545] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1550lb # [ 0.599167] PCI: CLS 0 bytes, default 641551worker1 # [ 0.589153] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1552worker1 # [ 0.590492] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1553lb # [ 0.600085] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1554lb # [ 0.601792] Trying to unpack rootfs image as initramfs...1555worker1 # [ 0.591852] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1556worker1 # [ 0.593172] TCP: Hash tables configured (established 16384 bind 16384)1557worker2 # [ 0.635878] Initialise system trusted keyrings1558worker2 # [ 0.637101] workingset: timestamp_bits=40 max_order=19 bucket_order=01559worker1 # [ 0.594361] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1560worker1 # [ 0.595768] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1561worker1 # [ 0.596988] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1562worker1 # [ 0.598282] NET: Registered PF_UNIX/PF_LOCAL protocol family1563worker1 # [ 0.599309] NET: Registered PF_XDP protocol family1564worker1 # [ 0.600186] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1565worker1 # [ 0.601240] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1566worker1 # [ 0.602298] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1567worker1 # [ 0.603453] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1568worker1 # [ 0.604600] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1569worker1 # [ 0.605739] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1570worker1 # [ 0.607572] ACPI: \_SB_.GSIA: Enabled at IRQ 161571worker1 # [ 0.609702] ACPI: \_SB_.GSIB: Enabled at IRQ 171572client # [ 0.638760] Initialise system trusted keyrings1573worker1 # [ 0.611777] ACPI: \_SB_.GSIC: Enabled at IRQ 181574client # [ 0.639703] workingset: timestamp_bits=40 max_order=19 bucket_order=01575worker1 # [ 0.613780] ACPI: \_SB_.GSID: Enabled at IRQ 191576worker1 # [ 0.615594] PCI: CLS 0 bytes, default 641577worker1 # [ 0.616552] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1578worker2 # [ 0.660455] Key type asymmetric registered1579worker1 # [ 0.618348] Trying to unpack rootfs image as initramfs...1580worker2 # [ 0.661152] Asymmetric key parser 'x509' registered1581worker2 # [ 0.662021] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1582worker2 # [ 0.664569] io scheduler mq-deadline registered1583worker2 # [ 0.666415] io scheduler kyber registered1584worker2 # [ 0.669104] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1585worker2 # [ 0.672416] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1586worker2 # [ 0.676029] Linux agpgart interface v0.1031587worker2 # [ 0.676811] ACPI: bus type drm_connector registered1588client # [ 0.663408] Key type asymmetric registered1589lb # [ 0.645953] Initialise system trusted keyrings1590client # [ 0.664082] Asymmetric key parser 'x509' registered1591worker2 # [ 0.679870] usbcore: registered new interface driver usbserial_generic1592client # [ 0.664968] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1593worker2 # [ 0.680996] usbserial: USB Serial support registered for generic1594lb # [ 0.649584] workingset: timestamp_bits=40 max_order=19 bucket_order=01595client # [ 0.669490] io scheduler mq-deadline registered1596worker2 # [ 0.684418] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1597client # [ 0.670259] io scheduler kyber registered1598worker2 # [ 0.684418] Please enable it if your BIOS has the CPPC option.1599worker2 # [ 0.686674] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1600client # [ 0.672929] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1601client # [ 0.674263] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1602worker2 # [ 0.690549] drop_monitor: Initializing network drop monitor service1603worker2 # [ 0.691779] NET: Registered PF_INET6 protocol family1604worker2 # [ 0.694945] Segment Routing with IPv61605client # [ 0.680131] Linux agpgart interface v0.1031606client # [ 0.680927] ACPI: bus type drm_connector registered1607worker2 # [ 0.697490] In-situ OAM (IOAM) with IPv61608worker2 # [ 0.698557] IPI shorthand broadcast: enabled1609client # [ 0.683823] usbcore: registered new interface driver usbserial_generic1610client # [ 0.684974] usbserial: USB Serial support registered for generic1611client # [ 0.687378] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1612lb # [ 0.671103] Key type asymmetric registered1613client # [ 0.687378] Please enable it if your BIOS has the CPPC option.1614lb # [ 0.671829] Asymmetric key parser 'x509' registered1615client # [ 0.689635] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1616worker1 # [ 0.663050] Initialise system trusted keyrings1617worker2 # [ 0.706698] sched_clock: Marking stable (572013800, 134387898)->(776729016, -70327318)1618lb # [ 0.674618] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1619worker1 # [ 0.666067] workingset: timestamp_bits=40 max_order=19 bucket_order=01620client # [ 0.693475] drop_monitor: Initializing network drop monitor service1621client # [ 0.694636] NET: Registered PF_INET6 protocol family1622lb # [ 0.677615] io scheduler mq-deadline registered1623lb # [ 0.678348] io scheduler kyber registered1624worker2 # [ 0.711656] registered taskstats version 11625worker2 # [ 0.712636] Loading compiled-in X.509 certificates1626lb # [ 0.681108] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1627client # [ 0.699672] Segment Routing with IPv61628lb # [ 0.682335] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1629client # [ 0.702385] In-situ OAM (IOAM) with IPv61630client # [ 0.703456] IPI shorthand broadcast: enabled1631lb # [ 0.686317] Linux agpgart interface v0.1031632lb # [ 0.687073] ACPI: bus type drm_connector registered1633lb # [ 0.690010] usbcore: registered new interface driver usbserial_generic1634lb # [ 0.691040] usbserial: USB Serial support registered for generic1635lb # [ 0.693562] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1636client # [ 0.711725] sched_clock: Marking stable (577013980, 134340459)->(782319158, -70964719)1637lb # [ 0.693562] Please enable it if your BIOS has the CPPC option.1638lb # [ 0.695653] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1639worker1 # [ 0.688185] Key type asymmetric registered1640worker2 # [ 0.731417] Demotion targets for Node 0: null1641worker1 # [ 0.688915] Asymmetric key parser 'x509' registered1642client # [ 0.716538] registered taskstats version 11643worker2 # [ 0.732261] Key type .fscrypt registered1644worker1 # [ 0.689797] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1645worker2 # [ 0.732935] Key type fscrypt-provisioning registered1646client # [ 0.717495] Loading compiled-in X.509 certificates1647lb # [ 0.699968] drop_monitor: Initializing network drop monitor service1648lb # [ 0.701123] NET: Registered PF_INET6 protocol family1649worker2 # [ 0.735521] ima: No TPM chip found, activating TPM-bypass!1650worker2 # [ 0.736450] ima: Allocated hash algorithm: sha11651worker2 # [ 0.737281] ima: No architecture policies found1652worker1 # [ 0.695161] io scheduler mq-deadline registered1653lb # [ 0.705229] Segment Routing with IPv61654worker1 # [ 0.695976] io scheduler kyber registered1655lb # [ 0.705984] In-situ OAM (IOAM) with IPv61656worker1 # [ 0.698620] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1657worker2 # [ 0.741702] PM: Magic number: 10:622:9321658lb # [ 0.708853] IPI shorthand broadcast: enabled1659worker1 # [ 0.699942] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1660worker2 # [ 0.745653] RAS: Correctable Errors collector initialized.1661worker1 # [ 0.705890] Linux agpgart interface v0.1031662worker1 # [ 0.706728] ACPI: bus type drm_connector registered1663client # [ 0.733874] Demotion targets for Node 0: null1664lb # [ 0.717214] sched_clock: Marking stable (582015881, 134517512)->(795924640, -79391247)1665client # [ 0.737414] Key type .fscrypt registered1666client # [ 0.738088] Key type fscrypt-provisioning registered1667lb # [ 0.720731] registered taskstats version 11668worker2 # [ 0.754410] clk: Disabling unused clocks1669worker1 # [ 0.711514] usbcore: registered new interface driver usbserial_generic1670client # [ 0.739037] ima: No TPM chip found, activating TPM-bypass!1671lb # [ 0.721708] Loading compiled-in X.509 certificates1672worker2 # [ 0.755077] PM: genpd: Disabling unused power domains1673worker1 # [ 0.712565] usbserial: USB Serial support registered for generic1674client # [ 0.741366] ima: Allocated hash algorithm: sha11675client # [ 0.742189] ima: No architecture policies found1676worker1 # [ 0.716031] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1677client # [ 0.744378] PM: Magic number: 10:622:9321678worker1 # [ 0.716031] Please enable it if your BIOS has the CPPC option.1679worker1 # [ 0.718416] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1680client # [ 0.745914] RAS: Correctable Errors collector initialized.1681worker1 # [ 0.719832] drop_monitor: Initializing network drop monitor service1682worker1 # [ 0.721139] NET: Registered PF_INET6 protocol family1683worker1 # [ 0.726360] Segment Routing with IPv61684worker1 # [ 0.727093] In-situ OAM (IOAM) with IPv61685client # [ 0.755011] clk: Disabling unused clocks1686worker1 # [ 0.729307] IPI shorthand broadcast: enabled1687lb # [ 0.740547] Demotion targets for Node 0: null1688lb # [ 0.741415] Key type .fscrypt registered1689client # [ 0.759371] PM: genpd: Disabling unused power domains1690lb # [ 0.742101] Key type fscrypt-provisioning registered1691lb # [ 0.745652] ima: No TPM chip found, activating TPM-bypass!1692lb # [ 0.746552] ima: Allocated hash algorithm: sha11693lb # [ 0.747303] ima: No architecture policies found1694worker1 # [ 0.737649] sched_clock: Marking stable (599014936, 137999240)->(816901415, -79887239)1695lb # [ 0.749739] PM: Magic number: 10:622:9321696lb # [ 0.751358] RAS: Correctable Errors collector initialized.1697worker1 # [ 0.743150] registered taskstats version 11698worker1 # [ 0.744165] Loading compiled-in X.509 certificates1699lb # [ 0.761262] clk: Disabling unused clocks1700lb # [ 0.763551] PM: genpd: Disabling unused power domains1701worker1 # [ 0.761037] Demotion targets for Node 0: null1702worker1 # [ 0.763070] Key type .fscrypt registered1703worker1 # [ 0.763796] Key type fscrypt-provisioning registered1704worker1 # [ 0.764797] ima: No TPM chip found, activating TPM-bypass!1705worker1 # [ 0.769046] ima: Allocated hash algorithm: sha11706worker1 # [ 0.769889] ima: No architecture policies found1707worker1 # [ 0.772028] PM: Magic number: 10:622:9321708worker1 # [ 0.773783] RAS: Correctable Errors collector initialized.1709worker1 # [ 0.783715] clk: Disabling unused clocks1710worker1 # [ 0.786029] PM: genpd: Disabling unused power domains1711client # [ 0.920616] Freeing initrd memory: 29096K1712worker2 # [ 0.937474] Freeing initrd memory: 29092K1713client # [ 0.923777] Freeing unused decrypted memory: 2028K1714worker2 # [ 0.940878] Freeing unused decrypted memory: 2028K1715client # [ 0.926389] Freeing unused kernel image (initmem) memory: 3644K1716client # [ 0.927514] Write protecting the kernel read-only data: 32768k1717worker2 # [ 0.943820] Freeing unused kernel image (initmem) memory: 3644K1718worker2 # [ 0.945002] Write protecting the kernel read-only data: 32768k1719client # [ 0.929427] Freeing unused kernel image (text/rodata gap) memory: 1208K1720client # [ 0.930975] Freeing unused kernel image (rodata/data gap) memory: 752K1721worker2 # [ 0.947009] Freeing unused kernel image (text/rodata gap) memory: 1208K1722worker2 # [ 0.948604] Freeing unused kernel image (rodata/data gap) memory: 752K1723lb # [ 0.936237] Freeing initrd memory: 29092K1724lb # [ 0.939501] Freeing unused decrypted memory: 2028K1725lb # [ 0.942204] Freeing unused kernel image (initmem) memory: 3644K1726lb # [ 0.943319] Write protecting the kernel read-only data: 32768k1727lb # [ 0.945247] Freeing unused kernel image (text/rodata gap) memory: 1208K1728lb # [ 0.946845] Freeing unused kernel image (rodata/data gap) memory: 752K1729client # [ 0.981879] x86/mm: Checked W+X mappings: passed, no W+X pages found.1730client # [ 0.982922] Run /init as init process1731worker1 # [ 0.956047] Freeing initrd memory: 29092K1732worker2 # [ 0.999596] x86/mm: Checked W+X mappings: passed, no W+X pages found.1733worker2 # [ 1.000796] Run /init as init process1734worker1 # [ 0.959282] Freeing unused decrypted memory: 2028K1735worker1 # [ 0.962021] Freeing unused kernel image (initmem) memory: 3644K1736worker1 # [ 0.963154] Write protecting the kernel read-only data: 32768k1737worker1 # [ 0.965101] Freeing unused kernel image (text/rodata gap) memory: 1208K1738client # [ 0.992971] systemd[1]: Inserted module 'autofs4'1739worker1 # [ 0.966669] Freeing unused kernel image (rodata/data gap) memory: 752K1740worker2 # [ 1.011877] systemd[1]: Inserted module 'autofs4'1741client # [ 1.009066] fuse: init (API version 7.45)1742worker2 # [ 1.029153] fuse: init (API version 7.45)1743client # [ 1.014914] ACPI: \_SB_.GSIG: Enabled at IRQ 221744lb # [ 0.998211] x86/mm: Checked W+X mappings: passed, no W+X pages found.1745lb # [ 0.999271] Run /init as init process1746client # [ 1.017245] ACPI: \_SB_.GSIH: Enabled at IRQ 231747worker2 # [ 1.035176] ACPI: \_SB_.GSIG: Enabled at IRQ 221748client # [ 1.020924] ACPI: \_SB_.GSIE: Enabled at IRQ 201749worker2 # [ 1.037967] ACPI: \_SB_.GSIH: Enabled at IRQ 231750client # [ 1.023830] ACPI: \_SB_.GSIF: Enabled at IRQ 211751worker2 # [ 1.041155] ACPI: \_SB_.GSIE: Enabled at IRQ 201752lb # [ 1.009698] systemd[1]: Inserted module 'autofs4'1753worker2 # [ 1.043674] ACPI: \_SB_.GSIF: Enabled at IRQ 211754lb # [ 1.026132] fuse: init (API version 7.45)1755worker1 # [ 1.018086] x86/mm: Checked W+X mappings: passed, no W+X pages found.1756worker1 # [ 1.019233] Run /init as init process1757lb # [ 1.032028] ACPI: \_SB_.GSIG: Enabled at IRQ 221758lb # [ 1.034485] ACPI: \_SB_.GSIH: Enabled at IRQ 231759client # [ 1.052094] systemd[1]: Successfully made /usr/ read-only.1760lb # [ 1.037488] ACPI: \_SB_.GSIE: Enabled at IRQ 201761worker2 # [ 1.070478] systemd[1]: Successfully made /usr/ read-only.1762worker1 # [ 1.030058] systemd[1]: Inserted module 'autofs4'1763lb # [ 1.040116] ACPI: \_SB_.GSIF: Enabled at IRQ 211764worker1 # [ 1.046298] fuse: init (API version 7.45)1765worker1 # [ 1.052320] ACPI: \_SB_.GSIG: Enabled at IRQ 221766worker1 # [ 1.055080] ACPI: \_SB_.GSIH: Enabled at IRQ 231767worker1 # [ 1.058269] ACPI: \_SB_.GSIE: Enabled at IRQ 201768lb # [ 1.067748] systemd[1]: Successfully made /usr/ read-only.1769worker1 # [ 1.060788] ACPI: \_SB_.GSIF: Enabled at IRQ 211770worker1 # [ 1.089790] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.388727] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772worker2 # [ 1.406926] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1773client # [ 1.400931] systemd[1]: Detected virtualization kvm.1774worker2 # [ 1.418725] systemd[1]: Detected virtualization kvm.1775client # [ 1.403122] systemd[1]: Detected architecture x86-64.1776client # [ 1.405275] systemd[1]: Running in initrd.1777worker2 # [ 1.420813] systemd[1]: Detected architecture x86-64.1778worker2 # [ 1.422883] systemd[1]: Running in initrd.1779client # [ 1.407823] systemd[1]: Initializing machine ID from random generator.1780worker2 # [ 1.425305] systemd[1]: Initializing machine ID from random generator.1781client # [ 1.410723] systemd[1]: Hostname set to <client>.1782worker2 # [ 1.428233] systemd[1]: Hostname set to <worker2>.1783lb # [ 1.403397] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784lb # [ 1.408259] systemd[1]: Detected virtualization kvm.1785lb # [ 1.409112] systemd[1]: Detected architecture x86-64.1786lb # [ 1.409965] systemd[1]: Running in initrd.1787lb # [ 1.411152] systemd[1]: Initializing machine ID from random generator.1788lb # [ 1.412367] systemd[1]: Hostname set to <lb>.1789worker1 # [ 1.426395] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker1 # [ 1.438539] systemd[1]: Detected virtualization kvm.1791worker1 # [ 1.440744] systemd[1]: Detected architecture x86-64.1792worker1 # [ 1.442891] systemd[1]: Running in initrd.1793worker1 # [ 1.445505] systemd[1]: Initializing machine ID from random generator.1794worker1 # [ 1.448487] systemd[1]: Hostname set to <worker1>.1795lb # [ 1.608125] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796client # [ 1.633074] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797worker2 # [ 1.649269] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798lb # [ 1.651024] systemd[1]: Queued start job for default target Initrd Default Target.1799lb # [ 1.654952] systemd[1]: Created slice Slice /system/modprobe.1800lb # [ 1.656109] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1801lb # [ 1.657470] systemd[1]: Expecting device /dev/disk/by-label/nixos...1802lb # [ 1.658595] systemd[1]: Reached target Path Units.1803lb # [ 1.659440] systemd[1]: Reached target Slice Units.1804lb # [ 1.660314] systemd[1]: Reached target Swaps.1805lb # [ 1.661115] systemd[1]: Reached target Timer Units.1806lb # [ 1.662108] systemd[1]: Listening on D-Bus System Message Bus Socket.1807lb # [ 1.663291] systemd[1]: Listening on Journal Socket (/dev/log).1808lb # [ 1.664423] systemd[1]: Listening on Journal Sockets.1809lb # [ 1.665413] systemd[1]: Listening on udev Control Socket.1810lb # [ 1.666422] systemd[1]: Listening on udev Kernel Socket.1811lb # [ 1.667364] systemd[1]: Reached target Socket Units.1812worker2 # [ 1.702100] systemd[1]: Queued start job for default target Initrd Default Target.1813lb # [ 1.669257] systemd[1]: Starting Create List of Static Device Nodes...1814client # [ 1.687891] systemd[1]: Queued start job for default target Initrd Default Target.1815worker2 # [ 1.706767] systemd[1]: Created slice Slice /system/modprobe.1816lb # [ 1.673603] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1817client # [ 1.691672] systemd[1]: Created slice Slice /system/modprobe.1818worker2 # [ 1.707898] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1819client # [ 1.692803] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1820worker2 # [ 1.709238] systemd[1]: Expecting device /dev/disk/by-label/nixos...1821client # [ 1.694164] systemd[1]: Expecting device /dev/disk/by-label/nixos...1822worker2 # [ 1.710285] systemd[1]: Reached target Path Units.1823client # [ 1.695242] systemd[1]: Reached target Path Units.1824worker2 # [ 1.711115] systemd[1]: Reached target Slice Units.1825client # [ 1.696081] systemd[1]: Reached target Slice Units.1826lb # [ 1.678604] systemd[1]: Starting Load Kernel Module configfs...1827worker2 # [ 1.711952] systemd[1]: Reached target Swaps.1828client # [ 1.696949] systemd[1]: Reached target Swaps.1829worker2 # [ 1.712742] systemd[1]: Reached target Timer Units.1830client # [ 1.697722] systemd[1]: Reached target Timer Units.1831worker2 # [ 1.713690] systemd[1]: Listening on D-Bus System Message Bus Socket.1832client # [ 1.698715] systemd[1]: Listening on D-Bus System Message Bus Socket.1833worker2 # [ 1.714872] systemd[1]: Listening on Journal Socket (/dev/log).1834client # [ 1.699890] systemd[1]: Listening on Journal Socket (/dev/log).1835worker2 # [ 1.716002] systemd[1]: Listening on Journal Sockets.1836client # [ 1.700992] systemd[1]: Listening on Journal Sockets.1837worker2 # [ 1.716983] systemd[1]: Listening on udev Control Socket.1838client # [ 1.701944] systemd[1]: Listening on udev Control Socket.1839worker2 # [ 1.717984] systemd[1]: Listening on udev Kernel Socket.1840client # [ 1.702927] systemd[1]: Listening on udev Kernel Socket.1841worker2 # [ 1.718919] systemd[1]: Reached target Socket Units.1842client # [ 1.703854] systemd[1]: Reached target Socket Units.1843worker1 # [ 1.675511] systemd[1]: bpf-restrict-fs: LSM BPF program attached1844worker2 # [ 1.720776] systemd[1]: Starting Create List of Static Device Nodes...1845client # [ 1.705708] systemd[1]: Starting Create List of Static Device Nodes...1846worker2 # [ 1.724476] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1847client # [ 1.709425] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1848lb # [ 1.692625] systemd[1]: Starting Journal Service...1849client # [ 1.715448] systemd[1]: Starting Load Kernel Module configfs...1850worker2 # [ 1.732583] systemd[1]: Starting Load Kernel Module configfs...1851lb # [ 1.703634] systemd[1]: Starting Load Kernel Modules...1852client # [ 1.728495] systemd[1]: Starting Journal Service...1853worker2 # [ 1.746468] systemd[1]: Starting Journal Service...1854client # [ 1.738430] systemd[1]: Starting Load Kernel Modules...1855lb # [ 1.712603] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1856worker2 # [ 1.759473] systemd[1]: Starting Load Kernel Modules...1857lb # [ 1.732321] netfs: FS-Cache loaded1858lb # [ 1.737030] 9pnet: Installing 9P2000 support1859client # [ 1.744411] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1860worker1 # [ 1.729366] systemd[1]: Queued start job for default target Initrd Default Target.1861worker1 # [ 1.733449] systemd[1]: Created slice Slice /system/modprobe.1862worker2 # [ 1.764683] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1863worker1 # [ 1.734593] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1864worker1 # [ 1.735975] systemd[1]: Expecting device /dev/disk/by-label/nixos...1865worker1 # [ 1.737083] systemd[1]: Reached target Path Units.1866lb # [ 1.746646] systemd[1]: Starting Coldplug All udev Devices...1867worker1 # [ 1.737976] systemd[1]: Reached target Slice Units.1868worker1 # [ 1.738915] systemd[1]: Reached target Swaps.1869worker2 # [ 1.782280] netfs: FS-Cache loaded1870client # [ 1.766791] netfs: FS-Cache loaded1871worker1 # [ 1.739712] systemd[1]: Reached target Timer Units.1872worker1 # [ 1.740644] systemd[1]: Listening on D-Bus System Message Bus Socket.1873worker1 # [ 1.741844] systemd[1]: Listening on Journal Socket (/dev/log).1874worker1 # [ 1.743054] systemd[1]: Listening on Journal Sockets.1875worker1 # [ 1.744114] systemd[1]: Listening on udev Control Socket.1876client # [ 1.771514] 9pnet: Installing 9P2000 support1877worker1 # [ 1.745170] systemd[1]: Listening on udev Kernel Socket.1878worker1 # [ 1.746102] systemd[1]: Reached target Socket Units.1879worker2 # [ 1.789276] 9pnet: Installing 9P2000 support1880worker1 # [ 1.747862] systemd[1]: Starting Create List of Static Device Nodes...1881worker1 # [ 1.752084] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1882lb # [ 1.761727] systemd[1]: Finished Create List of Static Device Nodes.1883client # [ 1.780572] systemd[1]: Starting Coldplug All udev Devices...1884lb # [ 1.768178] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1885worker1 # [ 1.760084] systemd[1]: Starting Load Kernel Module configfs...1886worker2 # [ 1.804134] systemd[1]: Starting Coldplug All udev Devices...1887worker2 # [ 1.807137] systemd-journald[67]: Collecting audit messages is disabled.1888lb # [ 1.776979] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1889client # [ 1.796481] systemd[1]: Finished Create List of Static Device Nodes.1890lb # [ 1.780282] systemd-journald[67]: Collecting audit messages is disabled.1891worker1 # [ 1.774268] systemd[1]: Starting Journal Service...1892lb # [ 1.783974] systemd[1]: modprobe@configfs.service: Deactivated successfully.1893client # [ 1.801936] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1894worker2 # [ 1.818820] systemd[1]: Finished Create List of Static Device Nodes.1895lb # [ 1.788797] systemd[1]: Finished Load Kernel Module configfs.1896worker2 # [ 1.824004] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897client # [ 1.809749] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1898lb # [ 1.793065] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1899client # [ 1.814764] systemd[1]: modprobe@configfs.service: Deactivated successfully.1900worker1 # [ 1.787972] systemd[1]: Starting Load Kernel Modules...1901worker2 # [ 1.831826] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1902lb # [ 1.798317] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1903client # [ 1.817604] systemd-journald[67]: Collecting audit messages is disabled.1904client # [ 1.820744] systemd[1]: Finished Load Kernel Module configfs.1905worker2 # [ 1.837792] systemd[1]: modprobe@configfs.service: Deactivated successfully.1906lb # [ 1.805782] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1907lb # [ 1.807267] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1908client # [ 1.827872] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1909worker2 # [ 1.844760] systemd[1]: Finished Load Kernel Module configfs.1910client # [ 1.832724] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1911worker2 # [ 1.849117] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1912worker1 # [ 1.794129] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1913worker2 # [ 1.851627] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1914client # [ 1.838432] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1915client # [ 1.842393] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1916worker1 # [ 1.816886] netfs: FS-Cache loaded1917worker2 # [ 1.860539] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1918worker2 # [ 1.862183] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1919worker1 # [ 1.822248] 9pnet: Installing 9P2000 support1920lb # [ 1.840102] systemd[1]: Finished Load Kernel Modules.1921lb # [ 1.847769] systemd[1]: Starting Apply Kernel Variables...1922worker1 # [ 1.839110] systemd[1]: Starting Coldplug All udev Devices...1923client # [ 1.872864] systemd[1]: Finished Load Kernel Modules.1924worker2 # [ 1.889991] systemd[1]: Finished Load Kernel Modules.1925lb # [ 1.859636] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1926worker1 # [ 1.850173] systemd-journald[67]: Collecting audit messages is disabled.1927client # [ 1.878541] systemd[1]: Starting Apply Kernel Variables...1928worker1 # [ 1.854507] systemd[1]: Finished Create List of Static Device Nodes.1929worker2 # [ 1.898599] systemd[1]: Starting Apply Kernel Variables...1930lb # [ 1.867799] systemd[1]: Starting Create Static Device Nodes in /dev...1931worker1 # [ 1.860617] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1932client # [ 1.889472] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1933worker2 # [ 1.908550] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1934worker1 # [ 1.868448] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1935client # [ 1.898547] systemd[1]: Starting Create Static Device Nodes in /dev...1936worker1 # [ 1.874404] systemd[1]: modprobe@configfs.service: Deactivated successfully.1937worker2 # [ 1.918607] systemd[1]: Starting Create Static Device Nodes in /dev...1938lb # [ 1.888675] systemd[1]: Finished Apply Kernel Variables.1939worker1 # [ 1.880069] systemd[1]: Finished Load Kernel Module configfs.1940worker2 # [ 1.792335] systemd-modules-load[68]: Inserted module 'dm_mod'1941worker2 # [ 1.794169] systemd-modules-load[68]: Inserted module 'virtio_balloon'1942worker1 # [ 1.885440] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1943worker1 # [ 1.887897] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1944worker2 # [ 1.798267] systemd-modules-load[68]: Inserted module 'virtio_gpu'1945client # [ 1.917908] systemd[1]: Finished Apply Kernel Variables.1946worker2 # [ 1.934568] systemd[1]: Started Journal Service.1947lb # [ 1.902012] systemd[1]: Started Journal Service.1948worker1 # [ 1.895175] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1949lb # [ 1.770115] systemd-modules-load[68]: Inserted module 'dm_mod'1950lb # [ 1.772337] systemd-modules-load[68]: Inserted module 'virtio_balloon'1951worker2 # [ 1.805035] systemd[1]: Finished Apply Kernel Variables.1952lb # [ 1.773405] systemd-modules-load[68]: Inserted module 'virtio_gpu'1953worker1 # [ 1.899141] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1954lb # [ 1.776367] systemd[1]: Finished Create Static Device Nodes in /dev.1955lb # [ 1.777556] systemd[1]: Reached target Preparation for Local File Systems.1956lb # [ 1.779356] systemd[1]: Reached target Local File Systems.1957client # [ 1.932938] systemd[1]: Finished Create Static Device Nodes in /dev.1958lb # [ 1.782287] systemd[1]: Starting Create System Files and Directories...1959client # [ 1.801213] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.936711] systemd[1]: Started Journal Service.1960client # 1961client # [ 1.803278] systemd-modules-load[68]: Inserted module 'virtio_balloon'1962lb # [ 1.786219] systemd[1]: Starting Rule-based Manager for Device Events and Files...1963client # [ 1.804349] systemd-modules-load[68]: Inserted module 'virtio_gpu'1964worker2 # [ 1.822146] systemd[1]: Finished Create Static Device Nodes in /dev.1965worker2 # [ 1.823132] systemd[1]: Reached target Preparation for Local File Systems.1966worker2 # [ 1.824124] systemd[1]: Reached target Local File Systems.1967worker2 # [ 1.829081] systemd[1]: Starting Create System Files and Directories...1968client # [ 1.814109] systemd[1]: Reached target Preparation for Local File Systems.1969worker2 # [ 1.830208] systemd[1]: Starting Rule-based Manager for Device Events and Files...1970client # [ 1.815155] systemd[1]: Reached target Local File Systems.1971client # [ 1.818101] systemd[1]: Starting Create System Files and Directories...1972worker1 # [ 1.926557] systemd[1]: Finished Load Kernel Modules.1973client # [ 1.822824] systemd[1]: Starting Rule-based Manager for Device Events and Files...1974worker1 # [ 1.935071] systemd[1]: Starting Apply Kernel Variables...1975lb # [ 1.815258] systemd[1]: Finished Create System Files and Directories.1976worker2 # [ 1.853208] systemd[1]: Finished Create System Files and Directories.1977worker1 # [ 1.946129] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1978worker1 # [ 1.954390] systemd[1]: Starting Create Static Device Nodes in /dev...1979client # [ 1.848183] systemd[1]: Finished Create System Files and Directories.1980lb # [ 1.845220] systemd-udevd[76]: Using default interface naming scheme 'v261'.1981worker2 # [ 1.878756] systemd-udevd[83]: Using default interface naming scheme 'v261'.1982worker1 # [ 1.834235] systemd-modules-load[68]: Inserted module 'dm_mod'1983worker1 # [ 1.835465] systemd-modules-load[68]: Inserted module 'virtio_balloon'1984worker1 # [ 1.974436] systemd[1]: Started Journal Service.1985worker1 # [ 1.837352] systemd-modules-load[68]: Inserted module 'virtio_gpu'1986client # [ 1.870607] systemd-udevd[83]: Using default interface naming scheme 'v261'.1987worker1 # [ 1.845385] systemd[1]: Finished Apply Kernel Variables.1988worker1 # [ 1.855089] systemd[1]: Finished Create Static Device Nodes in /dev.1989worker2 # [ 1.903781] systemd[1]: Started Rule-based Manager for Device Events and Files.1990worker1 # [ 1.857790] systemd[1]: Reached target Preparation for Local File Systems.1991worker1 # [ 1.858885] systemd[1]: Reached target Local File Systems.1992lb # [ 1.872080] systemd[1]: Started Rule-based Manager for Device Events and Files.1993worker1 # [ 1.860061] systemd[1]: Starting Create System Files and Directories...1994client # [ 1.894880] systemd[1]: Started Rule-based Manager for Device Events and Files.1995worker1 # [ 1.864948] systemd[1]: Starting Rule-based Manager for Device Events and Files...1996worker1 # [ 1.890294] systemd[1]: Finished Create System Files and Directories.1997lb # [ 1.923075] systemd[1]: Finished Coldplug All udev Devices.1998worker2 # [ 1.956755] systemd[1]: Finished Coldplug All udev Devices.1999client # [ 1.942351] systemd[1]: Finished Coldplug All udev Devices.2000lb # [ 1.925379] systemd[1]: Reached target System Initialization.
loading 2,722 hidden lines…
4723client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4724client # farm-top> LOG-farm-top-t14725worker2 # [ 46.382131] nix-daemon[859]: reaped child process 968, status = succeeded4726worker2 # [ 46.523194] nix-daemon[859]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4727worker2 # [ 46.535662] nix-daemon[859]: reaped child process 987, status = succeeded4728lb # [ 46.511271] niks3-server[1003]: 2026/09/22 10:56:17 INFO Received uploads request method=POST path=/api/pending_closures4729worker2 # [ 46.554989] nix-grpc-daemon[872]: time=2026-09-22T10:56:18.710Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4730worker2 # [ 46.556452] nix-grpc-daemon[872]: time=2026-09-22T10:56:18.712Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4731lb # [ 46.543272] niks3-server[1003]: 2026/09/22 10:56:17 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4732lb # [ 46.562157] niks3-server[1003]: 2026/09/22 10:56:17 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4733lb # [ 46.570792] niks3-server[1003]: 2026/09/22 10:56:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4734worker2 # [ 46.609030] nix-grpc-daemon[872]: time=2026-09-22T10:56:18.764Z level=INFO msg="Uploading 1 narinfos"4735lb # [ 46.576238] niks3-server[1003]: 2026/09/22 10:56:18 INFO Signed narinfos id=6 count=14736lb # [ 46.580267] niks3-server[1003]: 2026/09/22 10:56:18 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4737lb # [ 46.588456] niks3-server[1003]: 2026/09/22 10:56:18 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4738lb # [ 46.594446] niks3-server[1003]: 2026/09/22 10:56:18 INFO Completed upload id=64739worker2 # [ 46.628068] nix-grpc-daemon[872]: time=2026-09-22T10:56:18.783Z level=INFO msg="Upload complete. (252ms)"4740lb # [ 46.596048] niks3-server[1003]: 2026/09/22 10:56:18 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4741worker2 # [ 46.635524] nix-grpc-daemon[790]: ts=2026-09-22T10:56:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14742worker2 # [ 46.647750] nix-daemon[859]: reaped child process 966, status = succeeded4743client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 7.32 seconds)4744client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4745client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4746client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4747client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4748client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4749client # copying 3 paths...4750client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4751client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4752client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4753client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4754(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.86 seconds)4755subtest: repeat is answered Cached by the scheduler without building4756worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4757worker1 # finding garbage collector roots...4758worker1 # deleting specified paths...4759worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4760worker1 # deleting unused links...4761worker1 # note: hard linking is currently saving 0.0 KiB4762worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.28 seconds)4763worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4764worker2 # finding garbage collector roots...4765worker2 # deleting specified paths...4766worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4767worker2 # deleting unused links...4768worker2 # note: hard linking is currently saving 0.0 KiB4769worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.24 seconds)4770client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24771client # don't know how to build these paths:4772client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4773worker1 # [ 47.912560] nix-grpc-daemon[784]: ts=2026-09-22T10:56:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418044774worker1 # [ 47.917957] nix-grpc-daemon[784]: ts=2026-09-22T10:56:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418044775worker1 # [ 47.943765] nix-grpc-daemon[784]: ts=2026-09-22T10:56:19Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14776client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.26 seconds)4777client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4778client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4779lb # [ 48.232378] envoy[921]: conn peer=[2001:db8:1::1]:59960 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=26214780lb # [ 48.236173] envoy[921]: conn peer=[2001:db8:1::1]:59970 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=363 ms=604781lb # [ 48.239417] envoy[921]: conn peer=[2001:db8:1::1]:59980 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134782lb: must succeed: curl -sf localhost:9901/clusters4783lb # [ 48.242656] envoy[921]: conn peer=[2001:db8:1::1]:59988 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=94784lb # [ 48.247362] envoy[921]: conn peer=[2001:db8:1::1]:60004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9537 tx=4444 ms=70994785lb # [ 48.250876] envoy[921]: conn peer=[2001:db8:1::1]:51422 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=494786lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4787lb: must succeed: curl -sf localhost:9901/clusters4788lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4789worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4790worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4791(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.22 seconds)4792subtest: inputs already in the cache are not sent to the scheduler4793lb: must succeed: curl -sf localhost:9901/clusters4794lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4795lb: must succeed: curl -sf localhost:9901/clusters4796lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4797worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4798worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4799client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24800worker1 # [ 48.526093] nix-grpc-daemon[784]: E0000 00:00:1790074580.433835 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4801worker1 # [ 48.537877] nix-grpc-daemon[784]: E0000 00:00:1790074580.446083 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4802worker1 # [ 48.542050] nix-grpc-daemon[784]: E0000 00:00:1790074580.448771 826 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4803worker1 # [ 48.551281] nix-grpc-daemon[784]: E0000 00:00:1790074580.459819 826 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4804client # don't know how to build these paths:4805client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4806worker2 # [ 48.761098] nix-grpc-daemon[790]: ts=2026-09-22T10:56:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570004807worker2 # [ 48.766916] nix-grpc-daemon[790]: ts=2026-09-22T10:56:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570004808worker2 # [ 48.773096] nix-grpc-daemon[872]: time=2026-09-22T10:56:20.926Z level=INFO msg="All 2 paths already cached"4809worker1 # [ 48.734796] nix-grpc-daemon[784]: ts=2026-09-22T10:56:20Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04810worker1 # [ 48.738999] nix-grpc-daemon[784]: ts=2026-09-22T10:56:20Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14811worker2 # [ 48.824446] nix-grpc-daemon[790]: ts=2026-09-22T10:56:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570004812worker2 # [ 48.832913] nix-grpc-daemon[790]: ts=2026-09-22T10:56:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570004813worker2 # [ 48.840863] nix-grpc-daemon[872]: time=2026-09-22T10:56:20.996Z level=INFO msg="All 2 paths already cached"4814client # copying 1 paths to 'lb:50051'...4815worker2 # [ 48.846973] nix-grpc-daemon[790]: ts=2026-09-22T10:56:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570004816client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4817worker2 # [ 48.852718] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4818worker2 # [ 49.003350] nix-daemon[859]: accepted connection from pid 1004, user nix-grpc-daemon (trusted)4819worker2 # [ 49.016402] nix-daemon[859]: reaped child process 1007, status = succeeded4820worker2 # [ 49.162290] nix-daemon[859]: accepted connection from pid 1009, user nix-grpc-daemon (trusted)4821worker2 # [ 49.174647] nix-daemon[859]: reaped child process 1012, status = succeeded4822lb # [ 49.155619] niks3-server[1003]: 2026/09/22 10:56:20 INFO Received uploads request method=POST path=/api/pending_closures4823worker2 # [ 49.193879] nix-grpc-daemon[872]: time=2026-09-22T10:56:21.349Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4824worker2 # [ 49.195274] nix-grpc-daemon[872]: time=2026-09-22T10:56:21.349Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4825lb # [ 49.169403] niks3-server[1003]: 2026/09/22 10:56:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4826lb # [ 49.171252] niks3-server[1003]: 2026/09/22 10:56:20 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4827lb # [ 49.173572] niks3-server[1003]: 2026/09/22 10:56:20 INFO Signed narinfos id=7 count=14828worker2 # [ 49.207202] nix-grpc-daemon[872]: time=2026-09-22T10:56:21.362Z level=INFO msg="Uploading 1 narinfos"4829lb # [ 49.177478] niks3-server[1003]: 2026/09/22 10:56:20 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4830lb # [ 49.181153] niks3-server[1003]: 2026/09/22 10:56:20 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4831lb # [ 49.184803] niks3-server[1003]: 2026/09/22 10:56:20 INFO Completed upload id=74832worker2 # [ 49.218337] nix-grpc-daemon[872]: time=2026-09-22T10:56:21.374Z level=INFO msg="Upload complete. (359ms)"4833lb # [ 49.186561] niks3-server[1003]: 2026/09/22 10:56:20 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4834worker2 # [ 49.220444] nix-grpc-daemon[790]: ts=2026-09-22T10:56:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_in=7524835worker2 # [ 49.224226] nix-grpc-daemon[790]: ts=2026-09-22T10:56:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570004836worker2 # [ 49.230119] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4837worker2 # [ 49.232279] nix-daemon[859]: reaped child process 1002, status = succeeded4838worker2 # [ 49.237552] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)4839client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4840client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4841client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4842client # farm-top> LOG-farm-top-t1b4843worker2 # [ 50.853123] nix-daemon[859]: reaped child process 1016, status = succeeded4844worker2 # [ 50.989302] nix-daemon[859]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4845worker2 # [ 51.001211] nix-daemon[859]: reaped child process 1035, status = succeeded4846lb # [ 50.984548] niks3-server[1003]: 2026/09/22 10:56:22 INFO Received uploads request method=POST path=/api/pending_closures4847worker2 # [ 51.026659] nix-grpc-daemon[872]: time=2026-09-22T10:56:23.182Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4848worker2 # [ 51.028442] nix-grpc-daemon[872]: time=2026-09-22T10:56:23.183Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4849lb # [ 51.029328] niks3-server[1003]: 2026/09/22 10:56:22 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4850lb # [ 51.034946] niks3-server[1003]: 2026/09/22 10:56:22 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4851lb # [ 51.039721] niks3-server[1003]: 2026/09/22 10:56:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4852lb # [ 51.044437] niks3-server[1003]: 2026/09/22 10:56:22 INFO Signed narinfos id=8 count=14853worker2 # [ 51.082200] nix-grpc-daemon[872]: time=2026-09-22T10:56:23.237Z level=INFO msg="Uploading 1 narinfos"4854lb # [ 51.052037] niks3-server[1003]: 2026/09/22 10:56:22 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4855lb # [ 51.062348] niks3-server[1003]: 2026/09/22 10:56:22 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4856lb # [ 51.080587] niks3-server[1003]: 2026/09/22 10:56:22 INFO Completed upload id=84857worker2 # [ 51.114534] nix-grpc-daemon[872]: time=2026-09-22T10:56:23.269Z level=INFO msg="Upload complete. (268ms)"4858lb # [ 51.087563] niks3-server[1003]: 2026/09/22 10:56:22 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4859worker2 # [ 51.122312] nix-grpc-daemon[790]: ts=2026-09-22T10:56:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14860worker2 # [ 51.135852] nix-daemon[859]: reaped child process 1014, status = succeeded4861client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.60 seconds)4862client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4863client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4864lb: must succeed: curl -sf localhost:9901/clusters4865lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4866lb: must succeed: curl -sf localhost:9901/clusters4867lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4868worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4869worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4870(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.05 seconds)4871subtest: outputs read back through any worker carry the cache signature4872client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag sig >&24873worker1 # [ 51.670943] nix-grpc-daemon[784]: ts=2026-09-22T10:56:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418044874client # don't know how to build these paths:4875client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4876worker1 # [ 51.679467] nix-grpc-daemon[784]: ts=2026-09-22T10:56:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418044877worker1 # [ 51.698625] nix-grpc-daemon[784]: ts=2026-09-22T10:56:23Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04878worker1 # [ 51.704686] nix-grpc-daemon[784]: ts=2026-09-22T10:56:23Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=14879worker1 # [ 51.709299] nix-grpc-daemon[784]: ts=2026-09-22T10:56:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418044880worker1 # [ 51.722299] nix-grpc-daemon[784]: ts=2026-09-22T10:56:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418044881client # copying 1 paths to 'lb:50051'...4882client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4883worker1 # [ 51.733708] nix-grpc-daemon[784]: ts=2026-09-22T10:56:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418044884worker1 # [ 51.742843] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4885worker1 # [ 51.765223] nix-grpc-daemon[784]: ts=2026-09-22T10:56:23Z level=info event=niks3_push_started pid=9194886worker1 # [ 52.021942] nix-grpc-daemon[919]: time=2026-09-22T10:56:23.927Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""4887worker1 # [ 52.228094] nix-daemon[866]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4888worker1 # [ 52.242728] nix-daemon[866]: reaped child process 934, status = succeeded4889worker1 # [ 52.389139] nix-daemon[866]: accepted connection from pid 936, user nix-grpc-daemon (trusted)4890worker1 # [ 52.401137] nix-daemon[866]: reaped child process 939, status = succeeded4891lb # [ 52.429966] niks3-server[1003]: 2026/09/22 10:56:23 INFO Received uploads request method=POST path=/api/pending_closures4892worker1 # [ 52.425388] nix-grpc-daemon[919]: time=2026-09-22T10:56:24.333Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4893worker1 # [ 52.427651] nix-grpc-daemon[919]: time=2026-09-22T10:56:24.336Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4894lb # [ 52.471954] niks3-server[1003]: 2026/09/22 10:56:23 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4895lb # [ 52.476191] niks3-server[1003]: 2026/09/22 10:56:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4896lb # [ 52.484441] niks3-server[1003]: 2026/09/22 10:56:23 INFO Signed narinfos id=9 count=14897worker1 # [ 52.474684] nix-grpc-daemon[919]: time=2026-09-22T10:56:24.383Z level=INFO msg="Uploading 1 narinfos"4898lb # [ 52.490484] niks3-server[1003]: 2026/09/22 10:56:23 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4899lb # [ 52.501973] niks3-server[1003]: 2026/09/22 10:56:23 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4900lb # [ 52.508664] niks3-server[1003]: 2026/09/22 10:56:23 INFO Completed upload id=94901worker1 # [ 52.498097] nix-grpc-daemon[919]: time=2026-09-22T10:56:24.406Z level=INFO msg="Upload complete. (450ms)"4902lb # [ 52.511765] niks3-server[1003]: 2026/09/22 10:56:23 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4903worker1 # [ 52.500568] nix-grpc-daemon[784]: ts=2026-09-22T10:56:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_in=5044904worker1 # [ 52.506640] nix-grpc-daemon[784]: ts=2026-09-22T10:56:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418044905worker1 # [ 52.511954] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4906worker1 # [ 52.514227] nix-daemon[866]: reaped child process 917, status = succeeded4907client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4908worker1 # [ 52.519692] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4909client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4910client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4911client # farm-a> LOG-farm-a-sig4912worker1 # [ 53.349917] nix-daemon[866]: reaped child process 943, status = succeeded4913worker1 # [ 53.476808] nix-daemon[866]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4914worker1 # [ 53.489108] nix-daemon[866]: reaped child process 962, status = succeeded4915lb # [ 53.518408] niks3-server[1003]: 2026/09/22 10:56:24 INFO Received uploads request method=POST path=/api/pending_closures4916worker1 # [ 53.514506] nix-grpc-daemon[919]: time=2026-09-22T10:56:25.422Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4917worker1 # [ 53.516259] nix-grpc-daemon[919]: time=2026-09-22T10:56:25.424Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4918lb # [ 53.556194] niks3-server[1003]: 2026/09/22 10:56:24 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4919lb # [ 53.570370] niks3-server[1003]: 2026/09/22 10:56:24 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4920lb # [ 53.580132] niks3-server[1003]: 2026/09/22 10:56:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4921worker1 # [ 53.573818] nix-grpc-daemon[919]: time=2026-09-22T10:56:25.481Z level=INFO msg="Uploading 1 narinfos"4922lb # [ 53.586554] niks3-server[1003]: 2026/09/22 10:56:25 INFO Signed narinfos id=10 count=14923lb # [ 53.590821] niks3-server[1003]: 2026/09/22 10:56:25 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4924lb # [ 53.599278] niks3-server[1003]: 2026/09/22 10:56:25 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4925lb # [ 53.603796] niks3-server[1003]: 2026/09/22 10:56:25 INFO Completed upload id=104926worker1 # [ 53.591084] nix-grpc-daemon[919]: time=2026-09-22T10:56:25.499Z level=INFO msg="Upload complete. (262ms)"4927lb # [ 53.606233] niks3-server[1003]: 2026/09/22 10:56:25 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4928worker1 # [ 53.597843] nix-grpc-daemon[784]: ts=2026-09-22T10:56:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14929worker1 # [ 53.608922] nix-grpc-daemon[784]: ts=2026-09-22T10:56:25Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04930worker1 # [ 53.612221] nix-grpc-daemon[784]: ts=2026-09-22T10:56:25Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=14931worker1 # [ 53.620697] nix-daemon[866]: reaped child process 941, status = succeeded4932worker1 # [ 53.652451] nix-grpc-daemon[784]: ts=2026-09-22T10:56:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418044933worker1 # [ 53.664871] nix-grpc-daemon[784]: ts=2026-09-22T10:56:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418044934worker1 # [ 53.677222] nix-grpc-daemon[919]: time=2026-09-22T10:56:25.585Z level=INFO msg="All 1 paths already cached"4935client # copying 1 paths to 'lb:50051'...4936client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4937worker1 # [ 53.685820] nix-grpc-daemon[784]: ts=2026-09-22T10:56:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418044938worker1 # [ 53.694390] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4939worker1 # [ 53.865344] nix-daemon[866]: accepted connection from pid 969, user nix-grpc-daemon (trusted)4940worker1 # [ 53.879627] nix-daemon[866]: reaped child process 972, status = succeeded4941worker1 # [ 54.026675] nix-daemon[866]: accepted connection from pid 974, user nix-grpc-daemon (trusted)4942worker1 # [ 54.039314] nix-daemon[866]: reaped child process 977, status = succeeded4943lb # [ 54.064627] niks3-server[1003]: 2026/09/22 10:56:25 INFO Received uploads request method=POST path=/api/pending_closures4944worker1 # [ 54.070687] nix-grpc-daemon[919]: time=2026-09-22T10:56:25.979Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4945worker1 # [ 54.072220] nix-grpc-daemon[919]: time=2026-09-22T10:56:25.980Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4946lb # [ 54.113074] niks3-server[1003]: 2026/09/22 10:56:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4947lb # [ 54.118685] niks3-server[1003]: 2026/09/22 10:56:25 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4948lb # [ 54.125653] niks3-server[1003]: 2026/09/22 10:56:25 INFO Signed narinfos id=11 count=14949worker1 # [ 54.116056] nix-grpc-daemon[919]: time=2026-09-22T10:56:26.024Z level=INFO msg="Uploading 1 narinfos"4950lb # [ 54.138624] niks3-server[1003]: 2026/09/22 10:56:25 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4951lb # [ 54.143095] niks3-server[1003]: 2026/09/22 10:56:25 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4952worker1 # [ 54.150731] nix-grpc-daemon[919]: time=2026-09-22T10:56:26.059Z level=INFO msg="Upload complete. (442ms)"4953lb # [ 54.163537] niks3-server[1003]: 2026/09/22 10:56:25 INFO Completed upload id=114954worker1 # [ 54.153390] nix-grpc-daemon[784]: ts=2026-09-22T10:56:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_in=6244955worker1 # [ 54.156731] nix-grpc-daemon[784]: ts=2026-09-22T10:56:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418044956worker1 # [ 54.164197] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4957worker1 # [ 54.165965] nix-daemon[866]: reaped child process 966, status = succeeded4958lb # [ 54.180648] niks3-server[1003]: 2026/09/22 10:56:25 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4959worker1 # [ 54.170889] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4960client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4961client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4962client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4963client # farm-b> LOG-farm-b-sig4964worker1 # [ 55.392517] nix-daemon[866]: reaped child process 981, status = succeeded4965worker1 # [ 55.517090] nix-daemon[866]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4966worker1 # [ 55.528271] nix-daemon[866]: reaped child process 1000, status = succeeded4967lb # [ 55.553616] niks3-server[1003]: 2026/09/22 10:56:26 INFO Received uploads request method=POST path=/api/pending_closures4968worker1 # [ 55.548105] nix-grpc-daemon[919]: time=2026-09-22T10:56:27.456Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4969worker1 # [ 55.549567] nix-grpc-daemon[919]: time=2026-09-22T10:56:27.458Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4970lb # [ 55.584171] niks3-server[1003]: 2026/09/22 10:56:27 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4971lb # [ 55.596893] niks3-server[1003]: 2026/09/22 10:56:27 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4972lb # [ 55.606954] niks3-server[1003]: 2026/09/22 10:56:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4973lb # [ 55.613110] niks3-server[1003]: 2026/09/22 10:56:27 INFO Signed narinfos id=12 count=14974worker1 # [ 55.603190] nix-grpc-daemon[919]: time=2026-09-22T10:56:27.511Z level=INFO msg="Uploading 1 narinfos"4975lb # [ 55.625655] niks3-server[1003]: 2026/09/22 10:56:27 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4976lb # [ 55.684170] niks3-server[1003]: 2026/09/22 10:56:27 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4977lb # [ 55.691655] niks3-server[1003]: 2026/09/22 10:56:27 INFO Completed upload id=124978worker1 # [ 55.679124] nix-grpc-daemon[919]: time=2026-09-22T10:56:27.587Z level=INFO msg="Upload complete. (301ms)"4979lb # [ 55.696664] niks3-server[1003]: 2026/09/22 10:56:27 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4980worker1 # [ 55.685631] nix-grpc-daemon[784]: ts=2026-09-22T10:56:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14981worker1 # [ 55.700260] nix-grpc-daemon[784]: ts=2026-09-22T10:56:27Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04982worker1 # [ 55.705698] nix-grpc-daemon[784]: ts=2026-09-22T10:56:27Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=14983worker1 # [ 55.715563] nix-daemon[866]: reaped child process 979, status = succeeded4984worker1 # [ 55.744264] nix-grpc-daemon[784]: ts=2026-09-22T10:56:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418044985worker1 # [ 55.756878] nix-grpc-daemon[784]: ts=2026-09-22T10:56:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418044986worker1 # [ 55.769698] nix-grpc-daemon[919]: time=2026-09-22T10:56:27.678Z level=INFO msg="All 2 paths already cached"4987client # copying 1 paths to 'lb:50051'...4988client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4989worker1 # [ 55.777840] nix-grpc-daemon[784]: ts=2026-09-22T10:56:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418044990worker1 # [ 55.788173] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4991worker1 # [ 55.955390] nix-daemon[866]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4992worker1 # [ 55.967977] nix-daemon[866]: reaped child process 1009, status = succeeded4993worker1 # [ 56.111959] nix-daemon[866]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4994worker1 # [ 56.124426] nix-daemon[866]: reaped child process 1014, status = succeeded4995lb # [ 56.148230] niks3-server[1003]: 2026/09/22 10:56:27 INFO Received uploads request method=POST path=/api/pending_closures4996worker1 # [ 56.139487] nix-grpc-daemon[919]: time=2026-09-22T10:56:28.048Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4997worker1 # [ 56.140959] nix-grpc-daemon[919]: time=2026-09-22T10:56:28.049Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4998lb # [ 56.171805] niks3-server[1003]: 2026/09/22 10:56:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4999worker1 # [ 56.165683] nix-grpc-daemon[919]: time=2026-09-22T10:56:28.074Z level=INFO msg="Uploading 1 narinfos"5000lb # [ 56.178230] niks3-server[1003]: 2026/09/22 10:56:27 INFO Signed narinfos id=13 count=15001lb # [ 56.181263] niks3-server[1003]: 2026/09/22 10:56:27 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst5002lb # [ 56.187503] niks3-server[1003]: 2026/09/22 10:56:27 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls5003lb # [ 56.195115] niks3-server[1003]: 2026/09/22 10:56:27 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5004lb # [ 56.201927] niks3-server[1003]: 2026/09/22 10:56:27 INFO Completed upload id=135005worker1 # [ 56.188993] nix-grpc-daemon[919]: time=2026-09-22T10:56:28.097Z level=INFO msg="Upload complete. (389ms)"5006worker1 # [ 56.191110] nix-grpc-daemon[784]: ts=2026-09-22T10:56:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_in=7525007lb # [ 56.204864] niks3-server[1003]: 2026/09/22 10:56:27 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo5008worker1 # [ 56.194419] nix-grpc-daemon[784]: ts=2026-09-22T10:56:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045009worker1 # [ 56.201157] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5010worker1 # [ 56.203351] nix-daemon[866]: reaped child process 1004, status = succeeded5011client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...5012client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5013worker1 # [ 56.208489] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5014client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5015client # farm-top> LOG-farm-top-sig5016worker1 # [ 57.423811] nix-daemon[866]: reaped child process 1018, status = succeeded5017worker1 # [ 57.544436] nix-daemon[866]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5018worker1 # [ 57.558434] nix-daemon[866]: reaped child process 1037, status = succeeded5019lb # [ 57.585436] niks3-server[1003]: 2026/09/22 10:56:29 INFO Received uploads request method=POST path=/api/pending_closures5020worker1 # [ 57.577644] nix-grpc-daemon[919]: time=2026-09-22T10:56:29.485Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5021worker1 # [ 57.579933] nix-grpc-daemon[919]: time=2026-09-22T10:56:29.488Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5022lb # [ 57.614821] niks3-server[1003]: 2026/09/22 10:56:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5023lb # [ 57.623465] niks3-server[1003]: 2026/09/22 10:56:29 INFO Signed narinfos id=14 count=15024worker1 # [ 57.612584] nix-grpc-daemon[919]: time=2026-09-22T10:56:29.521Z level=INFO msg="Uploading 1 narinfos"5025lb # [ 57.631498] niks3-server[1003]: 2026/09/22 10:56:29 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5026lb # [ 57.646233] niks3-server[1003]: 2026/09/22 10:56:29 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5027lb # [ 57.651521] niks3-server[1003]: 2026/09/22 10:56:29 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5028lb # [ 57.654047] niks3-server[1003]: 2026/09/22 10:56:29 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5029lb # [ 57.658125] niks3-server[1003]: 2026/09/22 10:56:29 INFO Completed upload id=145030worker1 # [ 57.645267] nix-grpc-daemon[919]: time=2026-09-22T10:56:29.553Z level=INFO msg="Upload complete. (230ms)"5031lb # [ 57.661318] niks3-server[1003]: 2026/09/22 10:56:29 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5032worker1 # [ 57.649383] nix-grpc-daemon[784]: ts=2026-09-22T10:56:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15033worker1 # [ 57.657385] nix-daemon[866]: reaped child process 1016, status = succeeded5034client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag sig >&2, in 6.23 seconds)5035client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag sig outPath5036client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag sig outPath, in 0.14 seconds)5037client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&ca-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5038worker1 # [ 57.964410] nix-grpc-daemon[784]: ts=2026-09-22T10:56:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:348145039worker1 # [ 57.970968] nix-grpc-daemon[784]: ts=2026-09-22T10:56:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:348145040client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&ca-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.22 seconds)5041client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&ca-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5042worker2 # [ 58.258349] nix-grpc-daemon[790]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:372465043lb # [ 58.233416] envoy[921]: conn peer=[2001:db8:1::1]:51424 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1918 ms=23735044lb # [ 58.234948] envoy[921]: conn peer=[2001:db8:1::1]:51434 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9567 tx=4406 ms=59925045client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5046client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5047client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5048client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5049worker2 # [ 58.303765] nix-grpc-daemon[790]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:372465050worker2 # [ 58.311596] nix-grpc-daemon[790]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5051worker2 # [ 58.324582] nix-grpc-daemon[790]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5052worker2 # [ 58.333610] nix-grpc-daemon[790]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5053client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&ca-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.27 seconds)5054client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5055client # finding garbage collector roots...5056client # removing stale temporary roots file "/nix/var/nix/temproots/868"5057client # removing stale temporary roots file "/nix/var/nix/temproots/889"5058client # removing stale temporary roots file "/nix/var/nix/temproots/904"5059client # removing stale temporary roots file "/nix/var/nix/temproots/811"5060client # removing stale temporary roots file "/nix/var/nix/temproots/860"5061client # removing stale temporary roots file "/nix/var/nix/temproots/881"5062client # removing stale temporary roots file "/nix/var/nix/temproots/848"5063client # removing stale temporary roots file "/nix/var/nix/temproots/827"5064client # deleting specified paths...5065client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5066client # deleting unused links...5067client # note: hard linking is currently saving 0.0 KiB5068worker2 # [ 58.736256] nix-grpc-daemon[790]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570005069worker1 # [ 58.697831] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418045070worker1 # [ 58.704356] nix-grpc-daemon[919]: time=2026-09-22T10:56:30.612Z level=INFO msg="All 1 paths already cached"5071worker1 # [ 58.709991] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418045072worker2 # [ 58.762526] nix-grpc-daemon[790]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:570005073client # copying 3 paths...5074worker2 # [ 58.771916] nix-grpc-daemon[790]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:570005075client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5076worker1 # [ 58.744181] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418045077worker1 # [ 58.749185] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_out=2325078worker2 # [ 58.797488] nix-grpc-daemon[790]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:570005079worker2 # [ 58.802237] nix-grpc-daemon[790]: ts=2026-09-22T10:56:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_out=1765080worker1 # [ 58.757835] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418045081worker1 # [ 58.762469] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_out=1285082client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5083client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5084client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.52 seconds)5085(finished: subtest: outputs read back through any worker carry the cache signature, in 7.39 seconds)5086subtest: two clients wanting the same drv build it once5087client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup5088client # [ 58.896472] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.5089client # Running as unit: dedup1.service; invocation ID: ba8c777fd34746d2bfd948a049edbb645090client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.03 seconds)5091client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup5092client # [ 59.070769] nix[956]: don't know how to build these paths:5093client # [ 59.071621] nix[956]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5094worker2 # [ 59.085515] nix-grpc-daemon[790]: ts=2026-09-22T10:56:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570005095worker1 # [ 59.049330] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05096worker1 # [ 59.054330] nix-grpc-daemon[784]: ts=2026-09-22T10:56:30Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=15097worker2 # [ 59.143354] nix-grpc-daemon[790]: ts=2026-09-22T10:56:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005098worker2 # [ 59.153196] nix-grpc-daemon[790]: ts=2026-09-22T10:56:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570005099client # [ 59.146130] nix[956]: copying 1 paths to 'lb:50051'...5100client # [ 59.147594] nix[956]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5101worker2 # [ 59.162107] nix-grpc-daemon[790]: ts=2026-09-22T10:56:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570005102worker2 # [ 59.170839] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5103worker2 # [ 59.323071] nix-daemon[859]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)5104worker2 # [ 59.335822] nix-daemon[859]: reaped child process 1045, status = succeeded5105worker2 # [ 59.488293] nix-daemon[859]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)5106worker2 # [ 59.500082] nix-daemon[859]: reaped child process 1050, status = succeeded5107lb # [ 59.481357] niks3-server[1003]: 2026/09/22 10:56:30 INFO Received uploads request method=POST path=/api/pending_closures5108worker2 # [ 59.523300] nix-grpc-daemon[872]: time=2026-09-22T10:56:31.678Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5109worker2 # [ 59.524646] nix-grpc-daemon[872]: time=2026-09-22T10:56:31.678Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5110lb # [ 59.508734] niks3-server[1003]: 2026/09/22 10:56:30 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5111lb # [ 59.516696] niks3-server[1003]: 2026/09/22 10:56:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5112lb # [ 59.522339] niks3-server[1003]: 2026/09/22 10:56:30 INFO Signed narinfos id=15 count=15113worker2 # [ 59.560850] nix-grpc-daemon[872]: time=2026-09-22T10:56:31.716Z level=INFO msg="Uploading 1 narinfos"5114lb # [ 59.531370] niks3-server[1003]: 2026/09/22 10:56:30 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5115lb # [ 59.544211] niks3-server[1003]: 2026/09/22 10:56:30 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5116lb # [ 59.548982] niks3-server[1003]: 2026/09/22 10:56:30 INFO Completed upload id=155117worker2 # [ 59.584959] nix-grpc-daemon[872]: time=2026-09-22T10:56:31.740Z level=INFO msg="Upload complete. (400ms)"5118lb # [ 59.552114] niks3-server[1003]: 2026/09/22 10:56:30 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5119worker2 # [ 59.587452] nix-grpc-daemon[790]: ts=2026-09-22T10:56:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_in=4325120worker2 # [ 59.591239] nix-grpc-daemon[790]: ts=2026-09-22T10:56:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005121worker2 # [ 59.597093] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5122worker2 # [ 59.599568] nix-daemon[859]: reaped child process 1040, status = succeeded5123client # [ 59.587826] nix[956]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5124worker2 # [ 59.604174] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5125client # [ 59.958544] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.5126client # Running as unit: dedup2.service; invocation ID: 3496189fff654acc881b58d177107ffa5127client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)5128client # [ 60.185154] nix[972]: don't know how to build these paths:5129client # [ 60.185948] nix[972]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5130worker1 # [ 60.153803] nix-grpc-daemon[784]: ts=2026-09-22T10:56:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418045131worker1 # [ 60.165157] nix-grpc-daemon[784]: ts=2026-09-22T10:56:32Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05132worker2 # [ 60.215766] nix-grpc-daemon[790]: ts=2026-09-22T10:56:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005133client # [ 60.202869] nix[972]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5134client: must succeed: sleep 25135client: (finished: must succeed: sleep 2, in 2.05 seconds)5136client: waiting for success: ! systemctl is-active dedup1 dedup25137worker2 # [ 63.632817] nix-daemon[859]: reaped child process 1054, status = succeeded5138worker2 # [ 63.801132] nix-daemon[859]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5139worker2 # [ 63.818751] nix-daemon[859]: reaped child process 1087, status = succeeded5140lb # [ 63.800417] niks3-server[1003]: 2026/09/22 10:56:35 INFO Received uploads request method=POST path=/api/pending_closures5141worker2 # [ 63.861968] nix-grpc-daemon[872]: time=2026-09-22T10:56:36.017Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5142worker2 # [ 63.865075] nix-grpc-daemon[872]: time=2026-09-22T10:56:36.019Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5143lb # [ 63.867435] niks3-server[1003]: 2026/09/22 10:56:35 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5144lb # [ 63.872395] niks3-server[1003]: 2026/09/22 10:56:35 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5145lb # [ 63.886896] niks3-server[1003]: 2026/09/22 10:56:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5146lb # [ 63.892607] niks3-server[1003]: 2026/09/22 10:56:35 INFO Signed narinfos id=16 count=15147lb # [ 63.895715] niks3-server[1003]: 2026/09/22 10:56:35 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5148worker2 # [ 63.929320] nix-grpc-daemon[872]: time=2026-09-22T10:56:36.084Z level=INFO msg="Uploading 1 narinfos"5149lb # [ 63.906791] niks3-server[1003]: 2026/09/22 10:56:35 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5150lb # [ 63.912078] niks3-server[1003]: 2026/09/22 10:56:35 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5151lb # [ 63.914363] niks3-server[1003]: 2026/09/22 10:56:35 INFO Completed upload id=165152worker2 # [ 63.947940] nix-grpc-daemon[872]: time=2026-09-22T10:56:36.103Z level=INFO msg="Upload complete. (331ms)"5153worker2 # [ 63.954992] nix-grpc-daemon[790]: ts=2026-09-22T10:56:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15154worker2 # [ 63.962412] nix-grpc-daemon[790]: ts=2026-09-22T10:56:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15155worker2 # [ 63.974959] nix-daemon[859]: reaped child process 1052, status = succeeded5156client # [ 63.993139] systemd[1]: dedup1.service: Deactivated successfully.5157client # [ 63.995660] systemd[1]: dedup1.service: Consumed 117ms CPU time over 5.098s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.5158client # [ 63.998871] systemd[1]: dedup2.service: Deactivated successfully.5159client # [ 64.000659] systemd[1]: dedup2.service: Consumed 133ms CPU time over 4.042s wall clock time, 33.2M memory peak, 6K incoming IP traffic, 5.6K outgoing IP traffic.5160client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.07 seconds)5161client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5162client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5163worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5164worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)5165worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5166worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)5167worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5168worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5169worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5170worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5171(finished: subtest: two clients wanting the same drv build it once, in 5.87 seconds)5172subtest: --timeout and --max-silent-time reach the builder5173worker2 # [ 65.001920] nix-grpc-daemon[790]: ts=2026-09-22T10:56:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570005174worker1 # [ 64.969874] nix-grpc-daemon[784]: ts=2026-09-22T10:56:36Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05175worker1 # [ 64.976386] nix-grpc-daemon[784]: ts=2026-09-22T10:56:36Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=15176worker1 # [ 64.980731] nix-grpc-daemon[784]: ts=2026-09-22T10:56:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045177worker1 # [ 64.991826] nix-grpc-daemon[784]: ts=2026-09-22T10:56:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418045178worker1 # [ 65.002899] nix-grpc-daemon[784]: ts=2026-09-22T10:56:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418045179worker1 # [ 65.014711] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5180worker1 # [ 65.197616] nix-daemon[866]: accepted connection from pid 1068, user nix-grpc-daemon (trusted)5181worker1 # [ 65.217158] nix-daemon[866]: reaped child process 1071, status = succeeded5182worker1 # [ 65.373066] nix-daemon[866]: accepted connection from pid 1073, user nix-grpc-daemon (trusted)5183worker1 # [ 65.387593] nix-daemon[866]: reaped child process 1076, status = succeeded5184lb # [ 65.413514] niks3-server[1003]: 2026/09/22 10:56:36 INFO Received uploads request method=POST path=/api/pending_closures5185worker1 # [ 65.406962] nix-grpc-daemon[919]: time=2026-09-22T10:56:37.314Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5186worker1 # [ 65.408618] nix-grpc-daemon[919]: time=2026-09-22T10:56:37.314Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5187lb # [ 65.434957] niks3-server[1003]: 2026/09/22 10:56:36 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5188lb # [ 65.440408] niks3-server[1003]: 2026/09/22 10:56:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5189lb # [ 65.445035] niks3-server[1003]: 2026/09/22 10:56:36 INFO Signed narinfos id=17 count=15190worker1 # [ 65.432632] nix-grpc-daemon[919]: time=2026-09-22T10:56:37.340Z level=INFO msg="Uploading 1 narinfos"5191lb # [ 65.450818] niks3-server[1003]: 2026/09/22 10:56:36 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5192lb # [ 65.461352] niks3-server[1003]: 2026/09/22 10:56:36 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5193lb # [ 65.466741] niks3-server[1003]: 2026/09/22 10:56:36 INFO Completed upload id=175194worker1 # [ 65.454486] nix-grpc-daemon[919]: time=2026-09-22T10:56:37.362Z level=INFO msg="Upload complete. (427ms)"5195worker1 # [ 65.458463] nix-grpc-daemon[784]: ts=2026-09-22T10:56:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_in=4245196lb # [ 65.472391] niks3-server[1003]: 2026/09/22 10:56:36 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5197worker1 # [ 65.463818] nix-grpc-daemon[784]: ts=2026-09-22T10:56:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045198worker1 # [ 65.473110] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5199worker1 # [ 65.480212] nix-daemon[866]: reaped child process 1066, status = succeeded5200worker1 # [ 65.488617] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5201lb # [ 68.234336] envoy[921]: conn peer=[2001:db8:1::1]:42420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2887 ms=1315202lb # [ 68.236902] envoy[921]: conn peer=[2001:db8:1::1]:42436 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=645203lb # [ 68.238923] envoy[921]: conn peer=[2001:db8:1::1]:42434 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=725204lb # [ 68.240994] envoy[921]: conn peer=[2001:db8:1::1]:42426 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=795205lb # [ 68.243369] envoy[921]: conn peer=[2001:db8:1::1]:42446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3028 tx=1554 ms=48855206lb # [ 68.245404] envoy[921]: conn peer=[2001:db8:1::1]:42458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=37705207worker1 # [ 72.107394] nix-grpc-daemon[784]: ts=2026-09-22T10:56:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05208worker1 # [ 72.120272] nix-daemon[866]: reaped child process 1078, status = succeeded5209worker1 # [ 72.121312] nix-daemon[866]: reaped child process 1080, status = succeeded5210worker1 # [ 72.343151] nix-grpc-daemon[784]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418045211worker1 # [ 72.349543] nix-grpc-daemon[784]: ts=2026-09-22T10:56:44Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05212worker1 # [ 72.351771] nix-grpc-daemon[784]: ts=2026-09-22T10:56:44Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=15213worker2 # [ 72.401078] nix-grpc-daemon[790]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005214worker2 # [ 72.410436] nix-grpc-daemon[790]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570005215worker2 # [ 72.417368] nix-grpc-daemon[790]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570005216worker2 # [ 72.423599] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5217worker2 # [ 72.555781] nix-daemon[859]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)5218worker2 # [ 72.570672] nix-daemon[859]: reaped child process 1103, status = succeeded5219worker2 # [ 72.708268] nix-daemon[859]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5220worker2 # [ 72.721131] nix-daemon[859]: reaped child process 1108, status = succeeded5221lb # [ 72.706392] niks3-server[1003]: 2026/09/22 10:56:44 INFO Received uploads request method=POST path=/api/pending_closures5222worker2 # [ 72.748279] nix-grpc-daemon[872]: time=2026-09-22T10:56:44.903Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5223worker2 # [ 72.749786] nix-grpc-daemon[872]: time=2026-09-22T10:56:44.903Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5224lb # [ 72.736794] niks3-server[1003]: 2026/09/22 10:56:44 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5225lb # [ 72.747541] niks3-server[1003]: 2026/09/22 10:56:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5226worker2 # [ 72.791065] nix-grpc-daemon[872]: time=2026-09-22T10:56:44.946Z level=INFO msg="Uploading 1 narinfos"5227lb # [ 72.757598] niks3-server[1003]: 2026/09/22 10:56:44 INFO Signed narinfos id=18 count=15228lb # [ 72.763970] niks3-server[1003]: 2026/09/22 10:56:44 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5229lb # [ 72.777260] niks3-server[1003]: 2026/09/22 10:56:44 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5230lb # [ 72.781769] niks3-server[1003]: 2026/09/22 10:56:44 INFO Completed upload id=185231worker2 # [ 72.815322] nix-grpc-daemon[872]: time=2026-09-22T10:56:44.970Z level=INFO msg="Upload complete. (385ms)"5232lb # [ 72.784159] niks3-server[1003]: 2026/09/22 10:56:44 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5233worker2 # [ 72.819612] nix-grpc-daemon[790]: ts=2026-09-22T10:56:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_in=4245234worker2 # [ 72.824247] nix-grpc-daemon[790]: ts=2026-09-22T10:56:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005235worker2 # [ 72.833642] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5236worker2 # [ 72.841765] nix-daemon[859]: reaped child process 1098, status = succeeded5237worker2 # [ 72.846055] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5238lb # [ 78.235217] envoy[921]: conn peer=[2001:db8:1::1]:56840 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2970 tx=951 ms=71675239worker2 # [ 78.658294] nix-grpc-daemon[790]: ts=2026-09-22T10:56:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=5 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05240worker2 # [ 78.670544] nix-daemon[859]: reaped child process 1110, status = succeeded5241worker2 # [ 78.671680] nix-daemon[859]: reaped child process 1112, status = succeeded5242(finished: subtest: --timeout and --max-silent-time reach the builder, in 14.10 seconds)5243subtest: the first client leaving does not fail the second5244client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share5245client # [ 78.859498] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share.5246client # Running as unit: share1.service; invocation ID: a4ed275d592a445a87993fbea12e22815247client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share, in 0.03 seconds)5248client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share5249worker2 # [ 79.048218] nix-grpc-daemon[790]: ts=2026-09-22T10:56:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570005250client # [ 79.035832] nix[1045]: don't know how to build these paths:5251client # [ 79.037344] nix[1045]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5252worker1 # [ 79.016172] nix-grpc-daemon[784]: ts=2026-09-22T10:56:50Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05253worker1 # [ 79.021847] nix-grpc-daemon[784]: ts=2026-09-22T10:56:50Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=15254worker1 # [ 79.027104] nix-grpc-daemon[784]: ts=2026-09-22T10:56:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045255worker1 # [ 79.038632] nix-grpc-daemon[784]: ts=2026-09-22T10:56:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418045256client # [ 79.079085] nix[1045]: copying 1 paths to 'lb:50051'...5257client # [ 79.079856] nix[1045]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5258worker1 # [ 79.049398] nix-grpc-daemon[784]: ts=2026-09-22T10:56:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418045259worker1 # [ 79.060082] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5260worker1 # [ 79.232229] nix-daemon[866]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5261worker1 # [ 79.244579] nix-daemon[866]: reaped child process 1110, status = succeeded5262worker1 # [ 79.403487] nix-daemon[866]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)5263worker1 # [ 79.416153] nix-daemon[866]: reaped child process 1115, status = succeeded5264lb # [ 79.445937] niks3-server[1003]: 2026/09/22 10:56:50 INFO Received uploads request method=POST path=/api/pending_closures5265worker1 # [ 79.442275] nix-grpc-daemon[919]: time=2026-09-22T10:56:51.350Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5266worker1 # [ 79.443728] nix-grpc-daemon[919]: time=2026-09-22T10:56:51.350Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5267lb # [ 79.474596] niks3-server[1003]: 2026/09/22 10:56:50 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5268lb # [ 79.487284] niks3-server[1003]: 2026/09/22 10:56:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5269lb # [ 79.492558] niks3-server[1003]: 2026/09/22 10:56:50 INFO Signed narinfos id=19 count=15270worker1 # [ 79.483419] nix-grpc-daemon[919]: time=2026-09-22T10:56:51.391Z level=INFO msg="Uploading 1 narinfos"5271lb # [ 79.500253] niks3-server[1003]: 2026/09/22 10:56:50 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5272lb # [ 79.509582] niks3-server[1003]: 2026/09/22 10:56:50 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5273lb # [ 79.515861] niks3-server[1003]: 2026/09/22 10:56:50 INFO Completed upload id=195274worker1 # [ 79.503124] nix-grpc-daemon[919]: time=2026-09-22T10:56:51.411Z level=INFO msg="Upload complete. (431ms)"5275worker1 # [ 79.506774] nix-grpc-daemon[784]: ts=2026-09-22T10:56:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_in=4325276worker1 # [ 79.511839] nix-grpc-daemon[784]: ts=2026-09-22T10:56:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045277worker1 # [ 79.522551] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5278worker1 # [ 79.526904] nix-daemon[866]: reaped child process 1105, status = succeeded5279worker1 # [ 79.531271] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5280client # [ 79.561651] nix[1045]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5281client # [ 79.926952] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share.5282client # Running as unit: share2.service; invocation ID: b05a7170b6e545baad584f9359ddf5275283client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share, in 1.08 seconds)5284worker2 # [ 80.151212] nix-grpc-daemon[790]: ts=2026-09-22T10:56:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570005285client # [ 80.141459] nix[1061]: don't know how to build these paths:5286client # [ 80.142500] nix[1061]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5287worker1 # [ 80.115098] nix-grpc-daemon[784]: ts=2026-09-22T10:56:52Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05288client # [ 80.148648] nix[1061]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5289worker1 # [ 80.117649] nix-grpc-daemon[784]: ts=2026-09-22T10:56:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045290client: must succeed: sleep 2; systemctl kill -s INT share15291client # [ 83.227101] systemd[1]: share1.service: Sent signal SIGINT to main process 1045 (nix) on client request.5292client # [ 83.234862] nix[1045]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5293client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.07 seconds)5294client: waiting for success: ! systemctl is-active share15295client # [ 83.243183] nix[1045]: error: interrupted by the user5296client # [ 83.283710] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5297client # [ 83.285048] systemd[1]: share1.service: Failed with result 'exit-code'.5298client # [ 83.286639] systemd[1]: share1.service: Consumed 115ms CPU time over 4.424s wall clock time, 43.4M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.5299client: (finished: waiting for success: ! systemctl is-active share1, in 0.05 seconds)5300client: waiting for success: ! systemctl is-active share25301worker1 # [ 83.561695] nix-daemon[866]: reaped child process 1119, status = succeeded5302worker1 # [ 83.722604] nix-daemon[866]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5303worker1 # [ 83.736065] nix-daemon[866]: reaped child process 1158, status = succeeded5304lb # [ 83.766294] niks3-server[1003]: 2026/09/22 10:56:55 INFO Received uploads request method=POST path=/api/pending_closures5305worker1 # [ 83.764386] nix-grpc-daemon[919]: time=2026-09-22T10:56:55.672Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5306worker1 # [ 83.765839] nix-grpc-daemon[919]: time=2026-09-22T10:56:55.672Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5307lb # [ 83.811947] niks3-server[1003]: 2026/09/22 10:56:55 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5308lb # [ 83.818588] niks3-server[1003]: 2026/09/22 10:56:55 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5309lb # [ 83.830438] niks3-server[1003]: 2026/09/22 10:56:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5310worker1 # [ 83.825030] nix-grpc-daemon[919]: time=2026-09-22T10:56:55.733Z level=INFO msg="Uploading 1 narinfos"5311lb # [ 83.837661] niks3-server[1003]: 2026/09/22 10:56:55 INFO Signed narinfos id=20 count=15312lb # [ 83.842312] niks3-server[1003]: 2026/09/22 10:56:55 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5313lb # [ 83.849319] niks3-server[1003]: 2026/09/22 10:56:55 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5314lb # [ 83.853861] niks3-server[1003]: 2026/09/22 10:56:55 INFO Completed upload id=205315lb # [ 83.855288] niks3-server[1003]: 2026/09/22 10:56:55 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5316worker1 # [ 83.841450] nix-grpc-daemon[919]: time=2026-09-22T10:56:55.749Z level=INFO msg="Upload complete. (299ms)"5317worker1 # [ 83.849762] nix-grpc-daemon[784]: ts=2026-09-22T10:56:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15318worker1 # [ 83.855328] nix-grpc-daemon[784]: ts=2026-09-22T10:56:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=3 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15319worker1 # [ 83.866629] nix-daemon[866]: reaped child process 1117, status = succeeded5320client # [ 83.946224] systemd[1]: share2.service: Deactivated successfully.5321client # [ 83.948793] systemd[1]: share2.service: Consumed 114ms CPU time over 4.022s wall clock time, 35.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.5322client: (finished: waiting for success: ! systemctl is-active share2, in 1.09 seconds)5323client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5324client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5325(finished: subtest: the first client leaving does not fail the second, in 5.79 seconds)5326subtest: build hook: nix-daemon with builders = grpc://lb5327client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log5328client # [ 84.770679] systemd[1]: Started Nix Daemon.5329client # [ 84.935706] nix-daemon[1108]: accepted connection from pid 1105, user root (trusted)5330worker2 # [ 85.144471] nix-grpc-daemon[790]: ts=2026-09-22T10:56:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:570005331worker2 # [ 85.147976] nix-grpc-daemon[790]: ts=2026-09-22T10:56:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570005332worker1 # [ 85.117897] nix-grpc-daemon[784]: ts=2026-09-22T10:56:57Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05333worker2 # [ 85.171052] nix-grpc-daemon[790]: ts=2026-09-22T10:56:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005334worker1 # [ 85.122804] nix-grpc-daemon[784]: ts=2026-09-22T10:56:57Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15335worker2 # [ 85.181142] nix-grpc-daemon[790]: ts=2026-09-22T10:56:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570005336worker2 # [ 85.188798] nix-grpc-daemon[790]: ts=2026-09-22T10:56:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570005337worker2 # [ 85.199830] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5338worker2 # [ 85.362769] nix-daemon[859]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)5339worker2 # [ 85.376849] nix-daemon[859]: reaped child process 1154, status = succeeded5340worker2 # [ 85.540259] nix-daemon[859]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)5341worker2 # [ 85.552954] nix-daemon[859]: reaped child process 1159, status = succeeded5342lb # [ 85.536705] niks3-server[1003]: 2026/09/22 10:56:56 INFO Received uploads request method=POST path=/api/pending_closures5343worker2 # [ 85.580640] nix-grpc-daemon[872]: time=2026-09-22T10:56:57.736Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5344worker2 # [ 85.582552] nix-grpc-daemon[872]: time=2026-09-22T10:56:57.737Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5345lb # [ 85.575394] niks3-server[1003]: 2026/09/22 10:56:57 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5346lb # [ 85.582429] niks3-server[1003]: 2026/09/22 10:56:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5347lb # [ 85.585411] niks3-server[1003]: 2026/09/22 10:56:57 INFO Signed narinfos id=21 count=15348worker2 # [ 85.618847] nix-grpc-daemon[872]: time=2026-09-22T10:56:57.774Z level=INFO msg="Uploading 1 narinfos"5349lb # [ 85.589161] niks3-server[1003]: 2026/09/22 10:56:57 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5350lb # [ 85.598098] niks3-server[1003]: 2026/09/22 10:56:57 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5351lb # [ 85.601738] niks3-server[1003]: 2026/09/22 10:56:57 INFO Completed upload id=215352worker2 # [ 85.635172] nix-grpc-daemon[872]: time=2026-09-22T10:56:57.790Z level=INFO msg="Upload complete. (423ms)"5353lb # [ 85.604062] niks3-server[1003]: 2026/09/22 10:56:57 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5354worker2 # [ 85.638178] nix-grpc-daemon[790]: ts=2026-09-22T10:56:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_in=5125355worker2 # [ 85.643251] nix-grpc-daemon[790]: ts=2026-09-22T10:56:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005356worker2 # [ 85.652573] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5357worker2 # [ 85.656263] nix-daemon[859]: reaped child process 1149, status = succeeded5358worker2 # [ 85.661311] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5359worker2 # [ 87.088524] nix-daemon[859]: reaped child process 1163, status = succeeded5360worker2 # [ 87.239847] nix-daemon[859]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)5361worker2 # [ 87.251188] nix-daemon[859]: reaped child process 1182, status = succeeded5362lb # [ 87.229854] niks3-server[1003]: 2026/09/22 10:56:58 INFO Received uploads request method=POST path=/api/pending_closures5363worker2 # [ 87.272486] nix-grpc-daemon[872]: time=2026-09-22T10:56:59.427Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5364worker2 # [ 87.274431] nix-grpc-daemon[872]: time=2026-09-22T10:56:59.429Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5365lb # [ 87.268049] niks3-server[1003]: 2026/09/22 10:56:58 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5366lb # [ 87.280780] niks3-server[1003]: 2026/09/22 10:56:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5367lb # [ 87.286127] niks3-server[1003]: 2026/09/22 10:56:58 INFO Signed narinfos id=22 count=15368worker2 # [ 87.323093] nix-grpc-daemon[872]: time=2026-09-22T10:56:59.478Z level=INFO msg="Uploading 1 narinfos"5369lb # [ 87.292226] niks3-server[1003]: 2026/09/22 10:56:58 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5370lb # [ 87.297529] niks3-server[1003]: 2026/09/22 10:56:58 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5371lb # [ 87.306884] niks3-server[1003]: 2026/09/22 10:56:58 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5372lb # [ 87.312477] niks3-server[1003]: 2026/09/22 10:56:58 INFO Completed upload id=225373worker2 # [ 87.345830] nix-grpc-daemon[872]: time=2026-09-22T10:56:59.501Z level=INFO msg="Upload complete. (276ms)"5374lb # [ 87.314271] niks3-server[1003]: 2026/09/22 10:56:58 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5375worker2 # [ 87.348599] nix-grpc-daemon[790]: ts=2026-09-22T10:56:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15376worker2 # [ 87.355272] nix-daemon[859]: reaped child process 1161, status = succeeded5377worker2 # [ 87.395969] nix-grpc-daemon[790]: ts=2026-09-22T10:56:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:570005378worker2 # [ 87.411803] nix-grpc-daemon[790]: ts=2026-09-22T10:56:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:570005379worker2 # [ 87.417143] nix-grpc-daemon[790]: ts=2026-09-22T10:56:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_out=1285380worker2 # [ 87.668294] nix-grpc-daemon[790]: ts=2026-09-22T10:56:59Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:570005381worker2 # [ 87.674624] nix-grpc-daemon[790]: ts=2026-09-22T10:56:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570005382worker2 # [ 87.680352] nix-grpc-daemon[872]: time=2026-09-22T10:56:59.834Z level=INFO msg="All 1 paths already cached"5383worker1 # [ 87.643853] nix-grpc-daemon[784]: ts=2026-09-22T10:56:59Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05384worker1 # [ 87.648495] nix-grpc-daemon[784]: ts=2026-09-22T10:56:59Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15385worker2 # [ 87.696244] nix-grpc-daemon[790]: ts=2026-09-22T10:56:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005386worker2 # [ 87.708090] nix-grpc-daemon[790]: ts=2026-09-22T10:56:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570005387worker2 # [ 87.719392] nix-grpc-daemon[872]: time=2026-09-22T10:56:59.874Z level=INFO msg="All 1 paths already cached"5388worker2 # [ 87.727129] nix-grpc-daemon[790]: ts=2026-09-22T10:56:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570005389worker2 # [ 87.737366] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5390worker2 # [ 87.903206] nix-daemon[859]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)5391worker2 # [ 87.916841] nix-daemon[859]: reaped child process 1190, status = succeeded5392worker2 # [ 88.069402] nix-daemon[859]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)5393worker2 # [ 88.081414] nix-daemon[859]: reaped child process 1195, status = succeeded5394lb # [ 88.057926] niks3-server[1003]: 2026/09/22 10:56:59 INFO Received uploads request method=POST path=/api/pending_closures5395worker2 # [ 88.100201] nix-grpc-daemon[872]: time=2026-09-22T10:57:00.255Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5396worker2 # [ 88.101643] nix-grpc-daemon[872]: time=2026-09-22T10:57:00.257Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5397lb # [ 88.087877] niks3-server[1003]: 2026/09/22 10:56:59 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5398lb # [ 88.098869] niks3-server[1003]: 2026/09/22 10:56:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5399lb # [ 88.103874] niks3-server[1003]: 2026/09/22 10:56:59 INFO Signed narinfos id=23 count=15400worker2 # [ 88.140248] nix-grpc-daemon[872]: time=2026-09-22T10:57:00.295Z level=INFO msg="Uploading 1 narinfos"5401lb # [ 88.110319] niks3-server[1003]: 2026/09/22 10:56:59 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5402lb # [ 88.123693] niks3-server[1003]: 2026/09/22 10:56:59 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5403lb # [ 88.128843] niks3-server[1003]: 2026/09/22 10:56:59 INFO Completed upload id=235404worker2 # [ 88.162320] nix-grpc-daemon[872]: time=2026-09-22T10:57:00.317Z level=INFO msg="Upload complete. (412ms)"5405lb # [ 88.131309] niks3-server[1003]: 2026/09/22 10:56:59 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5406worker2 # [ 88.166310] nix-grpc-daemon[790]: ts=2026-09-22T10:57:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_in=6245407worker2 # [ 88.171444] nix-grpc-daemon[790]: ts=2026-09-22T10:57:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005408worker2 # [ 88.183430] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5409worker2 # [ 88.186431] nix-daemon[859]: reaped child process 1185, status = succeeded5410worker2 # [ 88.190635] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5411lb # [ 88.235165] envoy[921]: conn peer=[2001:db8:1::1]:56848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=975 ms=63245412lb # [ 88.238750] envoy[921]: conn peer=[2001:db8:1::1]:35376 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=788 ms=42245413lb # [ 88.242333] envoy[921]: conn peer=[2001:db8:1::1]:35392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1453 tx=1197 ms=37925414lb # [ 88.245842] envoy[921]: conn peer=[2001:db8:1::1]:51364 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3429 tx=1940 ms=23265415lb # [ 88.249347] envoy[921]: conn peer=[2001:db8:1::1]:51368 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=575416worker2 # [ 89.615182] nix-daemon[859]: reaped child process 1199, status = succeeded5417worker2 # [ 89.749530] nix-daemon[859]: accepted connection from pid 1215, user nix-grpc-daemon (trusted)5418worker2 # [ 89.761554] nix-daemon[859]: reaped child process 1218, status = succeeded5419lb # [ 89.743981] niks3-server[1003]: 2026/09/22 10:57:01 INFO Received uploads request method=POST path=/api/pending_closures5420worker2 # [ 89.786931] nix-grpc-daemon[872]: time=2026-09-22T10:57:01.942Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5421worker2 # [ 89.788412] nix-grpc-daemon[872]: time=2026-09-22T10:57:01.944Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5422lb # [ 89.781314] niks3-server[1003]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5423lb # [ 89.795748] niks3-server[1003]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5424lb # [ 89.808867] niks3-server[1003]: 2026/09/22 10:57:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5425lb # [ 89.812214] niks3-server[1003]: 2026/09/22 10:57:01 INFO Signed narinfos id=24 count=15426worker2 # [ 89.848245] nix-grpc-daemon[872]: time=2026-09-22T10:57:02.003Z level=INFO msg="Uploading 1 narinfos"5427lb # [ 89.817074] niks3-server[1003]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5428lb # [ 89.828185] niks3-server[1003]: 2026/09/22 10:57:01 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5429lb # [ 89.831288] niks3-server[1003]: 2026/09/22 10:57:01 INFO Completed upload id=245430worker2 # [ 89.864745] nix-grpc-daemon[872]: time=2026-09-22T10:57:02.020Z level=INFO msg="Upload complete. (264ms)"5431lb # [ 89.832926] niks3-server[1003]: 2026/09/22 10:57:01 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5432worker2 # [ 89.870742] nix-grpc-daemon[790]: ts=2026-09-22T10:57:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15433worker2 # [ 89.877868] nix-grpc-daemon[790]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:570005434worker2 # [ 89.888399] nix-daemon[859]: reaped child process 1197, status = succeeded5435worker1 # [ 89.849887] nix-grpc-daemon[784]: ts=2026-09-22T10:57:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418045436worker1 # [ 89.869407] nix-grpc-daemon[784]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5437worker1 # [ 89.885341] nix-grpc-daemon[784]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5438worker1 # [ 89.897933] nix-grpc-daemon[784]: ts=2026-09-22T10:57:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_out=1765439worker1 # [ 90.165981] nix-grpc-daemon[784]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418045440worker1 # [ 90.170968] nix-grpc-daemon[784]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418045441worker1 # [ 90.189978] nix-grpc-daemon[919]: time=2026-09-22T10:57:02.098Z level=INFO msg="All 2 paths already cached"5442worker1 # [ 90.201344] nix-grpc-daemon[784]: ts=2026-09-22T10:57:02Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05443worker1 # [ 90.203316] nix-grpc-daemon[784]: ts=2026-09-22T10:57:02Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15444worker2 # [ 90.251494] nix-grpc-daemon[790]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005445worker2 # [ 90.261388] nix-grpc-daemon[790]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570005446worker2 # [ 90.278600] nix-grpc-daemon[872]: time=2026-09-22T10:57:02.434Z level=INFO msg="All 2 paths already cached"5447worker2 # [ 90.287068] nix-grpc-daemon[790]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570005448worker2 # [ 90.295356] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5449worker2 # [ 90.458588] nix-daemon[859]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5450worker2 # [ 90.472207] nix-daemon[859]: reaped child process 1225, status = succeeded5451worker2 # [ 90.624303] nix-daemon[859]: accepted connection from pid 1227, user nix-grpc-daemon (trusted)5452worker2 # [ 90.635989] nix-daemon[859]: reaped child process 1230, status = succeeded5453lb # [ 90.618349] niks3-server[1003]: 2026/09/22 10:57:02 INFO Received uploads request method=POST path=/api/pending_closures5454worker2 # [ 90.660091] nix-grpc-daemon[872]: time=2026-09-22T10:57:02.815Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5455worker2 # [ 90.661637] nix-grpc-daemon[872]: time=2026-09-22T10:57:02.817Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5456lb # [ 90.653449] niks3-server[1003]: 2026/09/22 10:57:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5457lb # [ 90.660858] niks3-server[1003]: 2026/09/22 10:57:02 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5458lb # [ 90.666895] niks3-server[1003]: 2026/09/22 10:57:02 INFO Signed narinfos id=25 count=15459worker2 # [ 90.703148] nix-grpc-daemon[872]: time=2026-09-22T10:57:02.858Z level=INFO msg="Uploading 1 narinfos"5460lb # [ 90.674124] niks3-server[1003]: 2026/09/22 10:57:02 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5461lb # [ 90.685235] niks3-server[1003]: 2026/09/22 10:57:02 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5462worker2 # [ 90.724958] nix-grpc-daemon[872]: time=2026-09-22T10:57:02.880Z level=INFO msg="Upload complete. (416ms)"5463lb # [ 90.691700] niks3-server[1003]: 2026/09/22 10:57:02 INFO Completed upload id=255464lb # [ 90.694552] niks3-server[1003]: 2026/09/22 10:57:02 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5465worker2 # [ 90.728199] nix-grpc-daemon[790]: ts=2026-09-22T10:57:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_in=7525466worker2 # [ 90.734693] nix-grpc-daemon[790]: ts=2026-09-22T10:57:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005467worker2 # [ 90.744861] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5468worker2 # [ 90.748152] nix-daemon[859]: reaped child process 1220, status = succeeded5469worker2 # [ 90.753500] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5470worker2 # [ 91.966629] nix-daemon[859]: reaped child process 1234, status = succeeded5471worker2 # [ 92.106079] nix-daemon[859]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)5472worker2 # [ 92.118090] nix-daemon[859]: reaped child process 1253, status = succeeded5473lb # [ 92.098846] niks3-server[1003]: 2026/09/22 10:57:03 INFO Received uploads request method=POST path=/api/pending_closures5474worker2 # [ 92.137576] nix-grpc-daemon[872]: time=2026-09-22T10:57:04.292Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5475worker2 # [ 92.139042] nix-grpc-daemon[872]: time=2026-09-22T10:57:04.292Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5476lb # [ 92.115329] niks3-server[1003]: 2026/09/22 10:57:03 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5477lb # [ 92.119067] niks3-server[1003]: 2026/09/22 10:57:03 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5478lb # [ 92.123442] niks3-server[1003]: 2026/09/22 10:57:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5479lb # [ 92.125856] niks3-server[1003]: 2026/09/22 10:57:03 INFO Signed narinfos id=26 count=15480worker2 # [ 92.159387] nix-grpc-daemon[872]: time=2026-09-22T10:57:04.314Z level=INFO msg="Uploading 1 narinfos"5481lb # [ 92.129219] niks3-server[1003]: 2026/09/22 10:57:03 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5482lb # [ 92.133895] niks3-server[1003]: 2026/09/22 10:57:03 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5483lb # [ 92.137480] niks3-server[1003]: 2026/09/22 10:57:03 INFO Completed upload id=265484worker2 # [ 92.170892] nix-grpc-daemon[872]: time=2026-09-22T10:57:04.326Z level=INFO msg="Upload complete. (210ms)"5485lb # [ 92.139278] niks3-server[1003]: 2026/09/22 10:57:03 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5486worker2 # [ 92.173588] nix-grpc-daemon[790]: ts=2026-09-22T10:57:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15487worker2 # [ 92.180478] nix-grpc-daemon[790]: ts=2026-09-22T10:57:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:570005488worker2 # [ 92.183242] nix-daemon[859]: reaped child process 1232, status = succeeded5489worker1 # [ 92.150281] nix-grpc-daemon[784]: ts=2026-09-22T10:57:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418045490worker1 # [ 92.156210] nix-grpc-daemon[784]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5491worker1 # [ 92.162356] nix-grpc-daemon[784]: ts=2026-09-22T10:57:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_out=2325492client # [ 92.299681] nix-daemon[1108]: reaped child process 1110, status = succeeded5493client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.70 seconds)5494client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5495client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5496client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5497client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)5498client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5499client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5500(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.79 seconds)5501subtest: an input only one worker has still reaches the builder5502client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5503client # this derivation will be built:5504client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5505client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5506client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.38 seconds)5507client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5508client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.18 seconds)5509worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5510worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.22 seconds)5511worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5512worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5513client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25514client # [ 93.394164] nix-daemon[1108]: accepted connection from pid 1191, user root (trusted)5515client # this derivation will be built:5516client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5517worker2 # [ 93.588132] nix-grpc-daemon[790]: ts=2026-09-22T10:57:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:570005518client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5519worker1 # [ 93.544915] nix-grpc-daemon[784]: ts=2026-09-22T10:57:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418045520worker1 # [ 93.702489] nix-daemon[866]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5521worker1 # [ 93.713586] nix-daemon[866]: reaped child process 1177, status = succeeded5522lb # [ 93.741715] niks3-server[1003]: 2026/09/22 10:57:05 INFO Received uploads request method=POST path=/api/pending_closures5523worker1 # [ 93.734390] nix-grpc-daemon[919]: time=2026-09-22T10:57:05.642Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5524worker1 # [ 93.735949] nix-grpc-daemon[919]: time=2026-09-22T10:57:05.644Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5525lb # [ 93.762281] niks3-server[1003]: 2026/09/22 10:57:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5526lb # [ 93.764691] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5527lb # [ 93.766540] niks3-server[1003]: 2026/09/22 10:57:05 INFO Signed narinfos id=27 count=15528worker1 # [ 93.754863] nix-grpc-daemon[919]: time=2026-09-22T10:57:05.663Z level=INFO msg="Uploading 1 narinfos"5529lb # [ 93.771163] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5530lb # [ 93.776022] niks3-server[1003]: 2026/09/22 10:57:05 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5531lb # [ 93.779829] niks3-server[1003]: 2026/09/22 10:57:05 INFO Completed upload id=275532worker1 # [ 93.767033] nix-grpc-daemon[919]: time=2026-09-22T10:57:05.675Z level=INFO msg="Upload complete. (217ms)"5533lb # [ 93.782790] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5534worker1 # [ 93.781045] nix-grpc-daemon[784]: ts=2026-09-22T10:57:05Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05535worker1 # [ 93.783221] nix-grpc-daemon[784]: ts=2026-09-22T10:57:05Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15536worker1 # [ 93.787530] nix-grpc-daemon[784]: ts=2026-09-22T10:57:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045537worker1 # [ 93.793927] nix-grpc-daemon[784]: ts=2026-09-22T10:57:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418045538client # copying 2 paths to 'lb:50051'...5539worker1 # [ 93.803163] nix-grpc-daemon[784]: ts=2026-09-22T10:57:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418045540client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5541client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5542worker1 # [ 93.809801] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5543worker1 # [ 93.967501] nix-daemon[866]: accepted connection from pid 1183, user nix-grpc-daemon (trusted)5544worker1 # [ 93.980678] nix-daemon[866]: reaped child process 1186, status = succeeded5545worker1 # [ 94.116994] nix-daemon[866]: accepted connection from pid 1188, user nix-grpc-daemon (trusted)5546worker1 # [ 94.129104] nix-daemon[866]: reaped child process 1191, status = succeeded5547lb # [ 94.157252] niks3-server[1003]: 2026/09/22 10:57:05 INFO Received uploads request method=POST path=/api/pending_closures5548lb # [ 94.165839] niks3-server[1003]: 2026/09/22 10:57:05 INFO Received uploads request method=POST path=/api/pending_closures5549worker1 # [ 94.160909] nix-grpc-daemon[919]: time=2026-09-22T10:57:06.068Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5550worker1 # [ 94.162352] nix-grpc-daemon[919]: time=2026-09-22T10:57:06.068Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5551worker1 # [ 94.163950] nix-grpc-daemon[919]: time=2026-09-22T10:57:06.069Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5552lb # [ 94.205076] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5553lb # [ 94.210393] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5554lb # [ 94.218932] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5555lb # [ 94.224277] niks3-server[1003]: 2026/09/22 10:57:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5556lb # [ 94.228814] niks3-server[1003]: 2026/09/22 10:57:05 INFO Signed narinfos id=28 count=15557lb # [ 94.232211] niks3-server[1003]: 2026/09/22 10:57:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5558lb # [ 94.235231] niks3-server[1003]: 2026/09/22 10:57:05 INFO Signed narinfos id=29 count=25559lb # [ 94.237037] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5560worker1 # [ 94.224551] nix-grpc-daemon[919]: time=2026-09-22T10:57:06.132Z level=INFO msg="Uploading 3 narinfos"5561lb # [ 94.249386] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5562lb # [ 94.253055] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5563lb # [ 94.254971] niks3-server[1003]: 2026/09/22 10:57:05 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5564lb # [ 94.258243] niks3-server[1003]: 2026/09/22 10:57:05 INFO Completed upload id=285565lb # [ 94.259534] niks3-server[1003]: 2026/09/22 10:57:05 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5566lb # [ 94.261443] niks3-server[1003]: 2026/09/22 10:57:05 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5567lb # [ 94.264156] niks3-server[1003]: 2026/09/22 10:57:05 INFO Completed upload id=295568worker1 # [ 94.252582] nix-grpc-daemon[919]: time=2026-09-22T10:57:06.161Z level=INFO msg="Upload complete. (435ms)"5569worker1 # [ 94.257419] nix-grpc-daemon[784]: ts=2026-09-22T10:57:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=2 nar_bytes_in=9765570worker1 # [ 94.263356] nix-grpc-daemon[784]: ts=2026-09-22T10:57:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045571worker1 # [ 94.275173] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5572worker1 # [ 94.283611] nix-daemon[866]: reaped child process 1181, status = succeeded5573client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5574worker1 # [ 94.292515] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5575client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5576client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5577worker1 # [ 95.522388] nix-daemon[866]: reaped child process 1195, status = succeeded5578worker1 # [ 95.650917] nix-daemon[866]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)5579worker1 # [ 95.664739] nix-daemon[866]: reaped child process 1214, status = succeeded5580lb # [ 95.694332] niks3-server[1003]: 2026/09/22 10:57:07 INFO Received uploads request method=POST path=/api/pending_closures5581worker1 # [ 95.690398] nix-grpc-daemon[919]: time=2026-09-22T10:57:07.598Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5582lb # [ 95.726246] niks3-server[1003]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5583lb # [ 95.732231] niks3-server[1003]: 2026/09/22 10:57:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5584lb # [ 95.738335] niks3-server[1003]: 2026/09/22 10:57:07 INFO Signed narinfos id=30 count=15585worker1 # [ 95.730239] nix-grpc-daemon[919]: time=2026-09-22T10:57:07.638Z level=INFO msg="Uploading 1 narinfos"5586lb # [ 95.746449] niks3-server[1003]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5587lb # [ 95.757138] niks3-server[1003]: 2026/09/22 10:57:07 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5588lb # [ 95.762840] niks3-server[1003]: 2026/09/22 10:57:07 INFO Completed upload id=305589worker1 # [ 95.750024] nix-grpc-daemon[919]: time=2026-09-22T10:57:07.658Z level=INFO msg="Upload complete. (246ms)"5590lb # [ 95.766553] niks3-server[1003]: 2026/09/22 10:57:07 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5591worker1 # [ 95.753470] nix-grpc-daemon[784]: ts=2026-09-22T10:57:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15592client # copying 1 paths...5593worker1 # [ 95.761340] nix-daemon[866]: reaped child process 1193, status = succeeded5594worker2 # [ 95.806786] nix-grpc-daemon[790]: ts=2026-09-22T10:57:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:570005595worker2 # [ 95.813103] nix-grpc-daemon[790]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5596client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5597worker2 # [ 95.836744] nix-grpc-daemon[790]: ts=2026-09-22T10:57:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:570005598worker2 # [ 95.841221] nix-grpc-daemon[790]: ts=2026-09-22T10:57:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_out=1205599client # [ 95.932806] nix-daemon[1108]: reaped child process 1194, status = succeeded5600client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.73 seconds)5601client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25602client # [ 96.075557] nix-daemon[1108]: accepted connection from pid 1213, user root (trusted)5603client # this derivation will be built:5604client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5605worker1 # [ 96.224717] nix-grpc-daemon[784]: ts=2026-09-22T10:57:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418045606client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5607worker2 # [ 96.274399] nix-grpc-daemon[790]: ts=2026-09-22T10:57:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570005608worker1 # [ 96.251855] nix-grpc-daemon[784]: ts=2026-09-22T10:57:08Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05609worker2 # [ 96.301118] nix-grpc-daemon[790]: ts=2026-09-22T10:57:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005610worker1 # [ 96.256559] nix-grpc-daemon[784]: ts=2026-09-22T10:57:08Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15611worker2 # [ 96.312895] nix-grpc-daemon[790]: ts=2026-09-22T10:57:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570005612client # copying 1 paths to 'lb:50051'...5613client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5614worker2 # [ 96.330664] nix-grpc-daemon[790]: ts=2026-09-22T10:57:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570005615worker2 # [ 96.341588] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5616worker2 # [ 96.357552] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 1/5)5617worker2 # [ 96.434491] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 2/5)5618worker2 # [ 96.546619] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 233 ms (attempt 3/5)5619worker2 # [ 96.785553] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 414 ms (attempt 4/5)5620worker2 # [ 97.205739] nix-grpc-daemon[790]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5621worker2 # [ 97.207755] nix-grpc-daemon[790]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5622worker2 # [ 97.210891] nix-grpc-daemon[790]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5623worker2 # [ 97.392349] nix-daemon[859]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)5624worker2 # [ 97.404308] nix-daemon[859]: reaped child process 1269, status = succeeded5625worker2 # [ 97.568434] nix-daemon[859]: accepted connection from pid 1271, user nix-grpc-daemon (trusted)5626worker2 # [ 97.580716] nix-daemon[859]: reaped child process 1274, status = succeeded5627lb # [ 97.565592] niks3-server[1003]: 2026/09/22 10:57:08 INFO Received uploads request method=POST path=/api/pending_closures5628worker2 # [ 97.607634] nix-grpc-daemon[872]: time=2026-09-22T10:57:09.762Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5629worker2 # [ 97.609186] nix-grpc-daemon[872]: time=2026-09-22T10:57:09.762Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5630lb # [ 97.591775] niks3-server[1003]: 2026/09/22 10:57:09 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5631lb # [ 97.602885] niks3-server[1003]: 2026/09/22 10:57:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5632lb # [ 97.608368] niks3-server[1003]: 2026/09/22 10:57:09 INFO Signed narinfos id=31 count=15633worker2 # [ 97.645613] nix-grpc-daemon[872]: time=2026-09-22T10:57:09.801Z level=INFO msg="Uploading 1 narinfos"5634lb # [ 97.614168] niks3-server[1003]: 2026/09/22 10:57:09 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5635lb # [ 97.625333] niks3-server[1003]: 2026/09/22 10:57:09 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5636worker2 # [ 97.665272] nix-grpc-daemon[872]: time=2026-09-22T10:57:09.820Z level=INFO msg="Upload complete. (434ms)"5637lb # [ 97.632090] niks3-server[1003]: 2026/09/22 10:57:09 INFO Completed upload id=315638worker2 # [ 97.667667] nix-grpc-daemon[790]: ts=2026-09-22T10:57:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=1 paths=1 nar_bytes_in=5605639lb # [ 97.635504] niks3-server[1003]: 2026/09/22 10:57:09 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5640worker2 # [ 97.673572] nix-grpc-daemon[790]: ts=2026-09-22T10:57:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570005641worker2 # [ 97.682059] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5642worker2 # [ 97.685489] nix-daemon[859]: reaped child process 1259, status = succeeded5643worker2 # [ 97.689084] nix-grpc-daemon[790]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5644worker2 # [ 97.698354] nix-daemon[859]: accepted connection from pid 790, user nix-grpc-daemon (trusted)5645client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5646client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5647lb # [ 98.235075] envoy[921]: conn peer=[2001:db8:1::1]:51384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=995648lb # [ 98.236769] envoy[921]: conn peer=[2001:db8:1::1]:51382 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=2180 ms=23285649lb # [ 98.238204] envoy[921]: conn peer=[2001:db8:1::1]:51410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=645650lb # [ 98.239585] envoy[921]: conn peer=[2001:db8:1::1]:51398 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4329 tx=2387 ms=20505651lb # [ 98.241022] envoy[921]: conn peer=[2001:db8:1::1]:38300 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3865 tx=2080 ms=23025652lb # [ 98.242512] envoy[921]: conn peer=[2001:db8:1::1]:38312 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=555653client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5654worker2 # [ 98.917571] nix-daemon[859]: reaped child process 1279, status = succeeded5655worker2 # [ 99.043077] nix-daemon[859]: accepted connection from pid 1295, user nix-grpc-daemon (trusted)5656worker2 # [ 99.054714] nix-daemon[859]: reaped child process 1298, status = succeeded5657lb # [ 99.037293] niks3-server[1003]: 2026/09/22 10:57:10 INFO Received uploads request method=POST path=/api/pending_closures5658worker2 # [ 99.075221] nix-grpc-daemon[872]: time=2026-09-22T10:57:11.230Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5659lb # [ 99.051773] niks3-server[1003]: 2026/09/22 10:57:10 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5660lb # [ 99.054365] niks3-server[1003]: 2026/09/22 10:57:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5661lb # [ 99.055995] niks3-server[1003]: 2026/09/22 10:57:10 INFO Signed narinfos id=32 count=15662worker2 # [ 99.089435] nix-grpc-daemon[872]: time=2026-09-22T10:57:11.245Z level=INFO msg="Uploading 1 narinfos"5663lb # [ 99.059939] niks3-server[1003]: 2026/09/22 10:57:10 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5664lb # [ 99.065240] niks3-server[1003]: 2026/09/22 10:57:10 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5665lb # [ 99.071712] niks3-server[1003]: 2026/09/22 10:57:10 INFO Completed upload id=325666worker2 # [ 99.105169] nix-grpc-daemon[872]: time=2026-09-22T10:57:11.260Z level=INFO msg="Upload complete. (200ms)"5667lb # [ 99.073773] niks3-server[1003]: 2026/09/22 10:57:10 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5668worker2 # [ 99.107704] nix-grpc-daemon[790]: ts=2026-09-22T10:57:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15669worker2 # [ 99.114087] nix-daemon[859]: reaped child process 1276, status = succeeded5670client # copying 1 paths...5671worker2 # [ 99.153484] nix-grpc-daemon[790]: ts=2026-09-22T10:57:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:570005672client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5673worker1 # [ 99.121608] nix-grpc-daemon[784]: ts=2026-09-22T10:57:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:418045674worker1 # [ 99.129703] nix-grpc-daemon[784]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5675worker1 # [ 99.138243] nix-grpc-daemon[784]: ts=2026-09-22T10:57:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_out=1205676client # [ 99.258595] nix-daemon[1108]: reaped child process 1216, status = succeeded5677client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.33 seconds)5678client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25679client # [ 99.443242] nix-daemon[1108]: accepted connection from pid 1235, user root (trusted)5680client # this derivation will be built:5681client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5682worker1 # [ 99.607082] nix-grpc-daemon[784]: ts=2026-09-22T10:57:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:418045683client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5684worker1 # [ 99.612947] nix-grpc-daemon[784]: ts=2026-09-22T10:57:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418045685worker1 # [ 99.618833] nix-grpc-daemon[919]: time=2026-09-22T10:57:11.525Z level=INFO msg="All 1 paths already cached"5686worker1 # [ 99.627856] nix-grpc-daemon[784]: ts=2026-09-22T10:57:11Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05687worker1 # [ 99.634328] nix-grpc-daemon[784]: ts=2026-09-22T10:57:11Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15688worker1 # [ 99.639291] nix-grpc-daemon[784]: ts=2026-09-22T10:57:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045689worker1 # [ 99.650253] nix-grpc-daemon[784]: ts=2026-09-22T10:57:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418045690worker1 # [ 99.661965] nix-grpc-daemon[919]: time=2026-09-22T10:57:11.570Z level=INFO msg="All 1 paths already cached"5691client # copying 1 paths to 'lb:50051'...5692client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5693worker1 # [ 99.669837] nix-grpc-daemon[784]: ts=2026-09-22T10:57:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418045694worker1 # [ 99.680647] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5695worker1 # [ 99.838170] nix-daemon[866]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5696worker1 # [ 99.852718] nix-daemon[866]: reaped child process 1226, status = succeeded5697worker1 # [ 100.031484] nix-daemon[866]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5698worker1 # [ 100.044794] nix-daemon[866]: reaped child process 1231, status = succeeded5699lb # [ 100.076540] niks3-server[1003]: 2026/09/22 10:57:11 INFO Received uploads request method=POST path=/api/pending_closures5700worker1 # [ 100.071900] nix-grpc-daemon[919]: time=2026-09-22T10:57:11.980Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5701worker1 # [ 100.073331] nix-grpc-daemon[919]: time=2026-09-22T10:57:11.980Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5702lb # [ 100.104177] niks3-server[1003]: 2026/09/22 10:57:11 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5703lb # [ 100.113246] niks3-server[1003]: 2026/09/22 10:57:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5704worker1 # [ 100.106550] nix-grpc-daemon[919]: time=2026-09-22T10:57:12.014Z level=INFO msg="Uploading 1 narinfos"5705lb # [ 100.119809] niks3-server[1003]: 2026/09/22 10:57:11 INFO Signed narinfos id=33 count=15706lb # [ 100.124597] niks3-server[1003]: 2026/09/22 10:57:11 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5707lb # [ 100.136072] niks3-server[1003]: 2026/09/22 10:57:11 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5708worker1 # [ 100.128201] nix-grpc-daemon[919]: time=2026-09-22T10:57:12.036Z level=INFO msg="Upload complete. (437ms)"5709lb # [ 100.141164] niks3-server[1003]: 2026/09/22 10:57:11 INFO Completed upload id=335710worker1 # [ 100.130880] nix-grpc-daemon[784]: ts=2026-09-22T10:57:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=0 paths=1 nar_bytes_in=5605711lb # [ 100.144622] niks3-server[1003]: 2026/09/22 10:57:11 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5712worker1 # [ 100.135835] nix-grpc-daemon[784]: ts=2026-09-22T10:57:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418045713worker1 # [ 100.142402] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5714worker1 # [ 100.144887] nix-daemon[866]: reaped child process 1221, status = succeeded5715client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5716worker1 # [ 100.150528] nix-daemon[866]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5717client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5718client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5719worker1 # [ 100.978821] nix-daemon[866]: reaped child process 1235, status = succeeded5720worker1 # [ 101.114070] nix-daemon[866]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5721worker1 # [ 101.126104] nix-daemon[866]: reaped child process 1254, status = succeeded5722lb # [ 101.152369] niks3-server[1003]: 2026/09/22 10:57:12 INFO Received uploads request method=POST path=/api/pending_closures5723worker1 # [ 101.147679] nix-grpc-daemon[919]: time=2026-09-22T10:57:13.056Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5724lb # [ 101.180700] niks3-server[1003]: 2026/09/22 10:57:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5725lb # [ 101.189115] niks3-server[1003]: 2026/09/22 10:57:12 INFO Signed narinfos id=34 count=15726worker1 # [ 101.179397] nix-grpc-daemon[919]: time=2026-09-22T10:57:13.087Z level=INFO msg="Uploading 1 narinfos"5727lb # [ 101.195047] niks3-server[1003]: 2026/09/22 10:57:12 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5728lb # [ 101.201211] niks3-server[1003]: 2026/09/22 10:57:12 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5729lb # [ 101.212901] niks3-server[1003]: 2026/09/22 10:57:12 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5730lb # [ 101.215355] niks3-server[1003]: 2026/09/22 10:57:12 INFO Completed upload id=345731worker1 # [ 101.203961] nix-grpc-daemon[919]: time=2026-09-22T10:57:13.112Z level=INFO msg="Upload complete. (246ms)"5732lb # [ 101.217163] niks3-server[1003]: 2026/09/22 10:57:12 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5733worker1 # [ 101.207791] nix-grpc-daemon[784]: ts=2026-09-22T10:57:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41804 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15734worker1 # [ 101.214522] nix-daemon[866]: reaped child process 1233, status = succeeded5735client # copying 1 paths...5736worker2 # [ 101.300461] nix-grpc-daemon[790]: ts=2026-09-22T10:57:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:570005737worker2 # [ 101.307260] nix-grpc-daemon[790]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5738client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5739worker2 # [ 101.331360] nix-grpc-daemon[790]: ts=2026-09-22T10:57:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:570005740worker2 # [ 101.333802] nix-grpc-daemon[790]: ts=2026-09-22T10:57:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57000 duration_s=0 paths=1 nar_bytes_out=1205741client # [ 101.424621] nix-daemon[1108]: reaped child process 1238, status = succeeded5742client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.16 seconds)5743(finished: subtest: an input only one worker has still reaches the builder, in 9.03 seconds)5744subtest: upload whose reference the worker lacks is completed from the cache5745client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5746client # this derivation will be built:5747client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5748client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5749client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.40 seconds)5750client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5751client # this derivation will be built:5752client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5753client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5754client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.41 seconds)5755client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5756client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.19 seconds)5757worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5758worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.23 seconds)5759worker2 # [ 102.726377] systemd[1]: Stopping Nix worker-protocol over gRPC...5760worker2 # [ 103.670453] nix-grpc-daemon[790]: ts=2026-09-22T10:57:15Z level=info event=signal_exit5761worker2 # [ 104.745992] nix-grpc-daemon[790]: ts=2026-09-22T10:57:16Z level=info event=scheduler_restarting5762worker1 # [ 104.706267] nix-grpc-daemon[784]: ts=2026-09-22T10:57:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:417945763worker1 # [ 104.709372] nix-grpc-daemon[784]: ts=2026-09-22T10:57:16Z level=info event=worker_gone addr=192.168.1.4:500515764worker2 # [ 104.758545] nix-grpc-daemon[790]: ts=2026-09-22T10:57:16Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5765worker2 # [ 104.765621] nix-daemon[859]: reaped child process 861, status = succeeded5766worker2 # [ 104.806820] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5767worker2 # [ 104.808647] systemd[1]: Stopped Nix worker-protocol over gRPC.5768worker2 # [ 104.810304] systemd[1]: nix-grpc-daemon.service: Consumed 3.045s CPU time over 1min 26.421s wall clock time, 61.6M memory peak, 100.6K incoming IP traffic, 235K outgoing IP traffic.5769worker2 # [ 104.813912] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5770worker2 # [ 104.815650] systemd[1]: Closed Nix worker-protocol over gRPC.5771worker2 # [ 104.817135] systemd[1]: nix-grpc-daemon.socket: Consumed 283us CPU time over 1min 35.555s wall clock time, 256K memory peak, 102.8K incoming IP traffic, 91.5K outgoing IP traffic.5772lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5773lb: must succeed: curl -sf localhost:9901/clusters5774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5775lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5776client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25777worker1 # [ 105.052687] nix-grpc-daemon[784]: ts=2026-09-22T10:57:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:418045778worker1 # [ 105.059745] nix-grpc-daemon[784]: ts=2026-09-22T10:57:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:418045779client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5780client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.24 seconds)5781worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5782worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5783worker2 # [ 105.181750] systemd[1]: Listening on Nix worker-protocol over gRPC.5784worker1 # [ 105.197313] systemd[1]: Stopping Nix worker-protocol over gRPC...5785worker1 # [ 105.528353] nix-grpc-daemon[784]: ts=2026-09-22T10:57:17Z level=info event=signal_exit5786worker2 # [ 105.830871] systemd[1]: Starting scheduler-token.service...5787lb # [ 105.816475] niks3-server[1003]: 2026/09/22 10:57:17 INFO lead: released remote=192.168.1.3:336405788worker2 # [ 105.921754] systemd[1]: scheduler-token.service: Deactivated successfully.5789worker2 # [ 105.923600] systemd[1]: Finished scheduler-token.service.5790worker2 # [ 105.924758] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 93ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5791worker2 # [ 105.929965] systemd[1]: Starting Nix worker-protocol over gRPC...5792worker2 # [ 106.098440] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5793worker2 # [ 106.100644] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5794worker2 # [ 106.104342] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15795worker2 # [ 106.111065] nix-grpc-daemon[1323]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5796worker2 # [ 106.112345] nix-grpc-daemon[1323]: E0000 00:00:1790074638.266231 1323 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5797worker2 # [ 106.117949] nix-grpc-daemon[1323]: E0000 00:00:1790074638.271192 1323 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5798worker2 # [ 106.121114] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5799worker2 # [ 106.124291] systemd[1]: Started Nix worker-protocol over gRPC.5800lb # [ 106.094446] niks3-server[1003]: 2026/09/22 10:57:17 INFO lead: acquired remote=192.168.1.4:424605801worker2 # [ 106.128056] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=scheduler_take_over5802worker2 # [ 106.133868] nix-grpc-daemon[1323]: E0000 00:00:1790074638.288501 1365 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5803worker2 # [ 106.144614] nix-grpc-daemon[1323]: E0000 00:00:1790074638.300259 1365 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5804worker2 # [ 106.148902] nix-grpc-daemon[1323]: E0000 00:00:1790074638.302439 1365 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5805worker2 # [ 106.158572] nix-grpc-daemon[1323]: E0000 00:00:1790074638.314218 1365 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5806worker2 # [ 106.167205] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=scheduler_connected addr=lb:500515807worker2 # [ 106.168740] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5808worker2 # [ 106.370202] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=scheduler_connected addr=lb:500515809worker2 # [ 106.374156] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5810worker2 # [ 106.575130] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=scheduler_connected addr=lb:500515811worker2 # [ 106.578918] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5812worker2 # [ 106.779835] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=scheduler_connected addr=lb:500515813worker2 # [ 106.783718] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5814worker1 # [ 106.802894] nix-grpc-daemon[784]: ts=2026-09-22T10:57:18Z level=info event=scheduler_restarting5815worker1 # [ 106.807626] nix-grpc-daemon[784]: ts=2026-09-22T10:57:18Z level=info event=scheduler_restarting addr=lb:500515816worker1 # [ 106.809553] nix-grpc-daemon[784]: ts=2026-09-22T10:57:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:417945817worker1 # [ 106.811189] nix-grpc-daemon[784]: ts=2026-09-22T10:57:18Z level=info event=worker_gone addr=192.168.1.3:500515818worker1 # [ 106.824096] nix-daemon[866]: reaped child process 869, status = succeeded5819worker2 # [ 106.984820] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:19Z level=info event=scheduler_connected addr=lb:500515820worker2 # [ 106.988649] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5821worker1 # [ 107.011560] nix-grpc-daemon[784]: ts=2026-09-22T10:57:18Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5822worker2 # [ 107.190451] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:19Z level=info event=scheduler_connected addr=lb:500515823worker2 # [ 107.203142] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:19Z level=info event=oidc_jwks provider=mock keys=15824worker2 # [ 107.206350] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:19Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:380825825worker2 # [ 107.211685] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05826worker1 # [ 107.719593] nix-grpc-daemon[784]: E0000 00:00:1790074639.627985 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5827worker1 # [ 107.724472] nix-grpc-daemon[784]: E0000 00:00:1790074639.628108 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5828lb # [ 108.235869] envoy[921]: conn peer=[2001:db8:1::1]:38326 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2126 ms=29655829lb # [ 108.237605] envoy[921]: conn peer=[2001:db8:1::1]:38340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=665830lb # [ 108.239053] envoy[921]: conn peer=[2001:db8:1::1]:38346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3742 tx=2124 ms=17335831lb # [ 108.240501] envoy[921]: conn peer=[2001:db8:1::1]:38362 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545832lb # [ 108.241845] envoy[921]: conn peer=[2001:db8:1::4]:51838 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=66358 tx=3571 ms=860285833lb # [ 108.243377] envoy[921]: conn peer=[2001:db8:1::1]:55024 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215834lb # [ 108.244720] envoy[921]: conn peer=[2001:db8:1::3]:36810 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=16267 tx=3265 ms=874285835worker1 # [ 108.721723] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5836worker1 # [ 108.727145] systemd[1]: Stopped Nix worker-protocol over gRPC.5837worker1 # [ 108.729290] systemd[1]: nix-grpc-daemon.service: Consumed 2.670s CPU time over 1min 30.511s wall clock time, 59.2M memory peak, 94.8K incoming IP traffic, 203.4K outgoing IP traffic.5838worker1 # [ 108.740515] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5839worker1 # [ 108.744153] systemd[1]: Closed Nix worker-protocol over gRPC.5840worker1 # [ 108.747455] systemd[1]: nix-grpc-daemon.socket: Consumed 307us CPU time over 1min 39.148s wall clock time, 256K memory peak, 117.3K incoming IP traffic, 100.9K outgoing IP traffic.5841lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5842lb: must succeed: curl -sf localhost:9901/clusters5843lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5844lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5845client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25846worker2 # [ 109.068428] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380945847worker2 # [ 109.073733] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5848worker2 # [ 109.081483] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 1/5)5849worker2 # [ 109.159967] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 2/5)5850worker2 # [ 109.287658] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 366 ms (attempt 3/5)5851worker2 # [ 109.656614] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 313 ms (attempt 4/5)5852worker2 # [ 109.975809] nix-grpc-daemon[1323]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5853worker2 # [ 109.979634] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5854client # copying 2 paths...5855client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5856worker2 # [ 110.001533] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380945857client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5858worker2 # [ 110.010308] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5859worker2 # [ 110.022426] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:22Z level=info event=niks3_push_started pid=13835860worker2 # [ 110.185345] nix-grpc-daemon[1383]: time=2026-09-22T10:57:22.340Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5861worker2 # [ 110.352454] nix-daemon[859]: accepted connection from pid 1396, user nix-grpc-daemon (trusted)5862worker2 # [ 110.363143] nix-daemon[859]: reaped child process 1399, status = succeeded5863lb # [ 110.346550] niks3-server[1003]: 2026/09/22 10:57:21 INFO Received uploads request method=POST path=/api/pending_closures5864lb # [ 110.357620] niks3-server[1003]: 2026/09/22 10:57:21 INFO Received uploads request method=POST path=/api/pending_closures5865worker2 # [ 110.398216] nix-grpc-daemon[1383]: time=2026-09-22T10:57:22.553Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5866worker2 # [ 110.399763] nix-grpc-daemon[1383]: time=2026-09-22T10:57:22.555Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5867worker2 # [ 110.401746] nix-grpc-daemon[1383]: time=2026-09-22T10:57:22.557Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5868lb # [ 110.406480] niks3-server[1003]: 2026/09/22 10:57:21 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5869lb # [ 110.413916] niks3-server[1003]: 2026/09/22 10:57:21 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5870lb # [ 110.419391] niks3-server[1003]: 2026/09/22 10:57:21 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5871lb # [ 110.425853] niks3-server[1003]: 2026/09/22 10:57:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5872lb # [ 110.429083] niks3-server[1003]: 2026/09/22 10:57:21 INFO Signed narinfos id=35 count=15873lb # [ 110.434313] niks3-server[1003]: 2026/09/22 10:57:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5874lb # [ 110.437682] niks3-server[1003]: 2026/09/22 10:57:21 INFO Signed narinfos id=36 count=25875lb # [ 110.439720] niks3-server[1003]: 2026/09/22 10:57:21 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5876worker2 # [ 110.473161] nix-grpc-daemon[1383]: time=2026-09-22T10:57:22.628Z level=INFO msg="Uploading 3 narinfos"5877lb # [ 110.453621] niks3-server[1003]: 2026/09/22 10:57:21 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5878lb # [ 110.457034] niks3-server[1003]: 2026/09/22 10:57:21 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5879lb # [ 110.461415] niks3-server[1003]: 2026/09/22 10:57:21 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5880lb # [ 110.464401] niks3-server[1003]: 2026/09/22 10:57:21 INFO Completed upload id=355881lb # [ 110.466063] niks3-server[1003]: 2026/09/22 10:57:21 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5882lb # [ 110.467978] niks3-server[1003]: 2026/09/22 10:57:21 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5883lb # [ 110.470878] niks3-server[1003]: 2026/09/22 10:57:21 INFO Completed upload id=365884worker2 # [ 110.504654] nix-grpc-daemon[1383]: time=2026-09-22T10:57:22.660Z level=INFO msg="Upload complete. (315ms)"5885worker2 # [ 110.511342] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38094 duration_s=0 paths=2 nar_bytes_in=3125886worker2 # [ 110.523176] nix-daemon[859]: reaped child process 1381, status = succeeded5887client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.67 seconds)5888worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5889worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5890worker1 # [ 110.567474] systemd[1]: Listening on Nix worker-protocol over gRPC.5891worker1 # [ 110.574526] systemd[1]: Starting Nix worker-protocol over gRPC...5892worker1 # [ 110.729858] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5893worker1 # [ 110.732179] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5894worker1 # [ 110.736056] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15895worker1 # [ 110.744067] nix-grpc-daemon[1277]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5896worker1 # [ 110.745476] nix-grpc-daemon[1277]: E0000 00:00:1790074642.651720 1277 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5897worker1 # [ 110.752069] nix-grpc-daemon[1277]: E0000 00:00:1790074642.657083 1277 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5898worker1 # [ 110.754201] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:22Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5899worker1 # [ 110.755739] systemd[1]: Started Nix worker-protocol over gRPC.5900worker1 # [ 110.766332] nix-grpc-daemon[1277]: E0000 00:00:1790074642.674815 1318 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5901lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5902lb: must succeed: curl -sf localhost:9901/clusters5903worker1 # [ 110.780061] nix-grpc-daemon[1277]: E0000 00:00:1790074642.687562 1318 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5904worker1 # [ 110.782260] nix-grpc-daemon[1277]: E0000 00:00:1790074642.687718 1318 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5905worker1 # [ 110.793030] nix-grpc-daemon[1277]: E0000 00:00:1790074642.701559 1318 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5906worker1 # [ 110.800256] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:22Z level=info event=scheduler_connected addr=lb:500515907worker2 # [ 110.847776] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:23Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:380825908worker2 # [ 110.852873] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05909lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5910lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5911(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.42 seconds)5912subtest: a niks3 restart does not cost the leader its role5913lb: must succeed: curl -sf localhost:9901/clusters5914lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5915lb: must succeed: curl -sf localhost:9901/clusters5916lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5917lb # [ 110.948092] niks3-server[1003]: 2026/09/22 10:57:22 INFO Shutdown signal received, draining in-flight requests timeout=10s5918lb # [ 110.951264] systemd[1]: Stopping niks3 server...5919lb # [ 110.954662] niks3-server[1003]: 2026/09/22 10:57:22 INFO lead: released remote=192.168.1.4:424605920worker2 # [ 110.988456] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:23Z level=info event=scheduler_lock_lost status=2005921worker1 # [ 110.942347] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:22Z level=info event=scheduler_lock_lost status=2005922lb # [ 110.966953] systemd[1]: niks3.service: Deactivated successfully.5923lb # [ 110.968572] systemd[1]: Stopped niks3 server.5924lb # [ 110.969417] systemd[1]: niks3.service: Consumed 742ms CPU time over 1min 27.664s wall clock time, 22.3M memory peak, 97.9K incoming IP traffic, 139.7K outgoing IP traffic.5925lb # [ 110.978230] systemd[1]: Starting niks3 server...5926lb # [ 111.048354] niks3-server[1141]: 2026/09/22 10:57:22 goose: no migrations to run. current version: 202609200000005927lb # [ 111.054305] niks3-server[1141]: 2026/09/22 10:57:22 OK 1_commit_pending_closure.sql (4.62ms)5928lb # [ 111.058163] niks3-server[1141]: 2026/09/22 10:57:22 OK 2_object_stats_trigger.sql (2.23ms)5929lb # [ 111.059274] niks3-server[1141]: 2026/09/22 10:57:22 goose: up to current file version: 25930lb # [ 111.062421] niks3-server[1141]: 2026/09/22 10:57:22 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key5931lb # [ 111.074953] niks3-server[1141]: 2026/09/22 10:57:22 INFO Read proxy enabled — serving cache objects from S35932lb # [ 111.076270] niks3-server[1141]: 2026/09/22 10:57:22 INFO Using socket-activated listener address=0.0.0.0:57515933lb # [ 111.078227] niks3-server[1141]: 2026/09/22 10:57:22 INFO systemd watchdog enabled interval=15s5934lb # [ 111.079436] niks3-server[1141]: 2026/09/22 10:57:22 INFO Starting HTTP server address=0.0.0.0:57515935lb # [ 111.080788] systemd[1]: Started niks3 server.5936lb: waiting for unit niks3.service5937lb: (finished: waiting for unit niks3.service, in 0.04 seconds)5938lb: must succeed: curl -sf localhost:9901/clusters5939lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5940worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5941worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5942lb: must succeed: curl -sf localhost:9901/clusters5943lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5944lb: must succeed: curl -sf localhost:9901/clusters5945lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5946worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5947worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5948client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart >&25949client # don't know how to build these paths:5950client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5951worker2 # [ 111.578461] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380945952worker1 # [ 111.539100] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355685953worker1 # [ 111.550926] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)5954worker1 # [ 111.568338] nix-grpc-daemon[1277]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5)5955worker1 # [ 111.653338] nix-grpc-daemon[1277]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 2/5)5956worker1 # [ 111.717648] nix-grpc-daemon[1277]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 376 ms (attempt 3/5)5957lb # [ 111.962178] niks3-server[1141]: 2026/09/22 10:57:23 INFO lead: acquired remote=192.168.1.4:425125958worker1 # [ 112.100388] nix-grpc-daemon[1277]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 559 ms (attempt 4/5)5959worker1 # [ 112.665488] nix-grpc-daemon[1277]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5960worker1 # [ 112.669572] nix-grpc-daemon[1277]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5961worker2 # [ 112.757272] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:24Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05962worker2 # [ 112.761237] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:24Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15963worker2 # [ 112.798854] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380945964worker2 # [ 112.805697] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380945965client # copying 1 paths to 'lb:50051'...5966client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5967worker2 # [ 112.814067] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380945968worker2 # [ 112.820404] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5969worker2 # [ 112.984294] nix-daemon[859]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5970worker2 # [ 112.996541] nix-daemon[859]: reaped child process 1421, status = succeeded5971worker2 # [ 113.157216] nix-daemon[859]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5972worker2 # [ 113.168875] nix-daemon[859]: reaped child process 1426, status = succeeded5973lb # [ 113.148680] niks3-server[1141]: 2026/09/22 10:57:24 INFO Received uploads request method=POST path=/api/pending_closures5974worker2 # [ 113.194048] nix-grpc-daemon[1383]: time=2026-09-22T10:57:25.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5975worker2 # [ 113.195562] nix-grpc-daemon[1383]: time=2026-09-22T10:57:25.349Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5976lb # [ 113.190655] niks3-server[1141]: 2026/09/22 10:57:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5977lb # [ 113.197934] niks3-server[1141]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5978lb # [ 113.218078] niks3-server[1141]: 2026/09/22 10:57:24 INFO Signed narinfos id=37 count=15979worker2 # [ 113.253455] nix-grpc-daemon[1383]: time=2026-09-22T10:57:25.408Z level=INFO msg="Uploading 1 narinfos"5980lb # [ 113.224564] niks3-server[1141]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5981lb # [ 113.227138] niks3-server[1141]: 2026/09/22 10:57:24 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5982lb # [ 113.232995] niks3-server[1141]: 2026/09/22 10:57:24 INFO Completed upload id=375983worker2 # [ 113.266628] nix-grpc-daemon[1383]: time=2026-09-22T10:57:25.422Z level=INFO msg="Upload complete. (441ms)"5984lb # [ 113.236951] niks3-server[1141]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5985worker2 # [ 113.271709] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38094 duration_s=0 paths=1 nar_bytes_in=5365986worker2 # [ 113.277502] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380945987worker2 # [ 113.287478] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5988worker2 # [ 113.297419] nix-daemon[859]: reaped child process 1416, status = succeeded5989client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5990worker2 # [ 113.305322] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5991client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5992client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5993client # farm-a> LOG-farm-a-after-niks3-restart5994worker2 # [ 114.316921] nix-daemon[859]: reaped child process 1430, status = succeeded5995worker2 # [ 114.455350] nix-daemon[859]: accepted connection from pid 1446, user nix-grpc-daemon (trusted)5996worker2 # [ 114.467212] nix-daemon[859]: reaped child process 1449, status = succeeded5997lb # [ 114.452531] niks3-server[1141]: 2026/09/22 10:57:25 INFO Received uploads request method=POST path=/api/pending_closures5998worker2 # [ 114.495686] nix-grpc-daemon[1383]: time=2026-09-22T10:57:26.651Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5999worker2 # [ 114.499511] nix-grpc-daemon[1383]: time=2026-09-22T10:57:26.652Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"6000lb # [ 114.487604] niks3-server[1141]: 2026/09/22 10:57:25 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6001lb # [ 114.497305] niks3-server[1141]: 2026/09/22 10:57:25 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst6002lb # [ 114.508242] niks3-server[1141]: 2026/09/22 10:57:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6003worker2 # [ 114.547334] nix-grpc-daemon[1383]: time=2026-09-22T10:57:26.702Z level=INFO msg="Uploading 1 narinfos"6004lb # [ 114.513509] niks3-server[1141]: 2026/09/22 10:57:25 INFO Signed narinfos id=38 count=16005lb # [ 114.517057] niks3-server[1141]: 2026/09/22 10:57:25 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls6006lb # [ 114.527517] niks3-server[1141]: 2026/09/22 10:57:25 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6007lb # [ 114.531936] niks3-server[1141]: 2026/09/22 10:57:25 INFO Completed upload id=386008worker2 # [ 114.565401] nix-grpc-daemon[1383]: time=2026-09-22T10:57:26.721Z level=INFO msg="Upload complete. (254ms)"6009lb # [ 114.534131] niks3-server[1141]: 2026/09/22 10:57:25 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6010worker2 # [ 114.568146] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38094 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16011worker2 # [ 114.578174] nix-daemon[859]: reaped child process 1428, status = succeeded6012worker2 # [ 114.584544] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:26Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06013worker2 # [ 114.586202] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:26Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=16014worker2 # [ 114.631630] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380946015worker2 # [ 114.640178] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380946016worker2 # [ 114.649478] nix-grpc-daemon[1383]: time=2026-09-22T10:57:26.805Z level=INFO msg="All 1 paths already cached"6017client # copying 1 paths to 'lb:50051'...6018client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6019worker2 # [ 114.655244] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380946020worker2 # [ 114.661411] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6021worker2 # [ 114.819852] nix-daemon[859]: accepted connection from pid 1455, user nix-grpc-daemon (trusted)6022worker2 # [ 114.831955] nix-daemon[859]: reaped child process 1458, status = succeeded6023worker2 # [ 114.995284] nix-daemon[859]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)6024worker2 # [ 115.007451] nix-daemon[859]: reaped child process 1463, status = succeeded6025lb # [ 114.987448] niks3-server[1141]: 2026/09/22 10:57:26 INFO Received uploads request method=POST path=/api/pending_closures6026worker2 # [ 115.031352] nix-grpc-daemon[1383]: time=2026-09-22T10:57:27.186Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6027worker2 # [ 115.032763] nix-grpc-daemon[1383]: time=2026-09-22T10:57:27.186Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6028lb # [ 115.014689] niks3-server[1141]: 2026/09/22 10:57:26 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6029lb # [ 115.023520] niks3-server[1141]: 2026/09/22 10:57:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6030lb # [ 115.028365] niks3-server[1141]: 2026/09/22 10:57:26 INFO Signed narinfos id=39 count=16031worker2 # [ 115.064822] nix-grpc-daemon[1383]: time=2026-09-22T10:57:27.220Z level=INFO msg="Uploading 1 narinfos"6032lb # [ 115.040918] niks3-server[1141]: 2026/09/22 10:57:26 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6033lb # [ 115.047606] niks3-server[1141]: 2026/09/22 10:57:26 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6034lb # [ 115.053121] niks3-server[1141]: 2026/09/22 10:57:26 INFO Completed upload id=396035worker2 # [ 115.088798] nix-grpc-daemon[1383]: time=2026-09-22T10:57:27.244Z level=INFO msg="Upload complete. (422ms)"6036worker2 # [ 115.091170] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38094 duration_s=0 paths=1 nar_bytes_in=6566037lb # [ 115.057527] niks3-server[1141]: 2026/09/22 10:57:26 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6038worker2 # [ 115.096695] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380946039worker2 # [ 115.104519] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6040worker2 # [ 115.106827] nix-daemon[859]: reaped child process 1453, status = succeeded6041worker2 # [ 115.112072] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6042client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6043client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6044client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6045client # farm-b> LOG-farm-b-after-niks3-restart6046worker2 # [ 115.725247] nix-daemon[859]: reaped child process 1467, status = succeeded6047worker2 # [ 115.840532] nix-daemon[859]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)6048worker2 # [ 115.852342] nix-daemon[859]: reaped child process 1486, status = succeeded6049lb # [ 115.833297] niks3-server[1141]: 2026/09/22 10:57:27 INFO Received uploads request method=POST path=/api/pending_closures6050worker2 # [ 115.877182] nix-grpc-daemon[1383]: time=2026-09-22T10:57:28.032Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6051worker2 # [ 115.878540] nix-grpc-daemon[1383]: time=2026-09-22T10:57:28.032Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6052lb # [ 115.868156] niks3-server[1141]: 2026/09/22 10:57:27 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6053lb # [ 115.879722] niks3-server[1141]: 2026/09/22 10:57:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6054worker2 # [ 115.917689] nix-grpc-daemon[1383]: time=2026-09-22T10:57:28.073Z level=INFO msg="Uploading 1 narinfos"6055lb # [ 115.884672] niks3-server[1141]: 2026/09/22 10:57:27 INFO Signed narinfos id=40 count=16056lb # [ 115.886638] niks3-server[1141]: 2026/09/22 10:57:27 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6057lb # [ 115.892408] niks3-server[1141]: 2026/09/22 10:57:27 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6058lb # [ 115.897955] niks3-server[1141]: 2026/09/22 10:57:27 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6059lb # [ 115.900791] niks3-server[1141]: 2026/09/22 10:57:27 INFO Completed upload id=406060worker2 # [ 115.934244] nix-grpc-daemon[1383]: time=2026-09-22T10:57:28.089Z level=INFO msg="Upload complete. (216ms)"6061lb # [ 115.902913] niks3-server[1141]: 2026/09/22 10:57:27 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6062worker2 # [ 115.937574] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38094 duration_s=0 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16063worker2 # [ 115.948401] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:28Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06064worker2 # [ 115.950358] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:28Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=16065worker2 # [ 115.954972] nix-daemon[859]: reaped child process 1465, status = succeeded6066worker2 # [ 115.992941] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380946067worker2 # [ 116.005488] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380946068worker2 # [ 116.018921] nix-grpc-daemon[1383]: time=2026-09-22T10:57:28.174Z level=INFO msg="All 2 paths already cached"6069client # copying 1 paths to 'lb:50051'...6070client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6071worker2 # [ 116.028336] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380946072worker2 # [ 116.036563] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6073worker2 # [ 116.203625] nix-daemon[859]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)6074worker2 # [ 116.216319] nix-daemon[859]: reaped child process 1495, status = succeeded6075worker2 # [ 116.376455] nix-daemon[859]: accepted connection from pid 1497, user nix-grpc-daemon (trusted)6076worker2 # [ 116.388385] nix-daemon[859]: reaped child process 1500, status = succeeded6077lb # [ 116.369117] niks3-server[1141]: 2026/09/22 10:57:27 INFO Received uploads request method=POST path=/api/pending_closures6078worker2 # [ 116.410692] nix-grpc-daemon[1383]: time=2026-09-22T10:57:28.566Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6079worker2 # [ 116.412535] nix-grpc-daemon[1383]: time=2026-09-22T10:57:28.567Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6080lb # [ 116.398845] niks3-server[1141]: 2026/09/22 10:57:27 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6081lb # [ 116.407296] niks3-server[1141]: 2026/09/22 10:57:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6082worker2 # [ 116.445139] nix-grpc-daemon[1383]: time=2026-09-22T10:57:28.600Z level=INFO msg="Uploading 1 narinfos"6083lb # [ 116.413247] niks3-server[1141]: 2026/09/22 10:57:27 INFO Signed narinfos id=41 count=16084lb # [ 116.417914] niks3-server[1141]: 2026/09/22 10:57:27 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6085lb # [ 116.427657] niks3-server[1141]: 2026/09/22 10:57:27 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6086lb # [ 116.435783] niks3-server[1141]: 2026/09/22 10:57:27 INFO Completed upload id=416087worker2 # [ 116.469171] nix-grpc-daemon[1383]: time=2026-09-22T10:57:28.624Z level=INFO msg="Upload complete. (419ms)"6088worker2 # [ 116.472866] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38094 duration_s=0 paths=1 nar_bytes_in=7846089lb # [ 116.440656] niks3-server[1141]: 2026/09/22 10:57:27 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6090worker2 # [ 116.478406] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380946091worker2 # [ 116.487145] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6092worker2 # [ 116.492060] nix-daemon[859]: reaped child process 1490, status = succeeded6093worker2 # [ 116.497059] nix-daemon[859]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)6094client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6095client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6096client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6097client # farm-top> LOG-farm-top-after-niks3-restart6098worker2 # [ 117.523722] nix-daemon[859]: reaped child process 1504, status = succeeded6099worker2 # [ 117.675831] nix-daemon[859]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)6100worker2 # [ 117.686757] nix-daemon[859]: reaped child process 1523, status = succeeded6101lb # [ 117.669403] niks3-server[1141]: 2026/09/22 10:57:29 INFO Received uploads request method=POST path=/api/pending_closures6102worker2 # [ 117.707024] nix-grpc-daemon[1383]: time=2026-09-22T10:57:29.862Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6103worker2 # [ 117.708571] nix-grpc-daemon[1383]: time=2026-09-22T10:57:29.864Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6104lb # [ 117.690354] niks3-server[1141]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6105lb # [ 117.693614] niks3-server[1141]: 2026/09/22 10:57:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6106lb # [ 117.696680] niks3-server[1141]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6107lb # [ 117.698277] niks3-server[1141]: 2026/09/22 10:57:29 INFO Signed narinfos id=42 count=16108lb # [ 117.700392] niks3-server[1141]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6109worker2 # [ 117.732964] nix-grpc-daemon[1383]: time=2026-09-22T10:57:29.888Z level=INFO msg="Uploading 1 narinfos"6110lb # [ 117.705024] niks3-server[1141]: 2026/09/22 10:57:29 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6111lb # [ 117.708845] niks3-server[1141]: 2026/09/22 10:57:29 INFO Completed upload id=426112worker2 # [ 117.741991] nix-grpc-daemon[1383]: time=2026-09-22T10:57:29.897Z level=INFO msg="Upload complete. (236ms)"6113lb # [ 117.710586] niks3-server[1141]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6114worker2 # [ 117.744575] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38094 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16115worker2 # [ 117.751865] nix-daemon[859]: reaped child process 1502, status = succeeded6116client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart >&2, in 6.40 seconds)6117client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart outPath6118client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)6119(finished: subtest: a niks3 restart does not cost the leader its role, in 7.10 seconds)6120subtest: scheduler down: the other node takes the lock and keeps it6121lb: must succeed: curl -sf localhost:9901/clusters6122lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6123lb: must succeed: curl -sf localhost:9901/clusters6124lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6125lb: must succeed: curl -sf localhost:9901/clusters6126lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6127lb: must succeed: curl -sf localhost:9901/clusters6128lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6129worker2 # [ 118.203700] systemd[1]: Stopping Nix worker-protocol over gRPC...6130lb # [ 118.237689] envoy[921]: conn peer=[2001:db8:1::1]:55032 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1013 tx=282 ms=14526131lb # [ 118.243254] envoy[921]: conn peer=[2001:db8:1::1]:55034 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9696 tx=4476 ms=61836132worker2 # [ 119.123260] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:31Z level=info event=signal_exit6133lb # [ 119.171805] niks3-server[1141]: 2026/09/22 10:57:30 INFO lead: released remote=192.168.1.4:425126134worker1 # [ 120.159966] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:32Z level=info event=scheduler_restarting addr=lb:500516135worker2 # [ 120.211900] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:32Z level=info event=scheduler_restarting6136worker2 # [ 120.215700] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:32Z level=info event=scheduler_restarting addr=lb:500516137worker2 # [ 120.219413] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:380826138worker2 # [ 120.223342] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:32Z level=info event=worker_gone addr=192.168.1.3:500516139worker2 # [ 120.226931] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:380826140worker2 # [ 120.230403] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:32Z level=info event=worker_gone addr=192.168.1.4:500516141worker2 # [ 120.247566] nix-daemon[859]: reaped child process 1375, status = succeeded6142worker1 # [ 120.360353] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:32Z level=info event=scheduler_connected addr=lb:500516143worker1 # [ 120.364309] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6144worker2 # [ 120.418585] nix-grpc-daemon[1323]: ts=2026-09-22T10:57:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6145worker1 # [ 120.564834] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:32Z level=info event=scheduler_connected addr=lb:500516146worker1 # [ 120.568142] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6147worker1 # [ 120.768676] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:32Z level=info event=scheduler_connected addr=lb:500516148worker1 # [ 120.772469] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6149worker1 # [ 120.973315] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:32Z level=info event=scheduler_connected addr=lb:500516150worker1 # [ 120.977293] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6151worker2 # [ 121.155588] nix-grpc-daemon[1323]: E0000 00:00:1790074653.311030 1368 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6152worker2 # [ 121.160522] nix-grpc-daemon[1323]: E0000 00:00:1790074653.311166 1368 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6153worker1 # [ 121.178825] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516154worker1 # [ 121.182620] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6155worker2 # [ 121.277578] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6156worker2 # [ 121.281514] systemd[1]: Stopped Nix worker-protocol over gRPC.6157worker2 # [ 121.284280] systemd[1]: nix-grpc-daemon.service: Consumed 1.023s CPU time over 15.346s wall clock time, 35.4M memory peak, 45.3K incoming IP traffic, 83.1K outgoing IP traffic.6158worker2 # [ 121.295362] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6159worker2 # [ 121.299515] systemd[1]: Closed Nix worker-protocol over gRPC.6160worker2 # [ 121.302706] systemd[1]: nix-grpc-daemon.socket: Consumed 293us CPU time over 16.121s wall clock time, 256K memory peak, 38.4K incoming IP traffic, 28.6K outgoing IP traffic.6161worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6162worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.04 seconds)6163worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6164worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6165worker1 # [ 121.380282] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516166worker1 # [ 121.382098] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6167worker1 # [ 121.583079] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516168worker1 # [ 121.586939] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6169worker1 # [ 121.787283] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516170worker1 # [ 121.790987] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6171worker1 # [ 121.951158] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_take_over6172lb # [ 121.964311] niks3-server[1141]: 2026/09/22 10:57:33 INFO lead: acquired remote=192.168.1.3:533006173worker1 # [ 121.991733] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_connected addr=lb:500516174worker1 # [ 121.995601] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6175worker1 # [ 122.197490] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:34Z level=info event=scheduler_connected addr=lb:500516176worker1 # [ 122.201229] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6177worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6178worker1 # [ 122.402137] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:34Z level=info event=scheduler_connected addr=lb:500516179worker1 # [ 122.406308] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6180worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6181worker1 # [ 122.607080] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:34Z level=info event=scheduler_connected addr=lb:500516182worker1 # [ 122.611208] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6183worker1 # [ 122.811842] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:34Z level=info event=scheduler_connected addr=lb:500516184worker1 # [ 122.818967] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:34Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:564406185worker1 # [ 122.822628] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06186worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6187worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6188client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&26189client # don't know how to build these paths:6190client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6191worker1 # [ 123.673424] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355686192worker1 # [ 123.676327] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355686193worker1 # [ 123.692099] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:35Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06194worker1 # [ 123.694832] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:35Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=16195worker1 # [ 123.736348] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355686196worker1 # [ 123.745035] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355686197client # copying 1 paths to 'lb:50051'...6198client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6199worker1 # [ 123.753121] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355686200worker1 # [ 123.758923] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6201worker1 # [ 123.770166] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:35Z level=info event=niks3_push_started pid=13566202worker1 # [ 123.951658] nix-grpc-daemon[1356]: time=2026-09-22T10:57:35.859Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6203worker1 # [ 124.103091] nix-daemon[866]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6204worker1 # [ 124.114561] nix-daemon[866]: reaped child process 1372, status = succeeded6205worker1 # [ 124.257365] nix-daemon[866]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)6206worker1 # [ 124.269036] nix-daemon[866]: reaped child process 1377, status = succeeded6207lb # [ 124.296362] niks3-server[1141]: 2026/09/22 10:57:35 INFO Received uploads request method=POST path=/api/pending_closures6208worker1 # [ 124.291344] nix-grpc-daemon[1356]: time=2026-09-22T10:57:36.199Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6209worker1 # [ 124.292860] nix-grpc-daemon[1356]: time=2026-09-22T10:57:36.201Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6210lb # [ 124.326587] niks3-server[1141]: 2026/09/22 10:57:35 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6211lb # [ 124.337517] niks3-server[1141]: 2026/09/22 10:57:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6212lb # [ 124.343843] niks3-server[1141]: 2026/09/22 10:57:35 INFO Signed narinfos id=43 count=16213worker1 # [ 124.333907] nix-grpc-daemon[1356]: time=2026-09-22T10:57:36.242Z level=INFO msg="Uploading 1 narinfos"6214lb # [ 124.352369] niks3-server[1141]: 2026/09/22 10:57:35 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6215lb # [ 124.363456] niks3-server[1141]: 2026/09/22 10:57:35 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6216worker1 # [ 124.357275] nix-grpc-daemon[1356]: time=2026-09-22T10:57:36.265Z level=INFO msg="Upload complete. (401ms)"6217lb # [ 124.370198] niks3-server[1141]: 2026/09/22 10:57:35 INFO Completed upload id=436218worker1 # [ 124.359690] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35568 duration_s=0 paths=1 nar_bytes_in=5206219lb # [ 124.373430] niks3-server[1141]: 2026/09/22 10:57:35 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6220worker1 # [ 124.364927] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355686221worker1 # [ 124.372307] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6222worker1 # [ 124.374594] nix-daemon[866]: reaped child process 1354, status = succeeded6223worker1 # [ 124.379608] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6224client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6225client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6226client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6227client # farm-a> LOG-farm-a-on-standby6228worker1 # [ 125.192423] nix-daemon[866]: reaped child process 1381, status = succeeded6229worker1 # [ 125.317915] nix-daemon[866]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)6230worker1 # [ 125.328761] nix-daemon[866]: reaped child process 1400, status = succeeded6231lb # [ 125.357986] niks3-server[1141]: 2026/09/22 10:57:36 INFO Received uploads request method=POST path=/api/pending_closures6232worker1 # [ 125.352990] nix-grpc-daemon[1356]: time=2026-09-22T10:57:37.261Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6233worker1 # [ 125.354501] nix-grpc-daemon[1356]: time=2026-09-22T10:57:37.263Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6234lb # [ 125.395965] niks3-server[1141]: 2026/09/22 10:57:36 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6235lb # [ 125.403160] niks3-server[1141]: 2026/09/22 10:57:36 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6236lb # [ 125.411961] niks3-server[1141]: 2026/09/22 10:57:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6237lb # [ 125.416112] niks3-server[1141]: 2026/09/22 10:57:36 INFO Signed narinfos id=44 count=16238worker1 # [ 125.406654] nix-grpc-daemon[1356]: time=2026-09-22T10:57:37.314Z level=INFO msg="Uploading 1 narinfos"6239lb # [ 125.425300] niks3-server[1141]: 2026/09/22 10:57:36 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6240lb # [ 125.434337] niks3-server[1141]: 2026/09/22 10:57:36 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6241lb # [ 125.441342] niks3-server[1141]: 2026/09/22 10:57:36 INFO Completed upload id=446242lb # [ 125.442863] niks3-server[1141]: 2026/09/22 10:57:36 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6243worker1 # [ 125.428803] nix-grpc-daemon[1356]: time=2026-09-22T10:57:37.337Z level=INFO msg="Upload complete. (243ms)"6244worker1 # [ 125.435712] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35568 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16245worker1 # [ 125.446574] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:37Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06246worker1 # [ 125.449134] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:37Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=16247worker1 # [ 125.455179] nix-daemon[866]: reaped child process 1379, status = succeeded6248worker1 # [ 125.488970] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355686249worker1 # [ 125.500250] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355686250worker1 # [ 125.508833] nix-grpc-daemon[1356]: time=2026-09-22T10:57:37.417Z level=INFO msg="All 1 paths already cached"6251client # copying 1 paths to 'lb:50051'...6252client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6253worker1 # [ 125.513391] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355686254worker1 # [ 125.518821] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6255worker1 # [ 125.653292] nix-daemon[866]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)6256worker1 # [ 125.665580] nix-daemon[866]: reaped child process 1409, status = succeeded6257worker1 # [ 125.823852] nix-daemon[866]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)6258worker1 # [ 125.836169] nix-daemon[866]: reaped child process 1414, status = succeeded6259lb # [ 125.859381] niks3-server[1141]: 2026/09/22 10:57:37 INFO Received uploads request method=POST path=/api/pending_closures6260worker1 # [ 125.855558] nix-grpc-daemon[1356]: time=2026-09-22T10:57:37.763Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6261worker1 # [ 125.856907] nix-grpc-daemon[1356]: time=2026-09-22T10:57:37.763Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6262lb # [ 125.889095] niks3-server[1141]: 2026/09/22 10:57:37 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6263lb # [ 125.896454] niks3-server[1141]: 2026/09/22 10:57:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6264lb # [ 125.902247] niks3-server[1141]: 2026/09/22 10:57:37 INFO Signed narinfos id=45 count=16265worker1 # [ 125.892586] nix-grpc-daemon[1356]: time=2026-09-22T10:57:37.800Z level=INFO msg="Uploading 1 narinfos"6266lb # [ 125.906132] niks3-server[1141]: 2026/09/22 10:57:37 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6267lb # [ 125.918601] niks3-server[1141]: 2026/09/22 10:57:37 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6268worker1 # [ 125.912994] nix-grpc-daemon[1356]: time=2026-09-22T10:57:37.821Z level=INFO msg="Upload complete. (389ms)"6269lb # [ 125.926073] niks3-server[1141]: 2026/09/22 10:57:37 INFO Completed upload id=456270lb # [ 125.929417] niks3-server[1141]: 2026/09/22 10:57:37 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6271worker1 # [ 125.916341] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35568 duration_s=0 paths=1 nar_bytes_in=6406272worker1 # [ 125.920736] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355686273worker1 # [ 125.929483] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6274worker1 # [ 125.934372] nix-daemon[866]: reaped child process 1404, status = succeeded6275worker1 # [ 125.939240] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6276client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6277client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6278client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6279client # farm-b> LOG-farm-b-on-standby6280worker1 # [ 127.365644] nix-daemon[866]: reaped child process 1418, status = succeeded6281worker1 # [ 127.519869] nix-daemon[866]: accepted connection from pid 1434, user nix-grpc-daemon (trusted)6282worker1 # [ 127.532170] nix-daemon[866]: reaped child process 1437, status = succeeded6283lb # [ 127.562620] niks3-server[1141]: 2026/09/22 10:57:38 INFO Received uploads request method=POST path=/api/pending_closures6284worker1 # [ 127.569853] nix-grpc-daemon[1356]: time=2026-09-22T10:57:39.478Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6285worker1 # [ 127.572095] nix-grpc-daemon[1356]: time=2026-09-22T10:57:39.479Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6286lb # [ 127.606336] niks3-server[1141]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6287lb # [ 127.619387] niks3-server[1141]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6288lb # [ 127.628753] niks3-server[1141]: 2026/09/22 10:57:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6289lb # [ 127.633671] niks3-server[1141]: 2026/09/22 10:57:39 INFO Signed narinfos id=46 count=16290worker1 # [ 127.621578] nix-grpc-daemon[1356]: time=2026-09-22T10:57:39.529Z level=INFO msg="Uploading 1 narinfos"6291lb # [ 127.640986] niks3-server[1141]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6292lb # [ 127.647587] niks3-server[1141]: 2026/09/22 10:57:39 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6293lb # [ 127.654501] niks3-server[1141]: 2026/09/22 10:57:39 INFO Completed upload id=466294worker1 # [ 127.641754] nix-grpc-daemon[1356]: time=2026-09-22T10:57:39.550Z level=INFO msg="Upload complete. (294ms)"6295lb # [ 127.656768] niks3-server[1141]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6296worker1 # [ 127.648611] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35568 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16297worker1 # [ 127.659779] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:39Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06298worker1 # [ 127.664576] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:39Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16299worker1 # [ 127.674654] nix-daemon[866]: reaped child process 1416, status = succeeded6300worker1 # [ 127.703807] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355686301worker1 # [ 127.716165] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355686302worker1 # [ 127.729289] nix-grpc-daemon[1356]: time=2026-09-22T10:57:39.637Z level=INFO msg="All 2 paths already cached"6303client # copying 1 paths to 'lb:50051'...6304client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6305worker1 # [ 127.738077] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355686306worker1 # [ 127.746324] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6307worker1 # [ 127.915063] nix-daemon[866]: accepted connection from pid 1443, user nix-grpc-daemon (trusted)6308worker1 # [ 127.927657] nix-daemon[866]: reaped child process 1446, status = succeeded6309worker1 # [ 128.098300] nix-daemon[866]: accepted connection from pid 1448, user nix-grpc-daemon (trusted)6310worker1 # [ 128.110614] nix-daemon[866]: reaped child process 1451, status = succeeded6311lb # [ 128.140745] niks3-server[1141]: 2026/09/22 10:57:39 INFO Received uploads request method=POST path=/api/pending_closures6312worker1 # [ 128.138624] nix-grpc-daemon[1356]: time=2026-09-22T10:57:40.046Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6313worker1 # [ 128.140019] nix-grpc-daemon[1356]: time=2026-09-22T10:57:40.046Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6314lb # [ 128.167909] niks3-server[1141]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6315lb # [ 128.179500] niks3-server[1141]: 2026/09/22 10:57:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6316worker1 # [ 128.172640] nix-grpc-daemon[1356]: time=2026-09-22T10:57:40.081Z level=INFO msg="Uploading 1 narinfos"6317lb # [ 128.185191] niks3-server[1141]: 2026/09/22 10:57:39 INFO Signed narinfos id=47 count=16318lb # [ 128.189799] niks3-server[1141]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6319lb # [ 128.200752] niks3-server[1141]: 2026/09/22 10:57:39 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6320worker1 # [ 128.192839] nix-grpc-daemon[1356]: time=2026-09-22T10:57:40.101Z level=INFO msg="Upload complete. (433ms)"6321lb # [ 128.206153] niks3-server[1141]: 2026/09/22 10:57:39 INFO Completed upload id=476322worker1 # [ 128.194948] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35568 duration_s=0 paths=1 nar_bytes_in=7686323worker1 # [ 128.198569] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355686324lb # [ 128.211516] niks3-server[1141]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6325worker1 # [ 128.205183] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6326worker1 # [ 128.207439] nix-daemon[866]: reaped child process 1441, status = succeeded6327worker1 # [ 128.212905] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6328client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6329client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6330lb # [ 128.238072] envoy[921]: conn peer=[2001:db8:1::4]:32792 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5880 tx=648 ms=142476331client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6332client # farm-top> LOG-farm-top-on-standby6333worker1 # [ 129.627728] nix-daemon[866]: reaped child process 1455, status = succeeded6334worker1 # [ 129.772091] nix-daemon[866]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6335worker1 # [ 129.783348] nix-daemon[866]: reaped child process 1474, status = succeeded6336lb # [ 129.811525] niks3-server[1141]: 2026/09/22 10:57:41 INFO Received uploads request method=POST path=/api/pending_closures6337worker1 # [ 129.807974] nix-grpc-daemon[1356]: time=2026-09-22T10:57:41.716Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6338worker1 # [ 129.810058] nix-grpc-daemon[1356]: time=2026-09-22T10:57:41.718Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6339lb # [ 129.846616] niks3-server[1141]: 2026/09/22 10:57:41 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6340lb # [ 129.857854] niks3-server[1141]: 2026/09/22 10:57:41 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6341lb # [ 129.864348] niks3-server[1141]: 2026/09/22 10:57:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6342lb # [ 129.868938] niks3-server[1141]: 2026/09/22 10:57:41 INFO Signed narinfos id=48 count=16343worker1 # [ 129.859281] nix-grpc-daemon[1356]: time=2026-09-22T10:57:41.767Z level=INFO msg="Uploading 1 narinfos"6344lb # [ 129.876327] niks3-server[1141]: 2026/09/22 10:57:41 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6345lb # [ 129.884343] niks3-server[1141]: 2026/09/22 10:57:41 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6346lb # [ 129.890389] niks3-server[1141]: 2026/09/22 10:57:41 INFO Completed upload id=486347worker1 # [ 129.877773] nix-grpc-daemon[1356]: time=2026-09-22T10:57:41.786Z level=INFO msg="Upload complete. (257ms)"6348lb # [ 129.893716] niks3-server[1141]: 2026/09/22 10:57:41 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6349worker1 # [ 129.884987] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35568 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16350worker1 # [ 129.896275] nix-daemon[866]: reaped child process 1453, status = succeeded6351client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.46 seconds)6352client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath6353client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.15 seconds)6354worker2 # [ 130.161951] systemd[1]: Listening on Nix worker-protocol over gRPC.6355worker2 # [ 130.166092] systemd[1]: Starting scheduler-token.service...6356worker2 # [ 130.219625] systemd[1]: scheduler-token.service: Deactivated successfully.6357worker2 # [ 130.221285] systemd[1]: Finished scheduler-token.service.6358worker2 # [ 130.223075] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 55ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6359worker2 # [ 130.227448] systemd[1]: Starting Nix worker-protocol over gRPC...6360worker2 # [ 130.369772] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:42Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6361worker2 # [ 130.372129] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:42Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6362worker2 # [ 130.376130] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:42Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16363worker2 # [ 130.383065] nix-grpc-daemon[1544]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6364worker2 # [ 130.384439] nix-grpc-daemon[1544]: E0000 00:00:1790074662.538659 1544 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6365worker2 # [ 130.390116] nix-grpc-daemon[1544]: E0000 00:00:1790074662.543983 1544 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6366worker2 # [ 130.394304] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:42Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6367worker2 # [ 130.398733] systemd[1]: Started Nix worker-protocol over gRPC.6368worker2 # [ 130.406244] nix-grpc-daemon[1544]: E0000 00:00:1790074662.561939 1585 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6369lb: must succeed: curl -sf localhost:9901/clusters6370worker2 # [ 130.419687] nix-grpc-daemon[1544]: E0000 00:00:1790074662.575274 1585 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6371worker2 # [ 130.424131] nix-grpc-daemon[1544]: E0000 00:00:1790074662.577583 1585 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6372worker2 # [ 130.434045] nix-grpc-daemon[1544]: E0000 00:00:1790074662.589665 1585 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6373worker2 # [ 130.441208] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:42Z level=info event=scheduler_connected addr=lb:500516374worker1 # [ 130.399101] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:42Z level=info event=oidc_jwks provider=mock keys=16375worker1 # [ 130.401189] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:564406376worker1 # [ 130.402974] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6378worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6379worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6380lb: must succeed: curl -sf localhost:9901/clusters6381lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6382lb: must succeed: curl -sf localhost:9901/clusters6383lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6384worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6385worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6386(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.67 seconds)6387subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6388client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs6389client # [ 130.673132] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.6390client # Running as unit: rs.service; invocation ID: 45ca95a8265f46a886d17428f195d5336391client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.06 seconds)6392worker2 # [ 130.905080] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444006393worker2 # [ 130.910529] nix-daemon[859]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6394client # [ 130.900517] nix[1362]: don't know how to build these paths:6395client # [ 130.901942] nix[1362]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6396worker1 # [ 130.876909] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:42Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06397worker1 # [ 130.879312] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:42Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16398worker1 # [ 130.921044] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355686399worker1 # [ 130.928863] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355686400client # [ 130.965985] nix[1362]: copying 1 paths to 'lb:50051'...6401client # [ 130.966816] nix[1362]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6402worker1 # [ 130.936022] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355686403worker1 # [ 130.941204] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6404worker1 # [ 131.090596] nix-daemon[866]: accepted connection from pid 1487, user nix-grpc-daemon (trusted)6405worker1 # [ 131.103079] nix-daemon[866]: reaped child process 1490, status = succeeded6406worker1 # [ 131.270152] nix-daemon[866]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)6407worker1 # [ 131.282874] nix-daemon[866]: reaped child process 1495, status = succeeded6408lb # [ 131.307898] niks3-server[1141]: 2026/09/22 10:57:42 INFO Received uploads request method=POST path=/api/pending_closures6409worker1 # [ 131.303196] nix-grpc-daemon[1356]: time=2026-09-22T10:57:43.211Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6410worker1 # [ 131.304628] nix-grpc-daemon[1356]: time=2026-09-22T10:57:43.211Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6411lb # [ 131.333124] niks3-server[1141]: 2026/09/22 10:57:42 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6412lb # [ 131.341931] niks3-server[1141]: 2026/09/22 10:57:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6413lb # [ 131.347108] niks3-server[1141]: 2026/09/22 10:57:42 INFO Signed narinfos id=49 count=16414worker1 # [ 131.337256] nix-grpc-daemon[1356]: time=2026-09-22T10:57:43.245Z level=INFO msg="Uploading 1 narinfos"6415lb # [ 131.354239] niks3-server[1141]: 2026/09/22 10:57:42 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6416lb # [ 131.362477] niks3-server[1141]: 2026/09/22 10:57:42 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6417worker1 # [ 131.356431] nix-grpc-daemon[1356]: time=2026-09-22T10:57:43.265Z level=INFO msg="Upload complete. (410ms)"6418lb # [ 131.369226] niks3-server[1141]: 2026/09/22 10:57:42 INFO Completed upload id=496419worker1 # [ 131.358710] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35568 duration_s=0 paths=1 nar_bytes_in=4166420lb # [ 131.375133] niks3-server[1141]: 2026/09/22 10:57:42 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6421worker1 # [ 131.365640] nix-daemon[866]: reaped child process 1485, status = succeeded6422worker1 # [ 131.366806] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355686423worker1 # [ 131.371058] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6424worker1 # [ 131.376445] nix-daemon[866]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6425client # [ 131.406809] nix[1362]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6426lb: must succeed: curl -sf localhost:9901/clusters6427lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6428lb: must succeed: curl -sf localhost:9901/clusters6429lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6430lb: must succeed: curl -sf localhost:9901/clusters6431lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6432lb: must succeed: curl -sf localhost:9901/clusters6433lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6434worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6435worker2 # [ 133.429532] systemd-journald[289]: Received client request to rotate journal, rotating.6436worker2 # [ 133.442348] systemd-journald[289]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e8c3ae1334704ef0876d23585af2aa0f.6437worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.04 seconds)6438worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6439worker1 # [ 133.308245] systemd[1]: Stopping Nix worker-protocol over gRPC...6440worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)6441client: must succeed: sleep 5.06442worker1 # [ 133.757242] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:45Z level=info event=draining6443lb # [ 138.239236] envoy[921]: conn peer=[2001:db8:1::1]:52294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9659 tx=4449 ms=62236444client: (finished: must succeed: sleep 5.0, in 5.04 seconds)6445worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6446worker1 # [ 138.597340] nix-daemon[866]: reaped child process 1499, status = succeeded6447worker1 # [ 138.728751] nix-daemon[866]: accepted connection from pid 1540, user nix-grpc-daemon (trusted)6448worker1 # [ 138.741141] nix-daemon[866]: reaped child process 1543, status = succeeded6449lb # [ 138.767366] niks3-server[1141]: 2026/09/22 10:57:50 INFO Received uploads request method=POST path=/api/pending_closures6450worker1 # [ 138.762223] nix-grpc-daemon[1356]: time=2026-09-22T10:57:50.670Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6451worker1 # [ 138.764616] nix-grpc-daemon[1356]: time=2026-09-22T10:57:50.672Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6452lb # [ 138.805617] niks3-server[1141]: 2026/09/22 10:57:50 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6453lb # [ 138.813472] niks3-server[1141]: 2026/09/22 10:57:50 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6454lb # [ 138.823827] niks3-server[1141]: 2026/09/22 10:57:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6455worker1 # [ 138.817152] nix-grpc-daemon[1356]: time=2026-09-22T10:57:50.725Z level=INFO msg="Uploading 1 narinfos"6456lb # [ 138.831133] niks3-server[1141]: 2026/09/22 10:57:50 INFO Signed narinfos id=50 count=16457lb # [ 138.839486] niks3-server[1141]: 2026/09/22 10:57:50 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6458lb # [ 138.843070] niks3-server[1141]: 2026/09/22 10:57:50 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6459worker1 # [ 138.836239] nix-grpc-daemon[1356]: time=2026-09-22T10:57:50.744Z level=INFO msg="Upload complete. (247ms)"6460lb # [ 138.849605] niks3-server[1141]: 2026/09/22 10:57:50 INFO Completed upload id=506461lb # [ 138.852943] niks3-server[1141]: 2026/09/22 10:57:50 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6462worker1 # [ 138.841689] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35568 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16463worker1 # [ 138.852718] nix-daemon[866]: reaped child process 1497, status = succeeded6464client # [ 138.904951] systemd[1]: rs.service: Deactivated successfully.6465client # [ 138.908117] systemd[1]: rs.service: Consumed 113ms CPU time over 8.234s wall clock time, 39.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.6466worker1 # [ 139.757567] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:51Z level=info event=signal_exit6467lb # [ 140.196863] niks3-server[1141]: 2026/09/22 10:57:51 INFO lead: released remote=192.168.1.3:533006468lb # [ 140.372279] niks3-server[1141]: 2026/09/22 10:57:51 INFO lead: acquired remote=192.168.1.4:396506469worker2 # [ 140.409264] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:52Z level=info event=scheduler_take_over6470worker2 # [ 141.230418] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:53Z level=info event=scheduler_restarting addr=lb:500516471worker1 # [ 141.188528] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:53Z level=info event=scheduler_restarting6472worker1 # [ 141.193263] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:53Z level=info event=scheduler_restarting addr=lb:500516473worker1 # [ 141.196504] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:564406474worker1 # [ 141.200160] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:53Z level=info event=worker_gone addr=192.168.1.4:500516475worker1 # [ 141.203706] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:564406476worker1 # [ 141.207245] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:53Z level=info event=worker_gone addr=192.168.1.3:500516477worker1 # [ 141.221238] nix-daemon[866]: reaped child process 1329, status = succeeded6478worker2 # [ 141.431770] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:53Z level=info event=scheduler_connected addr=lb:500516479worker1 # [ 141.397559] nix-grpc-daemon[1277]: ts=2026-09-22T10:57:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6480worker2 # [ 141.445148] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:53Z level=info event=oidc_jwks provider=mock keys=16481worker2 # [ 141.448683] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:539006482worker2 # [ 141.452557] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06483worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.14 seconds)6484client: waiting for success: ! systemctl is-active rs6485client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)6486client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6487client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)6488client: must fail: journalctl -u rs | grep -q 'reconnecting'6489worker1 # [ 141.813693] nix-grpc-daemon[1277]: E0000 00:00:1790074673.722121 1322 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6490client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)6491lb: must succeed: curl -sf localhost:9901/clusters6492worker1 # [ 141.817202] nix-grpc-daemon[1277]: E0000 00:00:1790074673.725773 1322 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6493lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6494worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6495worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6496worker1 # [ 142.502827] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6497worker1 # [ 142.505824] systemd[1]: Stopped Nix worker-protocol over gRPC.6498worker1 # [ 142.507549] systemd[1]: nix-grpc-daemon.service: Consumed 1.267s CPU time over 31.928s wall clock time, 35.1M memory peak, 47.6K incoming IP traffic, 103.1K outgoing IP traffic.6499worker1 # [ 142.520686] systemd[1]: Starting Nix worker-protocol over gRPC...6500worker1 # [ 142.695580] nix-grpc-daemon[1550]: ts=2026-09-22T10:57:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6501worker1 # [ 142.697384] nix-grpc-daemon[1550]: ts=2026-09-22T10:57:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6502worker1 # [ 142.701055] nix-grpc-daemon[1550]: ts=2026-09-22T10:57:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16503worker1 # [ 142.709076] nix-grpc-daemon[1550]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6504worker1 # [ 142.710471] nix-grpc-daemon[1550]: E0000 00:00:1790074674.616895 1550 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6505worker1 # [ 142.715829] nix-grpc-daemon[1550]: E0000 00:00:1790074674.622464 1550 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6506worker1 # [ 142.720103] nix-grpc-daemon[1550]: ts=2026-09-22T10:57:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6507worker1 # [ 142.722196] systemd[1]: Started Nix worker-protocol over gRPC.6508worker1 # [ 142.727779] nix-grpc-daemon[1550]: E0000 00:00:1790074674.636363 1589 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6509worker1 # [ 142.740045] nix-grpc-daemon[1550]: E0000 00:00:1790074674.648547 1589 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6510worker1 # [ 142.744155] nix-grpc-daemon[1550]: E0000 00:00:1790074674.651073 1589 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6511worker1 # [ 142.756412] nix-grpc-daemon[1550]: E0000 00:00:1790074674.664899 1589 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6512worker1 # [ 142.767039] nix-grpc-daemon[1550]: ts=2026-09-22T10:57:54Z level=info event=scheduler_connected addr=lb:500516513worker2 # [ 142.814726] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:539006514worker2 # [ 142.816388] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06515lb: must succeed: curl -sf localhost:9901/clusters6516lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6517worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6518worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6519(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.47 seconds)6520subtest: scheduler restart mid-build: build finishes, no double build6521client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid6522client # [ 143.147600] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.6523client # Running as unit: mid.service; invocation ID: 7427646c67624803800499e5fd7a022f6524client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)6525client # [ 143.355942] nix[1398]: don't know how to build these paths:6526client # [ 143.356952] nix[1398]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6527worker2 # [ 143.368639] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:444006528worker2 # [ 143.381206] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:55Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06529worker2 # [ 143.385562] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:55Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16530worker2 # [ 143.429223] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444006531worker2 # [ 143.440964] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 1/5)6532worker2 # [ 143.497718] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5)6533worker2 # [ 143.505197] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 3/5)6534worker2 # [ 143.669718] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 557 ms (attempt 4/5)6535worker2 # [ 144.230137] nix-grpc-daemon[1544]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6536worker2 # [ 144.231688] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6537worker2 # [ 144.244824] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:444006538client # [ 144.236203] nix[1398]: copying 1 paths to 'lb:50051'...6539worker2 # [ 144.255892] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:444006540client # [ 144.240166] nix[1398]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6541worker2 # [ 144.263784] nix-daemon[859]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6542worker2 # [ 144.274744] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:56Z level=info event=niks3_push_started pid=16726543worker2 # [ 144.457752] nix-grpc-daemon[1672]: time=2026-09-22T10:57:56.613Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6544worker2 # [ 144.620895] nix-daemon[859]: accepted connection from pid 1689, user nix-grpc-daemon (trusted)6545worker2 # [ 144.631655] nix-daemon[859]: reaped child process 1692, status = succeeded6546worker2 # [ 144.797992] nix-daemon[859]: accepted connection from pid 1694, user nix-grpc-daemon (trusted)6547worker2 # [ 144.810144] nix-daemon[859]: reaped child process 1697, status = succeeded6548lb # [ 144.795285] niks3-server[1141]: 2026/09/22 10:57:56 INFO Received uploads request method=POST path=/api/pending_closures6549worker2 # [ 144.837912] nix-grpc-daemon[1672]: time=2026-09-22T10:57:56.993Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6550worker2 # [ 144.840697] nix-grpc-daemon[1672]: time=2026-09-22T10:57:56.995Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6551lb # [ 144.831193] niks3-server[1141]: 2026/09/22 10:57:56 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6552lb # [ 144.838418] niks3-server[1141]: 2026/09/22 10:57:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6553lb # [ 144.843336] niks3-server[1141]: 2026/09/22 10:57:56 INFO Signed narinfos id=51 count=16554worker2 # [ 144.879957] nix-grpc-daemon[1672]: time=2026-09-22T10:57:57.035Z level=INFO msg="Uploading 1 narinfos"6555lb # [ 144.854107] niks3-server[1141]: 2026/09/22 10:57:56 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6556lb # [ 144.863190] niks3-server[1141]: 2026/09/22 10:57:56 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6557worker2 # [ 144.903824] nix-grpc-daemon[1672]: time=2026-09-22T10:57:57.059Z level=INFO msg="Upload complete. (441ms)"6558lb # [ 144.870414] niks3-server[1141]: 2026/09/22 10:57:56 INFO Completed upload id=516559worker2 # [ 144.906033] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44400 duration_s=0 paths=1 nar_bytes_in=4246560worker2 # [ 144.909202] nix-grpc-daemon[1544]: ts=2026-09-22T10:57:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:444006561lb # [ 144.881077] niks3-server[1141]: 2026/09/22 10:57:56 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6562worker2 # [ 144.917545] nix-daemon[859]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6563worker2 # [ 144.919238] nix-daemon[859]: reaped child process 1670, status = succeeded6564worker2 # [ 144.924506] nix-daemon[859]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6565client # [ 144.908898] nix[1398]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6566lb: must succeed: curl -sf localhost:9901/clusters6567lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6568lb: must succeed: curl -sf localhost:9901/clusters6569lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6570worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6571worker2 # [ 146.817599] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1544 (nix-grpc-daemon) on client request.6572worker2 # [ 146.823416] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6573lb # [ 146.795376] niks3-server[1141]: 2026/09/22 10:57:58 INFO lead: released remote=192.168.1.4:396506574client # [ 146.821265] nix[1398]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6575worker1 # [ 146.790122] nix-grpc-daemon[1550]: ts=2026-09-22T10:57:58Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6576client # [ 146.825965] nix[1398]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6577worker2 # [ 146.844457] systemd[1]: Starting scheduler-token.service...6578worker2 # [ 146.858290] nix-daemon[859]: reaped child process 1699, status = succeeded6579worker2 # [ 146.861873] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6580worker2 # [ 146.863326] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6581worker2 # [ 146.865242] systemd[1]: nix-grpc-daemon.service: Consumed 409ms CPU time over 16.633s wall clock time, 33.4M memory peak, 14.6K incoming IP traffic, 27.5K outgoing IP traffic.6582worker2 # [ 146.867958] nix-daemon[859]: reaped child process 1603, status = succeeded6583worker2 # [ 146.879540] nix-daemon[1701]: unexpected Nix daemon error: error: interrupted by the user6584worker2 # [ 146.884344] nix-daemon[859]: reaped child process 1701, status = failed with exit code 16585worker2 # [ 146.924690] systemd[1]: scheduler-token.service: Deactivated successfully.6586worker2 # [ 146.926264] systemd[1]: Finished scheduler-token.service.6587worker2 # [ 146.928223] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 81ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6588worker2 # [ 146.930526] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6589worker2 # [ 146.935076] systemd[1]: Starting scheduler-token.service...6590worker2 # [ 146.983085] systemd[1]: scheduler-token.service: Deactivated successfully.6591worker2 # [ 146.984140] systemd[1]: Finished scheduler-token.service.6592worker2 # [ 146.985420] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 48ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6593worker2 # [ 146.990250] systemd[1]: Starting Nix worker-protocol over gRPC...6594worker1 # [ 146.990634] nix-grpc-daemon[1550]: ts=2026-09-22T10:57:58Z level=info event=scheduler_connected addr=lb:500516595worker2 # [ 147.132960] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6596worker2 # [ 147.135233] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6597worker2 # [ 147.138789] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16598worker2 # [ 147.146070] nix-grpc-daemon[1744]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6599worker2 # [ 147.147365] nix-grpc-daemon[1744]: E0000 00:00:1790074679.301108 1744 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6600worker2 # [ 147.154142] nix-grpc-daemon[1744]: E0000 00:00:1790074679.306230 1744 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6601worker2 # [ 147.156274] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6602worker2 # [ 147.157820] systemd[1]: Started Nix worker-protocol over gRPC.6603worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.38 seconds)6604client: must succeed: sleep 5.06605worker2 # [ 147.166070] nix-grpc-daemon[1744]: E0000 00:00:1790074679.321591 1785 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6606lb # [ 147.132516] niks3-server[1141]: 2026/09/22 10:57:58 INFO lead: acquired remote=192.168.1.4:448866607worker2 # [ 147.170063] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=scheduler_take_over6608worker1 # [ 147.124325] nix-grpc-daemon[1550]: ts=2026-09-22T10:57:59Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6609worker2 # [ 147.179591] nix-grpc-daemon[1744]: E0000 00:00:1790074679.335235 1785 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6610worker2 # [ 147.183786] nix-grpc-daemon[1744]: E0000 00:00:1790074679.337404 1785 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6611worker2 # [ 147.193487] nix-grpc-daemon[1744]: E0000 00:00:1790074679.349133 1785 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6612worker2 # [ 147.202241] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=scheduler_connected addr=lb:500516613worker2 # [ 147.206897] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=oidc_jwks provider=mock keys=16614worker2 # [ 147.208247] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:528866615worker2 # [ 147.209839] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06616worker2 # [ 147.350142] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06617worker2 # [ 147.355364] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16618worker1 # [ 147.324859] nix-grpc-daemon[1550]: ts=2026-09-22T10:57:59Z level=info event=scheduler_connected addr=lb:500516619worker2 # [ 147.372666] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:528866620worker2 # [ 147.379206] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06621worker2 # [ 147.387737] nix-grpc-daemon[1744]: ts=2026-09-22T10:57:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528946622worker2 # [ 147.397166] nix-daemon[859]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6623worker2 # [ 147.411068] nix-daemon[859]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6624client # [ 147.405653] nix[1398]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6625worker2 # [ 147.422906] nix-daemon[859]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6626lb # [ 148.240257] envoy[921]: conn peer=[2001:db8:1::1]:52300 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=1515 ms=79986627lb # [ 148.244072] envoy[921]: conn peer=[2001:db8:1::3]:36400 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3984 tx=999 ms=305896628lb # [ 148.247777] envoy[921]: conn peer=[2001:db8:1::4]:54720 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2046 tx=311 ms=163966629client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6630client: waiting for success: ! systemctl is-active mid6631worker2 # [ 152.435574] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:04Z level=info event=niks3_push_started pid=18276632worker2 # [ 152.449543] nix-daemon[859]: reaped child process 1802, status = succeeded6633worker2 # [ 152.607826] nix-grpc-daemon[1827]: time=2026-09-22T10:58:04.763Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6634worker2 # [ 152.766245] nix-daemon[859]: accepted connection from pid 1840, user nix-grpc-daemon (trusted)6635worker2 # [ 152.778929] nix-daemon[859]: reaped child process 1843, status = succeeded6636lb # [ 152.760391] niks3-server[1141]: 2026/09/22 10:58:04 INFO Received uploads request method=POST path=/api/pending_closures6637worker2 # [ 152.801385] nix-grpc-daemon[1827]: time=2026-09-22T10:58:04.957Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6638worker2 # [ 152.802866] nix-grpc-daemon[1827]: time=2026-09-22T10:58:04.958Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6639lb # [ 152.787432] niks3-server[1141]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6640lb # [ 152.790652] niks3-server[1141]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6641lb # [ 152.793420] niks3-server[1141]: 2026/09/22 10:58:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6642lb # [ 152.795380] niks3-server[1141]: 2026/09/22 10:58:04 INFO Signed narinfos id=52 count=16643worker2 # [ 152.828891] nix-grpc-daemon[1827]: time=2026-09-22T10:58:04.984Z level=INFO msg="Uploading 1 narinfos"6644lb # [ 152.797784] niks3-server[1141]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6645lb # [ 152.803056] niks3-server[1141]: 2026/09/22 10:58:04 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6646worker2 # [ 152.839846] nix-grpc-daemon[1827]: time=2026-09-22T10:58:04.995Z level=INFO msg="Upload complete. (227ms)"6647lb # [ 152.806468] niks3-server[1141]: 2026/09/22 10:58:04 INFO Completed upload id=526648worker2 # [ 152.842332] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52894 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16649worker2 # [ 152.850880] nix-daemon[859]: reaped child process 1800, status = succeeded6650client # [ 152.862431] systemd[1]: mid.service: Deactivated successfully.6651client # [ 152.866124] systemd[1]: mid.service: Consumed 134ms CPU time over 9.718s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.4K outgoing IP traffic.6652client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)6653client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6654client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6655(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.40 seconds)6656subtest: interrupting the client stops the build on the worker6657client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6658client # [ 153.521445] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6659client # Running as unit: intr.service; invocation ID: 6d1c002abcd44d54a7ac3e722edf052a6660client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.03 seconds)6661worker2 # [ 153.716648] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:528946662client # [ 153.707336] nix[1434]: don't know how to build these paths:6663client # [ 153.708188] nix[1434]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6664worker2 # [ 153.730716] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:05Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06665worker2 # [ 153.734834] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:05Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16666worker1 # [ 153.733882] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535286667worker1 # [ 153.739075] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6668worker1 # [ 153.747617] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)6669worker1 # [ 153.757622] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 2/5)6670worker1 # [ 153.927693] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 374 ms (attempt 3/5)6671worker1 # [ 154.301801] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 4/5)6672worker1 # [ 154.351482] nix-grpc-daemon[1550]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6673worker1 # [ 154.353105] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6674worker1 # [ 154.364369] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535286675client # [ 154.401550] nix[1434]: copying 1 paths to 'lb:50051'...6676client # [ 154.403142] nix[1434]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6677worker1 # [ 154.373049] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535286678worker1 # [ 154.378442] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6679worker1 # [ 154.387139] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:06Z level=info event=niks3_push_started pid=16426680worker1 # [ 154.549789] nix-grpc-daemon[1642]: time=2026-09-22T10:58:06.457Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6681worker1 # [ 154.707175] nix-daemon[866]: accepted connection from pid 1655, user nix-grpc-daemon (trusted)6682worker1 # [ 154.718085] nix-daemon[866]: reaped child process 1662, status = succeeded6683worker1 # [ 154.872801] nix-daemon[866]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6684worker1 # [ 154.885428] nix-daemon[866]: reaped child process 1667, status = succeeded6685lb # [ 154.911778] niks3-server[1141]: 2026/09/22 10:58:06 INFO Received uploads request method=POST path=/api/pending_closures6686worker1 # [ 154.908074] nix-grpc-daemon[1642]: time=2026-09-22T10:58:06.815Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6687worker1 # [ 154.909428] nix-grpc-daemon[1642]: time=2026-09-22T10:58:06.816Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6688lb # [ 154.953197] niks3-server[1141]: 2026/09/22 10:58:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6689lb # [ 154.964620] niks3-server[1141]: 2026/09/22 10:58:06 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6690lb # [ 154.978733] niks3-server[1141]: 2026/09/22 10:58:06 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6691lb # [ 154.982367] niks3-server[1141]: 2026/09/22 10:58:06 INFO Signed narinfos id=53 count=16692worker1 # [ 154.972165] nix-grpc-daemon[1642]: time=2026-09-22T10:58:06.880Z level=INFO msg="Uploading 1 narinfos"6693lb # [ 154.991065] niks3-server[1141]: 2026/09/22 10:58:06 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6694lb # [ 154.998108] niks3-server[1141]: 2026/09/22 10:58:06 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6695lb # [ 155.002127] niks3-server[1141]: 2026/09/22 10:58:06 INFO Completed upload id=536696worker1 # [ 154.989517] nix-grpc-daemon[1642]: time=2026-09-22T10:58:06.897Z level=INFO msg="Upload complete. (435ms)"6697worker1 # [ 154.994749] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=4246698worker1 # [ 155.002632] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535286699worker1 # [ 155.010243] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6700worker1 # [ 155.021411] nix-daemon[866]: reaped child process 1640, status = succeeded6701worker1 # [ 155.025912] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6702client # [ 155.056436] nix[1434]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6703client: must succeed: systemctl kill -s INT intr6704client # [ 156.961413] systemd[1]: intr.service: Sent signal SIGINT to main process 1434 (nix) on client request.6705client # [ 156.964475] nix[1434]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6706worker1 # [ 156.934874] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:08Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6707client # [ 156.966936] nix[1434]: error: interrupted by the user6708client: (finished: must succeed: systemctl kill -s INT intr, in 0.03 seconds)6709client # [ 156.990543] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6710client # [ 156.992385] systemd[1]: intr.service: Failed with result 'exit-code'.6711client # [ 156.994070] systemd[1]: intr.service: Consumed 105ms CPU time over 3.470s wall clock time, 43.4M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.6712worker1 # [ 157.043729] nix-daemon[866]: reaped child process 1669, status = succeeded6713worker1 # [ 157.048262] nix-daemon[1671]: unexpected Nix daemon error: error: interrupted by the user6714worker1 # [ 157.051909] nix-daemon[866]: reaped child process 1671, status = failed with exit code 16715(finished: subtest: interrupting the client stops the build on the worker, in 4.66 seconds)6716subtest: a deploy mid-build drains: the build finishes, the new generation takes over6717client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6718client # [ 158.187805] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6719client # Running as unit: sw.service; invocation ID: 4722cd338b0849e8aad50a868a6b2e286720client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6721lb # [ 158.241207] envoy[921]: conn peer=[2001:db8:1::1]:41804 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3934 tx=1965 ms=94896722lb # [ 158.245088] envoy[921]: conn peer=[2001:db8:1::1]:44320 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=850 ms=32746723worker2 # [ 158.411264] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:528946724client # [ 158.401155] nix[1455]: don't know how to build these paths:6725client # [ 158.402054] nix[1455]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6726worker2 # [ 158.427073] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:10Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06727worker2 # [ 158.432640] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:10Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16728worker2 # [ 158.470659] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528946729worker2 # [ 158.482616] nix-grpc-daemon[1744]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)6730worker2 # [ 158.543425] nix-grpc-daemon[1744]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 2/5)6731worker2 # [ 158.569233] nix-grpc-daemon[1744]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 142 ms (attempt 3/5)6732worker2 # [ 158.717884] nix-grpc-daemon[1744]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 379 ms (attempt 4/5)6733worker2 # [ 159.104071] nix-grpc-daemon[1744]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6734worker2 # [ 159.108445] nix-grpc-daemon[1744]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6735worker2 # [ 159.127056] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528946736client # [ 159.122188] nix[1455]: copying 1 paths to 'lb:50051'...6737client # [ 159.126123] nix[1455]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6738worker2 # [ 159.141736] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:528946739worker2 # [ 159.153461] nix-daemon[859]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6740worker2 # [ 159.328620] nix-daemon[859]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6741worker2 # [ 159.340731] nix-daemon[859]: reaped child process 1886, status = succeeded6742worker2 # [ 159.489451] nix-daemon[859]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6743worker2 # [ 159.501323] nix-daemon[859]: reaped child process 1895, status = succeeded6744lb # [ 159.481482] niks3-server[1141]: 2026/09/22 10:58:10 INFO Received uploads request method=POST path=/api/pending_closures6745worker2 # [ 159.525947] nix-grpc-daemon[1827]: time=2026-09-22T10:58:11.681Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6746worker2 # [ 159.527469] nix-grpc-daemon[1827]: time=2026-09-22T10:58:11.681Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6747lb # [ 159.513312] niks3-server[1141]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6748lb # [ 159.528352] niks3-server[1141]: 2026/09/22 10:58:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6749worker2 # [ 159.569201] nix-grpc-daemon[1827]: time=2026-09-22T10:58:11.724Z level=INFO msg="Uploading 1 narinfos"6750lb # [ 159.535763] niks3-server[1141]: 2026/09/22 10:58:10 INFO Signed narinfos id=54 count=16751lb # [ 159.543834] niks3-server[1141]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6752lb # [ 159.553319] niks3-server[1141]: 2026/09/22 10:58:10 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6753lb # [ 159.556599] niks3-server[1141]: 2026/09/22 10:58:10 INFO Completed upload id=546754lb # [ 159.558286] niks3-server[1141]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6755worker2 # [ 159.591509] nix-grpc-daemon[1827]: time=2026-09-22T10:58:11.747Z level=INFO msg="Upload complete. (425ms)"6756worker2 # [ 159.597292] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52894 duration_s=0 paths=1 nar_bytes_in=4166757worker2 # [ 159.603294] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528946758worker2 # [ 159.612130] nix-daemon[859]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6759worker2 # [ 159.621278] nix-daemon[859]: reaped child process 1881, status = succeeded6760worker2 # [ 159.624186] nix-daemon[859]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6761client # [ 159.608457] nix[1455]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6762worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26763worker2 # Checking switch inhibitors... done6764worker2 # [ 161.470086] nixos[1927]: switching to system configuration /nix/store/6q1lny800p9ad7ly0m09l74vha5wlg0f-nixos-system-worker2-test6765worker2 # [ 161.472366] systemd[1]: Stopped target Remote File Systems.6766worker2 # activating the configuration...6767worker2 # [ 161.473850] systemd[1]: Stopped target Local File Systems.6768worker2 # setting up /etc...6769worker2 # [ 162.159510] systemd[1]: Reload requested from client PID 1927 ('.switch-to-conf') (unit backdoor.service)...6770worker2 # [ 162.161100] systemd[1]: Reloading...6771worker2 # [ 162.877645] systemd[1]: Reloading finished in 714 ms.6772worker2 # restarting sysinit-reactivation.target6773worker2 # [ 162.935343] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6774worker2 # [ 163.078080] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6775worker2 # [ 163.081386] systemd[1]: Reached target Reactivate sysinit units.6776worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6777worker2 # [ 163.087565] systemd[1]: Reloading D-Bus System Message Bus...6778worker2 # [ 163.088624] dbus-broker-launch[537]: Caught SIGHUP, trigger reload.6779worker2 # [ 163.092664] systemd[1]: Reloading Nix worker-protocol over gRPC...6780worker2 # [ 163.126447] dbus-broker-launch[537]: Looking up NSS user entry for 'systemd-timesync'...6781worker2 # [ 163.128700] dbus-broker-launch[537]: NSS returned no entry for 'systemd-timesync'6782worker2 # [ 163.129967] dbus-broker-launch[537]: Invalid user-name in /nix/store/s93qka0z548yjbxqnmj4c4b32apww6jv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6783worker2 # [ 163.140495] systemd[1]: Reloaded Nix worker-protocol over gRPC.6784worker2 # [ 163.158335] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:15Z level=info event=draining6785worker2 # [ 163.163707] systemd[1]: Reloaded D-Bus System Message Bus.6786worker2 # [ 163.168114] systemd[1]: Reached target Remote File Systems.6787worker2 # [ 163.172605] systemd[1]: Reached target Local File Systems.6788worker2 # [ 163.175777] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6789worker2 # [ 163.177496] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6790worker2 # [ 163.191212] systemd[1]: Starting linger-users.service...6791worker2 # [ 163.193432] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6792worker2 # [ 163.199842] systemd[1]: Starting Create SUID/SGID Wrappers...6793worker2 # [ 163.200906] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6794worker2 # [ 163.203197] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676795worker2 # [ 163.205771] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6796worker2 # [ 163.207448] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6797worker2 # [ 163.211056] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6798worker2 # [ 163.257248] systemd[1]: linger-users.service: Deactivated successfully.6799worker2 # [ 163.258621] systemd[1]: Finished linger-users.service.6800worker2 # [ 163.607259] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6801worker2 # [ 163.609988] systemd[1]: Finished Create SUID/SGID Wrappers.6802worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6803worker2 # [ 163.868113] nixos[1927]: finished switching to system configuration /nix/store/6q1lny800p9ad7ly0m09l74vha5wlg0f-nixos-system-worker2-test6804worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.19 seconds)6805worker2: must succeed: pgrep -f 'read -t [9]0 x'6806worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6807client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26808worker1 # [ 164.085346] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:535286809client # don't know how to build these paths:6810client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6811worker1 # [ 164.096686] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535286812worker2 # [ 164.164516] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:16Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06813worker2 # [ 164.167158] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:16Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16814worker1 # [ 164.165464] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535286815worker1 # [ 164.183731] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535286816client # copying 1 paths to 'lb:50051'...6817worker1 # [ 164.196850] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535286818client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6819worker1 # [ 164.218645] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6820worker1 # [ 164.394760] nix-daemon[866]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6821worker1 # [ 164.408563] nix-daemon[866]: reaped child process 1725, status = succeeded6822worker1 # [ 164.555529] nix-daemon[866]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6823worker1 # [ 164.569826] nix-daemon[866]: reaped child process 1730, status = succeeded6824lb # [ 164.596695] niks3-server[1141]: 2026/09/22 10:58:16 INFO Received uploads request method=POST path=/api/pending_closures6825worker1 # [ 164.595071] nix-grpc-daemon[1642]: time=2026-09-22T10:58:16.503Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6826worker1 # [ 164.597291] nix-grpc-daemon[1642]: time=2026-09-22T10:58:16.505Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6827lb # [ 164.626758] niks3-server[1141]: 2026/09/22 10:58:16 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6828lb # [ 164.639574] niks3-server[1141]: 2026/09/22 10:58:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6829worker1 # [ 164.632756] nix-grpc-daemon[1642]: time=2026-09-22T10:58:16.541Z level=INFO msg="Uploading 1 narinfos"6830lb # [ 164.645591] niks3-server[1141]: 2026/09/22 10:58:16 INFO Signed narinfos id=55 count=16831lb # [ 164.651799] niks3-server[1141]: 2026/09/22 10:58:16 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6832lb # [ 164.661665] niks3-server[1141]: 2026/09/22 10:58:16 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6833worker1 # [ 164.655249] nix-grpc-daemon[1642]: time=2026-09-22T10:58:16.563Z level=INFO msg="Upload complete. (424ms)"6834lb # [ 164.668074] niks3-server[1141]: 2026/09/22 10:58:16 INFO Completed upload id=556835lb # [ 164.670822] niks3-server[1141]: 2026/09/22 10:58:16 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6836worker1 # [ 164.658924] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=5286837worker1 # [ 164.664755] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535286838worker1 # [ 164.673260] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6839worker1 # [ 164.679387] nix-daemon[866]: reaped child process 1720, status = succeeded6840worker1 # [ 164.684822] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6841client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6842client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6843client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6844client # farm-a> LOG-farm-a-during-drain6845worker1 # [ 165.712854] nix-daemon[866]: reaped child process 1734, status = succeeded6846worker1 # [ 165.908366] nix-daemon[866]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6847worker1 # [ 165.922568] nix-daemon[866]: reaped child process 1753, status = succeeded6848lb # [ 165.955611] niks3-server[1141]: 2026/09/22 10:58:17 INFO Received uploads request method=POST path=/api/pending_closures6849worker1 # [ 165.948600] nix-grpc-daemon[1642]: time=2026-09-22T10:58:17.856Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6850worker1 # [ 165.952063] nix-grpc-daemon[1642]: time=2026-09-22T10:58:17.858Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6851lb # [ 165.976995] niks3-server[1141]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6852lb # [ 165.981823] niks3-server[1141]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6853lb # [ 165.986911] niks3-server[1141]: 2026/09/22 10:58:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6854lb # [ 165.989873] niks3-server[1141]: 2026/09/22 10:58:17 INFO Signed narinfos id=56 count=16855worker1 # [ 165.976891] nix-grpc-daemon[1642]: time=2026-09-22T10:58:17.885Z level=INFO msg="Uploading 1 narinfos"6856lb # [ 165.992373] niks3-server[1141]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6857lb # [ 165.997526] niks3-server[1141]: 2026/09/22 10:58:17 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6858lb # [ 166.001434] niks3-server[1141]: 2026/09/22 10:58:17 INFO Completed upload id=566859worker1 # [ 165.988761] nix-grpc-daemon[1642]: time=2026-09-22T10:58:17.897Z level=INFO msg="Upload complete. (295ms)"6860lb # [ 166.003256] niks3-server[1141]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6861worker1 # [ 165.992440] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16862worker2 # [ 166.044472] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:18Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06863worker1 # [ 166.000789] nix-daemon[866]: reaped child process 1732, status = succeeded6864worker2 # [ 166.046410] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:18Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16865worker1 # [ 166.041121] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535286866worker1 # [ 166.050637] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535286867worker1 # [ 166.059800] nix-grpc-daemon[1642]: time=2026-09-22T10:58:17.968Z level=INFO msg="All 1 paths already cached"6868client # copying 1 paths to 'lb:50051'...6869worker1 # [ 166.066717] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535286870client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6871worker1 # [ 166.086113] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6872worker1 # [ 166.271357] nix-daemon[866]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6873worker1 # [ 166.285520] nix-daemon[866]: reaped child process 1760, status = succeeded6874worker1 # [ 166.472565] nix-daemon[866]: accepted connection from pid 1762, user nix-grpc-daemon (trusted)6875worker1 # [ 166.485272] nix-daemon[866]: reaped child process 1765, status = succeeded6876lb # [ 166.517374] niks3-server[1141]: 2026/09/22 10:58:17 INFO Received uploads request method=POST path=/api/pending_closures6877worker1 # [ 166.514122] nix-grpc-daemon[1642]: time=2026-09-22T10:58:18.422Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6878worker1 # [ 166.516202] nix-grpc-daemon[1642]: time=2026-09-22T10:58:18.424Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6879lb # [ 166.549289] niks3-server[1141]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6880lb # [ 166.559397] niks3-server[1141]: 2026/09/22 10:58:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6881lb # [ 166.565089] niks3-server[1141]: 2026/09/22 10:58:17 INFO Signed narinfos id=57 count=16882worker1 # [ 166.555291] nix-grpc-daemon[1642]: time=2026-09-22T10:58:18.463Z level=INFO msg="Uploading 1 narinfos"6883lb # [ 166.569581] niks3-server[1141]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6884lb # [ 166.578486] niks3-server[1141]: 2026/09/22 10:58:18 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6885lb # [ 166.582299] niks3-server[1141]: 2026/09/22 10:58:18 INFO Completed upload id=576886worker1 # [ 166.569487] nix-grpc-daemon[1642]: time=2026-09-22T10:58:18.478Z level=INFO msg="Upload complete. (473ms)"6887lb # [ 166.584096] niks3-server[1141]: 2026/09/22 10:58:18 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6888worker1 # [ 166.573295] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=6406889worker1 # [ 166.578113] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535286890worker1 # [ 166.585770] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6891worker1 # [ 166.591719] nix-daemon[866]: reaped child process 1755, status = succeeded6892worker1 # [ 166.596042] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6893client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6894client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6895client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6896client # farm-b> LOG-farm-b-during-drain6897worker1 # [ 168.024408] nix-daemon[866]: reaped child process 1769, status = succeeded6898worker1 # [ 168.195835] nix-daemon[866]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6899worker1 # [ 168.208373] nix-daemon[866]: reaped child process 1788, status = succeeded6900lb # [ 168.239409] niks3-server[1141]: 2026/09/22 10:58:19 INFO Received uploads request method=POST path=/api/pending_closures6901worker1 # [ 168.235898] nix-grpc-daemon[1642]: time=2026-09-22T10:58:20.144Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6902worker1 # [ 168.238306] nix-grpc-daemon[1642]: time=2026-09-22T10:58:20.146Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6903lb # [ 168.275821] niks3-server[1141]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6904lb # [ 168.290745] niks3-server[1141]: 2026/09/22 10:58:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6905lb # [ 168.295896] niks3-server[1141]: 2026/09/22 10:58:19 INFO Signed narinfos id=58 count=16906worker1 # [ 168.286369] nix-grpc-daemon[1642]: time=2026-09-22T10:58:20.194Z level=INFO msg="Uploading 1 narinfos"6907lb # [ 168.304204] niks3-server[1141]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6908lb # [ 168.310811] niks3-server[1141]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6909lb # [ 168.320425] niks3-server[1141]: 2026/09/22 10:58:19 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6910lb # [ 168.324731] niks3-server[1141]: 2026/09/22 10:58:19 INFO Completed upload id=586911lb # [ 168.326409] niks3-server[1141]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6912worker1 # [ 168.313380] nix-grpc-daemon[1642]: time=2026-09-22T10:58:20.221Z level=INFO msg="Upload complete. (309ms)"6913worker1 # [ 168.319853] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16914worker2 # [ 168.371651] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:20Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06915worker2 # [ 168.373704] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:20Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16916worker1 # [ 168.334958] nix-daemon[866]: reaped child process 1767, status = succeeded6917worker1 # [ 168.370624] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535286918worker1 # [ 168.384223] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535286919worker1 # [ 168.397978] nix-grpc-daemon[1642]: time=2026-09-22T10:58:20.306Z level=INFO msg="All 2 paths already cached"6920client # copying 1 paths to 'lb:50051'...6921worker1 # [ 168.407391] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535286922client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6923worker1 # [ 168.425304] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6924worker1 # [ 168.614521] nix-daemon[866]: accepted connection from pid 1792, user nix-grpc-daemon (trusted)6925worker1 # [ 168.627144] nix-daemon[866]: reaped child process 1795, status = succeeded6926worker1 # [ 168.802624] nix-daemon[866]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6927worker1 # [ 168.815087] nix-daemon[866]: reaped child process 1800, status = succeeded6928lb # [ 168.844456] niks3-server[1141]: 2026/09/22 10:58:20 INFO Received uploads request method=POST path=/api/pending_closures6929worker1 # [ 168.840079] nix-grpc-daemon[1642]: time=2026-09-22T10:58:20.748Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6930worker1 # [ 168.841586] nix-grpc-daemon[1642]: time=2026-09-22T10:58:20.750Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6931lb # [ 168.877125] niks3-server[1141]: 2026/09/22 10:58:20 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6932lb # [ 168.888154] niks3-server[1141]: 2026/09/22 10:58:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6933lb # [ 168.893799] niks3-server[1141]: 2026/09/22 10:58:20 INFO Signed narinfos id=59 count=16934worker1 # [ 168.884296] nix-grpc-daemon[1642]: time=2026-09-22T10:58:20.792Z level=INFO msg="Uploading 1 narinfos"6935lb # [ 168.900869] niks3-server[1141]: 2026/09/22 10:58:20 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6936lb # [ 168.912297] niks3-server[1141]: 2026/09/22 10:58:20 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6937lb # [ 168.919865] niks3-server[1141]: 2026/09/22 10:58:20 INFO Completed upload id=596938worker1 # [ 168.907158] nix-grpc-daemon[1642]: time=2026-09-22T10:58:20.815Z level=INFO msg="Upload complete. (467ms)"6939lb # [ 168.922755] niks3-server[1141]: 2026/09/22 10:58:20 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6940worker1 # [ 168.911507] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=7686941worker1 # [ 168.916651] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535286942worker1 # [ 168.927138] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6943worker1 # [ 168.932935] nix-daemon[866]: reaped child process 1790, status = succeeded6944client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6945worker1 # [ 168.937092] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6946client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6947client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6948client # farm-top> LOG-farm-top-during-drain6949worker1 # [ 170.151478] nix-daemon[866]: reaped child process 1804, status = succeeded6950worker1 # [ 170.303354] nix-daemon[866]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6951worker1 # [ 170.315901] nix-daemon[866]: reaped child process 1823, status = succeeded6952lb # [ 170.346636] niks3-server[1141]: 2026/09/22 10:58:21 INFO Received uploads request method=POST path=/api/pending_closures6953worker1 # [ 170.338820] nix-grpc-daemon[1642]: time=2026-09-22T10:58:22.247Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6954worker1 # [ 170.340359] nix-grpc-daemon[1642]: time=2026-09-22T10:58:22.248Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6955lb # [ 170.366513] niks3-server[1141]: 2026/09/22 10:58:21 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6956lb # [ 170.371668] niks3-server[1141]: 2026/09/22 10:58:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6957lb # [ 170.374286] niks3-server[1141]: 2026/09/22 10:58:21 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6958lb # [ 170.377086] niks3-server[1141]: 2026/09/22 10:58:21 INFO Signed narinfos id=60 count=16959worker1 # [ 170.364340] nix-grpc-daemon[1642]: time=2026-09-22T10:58:22.272Z level=INFO msg="Uploading 1 narinfos"6960lb # [ 170.378935] niks3-server[1141]: 2026/09/22 10:58:21 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6961lb # [ 170.384056] niks3-server[1141]: 2026/09/22 10:58:21 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6962lb # [ 170.387323] niks3-server[1141]: 2026/09/22 10:58:21 INFO Completed upload id=606963worker1 # [ 170.374319] nix-grpc-daemon[1642]: time=2026-09-22T10:58:22.282Z level=INFO msg="Upload complete. (229ms)"6964lb # [ 170.388978] niks3-server[1141]: 2026/09/22 10:58:21 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6965worker1 # [ 170.376873] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16966worker1 # [ 170.383953] nix-daemon[866]: reaped child process 1802, status = succeeded6967client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.55 seconds)6968client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6969client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6970worker2: must succeed: pkill -f 'read -t [9]0 x'6971worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6972client: waiting for success: ! systemctl is-active sw6973worker2 # [ 170.859337] nix-daemon[859]: reaped child process 1899, status = succeeded6974worker2 # [ 171.032494] nix-daemon[859]: accepted connection from pid 2073, user nix-grpc-daemon (trusted)6975worker2 # [ 171.043345] nix-daemon[859]: reaped child process 2076, status = succeeded6976lb # [ 171.029409] niks3-server[1141]: 2026/09/22 10:58:22 INFO Received uploads request method=POST path=/api/pending_closures6977worker2 # [ 171.069170] nix-grpc-daemon[1827]: time=2026-09-22T10:58:23.224Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6978worker2 # [ 171.070862] nix-grpc-daemon[1827]: time=2026-09-22T10:58:23.226Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6979lb # [ 171.051547] niks3-server[1141]: 2026/09/22 10:58:22 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6980lb # [ 171.055686] niks3-server[1141]: 2026/09/22 10:58:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6981lb # [ 171.057624] niks3-server[1141]: 2026/09/22 10:58:22 INFO Signed narinfos id=61 count=16982worker2 # [ 171.092748] nix-grpc-daemon[1827]: time=2026-09-22T10:58:23.247Z level=INFO msg="Uploading 1 narinfos"6983lb # [ 171.060968] niks3-server[1141]: 2026/09/22 10:58:22 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6984lb # [ 171.066919] niks3-server[1141]: 2026/09/22 10:58:22 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6985lb # [ 171.068414] niks3-server[1141]: 2026/09/22 10:58:22 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6986lb # [ 171.073506] niks3-server[1141]: 2026/09/22 10:58:22 INFO Completed upload id=616987worker2 # [ 171.106968] nix-grpc-daemon[1827]: time=2026-09-22T10:58:23.262Z level=INFO msg="Upload complete. (263ms)"6988lb # [ 171.075264] niks3-server[1141]: 2026/09/22 10:58:22 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6989worker2 # [ 171.109790] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52894 duration_s=11 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16990worker2 # [ 171.117687] nix-daemon[859]: reaped child process 1897, status = succeeded6991client # [ 171.136282] systemd[1]: sw.service: Deactivated successfully.6992client # [ 171.140148] systemd[1]: sw.service: Consumed 124ms CPU time over 12.951s wall clock time, 41.4M memory peak, 7.4K incoming IP traffic, 8.3K outgoing IP traffic.6993worker2 # [ 171.160263] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:23Z level=info event=signal_exit6994lb # [ 171.355543] niks3-server[1141]: 2026/09/22 10:58:22 INFO lead: released remote=192.168.1.4:448866995client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6996client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6997client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6998worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6999worker2 # [ 171.922440] systemd[1]: Starting scheduler-token.service...7000worker2 # [ 171.998588] systemd[1]: scheduler-token.service: Deactivated successfully.7001worker2 # [ 171.999653] systemd[1]: Finished scheduler-token.service.7002worker2 # [ 172.001394] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 76ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.7003worker2 # [ 172.387853] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:24Z level=info event=scheduler_restarting7004worker1 # [ 172.342930] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:24Z level=info event=scheduler_restarting addr=lb:500517005worker2 # [ 172.395847] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:528867006worker2 # [ 172.399701] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:24Z level=info event=worker_gone addr=192.168.1.3:500517007worker2 # [ 172.402955] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:24Z level=info event=scheduler_restarting addr=lb:500517008worker2 # [ 172.406198] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:528867009worker2 # [ 172.409765] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:24Z level=info event=worker_gone addr=192.168.1.4:500517010worker2 # [ 172.422777] nix-daemon[859]: reaped child process 1798, status = succeeded7011worker2 # [ 172.470324] nix-grpc-daemon[1744]: E0000 00:00:1790074704.625657 1789 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true7012worker2 # [ 172.475832] nix-grpc-daemon[1744]: E0000 00:00:1790074704.625925 1789 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true7013worker1 # [ 172.542916] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:24Z level=info event=scheduler_connected addr=lb:500517014worker1 # [ 172.546991] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7015worker2 # [ 172.605814] nix-grpc-daemon[1744]: ts=2026-09-22T10:58:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"7016lb # [ 172.737728] niks3-server[1141]: 2026/09/22 10:58:24 INFO lead: acquired remote=192.168.1.3:602787017worker1 # [ 172.724710] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:24Z level=info event=scheduler_take_over7018worker1 # [ 172.747583] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:24Z level=info event=scheduler_connected addr=lb:500517019worker1 # [ 172.751539] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7020worker1 # [ 172.952398] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:24Z level=info event=scheduler_connected addr=lb:500517021worker1 # [ 172.956525] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7022worker2 # [ 173.192604] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.7023worker2 # [ 173.196902] systemd[1]: nix-grpc-daemon.service: Consumed 643ms CPU time over 26.330s wall clock time, 33.5M memory peak, 24.3K incoming IP traffic, 32.4K outgoing IP traffic.7024worker1 # [ 173.157618] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=scheduler_connected addr=lb:500517025worker1 # [ 173.161647] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7026worker2 # [ 173.213579] systemd[1]: Starting Nix worker-protocol over gRPC...7027worker2 # [ 173.383928] nix-grpc-daemon[2093]: ts=2026-09-22T10:58:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"7028worker2 # [ 173.386236] nix-grpc-daemon[2093]: ts=2026-09-22T10:58:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc7029worker2 # [ 173.389760] nix-grpc-daemon[2093]: ts=2026-09-22T10:58:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=17030worker2 # [ 173.397068] nix-grpc-daemon[2093]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR7031worker2 # [ 173.398381] nix-grpc-daemon[2093]: E0000 00:00:1790074705.552101 2093 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7032worker2 # [ 173.403410] nix-grpc-daemon[2093]: E0000 00:00:1790074705.557137 2093 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7033worker2 # [ 173.408610] nix-grpc-daemon[2093]: ts=2026-09-22T10:58:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket7034worker2 # [ 173.410514] systemd[1]: Started Nix worker-protocol over gRPC.7035worker1 # [ 173.363903] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=scheduler_connected addr=lb:500517036worker1 # [ 173.367360] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7037worker2 # [ 173.415618] nix-grpc-daemon[2093]: E0000 00:00:1790074705.571304 2132 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7038worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.58 seconds)7039lb: waiting until exactly no endpoints of x86_64-linux are unhealthy7040lb: must succeed: curl -sf localhost:9901/clusters7041worker2 # [ 173.428449] nix-grpc-daemon[2093]: E0000 00:00:1790074705.584075 2132 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7042worker2 # [ 173.432139] nix-grpc-daemon[2093]: E0000 00:00:1790074705.586266 2132 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7043worker2 # [ 173.443096] nix-grpc-daemon[2093]: E0000 00:00:1790074705.598175 2132 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7044worker2 # [ 173.449871] nix-grpc-daemon[2093]: ts=2026-09-22T10:58:25Z level=info event=scheduler_connected addr=lb:500517045worker2 # [ 173.451309] nix-grpc-daemon[2093]: ts=2026-09-22T10:58:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7046lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7047worker1 # [ 173.564636] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=scheduler_connected addr=lb:500517048worker1 # [ 173.575147] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:502927049worker1 # [ 173.578728] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07050worker2 # [ 173.652832] nix-grpc-daemon[2093]: ts=2026-09-22T10:58:25Z level=info event=scheduler_connected addr=lb:500517051worker1 # [ 173.613164] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=oidc_jwks provider=mock keys=17052worker1 # [ 173.616227] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:502927053worker1 # [ 173.620522] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07054lb: must succeed: curl -sf localhost:9901/clusters7055lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7056lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)7057lb: must succeed: curl -sf localhost:9901/clusters7058lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7059worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7060worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7061(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.45 seconds)7062subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7063client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17064worker2 # [ 174.843818] nix-grpc-daemon[2093]: ts=2026-09-22T10:58:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:476247065worker2 # [ 174.854641] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7066worker1 # [ 174.823918] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287067worker1 # [ 174.849861] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:26Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07068worker1 # [ 174.854980] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:26Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17069worker1 # [ 174.859382] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:26Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07070worker1 # [ 174.863285] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:26Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07071worker1 # [ 174.894415] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287072worker1 # [ 174.907908] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287073worker1 # [ 174.919996] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535287074worker1 # [ 174.930479] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7075worker1 # [ 175.085376] nix-daemon[866]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)7076worker1 # [ 175.098338] nix-daemon[866]: reaped child process 1835, status = succeeded7077worker1 # [ 175.240949] nix-daemon[866]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)7078worker1 # [ 175.251559] nix-daemon[866]: reaped child process 1840, status = succeeded7079lb # [ 175.283296] niks3-server[1141]: 2026/09/22 10:58:26 INFO Received uploads request method=POST path=/api/pending_closures7080worker1 # [ 175.279386] nix-grpc-daemon[1642]: time=2026-09-22T10:58:27.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7081worker1 # [ 175.280951] nix-grpc-daemon[1642]: time=2026-09-22T10:58:27.189Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7082lb # [ 175.316160] niks3-server[1141]: 2026/09/22 10:58:26 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7083lb # [ 175.325472] niks3-server[1141]: 2026/09/22 10:58:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7084lb # [ 175.330631] niks3-server[1141]: 2026/09/22 10:58:26 INFO Signed narinfos id=62 count=17085worker1 # [ 175.320935] nix-grpc-daemon[1642]: time=2026-09-22T10:58:27.229Z level=INFO msg="Uploading 1 narinfos"7086lb # [ 175.337944] niks3-server[1141]: 2026/09/22 10:58:26 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7087lb # [ 175.347412] niks3-server[1141]: 2026/09/22 10:58:26 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7088worker1 # [ 175.340080] nix-grpc-daemon[1642]: time=2026-09-22T10:58:27.248Z level=INFO msg="Upload complete. (398ms)"7089lb # [ 175.352850] niks3-server[1141]: 2026/09/22 10:58:26 INFO Completed upload id=627090worker1 # [ 175.342382] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=5047091worker1 # [ 175.345332] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287092lb # [ 175.358267] niks3-server[1141]: 2026/09/22 10:58:26 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7093worker1 # [ 175.354034] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7094worker1 # [ 175.355325] nix-daemon[866]: reaped child process 1830, status = succeeded7095worker1 # [ 175.360649] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7096worker1 # [ 176.987438] nix-daemon[866]: reaped child process 1844, status = succeeded7097worker1 # [ 177.164097] nix-daemon[866]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)7098worker1 # [ 177.175531] nix-daemon[866]: reaped child process 1863, status = succeeded7099lb # [ 177.204885] niks3-server[1141]: 2026/09/22 10:58:28 INFO Received uploads request method=POST path=/api/pending_closures7100worker1 # [ 177.197177] nix-grpc-daemon[1642]: time=2026-09-22T10:58:29.105Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7101worker1 # [ 177.199902] nix-grpc-daemon[1642]: time=2026-09-22T10:58:29.107Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7102lb # [ 177.228985] niks3-server[1141]: 2026/09/22 10:58:28 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7103lb # [ 177.231832] niks3-server[1141]: 2026/09/22 10:58:28 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7104lb # [ 177.236284] niks3-server[1141]: 2026/09/22 10:58:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7105lb # [ 177.238565] niks3-server[1141]: 2026/09/22 10:58:28 INFO Signed narinfos id=63 count=17106worker1 # [ 177.225904] nix-grpc-daemon[1642]: time=2026-09-22T10:58:29.134Z level=INFO msg="Uploading 1 narinfos"7107lb # [ 177.241746] niks3-server[1141]: 2026/09/22 10:58:28 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7108lb # [ 177.246980] niks3-server[1141]: 2026/09/22 10:58:28 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7109lb # [ 177.249777] niks3-server[1141]: 2026/09/22 10:58:28 INFO Completed upload id=637110worker1 # [ 177.238276] nix-grpc-daemon[1642]: time=2026-09-22T10:58:29.146Z level=INFO msg="Upload complete. (269ms)"7111lb # [ 177.251730] niks3-server[1141]: 2026/09/22 10:58:28 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7112worker1 # [ 177.241119] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=17113worker1 # [ 177.247426] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:29Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17114worker1 # [ 177.250380] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:29Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07115worker1 # [ 177.255044] nix-daemon[866]: reaped child process 1842, status = succeeded7116worker1 # [ 177.289616] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287117worker1 # [ 177.296798] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287118worker1 # [ 177.304585] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535287119worker1 # [ 177.312838] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7120worker1 # [ 177.478321] nix-daemon[866]: accepted connection from pid 1869, user nix-grpc-daemon (trusted)7121worker1 # [ 177.490799] nix-daemon[866]: reaped child process 1872, status = succeeded7122worker1 # [ 177.656259] nix-daemon[866]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)7123worker1 # [ 177.668363] nix-daemon[866]: reaped child process 1877, status = succeeded7124lb # [ 177.694448] niks3-server[1141]: 2026/09/22 10:58:29 INFO Received uploads request method=POST path=/api/pending_closures7125worker1 # [ 177.686981] nix-grpc-daemon[1642]: time=2026-09-22T10:58:29.595Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7126worker1 # [ 177.688805] nix-grpc-daemon[1642]: time=2026-09-22T10:58:29.597Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7127lb # [ 177.712984] niks3-server[1141]: 2026/09/22 10:58:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7128lb # [ 177.716905] niks3-server[1141]: 2026/09/22 10:58:29 INFO Signed narinfos id=64 count=17129worker1 # [ 177.703930] nix-grpc-daemon[1642]: time=2026-09-22T10:58:29.612Z level=INFO msg="Uploading 1 narinfos"7130lb # [ 177.724555] niks3-server[1141]: 2026/09/22 10:58:29 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7131lb # [ 177.729869] niks3-server[1141]: 2026/09/22 10:58:29 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7132lb # [ 177.733532] niks3-server[1141]: 2026/09/22 10:58:29 INFO Completed upload id=647133worker1 # [ 177.720741] nix-grpc-daemon[1642]: time=2026-09-22T10:58:29.629Z level=INFO msg="Upload complete. (403ms)"7134worker1 # [ 177.722815] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=5047135lb # [ 177.736902] niks3-server[1141]: 2026/09/22 10:58:29 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7136worker1 # [ 177.727832] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287137lb # [ 177.741509] niks3-server[1141]: 2026/09/22 10:58:29 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7138worker1 # [ 177.733792] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7139worker1 # [ 177.736175] nix-daemon[866]: reaped child process 1867, status = succeeded7140worker1 # [ 177.741579] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7141lb # [ 178.245811] envoy[921]: conn peer=[2001:db8:1::1]:50240 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9641 tx=4409 ms=63117142lb # [ 178.250327] envoy[921]: conn peer=[2001:db8:1::1]:44334 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2936 tx=1544 ms=127137143lb # [ 178.253870] envoy[921]: conn peer=[2001:db8:1::4]:56746 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1627 tx=426 ms=253987144worker1 # [ 178.765643] nix-daemon[866]: reaped child process 1881, status = succeeded7145worker1 # [ 178.916639] nix-daemon[866]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7146worker1 # [ 178.928783] nix-daemon[866]: reaped child process 1900, status = succeeded7147lb # [ 178.959826] niks3-server[1141]: 2026/09/22 10:58:30 INFO Received uploads request method=POST path=/api/pending_closures7148worker1 # [ 178.958701] nix-grpc-daemon[1642]: time=2026-09-22T10:58:30.867Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7149worker1 # [ 178.960300] nix-grpc-daemon[1642]: time=2026-09-22T10:58:30.868Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7150lb # [ 179.004247] niks3-server[1141]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7151lb # [ 179.014953] niks3-server[1141]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7152lb # [ 179.023720] niks3-server[1141]: 2026/09/22 10:58:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7153lb # [ 179.029667] niks3-server[1141]: 2026/09/22 10:58:30 INFO Signed narinfos id=65 count=17154worker1 # [ 179.017265] nix-grpc-daemon[1642]: time=2026-09-22T10:58:30.925Z level=INFO msg="Uploading 1 narinfos"7155lb # [ 179.032994] niks3-server[1141]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7156lb # [ 179.044595] niks3-server[1141]: 2026/09/22 10:58:30 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7157lb # [ 179.048107] niks3-server[1141]: 2026/09/22 10:58:30 INFO Completed upload id=657158worker1 # [ 179.035647] nix-grpc-daemon[1642]: time=2026-09-22T10:58:30.944Z level=INFO msg="Upload complete. (287ms)"7159lb # [ 179.050202] niks3-server[1141]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7160worker1 # [ 179.047227] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=17161worker1 # [ 179.054665] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:30Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17162worker1 # [ 179.059278] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287163worker1 # [ 179.063335] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:30Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07164worker1 # [ 179.073746] nix-daemon[866]: reaped child process 1879, status = succeeded7165worker1 # [ 179.075075] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287166worker1 # [ 179.081964] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535287167worker1 # [ 179.087087] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7168worker1 # [ 179.245292] nix-daemon[866]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)7169worker1 # [ 179.257491] nix-daemon[866]: reaped child process 1909, status = succeeded7170worker1 # [ 179.408063] nix-daemon[866]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)7171worker1 # [ 179.420079] nix-daemon[866]: reaped child process 1914, status = succeeded7172lb # [ 179.450106] niks3-server[1141]: 2026/09/22 10:58:30 INFO Received uploads request method=POST path=/api/pending_closures7173worker1 # [ 179.442671] nix-grpc-daemon[1642]: time=2026-09-22T10:58:31.351Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7174worker1 # [ 179.444220] nix-grpc-daemon[1642]: time=2026-09-22T10:58:31.352Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7175lb # [ 179.468939] niks3-server[1141]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7176lb # [ 179.472832] niks3-server[1141]: 2026/09/22 10:58:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7177worker1 # [ 179.462389] nix-grpc-daemon[1642]: time=2026-09-22T10:58:31.371Z level=INFO msg="Uploading 1 narinfos"7178lb # [ 179.475604] niks3-server[1141]: 2026/09/22 10:58:30 INFO Signed narinfos id=66 count=17179lb # [ 179.479207] niks3-server[1141]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7180lb # [ 179.484710] niks3-server[1141]: 2026/09/22 10:58:30 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7181lb # [ 179.490355] niks3-server[1141]: 2026/09/22 10:58:30 INFO Completed upload id=667182worker1 # [ 179.477538] nix-grpc-daemon[1642]: time=2026-09-22T10:58:31.386Z level=INFO msg="Upload complete. (385ms)"7183lb # [ 179.491828] niks3-server[1141]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7184worker1 # [ 179.479831] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=5047185worker1 # [ 179.483583] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287186worker1 # [ 179.490847] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7187worker1 # [ 179.492507] nix-daemon[866]: reaped child process 1904, status = succeeded7188worker1 # [ 179.497158] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7189worker1 # [ 180.911587] nix-daemon[866]: reaped child process 1918, status = succeeded7190worker1 # [ 181.064095] nix-daemon[866]: accepted connection from pid 1934, user nix-grpc-daemon (trusted)7191worker1 # [ 181.076144] nix-daemon[866]: reaped child process 1937, status = succeeded7192lb # [ 181.107965] niks3-server[1141]: 2026/09/22 10:58:32 INFO Received uploads request method=POST path=/api/pending_closures7193worker1 # [ 181.103492] nix-grpc-daemon[1642]: time=2026-09-22T10:58:33.011Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7194worker1 # [ 181.105141] nix-grpc-daemon[1642]: time=2026-09-22T10:58:33.013Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7195lb # [ 181.139590] niks3-server[1141]: 2026/09/22 10:58:32 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7196lb # [ 181.156365] niks3-server[1141]: 2026/09/22 10:58:32 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7197lb # [ 181.163485] niks3-server[1141]: 2026/09/22 10:58:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7198lb # [ 181.168328] niks3-server[1141]: 2026/09/22 10:58:32 INFO Signed narinfos id=67 count=17199worker1 # [ 181.158491] nix-grpc-daemon[1642]: time=2026-09-22T10:58:33.066Z level=INFO msg="Uploading 1 narinfos"7200lb # [ 181.175543] niks3-server[1141]: 2026/09/22 10:58:32 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7201lb # [ 181.184493] niks3-server[1141]: 2026/09/22 10:58:32 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7202lb # [ 181.189760] niks3-server[1141]: 2026/09/22 10:58:32 INFO Completed upload id=677203worker1 # [ 181.177237] nix-grpc-daemon[1642]: time=2026-09-22T10:58:33.085Z level=INFO msg="Upload complete. (272ms)"7204lb # [ 181.192787] niks3-server[1141]: 2026/09/22 10:58:32 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7205worker1 # [ 181.191121] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=17206worker1 # [ 181.195770] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:33Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17207worker1 # [ 181.199712] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287208worker1 # [ 181.203185] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:33Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07209worker1 # [ 181.213504] nix-daemon[866]: reaped child process 1916, status = succeeded7210worker1 # [ 181.216227] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287211worker1 # [ 181.224238] nix-grpc-daemon[1642]: time=2026-09-22T10:58:33.132Z level=INFO msg="All 1 paths already cached"7212worker1 # [ 181.228812] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535287213worker1 # [ 181.233611] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7214worker1 # [ 181.403706] nix-daemon[866]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)7215worker1 # [ 181.416303] nix-daemon[866]: reaped child process 1946, status = succeeded7216worker1 # [ 181.597391] nix-daemon[866]: accepted connection from pid 1948, user nix-grpc-daemon (trusted)7217worker1 # [ 181.609849] nix-daemon[866]: reaped child process 1951, status = succeeded7218lb # [ 181.639541] niks3-server[1141]: 2026/09/22 10:58:33 INFO Received uploads request method=POST path=/api/pending_closures7219worker1 # [ 181.634191] nix-grpc-daemon[1642]: time=2026-09-22T10:58:33.542Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7220worker1 # [ 181.635724] nix-grpc-daemon[1642]: time=2026-09-22T10:58:33.544Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7221lb # [ 181.670898] niks3-server[1141]: 2026/09/22 10:58:33 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7222lb # [ 181.679650] niks3-server[1141]: 2026/09/22 10:58:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7223lb # [ 181.683563] niks3-server[1141]: 2026/09/22 10:58:33 INFO Signed narinfos id=68 count=17224worker1 # [ 181.674290] nix-grpc-daemon[1642]: time=2026-09-22T10:58:33.582Z level=INFO msg="Uploading 1 narinfos"7225lb # [ 181.688943] niks3-server[1141]: 2026/09/22 10:58:33 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7226lb # [ 181.700171] niks3-server[1141]: 2026/09/22 10:58:33 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7227worker1 # [ 181.694348] nix-grpc-daemon[1642]: time=2026-09-22T10:58:33.602Z level=INFO msg="Upload complete. (455ms)"7228lb # [ 181.707261] niks3-server[1141]: 2026/09/22 10:58:33 INFO Completed upload id=687229worker1 # [ 181.696586] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=6247230lb # [ 181.709691] niks3-server[1141]: 2026/09/22 10:58:33 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7231worker1 # [ 181.703439] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287232worker1 # [ 181.705197] nix-daemon[866]: reaped child process 1941, status = succeeded7233worker1 # [ 181.708284] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7234worker1 # [ 181.715988] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7235worker1 # [ 182.527232] nix-daemon[866]: reaped child process 1955, status = succeeded7236worker1 # [ 182.682663] nix-daemon[866]: accepted connection from pid 1971, user nix-grpc-daemon (trusted)7237worker1 # [ 182.695295] nix-daemon[866]: reaped child process 1974, status = succeeded7238lb # [ 182.727537] niks3-server[1141]: 2026/09/22 10:58:34 INFO Received uploads request method=POST path=/api/pending_closures7239worker1 # [ 182.724224] nix-grpc-daemon[1642]: time=2026-09-22T10:58:34.631Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7240worker1 # [ 182.725571] nix-grpc-daemon[1642]: time=2026-09-22T10:58:34.631Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7241lb # [ 182.766612] niks3-server[1141]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7242lb # [ 182.777246] niks3-server[1141]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7243lb # [ 182.785585] niks3-server[1141]: 2026/09/22 10:58:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7244worker1 # [ 182.778747] nix-grpc-daemon[1642]: time=2026-09-22T10:58:34.687Z level=INFO msg="Uploading 1 narinfos"7245lb # [ 182.791542] niks3-server[1141]: 2026/09/22 10:58:34 INFO Signed narinfos id=69 count=17246lb # [ 182.796899] niks3-server[1141]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7247lb # [ 182.806219] niks3-server[1141]: 2026/09/22 10:58:34 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7248lb # [ 182.811510] niks3-server[1141]: 2026/09/22 10:58:34 INFO Completed upload id=697249worker1 # [ 182.798778] nix-grpc-daemon[1642]: time=2026-09-22T10:58:34.707Z level=INFO msg="Upload complete. (275ms)"7250lb # [ 182.814064] niks3-server[1141]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7251worker1 # [ 182.806544] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=17252worker1 # [ 182.816341] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:34Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17253worker1 # [ 182.821252] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287254worker1 # [ 182.824318] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:34Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07255worker1 # [ 182.833875] nix-daemon[866]: reaped child process 1953, status = succeeded7256worker1 # [ 182.835368] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287257worker1 # [ 182.843215] nix-grpc-daemon[1642]: time=2026-09-22T10:58:34.751Z level=INFO msg="All 1 paths already cached"7258worker1 # [ 182.846922] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535287259worker1 # [ 182.851519] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7260worker1 # [ 183.018671] nix-daemon[866]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)7261worker1 # [ 183.031477] nix-daemon[866]: reaped child process 1983, status = succeeded7262worker1 # [ 183.211629] nix-daemon[866]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7263worker1 # [ 183.224169] nix-daemon[866]: reaped child process 1988, status = succeeded7264lb # [ 183.253969] niks3-server[1141]: 2026/09/22 10:58:34 INFO Received uploads request method=POST path=/api/pending_closures7265worker1 # [ 183.251240] nix-grpc-daemon[1642]: time=2026-09-22T10:58:35.159Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7266worker1 # [ 183.252708] nix-grpc-daemon[1642]: time=2026-09-22T10:58:35.159Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7267lb # [ 183.285735] niks3-server[1141]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7268lb # [ 183.295343] niks3-server[1141]: 2026/09/22 10:58:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7269lb # [ 183.300624] niks3-server[1141]: 2026/09/22 10:58:34 INFO Signed narinfos id=70 count=17270worker1 # [ 183.290813] nix-grpc-daemon[1642]: time=2026-09-22T10:58:35.199Z level=INFO msg="Uploading 1 narinfos"7271lb # [ 183.306440] niks3-server[1141]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7272lb # [ 183.318401] niks3-server[1141]: 2026/09/22 10:58:34 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7273lb # [ 183.323174] niks3-server[1141]: 2026/09/22 10:58:34 INFO Completed upload id=707274worker1 # [ 183.312455] nix-grpc-daemon[1642]: time=2026-09-22T10:58:35.221Z level=INFO msg="Upload complete. (455ms)"7275worker1 # [ 183.314957] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=6247276lb # [ 183.329249] niks3-server[1141]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7277worker1 # [ 183.320419] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287278worker1 # [ 183.328314] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7279worker1 # [ 183.329943] nix-daemon[866]: reaped child process 1978, status = succeeded7280worker1 # [ 183.335954] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7281worker1 # [ 183.948592] nix-daemon[866]: reaped child process 1992, status = succeeded7282worker1 # [ 184.093948] nix-daemon[866]: accepted connection from pid 2008, user nix-grpc-daemon (trusted)7283worker1 # [ 184.106576] nix-daemon[866]: reaped child process 2011, status = succeeded7284lb # [ 184.137660] niks3-server[1141]: 2026/09/22 10:58:35 INFO Received uploads request method=POST path=/api/pending_closures7285worker1 # [ 184.133262] nix-grpc-daemon[1642]: time=2026-09-22T10:58:36.041Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7286worker1 # [ 184.135150] nix-grpc-daemon[1642]: time=2026-09-22T10:58:36.043Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7287lb # [ 184.175083] niks3-server[1141]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7288lb # [ 184.183651] niks3-server[1141]: 2026/09/22 10:58:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7289lb # [ 184.187678] niks3-server[1141]: 2026/09/22 10:58:35 INFO Signed narinfos id=71 count=17290worker1 # [ 184.177595] nix-grpc-daemon[1642]: time=2026-09-22T10:58:36.086Z level=INFO msg="Uploading 1 narinfos"7291lb # [ 184.195238] niks3-server[1141]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7292lb # [ 184.198272] niks3-server[1141]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7293lb # [ 184.202703] niks3-server[1141]: 2026/09/22 10:58:35 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7294lb # [ 184.206123] niks3-server[1141]: 2026/09/22 10:58:35 INFO Completed upload id=717295worker1 # [ 184.193332] nix-grpc-daemon[1642]: time=2026-09-22T10:58:36.101Z level=INFO msg="Upload complete. (251ms)"7296lb # [ 184.207891] niks3-server[1141]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7297worker1 # [ 184.198580] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=17298worker1 # [ 184.206663] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:36Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17299worker1 # [ 184.210139] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:36Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07300worker1 # [ 184.215445] nix-daemon[866]: reaped child process 1990, status = succeeded7301worker1 # [ 184.247680] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287302worker1 # [ 184.259980] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287303worker1 # [ 184.273340] nix-grpc-daemon[1642]: time=2026-09-22T10:58:36.181Z level=INFO msg="All 1 paths already cached"7304worker1 # [ 184.280544] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535287305worker1 # [ 184.290160] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7306worker1 # [ 184.463190] nix-daemon[866]: accepted connection from pid 2017, user nix-grpc-daemon (trusted)7307worker1 # [ 184.475477] nix-daemon[866]: reaped child process 2020, status = succeeded7308worker1 # [ 184.648059] nix-daemon[866]: accepted connection from pid 2022, user nix-grpc-daemon (trusted)7309worker1 # [ 184.660270] nix-daemon[866]: reaped child process 2025, status = succeeded7310lb # [ 184.691495] niks3-server[1141]: 2026/09/22 10:58:36 INFO Received uploads request method=POST path=/api/pending_closures7311worker1 # [ 184.686700] nix-grpc-daemon[1642]: time=2026-09-22T10:58:36.595Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7312worker1 # [ 184.688357] nix-grpc-daemon[1642]: time=2026-09-22T10:58:36.596Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7313lb # [ 184.722907] niks3-server[1141]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7314lb # [ 184.733066] niks3-server[1141]: 2026/09/22 10:58:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7315lb # [ 184.739737] niks3-server[1141]: 2026/09/22 10:58:36 INFO Signed narinfos id=72 count=17316worker1 # [ 184.729832] nix-grpc-daemon[1642]: time=2026-09-22T10:58:36.638Z level=INFO msg="Uploading 1 narinfos"7317lb # [ 184.744375] niks3-server[1141]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7318lb # [ 184.754945] niks3-server[1141]: 2026/09/22 10:58:36 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7319lb # [ 184.760085] niks3-server[1141]: 2026/09/22 10:58:36 INFO Completed upload id=727320worker1 # [ 184.749605] nix-grpc-daemon[1642]: time=2026-09-22T10:58:36.658Z level=INFO msg="Upload complete. (449ms)"7321lb # [ 184.764950] niks3-server[1141]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7322worker1 # [ 184.753715] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=6247323worker1 # [ 184.758458] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287324worker1 # [ 184.767890] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7325worker1 # [ 184.773394] nix-daemon[866]: reaped child process 2015, status = succeeded7326worker1 # [ 184.778501] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7327worker1 # [ 185.592434] nix-daemon[866]: reaped child process 2029, status = succeeded7328worker1 # [ 185.741578] nix-daemon[866]: accepted connection from pid 2045, user nix-grpc-daemon (trusted)7329worker1 # [ 185.753989] nix-daemon[866]: reaped child process 2048, status = succeeded7330lb # [ 185.784929] niks3-server[1141]: 2026/09/22 10:58:37 INFO Received uploads request method=POST path=/api/pending_closures7331worker1 # [ 185.782313] nix-grpc-daemon[1642]: time=2026-09-22T10:58:37.689Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7332worker1 # [ 185.784063] nix-grpc-daemon[1642]: time=2026-09-22T10:58:37.689Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7333lb # [ 185.819553] niks3-server[1141]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7334lb # [ 185.829163] niks3-server[1141]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7335lb # [ 185.837932] niks3-server[1141]: 2026/09/22 10:58:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7336worker1 # [ 185.829913] nix-grpc-daemon[1642]: time=2026-09-22T10:58:37.738Z level=INFO msg="Uploading 1 narinfos"7337lb # [ 185.844059] niks3-server[1141]: 2026/09/22 10:58:37 INFO Signed narinfos id=73 count=17338lb # [ 185.848258] niks3-server[1141]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7339lb # [ 185.855946] niks3-server[1141]: 2026/09/22 10:58:37 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7340lb # [ 185.862991] niks3-server[1141]: 2026/09/22 10:58:37 INFO Completed upload id=737341worker1 # [ 185.850106] nix-grpc-daemon[1642]: time=2026-09-22T10:58:37.758Z level=INFO msg="Upload complete. (265ms)"7342lb # [ 185.865410] niks3-server[1141]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7343worker1 # [ 185.860360] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=17344worker1 # [ 185.866241] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:37Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17345worker1 # [ 185.869738] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287346worker1 # [ 185.873214] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:37Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07347worker1 # [ 185.881878] nix-daemon[866]: reaped child process 2027, status = succeeded7348worker1 # [ 185.883385] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287349worker1 # [ 185.892149] nix-grpc-daemon[1642]: time=2026-09-22T10:58:37.800Z level=INFO msg="All 2 paths already cached"7350worker1 # [ 185.896628] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535287351worker1 # [ 185.900489] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7352worker1 # [ 186.063100] nix-daemon[866]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)7353worker1 # [ 186.075693] nix-daemon[866]: reaped child process 2057, status = succeeded7354worker1 # [ 186.237755] nix-daemon[866]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)7355worker1 # [ 186.250187] nix-daemon[866]: reaped child process 2062, status = succeeded7356lb # [ 186.275624] niks3-server[1141]: 2026/09/22 10:58:37 INFO Received uploads request method=POST path=/api/pending_closures7357worker1 # [ 186.270590] nix-grpc-daemon[1642]: time=2026-09-22T10:58:38.179Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7358worker1 # [ 186.272152] nix-grpc-daemon[1642]: time=2026-09-22T10:58:38.180Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7359lb # [ 186.304643] niks3-server[1141]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7360lb # [ 186.313474] niks3-server[1141]: 2026/09/22 10:58:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7361lb # [ 186.317839] niks3-server[1141]: 2026/09/22 10:58:37 INFO Signed narinfos id=74 count=17362worker1 # [ 186.307878] nix-grpc-daemon[1642]: time=2026-09-22T10:58:38.216Z level=INFO msg="Uploading 1 narinfos"7363lb # [ 186.326113] niks3-server[1141]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7364lb # [ 186.337519] niks3-server[1141]: 2026/09/22 10:58:37 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7365lb # [ 186.343497] niks3-server[1141]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7366lb # [ 186.347791] niks3-server[1141]: 2026/09/22 10:58:37 INFO Completed upload id=747367worker1 # [ 186.337133] nix-grpc-daemon[1642]: time=2026-09-22T10:58:38.245Z level=INFO msg="Upload complete. (430ms)"7368worker1 # [ 186.339271] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=7527369worker1 # [ 186.342711] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287370worker1 # [ 186.348550] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7371worker1 # [ 186.352086] nix-daemon[866]: reaped child process 2052, status = succeeded7372worker1 # [ 186.357110] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7373worker1 # [ 187.173477] nix-daemon[866]: reaped child process 2066, status = succeeded7374worker1 # [ 187.331964] nix-daemon[866]: accepted connection from pid 2082, user nix-grpc-daemon (trusted)7375worker1 # [ 187.343152] nix-daemon[866]: reaped child process 2085, status = succeeded7376lb # [ 187.374709] niks3-server[1141]: 2026/09/22 10:58:38 INFO Received uploads request method=POST path=/api/pending_closures7377worker1 # [ 187.366121] nix-grpc-daemon[1642]: time=2026-09-22T10:58:39.274Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7378worker1 # [ 187.368247] nix-grpc-daemon[1642]: time=2026-09-22T10:58:39.276Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7379lb # [ 187.394244] niks3-server[1141]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7380lb # [ 187.397084] niks3-server[1141]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7381lb # [ 187.401854] niks3-server[1141]: 2026/09/22 10:58:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7382lb # [ 187.403919] niks3-server[1141]: 2026/09/22 10:58:38 INFO Signed narinfos id=75 count=17383worker1 # [ 187.391063] nix-grpc-daemon[1642]: time=2026-09-22T10:58:39.299Z level=INFO msg="Uploading 1 narinfos"7384lb # [ 187.406906] niks3-server[1141]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7385lb # [ 187.412134] niks3-server[1141]: 2026/09/22 10:58:38 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7386lb # [ 187.415715] niks3-server[1141]: 2026/09/22 10:58:38 INFO Completed upload id=757387worker1 # [ 187.402884] nix-grpc-daemon[1642]: time=2026-09-22T10:58:39.311Z level=INFO msg="Upload complete. (239ms)"7388lb # [ 187.417356] niks3-server[1141]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7389worker1 # [ 187.405542] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=17390worker1 # [ 187.410409] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:39Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17391worker1 # [ 187.415236] nix-daemon[866]: reaped child process 2064, status = succeeded7392worker1 # [ 187.452187] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287393worker1 # [ 187.460940] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287394worker1 # [ 187.470164] nix-grpc-daemon[1642]: time=2026-09-22T10:58:39.378Z level=INFO msg="All 2 paths already cached"7395worker1 # [ 187.476572] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535287396worker1 # [ 187.483294] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7397worker1 # [ 187.651594] nix-daemon[866]: accepted connection from pid 2089, user nix-grpc-daemon (trusted)7398worker1 # [ 187.663953] nix-daemon[866]: reaped child process 2092, status = succeeded7399worker1 # [ 187.832592] nix-daemon[866]: accepted connection from pid 2094, user nix-grpc-daemon (trusted)7400worker1 # [ 187.845982] nix-daemon[866]: reaped child process 2097, status = succeeded7401lb # [ 187.871513] niks3-server[1141]: 2026/09/22 10:58:39 INFO Received uploads request method=POST path=/api/pending_closures7402worker1 # [ 187.866681] nix-grpc-daemon[1642]: time=2026-09-22T10:58:39.775Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7403worker1 # [ 187.868244] nix-grpc-daemon[1642]: time=2026-09-22T10:58:39.776Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7404lb # [ 187.899452] niks3-server[1141]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7405lb # [ 187.912438] niks3-server[1141]: 2026/09/22 10:58:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7406lb # [ 187.918543] niks3-server[1141]: 2026/09/22 10:58:39 INFO Signed narinfos id=76 count=17407worker1 # [ 187.909184] nix-grpc-daemon[1642]: time=2026-09-22T10:58:39.817Z level=INFO msg="Uploading 1 narinfos"7408lb # [ 187.922493] niks3-server[1141]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7409lb # [ 187.936107] niks3-server[1141]: 2026/09/22 10:58:39 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7410lb # [ 187.940576] niks3-server[1141]: 2026/09/22 10:58:39 INFO Completed upload id=767411worker1 # [ 187.930144] nix-grpc-daemon[1642]: time=2026-09-22T10:58:39.838Z level=INFO msg="Upload complete. (437ms)"7412lb # [ 187.943314] niks3-server[1141]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7413worker1 # [ 187.934359] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=7527414worker1 # [ 187.938850] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287415worker1 # [ 187.949288] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7416worker1 # [ 187.953657] nix-daemon[866]: reaped child process 2087, status = succeeded7417worker1 # [ 187.958860] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7418worker1 # [ 189.377788] nix-daemon[866]: reaped child process 2101, status = succeeded7419worker1 # [ 189.548776] nix-daemon[866]: accepted connection from pid 2117, user nix-grpc-daemon (trusted)7420worker1 # [ 189.561415] nix-daemon[866]: reaped child process 2120, status = succeeded7421lb # [ 189.589271] niks3-server[1141]: 2026/09/22 10:58:41 INFO Received uploads request method=POST path=/api/pending_closures7422worker1 # [ 189.589262] nix-grpc-daemon[1642]: time=2026-09-22T10:58:41.497Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7423worker1 # [ 189.592059] nix-grpc-daemon[1642]: time=2026-09-22T10:58:41.499Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7424lb # [ 189.621433] niks3-server[1141]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7425lb # [ 189.637284] niks3-server[1141]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7426lb # [ 189.646297] niks3-server[1141]: 2026/09/22 10:58:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7427lb # [ 189.650590] niks3-server[1141]: 2026/09/22 10:58:41 INFO Signed narinfos id=77 count=17428worker1 # [ 189.640266] nix-grpc-daemon[1642]: time=2026-09-22T10:58:41.548Z level=INFO msg="Uploading 1 narinfos"7429lb # [ 189.656545] niks3-server[1141]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7430lb # [ 189.661406] niks3-server[1141]: 2026/09/22 10:58:41 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7431lb # [ 189.664298] niks3-server[1141]: 2026/09/22 10:58:41 INFO Completed upload id=777432worker1 # [ 189.651448] nix-grpc-daemon[1642]: time=2026-09-22T10:58:41.560Z level=INFO msg="Upload complete. (284ms)"7433lb # [ 189.667657] niks3-server[1141]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7434worker1 # [ 189.654219] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=17435worker1 # [ 189.658120] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:41Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17436worker1 # [ 189.661143] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287437worker1 # [ 189.667873] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287438worker1 # [ 189.669994] nix-daemon[866]: reaped child process 2099, status = succeeded7439worker1 # [ 189.673689] nix-grpc-daemon[1642]: time=2026-09-22T10:58:41.582Z level=INFO msg="All 2 paths already cached"7440worker1 # [ 189.677714] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:535287441worker1 # [ 189.687100] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7442worker1 # [ 189.848806] nix-daemon[866]: accepted connection from pid 2124, user nix-grpc-daemon (trusted)7443worker1 # [ 189.861625] nix-daemon[866]: reaped child process 2127, status = succeeded7444worker1 # [ 190.013781] nix-daemon[866]: accepted connection from pid 2129, user nix-grpc-daemon (trusted)7445worker1 # [ 190.026068] nix-daemon[866]: reaped child process 2132, status = succeeded7446lb # [ 190.053891] niks3-server[1141]: 2026/09/22 10:58:41 INFO Received uploads request method=POST path=/api/pending_closures7447worker1 # [ 190.049322] nix-grpc-daemon[1642]: time=2026-09-22T10:58:41.957Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7448worker1 # [ 190.050933] nix-grpc-daemon[1642]: time=2026-09-22T10:58:41.957Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7449lb # [ 190.081329] niks3-server[1141]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7450lb # [ 190.092288] niks3-server[1141]: 2026/09/22 10:58:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7451worker1 # [ 190.083825] nix-grpc-daemon[1642]: time=2026-09-22T10:58:41.992Z level=INFO msg="Uploading 1 narinfos"7452lb # [ 190.096584] niks3-server[1141]: 2026/09/22 10:58:41 INFO Signed narinfos id=78 count=17453lb # [ 190.105494] niks3-server[1141]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7454lb # [ 190.112220] niks3-server[1141]: 2026/09/22 10:58:41 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7455lb # [ 190.117951] niks3-server[1141]: 2026/09/22 10:58:41 INFO Completed upload id=787456worker1 # [ 190.107340] nix-grpc-daemon[1642]: time=2026-09-22T10:58:42.015Z level=INFO msg="Upload complete. (415ms)"7457worker1 # [ 190.109558] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=0 paths=1 nar_bytes_in=7527458lb # [ 190.121843] niks3-server[1141]: 2026/09/22 10:58:41 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7459worker1 # [ 190.115122] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:535287460worker1 # [ 190.122076] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7461worker1 # [ 190.124356] nix-daemon[866]: reaped child process 2122, status = succeeded7462worker1 # [ 190.129803] nix-daemon[866]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)7463worker1 # [ 191.145619] nix-daemon[866]: reaped child process 2136, status = succeeded7464worker1 # [ 191.279751] nix-daemon[866]: accepted connection from pid 2152, user nix-grpc-daemon (trusted)7465worker1 # [ 191.292504] nix-daemon[866]: reaped child process 2155, status = succeeded7466lb # [ 191.322471] niks3-server[1141]: 2026/09/22 10:58:42 INFO Received uploads request method=POST path=/api/pending_closures7467worker1 # [ 191.318310] nix-grpc-daemon[1642]: time=2026-09-22T10:58:43.226Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7468worker1 # [ 191.320048] nix-grpc-daemon[1642]: time=2026-09-22T10:58:43.228Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7469lb # [ 191.351582] niks3-server[1141]: 2026/09/22 10:58:42 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7470lb # [ 191.369858] niks3-server[1141]: 2026/09/22 10:58:42 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7471lb # [ 191.378485] niks3-server[1141]: 2026/09/22 10:58:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7472worker1 # [ 191.370345] nix-grpc-daemon[1642]: time=2026-09-22T10:58:43.278Z level=INFO msg="Uploading 1 narinfos"7473lb # [ 191.383134] niks3-server[1141]: 2026/09/22 10:58:42 INFO Signed narinfos id=79 count=17474lb # [ 191.390322] niks3-server[1141]: 2026/09/22 10:58:42 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7475lb # [ 191.399237] niks3-server[1141]: 2026/09/22 10:58:42 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7476lb # [ 191.405160] niks3-server[1141]: 2026/09/22 10:58:42 INFO Completed upload id=797477worker1 # [ 191.392506] nix-grpc-daemon[1642]: time=2026-09-22T10:58:43.301Z level=INFO msg="Upload complete. (256ms)"7478lb # [ 191.406832] niks3-server[1141]: 2026/09/22 10:58:42 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7479worker1 # [ 191.395300] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53528 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=17480worker1 # [ 191.403516] nix-daemon[866]: reaped child process 2134, status = succeeded7481client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.88 seconds)7482worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7483worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7484client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'7485client # [ 191.583128] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7486client # Running as unit: gpu.service; invocation ID: b4d71824cea54a32a89b435920b7f28e7487client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.07 seconds)7488lb: must succeed: curl -sf localhost:9901/clusters7489lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7490lb: must succeed: curl -sf localhost:9901/clusters7491lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7492worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7493worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7494client # [ 191.788985] nix[1535]: don't know how to build these paths:7495client # [ 191.790376] nix[1535]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7496worker2 # [ 191.803942] nix-grpc-daemon[2093]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:476247497worker1 # [ 191.761074] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:535287498client # [ 191.812284] nix[1535]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker7499worker1 # [ 191.780956] nix-grpc-daemon[1550]: ts=2026-09-22T10:58:43Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07500lb: must succeed: curl -sf localhost:9901/clusters7501lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7502lb: must succeed: curl -sf localhost:9901/clusters7503lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7504worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7505worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7506client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7507lb # [ 198.246076] envoy[921]: conn peer=[2001:db8:1::1]:36636 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27094 tx=12345 ms=166237508client # [ 221.813463] nix[1535]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}7509client # [ 221.820985] nix[1535]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7510client # [ 221.859293] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7511client # [ 221.862681] systemd[1]: gpu.service: Failed with result 'exit-code'.7512client # [ 221.867110] systemd[1]: gpu.service: Consumed 132ms CPU time over 30.283s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7513client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.66 seconds)7514client: must succeed: journalctl -u gpu -o cat7515client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7516lb: must succeed: curl -sf localhost:9901/clusters7517lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7518lb: must succeed: curl -sf localhost:9901/clusters7519lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7520worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7521worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7522(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.19 seconds)7523subtest: low disk drains a worker and builds go to the other7524worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7525worker1 # [ 223.745980] nix-grpc-daemon[1550]: ts=2026-09-22T10:59:15Z level=info event=unhealthy reason=min_free7526worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)7527worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7528worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.05 seconds)7529client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&27530worker1 # [ 224.027465] nix-grpc-daemon[1550]: ts=2026-09-22T10:59:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:535287531client # don't know how to build these paths:7532client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7533worker2 # [ 224.083866] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476247534worker2 # [ 224.090881] nix-grpc-daemon[2093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 1/5)7535worker2 # [ 224.166613] nix-grpc-daemon[2093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 183 ms (attempt 2/5)7536worker2 # [ 224.355720] nix-grpc-daemon[2093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 3/5)7537worker2 # [ 224.379187] nix-grpc-daemon[2093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 444 ms (attempt 4/5)7538worker2 # [ 224.823858] nix-grpc-daemon[2093]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7539worker2 # [ 224.827894] nix-grpc-daemon[2093]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7540worker1 # [ 224.809376] nix-grpc-daemon[1550]: ts=2026-09-22T10:59:16Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07541worker1 # [ 224.811281] nix-grpc-daemon[1550]: ts=2026-09-22T10:59:16Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17542worker2 # [ 224.900167] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476247543worker2 # [ 224.913478] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476247544client # copying 1 paths to 'lb:50051'...7545client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7546worker2 # [ 224.926283] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:476247547worker2 # [ 224.935898] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7548worker2 # [ 224.958266] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:17Z level=info event=niks3_push_started pid=21537549worker2 # [ 225.177605] nix-grpc-daemon[2153]: time=2026-09-22T10:59:17.332Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""7550worker2 # [ 225.362102] nix-daemon[859]: accepted connection from pid 2166, user nix-grpc-daemon (trusted)7551worker2 # [ 225.374939] nix-daemon[859]: reaped child process 2169, status = succeeded7552worker2 # [ 225.555334] nix-daemon[859]: accepted connection from pid 2171, user nix-grpc-daemon (trusted)7553worker2 # [ 225.569153] nix-daemon[859]: reaped child process 2174, status = succeeded7554lb # [ 225.554927] niks3-server[1141]: 2026/09/22 10:59:16 INFO Received uploads request method=POST path=/api/pending_closures7555worker2 # [ 225.598509] nix-grpc-daemon[2153]: time=2026-09-22T10:59:17.754Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7556worker2 # [ 225.602092] nix-grpc-daemon[2153]: time=2026-09-22T10:59:17.755Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7557lb # [ 225.590285] niks3-server[1141]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7558worker2 # [ 225.643457] nix-grpc-daemon[2153]: time=2026-09-22T10:59:17.798Z level=INFO msg="Uploading 1 narinfos"7559lb # [ 225.609561] niks3-server[1141]: 2026/09/22 10:59:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7560lb # [ 225.612713] niks3-server[1141]: 2026/09/22 10:59:17 INFO Signed narinfos id=80 count=17561lb # [ 225.622155] niks3-server[1141]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7562lb # [ 225.635291] niks3-server[1141]: 2026/09/22 10:59:17 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7563lb # [ 225.638843] niks3-server[1141]: 2026/09/22 10:59:17 INFO Completed upload id=807564worker2 # [ 225.673664] nix-grpc-daemon[2153]: time=2026-09-22T10:59:17.829Z level=INFO msg="Upload complete. (492ms)"7565lb # [ 225.641312] niks3-server[1141]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7566worker2 # [ 225.676849] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47624 duration_s=0 paths=1 nar_bytes_in=5127567worker2 # [ 225.679859] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476247568worker2 # [ 225.685113] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7569worker2 # [ 225.693161] nix-daemon[859]: reaped child process 2151, status = succeeded7570client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7571worker2 # [ 225.697140] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7572client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7573client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7574client # farm-a> LOG-farm-a-drain7575worker2 # [ 227.111767] nix-daemon[859]: reaped child process 2178, status = succeeded7576worker2 # [ 227.276231] nix-daemon[859]: accepted connection from pid 2194, user nix-grpc-daemon (trusted)7577worker2 # [ 227.290358] nix-daemon[859]: reaped child process 2197, status = succeeded7578lb # [ 227.276626] niks3-server[1141]: 2026/09/22 10:59:18 INFO Received uploads request method=POST path=/api/pending_closures7579worker2 # [ 227.321463] nix-grpc-daemon[2153]: time=2026-09-22T10:59:19.476Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7580worker2 # [ 227.322993] nix-grpc-daemon[2153]: time=2026-09-22T10:59:19.478Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7581lb # [ 227.321603] niks3-server[1141]: 2026/09/22 10:59:18 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7582lb # [ 227.338205] niks3-server[1141]: 2026/09/22 10:59:18 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7583lb # [ 227.343923] niks3-server[1141]: 2026/09/22 10:59:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7584lb # [ 227.346855] niks3-server[1141]: 2026/09/22 10:59:18 INFO Signed narinfos id=81 count=17585worker2 # [ 227.381239] nix-grpc-daemon[2153]: time=2026-09-22T10:59:19.536Z level=INFO msg="Uploading 1 narinfos"7586lb # [ 227.349841] niks3-server[1141]: 2026/09/22 10:59:18 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7587lb # [ 227.357265] niks3-server[1141]: 2026/09/22 10:59:18 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7588lb # [ 227.361848] niks3-server[1141]: 2026/09/22 10:59:18 INFO Completed upload id=817589worker2 # [ 227.395116] nix-grpc-daemon[2153]: time=2026-09-22T10:59:19.550Z level=INFO msg="Upload complete. (291ms)"7590lb # [ 227.364080] niks3-server[1141]: 2026/09/22 10:59:18 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7591worker2 # [ 227.400548] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47624 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17592worker1 # [ 227.361874] nix-grpc-daemon[1550]: ts=2026-09-22T10:59:19Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07593worker2 # [ 227.413600] nix-daemon[859]: reaped child process 2176, status = succeeded7594worker1 # [ 227.365999] nix-grpc-daemon[1550]: ts=2026-09-22T10:59:19Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17595worker2 # [ 227.450900] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476247596worker2 # [ 227.462190] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476247597worker2 # [ 227.474280] nix-grpc-daemon[2153]: time=2026-09-22T10:59:19.629Z level=INFO msg="All 1 paths already cached"7598client # copying 1 paths to 'lb:50051'...7599client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7600worker2 # [ 227.482685] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:476247601worker2 # [ 227.493185] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7602worker2 # [ 227.686366] nix-daemon[859]: accepted connection from pid 2201, user nix-grpc-daemon (trusted)7603worker2 # [ 227.700328] nix-daemon[859]: reaped child process 2204, status = succeeded7604worker2 # [ 227.889592] nix-daemon[859]: accepted connection from pid 2206, user nix-grpc-daemon (trusted)7605worker2 # [ 227.903496] nix-daemon[859]: reaped child process 2209, status = succeeded7606lb # [ 227.889243] niks3-server[1141]: 2026/09/22 10:59:19 INFO Received uploads request method=POST path=/api/pending_closures7607worker2 # [ 227.931050] nix-grpc-daemon[2153]: time=2026-09-22T10:59:20.086Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7608worker2 # [ 227.932709] nix-grpc-daemon[2153]: time=2026-09-22T10:59:20.088Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7609lb # [ 227.918606] niks3-server[1141]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7610lb # [ 227.932053] niks3-server[1141]: 2026/09/22 10:59:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7611lb # [ 227.937056] niks3-server[1141]: 2026/09/22 10:59:19 INFO Signed narinfos id=82 count=17612worker2 # [ 227.973573] nix-grpc-daemon[2153]: time=2026-09-22T10:59:20.129Z level=INFO msg="Uploading 1 narinfos"7613lb # [ 227.944259] niks3-server[1141]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7614lb # [ 227.953357] niks3-server[1141]: 2026/09/22 10:59:19 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7615lb # [ 227.959623] niks3-server[1141]: 2026/09/22 10:59:19 INFO Completed upload id=827616worker2 # [ 227.993162] nix-grpc-daemon[2153]: time=2026-09-22T10:59:20.148Z level=INFO msg="Upload complete. (488ms)"7617lb # [ 227.964440] niks3-server[1141]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7618worker2 # [ 227.997357] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47624 duration_s=0 paths=1 nar_bytes_in=6247619worker2 # [ 228.002195] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476247620worker2 # [ 228.012063] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7621worker2 # [ 228.017130] nix-daemon[859]: reaped child process 2199, status = succeeded7622worker2 # [ 228.022674] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7623client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7624client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7625lb # [ 228.248136] envoy[921]: conn peer=[2001:db8:1::1]:42700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300477626client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7627client # farm-b> LOG-farm-b-drain7628worker2 # [ 229.649910] nix-daemon[859]: reaped child process 2213, status = succeeded7629worker2 # [ 229.830739] nix-daemon[859]: accepted connection from pid 2229, user nix-grpc-daemon (trusted)7630worker2 # [ 229.843649] nix-daemon[859]: reaped child process 2232, status = succeeded7631lb # [ 229.827955] niks3-server[1141]: 2026/09/22 10:59:21 INFO Received uploads request method=POST path=/api/pending_closures7632worker2 # [ 229.865877] nix-grpc-daemon[2153]: time=2026-09-22T10:59:22.021Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7633worker2 # [ 229.867444] nix-grpc-daemon[2153]: time=2026-09-22T10:59:22.023Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7634lb # [ 229.848582] niks3-server[1141]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7635lb # [ 229.852057] niks3-server[1141]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7636lb # [ 229.854087] niks3-server[1141]: 2026/09/22 10:59:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7637lb # [ 229.856047] niks3-server[1141]: 2026/09/22 10:59:21 INFO Signed narinfos id=83 count=17638worker2 # [ 229.889584] nix-grpc-daemon[2153]: time=2026-09-22T10:59:22.045Z level=INFO msg="Uploading 1 narinfos"7639lb # [ 229.859778] niks3-server[1141]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7640lb # [ 229.863920] niks3-server[1141]: 2026/09/22 10:59:21 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7641lb # [ 229.866673] niks3-server[1141]: 2026/09/22 10:59:21 INFO Completed upload id=837642worker2 # [ 229.901218] nix-grpc-daemon[2153]: time=2026-09-22T10:59:22.056Z level=INFO msg="Upload complete. (267ms)"7643lb # [ 229.868605] niks3-server[1141]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7644worker2 # [ 229.904298] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47624 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17645worker2 # [ 229.912380] nix-daemon[859]: reaped child process 2211, status = succeeded7646worker1 # [ 229.865908] nix-grpc-daemon[1550]: ts=2026-09-22T10:59:21Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07647worker1 # [ 229.870326] nix-grpc-daemon[1550]: ts=2026-09-22T10:59:21Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17648worker2 # [ 229.954836] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476247649worker2 # [ 229.965445] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476247650worker2 # [ 229.974521] nix-grpc-daemon[2153]: time=2026-09-22T10:59:22.130Z level=INFO msg="All 2 paths already cached"7651client # copying 1 paths to 'lb:50051'...7652client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7653worker2 # [ 229.982192] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:476247654worker2 # [ 229.991166] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7655worker2 # [ 230.170464] nix-daemon[859]: accepted connection from pid 2236, user nix-grpc-daemon (trusted)7656worker2 # [ 230.184775] nix-daemon[859]: reaped child process 2239, status = succeeded7657worker2 # [ 230.367483] nix-daemon[859]: accepted connection from pid 2241, user nix-grpc-daemon (trusted)7658worker2 # [ 230.381861] nix-daemon[859]: reaped child process 2244, status = succeeded7659lb # [ 230.364509] niks3-server[1141]: 2026/09/22 10:59:21 INFO Received uploads request method=POST path=/api/pending_closures7660worker2 # [ 230.402294] nix-grpc-daemon[2153]: time=2026-09-22T10:59:22.557Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7661worker2 # [ 230.404315] nix-grpc-daemon[2153]: time=2026-09-22T10:59:22.559Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7662lb # [ 230.379342] niks3-server[1141]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7663lb # [ 230.382774] niks3-server[1141]: 2026/09/22 10:59:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7664lb # [ 230.384579] niks3-server[1141]: 2026/09/22 10:59:21 INFO Signed narinfos id=84 count=17665worker2 # [ 230.417981] nix-grpc-daemon[2153]: time=2026-09-22T10:59:22.573Z level=INFO msg="Uploading 1 narinfos"7666lb # [ 230.388237] niks3-server[1141]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7667lb # [ 230.392496] niks3-server[1141]: 2026/09/22 10:59:21 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7668lb # [ 230.395069] niks3-server[1141]: 2026/09/22 10:59:21 INFO Completed upload id=847669worker2 # [ 230.428502] nix-grpc-daemon[2153]: time=2026-09-22T10:59:22.584Z level=INFO msg="Upload complete. (428ms)"7670lb # [ 230.397151] niks3-server[1141]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7671worker2 # [ 230.430881] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47624 duration_s=0 paths=1 nar_bytes_in=7527672worker2 # [ 230.435208] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476247673worker2 # [ 230.441830] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7674worker2 # [ 230.444237] nix-daemon[859]: reaped child process 2234, status = succeeded7675client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7676worker2 # [ 230.450308] nix-daemon[859]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)7677client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7678client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7679client # farm-top> LOG-farm-top-drain7680worker2 # [ 231.266631] nix-daemon[859]: reaped child process 2248, status = succeeded7681worker2 # [ 231.425551] nix-daemon[859]: accepted connection from pid 2264, user nix-grpc-daemon (trusted)7682worker2 # [ 231.439870] nix-daemon[859]: reaped child process 2267, status = succeeded7683lb # [ 231.425155] niks3-server[1141]: 2026/09/22 10:59:22 INFO Received uploads request method=POST path=/api/pending_closures7684worker2 # [ 231.463477] nix-grpc-daemon[2153]: time=2026-09-22T10:59:23.618Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7685worker2 # [ 231.466176] nix-grpc-daemon[2153]: time=2026-09-22T10:59:23.620Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7686lb # [ 231.446301] niks3-server[1141]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7687lb # [ 231.450382] niks3-server[1141]: 2026/09/22 10:59:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7688lb # [ 231.452226] niks3-server[1141]: 2026/09/22 10:59:22 INFO Signed narinfos id=85 count=17689worker2 # [ 231.487072] nix-grpc-daemon[2153]: time=2026-09-22T10:59:23.642Z level=INFO msg="Uploading 1 narinfos"7690lb # [ 231.455041] niks3-server[1141]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7691lb # [ 231.458477] niks3-server[1141]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7692lb # [ 231.463258] niks3-server[1141]: 2026/09/22 10:59:22 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7693lb # [ 231.466844] niks3-server[1141]: 2026/09/22 10:59:22 INFO Completed upload id=857694worker2 # [ 231.500303] nix-grpc-daemon[2153]: time=2026-09-22T10:59:23.655Z level=INFO msg="Upload complete. (242ms)"7695lb # [ 231.468449] niks3-server[1141]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7696worker2 # [ 231.503418] nix-grpc-daemon[2093]: ts=2026-09-22T10:59:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47624 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17697worker2 # [ 231.511661] nix-daemon[859]: reaped child process 2246, status = succeeded7698client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 7.68 seconds)7699client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7700client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.20 seconds)7701worker1: must succeed: rm /nix/.rw-store/fill7702worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7703worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7704worker1 # [ 232.750255] nix-grpc-daemon[1550]: ts=2026-09-22T10:59:24Z level=info event=healthy reason=min_free7705worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)7706(finished: subtest: low disk drains a worker and builds go to the other, in 10.23 seconds)7707(finished: run the VM test script, in 233.90 seconds)7708test script finished in 234.14s7709cleanup7710kill QemuMachine (pid 49)7711client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7712kill QemuMachine (pid 50)7713lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7714kill QemuMachine (pid 51)7715worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7716kill QemuMachine (pid 52)7717worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7718(finished: cleanup, in 1.03 seconds)7719additionally exposed symbols:7720 client, lb, worker1, worker2,7721 vlan1,7722 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh