vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #490
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.GxoBY7voNY', fmt=raw size=107374182414worker2: QEMU running (pid 51)15worker2 # mke2fs 1.47.4 (6-Mar-2025)16client # Disk image does not exist, creating the virtualisation disk image...17client: QEMU running (pid 49)18client # Formatting '/build/vm-state-client/tmp.TRyWuNMI5H', fmt=raw size=107374182419worker2 # Discarding device blocks: 0/262144 done20client # mke2fs 1.47.4 (6-Mar-2025)21worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes22client # Discarding device blocks: 0/262144 done23worker2 # Filesystem UUID: cbec9c60-0d58-4c80-95cf-6588a41b141a24client # Creating filesystem with 262144 4k blocks and 65536 inodes25worker2 # Superblock backups stored on blocks:26client # Filesystem UUID: 0890b4c2-0ef3-45c7-b169-9d94c50e5fd227worker2 # 32768, 98304, 163840, 22937628client # Superblock backups stored on blocks:29worker2 # 30client # 32768, 98304, 163840, 22937631worker2 # Allocating group tables: 0/8 done32client # 33worker2 # Writing inode tables: 0/8 done34client # Allocating group tables: 0/8 done35worker2 # Creating journal (8192 blocks): done36worker1 # Disk image does not exist, creating the virtualisation disk image...37worker1: QEMU running (pid 52)38client # Writing inode tables: 0/8 done39worker2 # Writing superblocks and filesystem accounting information: 0/8 done40lb # Disk image does not exist, creating the virtualisation disk image...41lb: QEMU running (pid 50)42worker2 # 43lb # Formatting '/build/vm-state-lb/tmp.h769jcu1Hx', fmt=raw size=107374182444client # Creating journal (8192 blocks): done45lb # mke2fs 1.47.4 (6-Mar-2025)46lb # Discarding device blocks: 0/262144 done47worker1 # Formatting '/build/vm-state-worker1/tmp.5dxvLfZe9l', fmt=raw size=107374182448(finished: start all VMs, in 0.22 seconds)49lb # Creating filesystem with 262144 4k blocks and 65536 inodes50client # Writing superblocks and filesystem accounting information: 0/8 done51client # 52worker2 # Virtualisation disk image created.53client: waiting for unit network-addresses-eth1.service54worker1 # mke2fs 1.47.4 (6-Mar-2025)55client: waiting for the VM to finish booting56worker1 # Discarding device blocks: 0/262144 done57lb # Filesystem UUID: ba1ef3fe-c0df-411e-ba6f-7acaf31b897f58worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes59lb # Superblock backups stored on blocks:60worker1 # Filesystem UUID: 1867878c-9e92-4aa3-91f4-87565665507a61lb # 32768, 98304, 163840, 22937662worker1 # Superblock backups stored on blocks:63lb # 64worker1 # 32768, 98304, 163840, 22937665lb # Allocating group tables: 0/8 done66worker1 # 67lb # Writing inode tables: 0/8 done68worker1 # Allocating group tables: 0/8 done69lb # Creating journal (8192 blocks): done70worker1 # Writing inode tables: 0/8 done71lb # Writing superblocks and filesystem accounting information: 0/8 done72worker1 # Creating journal (8192 blocks): done73lb # 74worker1 # Writing superblocks and filesystem accounting information: 0/8 done75lb # Virtualisation disk image created.76worker1 # 77client # Virtualisation disk image created.78worker1 # Virtualisation disk image created.79client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83client # 84client # 85client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker1 # 87worker1 # 88lb # 89lb # 90worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0091lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0092client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...93client # 94client # 95client # 96client # 97worker2 # 98worker2 # 99client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00100worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00101worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...102worker1 # 103worker1 # 104worker1 # 105worker1 # 106lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...107lb # 108lb # 109lb # 110lb # 111worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00112lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00113worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...114worker2 # 115worker2 # 116worker2 # 117worker2 # 118worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...119worker1 # 120worker1 # 121client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...122client # 123client # 124worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00125lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...126lb # 127lb # 128worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker2 # 130worker2 # 131client # Booting from ROM...132lb # Booting from ROM...133worker1 # Booting from ROM...134worker2 # Booting from ROM...135client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test/init regInfo=/nix/store/nm58dyaq8zm0vqgkfdrxs5bqws6f19cb-closure-info/registration console=ttyS0,115200n8 console=tty0137client # [ 0.000000] BIOS-provided physical RAM map:138client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026141client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026143client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable144client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved145client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved146worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lv03q5h92lsnc65incxxasgvq4y82d3q-nixos-system-worker1-test/init regInfo=/nix/store/k2lpqpf1yl5ndvc9v5w1mjf3dnbawjds-closure-info/registration console=ttyS0,115200n8 console=tty0147client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved148worker1 # [ 0.000000] BIOS-provided physical RAM map:149worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026150lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mvx6cn1l6frdr55gdj0p01an8nxnxzn-nixos-system-lb-test/init regInfo=/nix/store/0a2f35z8lm3xm6ffdv71jfscn9vyk5l1-closure-info/registration console=ttyS0,115200n8 console=tty0151client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved152worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable153lb # [ 0.000000] BIOS-provided physical RAM map:154client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved155worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved156lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable157client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved158worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved159lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved160client # [ 0.000000] NX (Execute Disable) protection: active161worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable162client # [ 0.000000] APIC: Static calls initialized163client # [ 0.000000] SMBIOS 2.8 present.164lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved165worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved166worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m8qvh7i9ijf4y451014qsh100ca0dry7-nixos-system-worker2-test/init regInfo=/nix/store/2a07p09nzhh9mwi0id9lmsp9cpx2aiml-closure-info/registration console=ttyS0,115200n8 console=tty0167lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable168worker2 # [ 0.000000] BIOS-provided physical RAM map:169worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved170client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014171lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved172client # [ 0.000000] DMI: Memory slots populated: 1/1173worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable174worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved175client # [ 0.000000] Hypervisor detected: KVM176lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved177worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved178client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000179worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved180lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved181worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved182client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00183worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved184lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved185client # [ 0.000000] kvm-clock: using sched offset of 479613763 cycles186worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable187worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved188lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved189worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved190worker1 # [ 0.000000] NX (Execute Disable) protection: active191client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns192worker1 # [ 0.000000] APIC: Static calls initialized193lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved194worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved195client # [ 0.000005] tsc: Detected 2400.012 MHz processor196worker1 # [ 0.000000] SMBIOS 2.8 present.197lb # [ 0.000000] NX (Execute Disable) protection: active198client # [ 0.000825] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000199worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved200lb # [ 0.000000] APIC: Static calls initialized201worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014202lb # [ 0.000000] SMBIOS 2.8 present.203worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved204client # [ 0.000852] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs205worker1 # [ 0.000000] DMI: Memory slots populated: 1/1206worker1 # [ 0.000000] Hypervisor detected: KVM207worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved208lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014209client # [ 0.000855] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT210worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000211lb # [ 0.000000] DMI: Memory slots populated: 1/1212client # [ 0.002826] found SMP MP-table at [mem 0x000f5450-0x000f545f]213worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved214lb # [ 0.000000] Hypervisor detected: KVM215worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00216client # [ 0.002837] Using GB pages for direct mapping217worker2 # [ 0.000000] NX (Execute Disable) protection: active218lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000219client # [ 0.002906] RAMDISK: [mem 0x5e366000-0x5ffcffff]220worker1 # [ 0.000000] kvm-clock: using sched offset of 463815340 cycles221worker2 # [ 0.000000] APIC: Static calls initialized222worker2 # [ 0.000000] SMBIOS 2.8 present.223lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00224client # [ 0.002915] ACPI: Early table checksum verification disabled225lb # [ 0.000000] kvm-clock: using sched offset of 469288803 cycles226worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns227client # [ 0.002918] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )228worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor229worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014230worker2 # [ 0.000000] DMI: Memory slots populated: 1/1231worker1 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000232client # [ 0.002921] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)233lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns234worker2 # [ 0.000000] Hypervisor detected: KVM235lb # [ 0.000005] tsc: Detected 2400.012 MHz processor236worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000237client # [ 0.002925] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)238worker1 # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs239lb # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000240worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00241client # [ 0.002932] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)242worker1 # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT243worker2 # [ 0.000001] kvm-clock: using sched offset of 495865342 cycles244lb # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs245client # [ 0.002934] ACPI: FACS 0x000000005FFE0000 000040246worker1 # [ 0.002741] found SMP MP-table at [mem 0x000f5450-0x000f545f]247worker1 # [ 0.002752] Using GB pages for direct mapping248lb # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT249worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns250client # [ 0.002935] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)251worker1 # [ 0.002836] RAMDISK: [mem 0x5e367000-0x5ffcffff]252lb # [ 0.002750] found SMP MP-table at [mem 0x000f5450-0x000f545f]253worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor254worker1 # [ 0.002845] ACPI: Early table checksum verification disabled255lb # [ 0.002763] Using GB pages for direct mapping256client # [ 0.002937] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)257worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000258lb # [ 0.002909] RAMDISK: [mem 0x5e367000-0x5ffcffff]259worker1 # [ 0.002848] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )260client # [ 0.002939] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)261lb # [ 0.002918] ACPI: Early table checksum verification disabled262worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs263worker1 # [ 0.002852] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)264lb # [ 0.002922] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )265worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT266client # [ 0.002940] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)267worker1 # [ 0.002855] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)268worker2 # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]269lb # [ 0.002927] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)270client # [ 0.002941] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]271worker2 # [ 0.002748] Using GB pages for direct mapping272worker1 # [ 0.002862] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)273worker2 # [ 0.002911] RAMDISK: [mem 0x5e367000-0x5ffcffff]274lb # [ 0.002931] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)275client # [ 0.002942] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]276worker1 # [ 0.002864] ACPI: FACS 0x000000005FFE0000 000040277worker2 # [ 0.002919] ACPI: Early table checksum verification disabled278client # [ 0.002943] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]279lb # [ 0.002939] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)280worker1 # [ 0.002866] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)281worker2 # [ 0.002922] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )282lb # [ 0.002941] ACPI: FACS 0x000000005FFE0000 000040283client # [ 0.002944] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]284worker2 # [ 0.002926] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)285worker1 # [ 0.002867] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)286client # [ 0.002944] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]287lb # [ 0.002943] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)288worker2 # [ 0.002930] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)289worker1 # [ 0.002869] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)290client # [ 0.002944] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]291lb # [ 0.002944] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)292client # [ 0.002945] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]293worker2 # [ 0.002937] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)294worker1 # [ 0.002870] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)295client # [ 0.003202] No NUMA configuration found296lb # [ 0.002946] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)297worker2 # [ 0.002939] ACPI: FACS 0x000000005FFE0000 000040298worker1 # [ 0.002872] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]299client # [ 0.003203] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]300lb # [ 0.002947] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)301worker2 # [ 0.002941] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)302worker1 # [ 0.002873] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]303client # [ 0.003205] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]304client # [ 0.005450] Zone ranges:305lb # [ 0.002949] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]306worker1 # [ 0.002873] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]307worker2 # [ 0.002942] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)308client # [ 0.005451] DMA [mem 0x0000000000001000-0x0000000000ffffff]309lb # [ 0.002950] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]310worker1 # [ 0.002874] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]311worker2 # [ 0.002944] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)312client # [ 0.005452] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]313lb # [ 0.002950] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]314client # [ 0.005453] Normal empty315worker1 # [ 0.002874] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]316client # [ 0.005454] Device empty317worker2 # [ 0.002945] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)318lb # [ 0.002951] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]319client # [ 0.005455] Movable zone start for each node320worker1 # [ 0.002875] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]321client # [ 0.005455] Early memory node ranges322worker2 # [ 0.002946] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]323lb # [ 0.002951] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]324worker1 # [ 0.002875] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]325client # [ 0.005456] node 0: [mem 0x0000000000001000-0x000000000009efff]326worker2 # [ 0.002948] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]327lb # [ 0.002952] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]328worker1 # [ 0.003108] No NUMA configuration found329client # [ 0.005457] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]330worker2 # [ 0.002948] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]331worker1 # [ 0.003109] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]332lb # [ 0.002952] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]333client # [ 0.005458] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]334lb # [ 0.003188] No NUMA configuration found335worker1 # [ 0.003111] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]336worker2 # [ 0.002949] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]337worker1 # [ 0.005390] Zone ranges:338client # [ 0.005477] On node 0, zone DMA: 1 pages in unavailable ranges339lb # [ 0.003189] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]340worker2 # [ 0.002949] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]341client # [ 0.005749] On node 0, zone DMA: 97 pages in unavailable ranges342worker1 # [ 0.005391] DMA [mem 0x0000000000001000-0x0000000000ffffff]343lb # [ 0.003191] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]344lb # [ 0.005447] Zone ranges:345client # [ 0.031737] On node 0, zone DMA32: 40 pages in unavailable ranges346worker1 # [ 0.005392] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]347worker2 # [ 0.002950] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]348worker1 # [ 0.005394] Normal empty349client # [ 0.032204] ACPI: PM-Timer IO Port: 0x608350lb # [ 0.005448] DMA [mem 0x0000000000001000-0x0000000000ffffff]351worker1 # [ 0.005395] Device empty352worker2 # [ 0.002950] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]353client # [ 0.032216] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])354worker1 # [ 0.005395] Movable zone start for each node355worker2 # [ 0.003172] No NUMA configuration found356lb # [ 0.005450] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]357worker1 # [ 0.005396] Early memory node ranges358lb # [ 0.005451] Normal empty359client # [ 0.032242] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23360worker2 # [ 0.003173] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]361lb # [ 0.005452] Device empty362worker1 # [ 0.005396] node 0: [mem 0x0000000000001000-0x000000000009efff]363lb # [ 0.005452] Movable zone start for each node364worker2 # [ 0.003176] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]365client # [ 0.032244] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)366lb # [ 0.005453] Early memory node ranges367worker2 # [ 0.005680] Zone ranges:368worker1 # [ 0.005397] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]369client # [ 0.032246] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)370lb # [ 0.005454] node 0: [mem 0x0000000000001000-0x000000000009efff]371worker2 # [ 0.005681] DMA [mem 0x0000000000001000-0x0000000000ffffff]372worker1 # [ 0.005398] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]373client # [ 0.032247] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)374worker2 # [ 0.005682] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]375lb # [ 0.005455] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]376worker1 # [ 0.005418] On node 0, zone DMA: 1 pages in unavailable ranges377worker2 # [ 0.005684] Normal empty378client # [ 0.032248] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)379worker2 # [ 0.005685] Device empty380lb # [ 0.005456] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]381worker1 # [ 0.005690] On node 0, zone DMA: 97 pages in unavailable ranges382worker2 # [ 0.005685] Movable zone start for each node383client # [ 0.032249] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)384lb # [ 0.005482] On node 0, zone DMA: 1 pages in unavailable ranges385worker2 # [ 0.005686] Early memory node ranges386worker1 # [ 0.032439] On node 0, zone DMA32: 40 pages in unavailable ranges387worker1 # [ 0.032906] ACPI: PM-Timer IO Port: 0x608388client # [ 0.032251] ACPI: Using ACPI (MADT) for SMP configuration information389lb # [ 0.005746] On node 0, zone DMA: 97 pages in unavailable ranges390worker2 # [ 0.005686] node 0: [mem 0x0000000000001000-0x000000000009efff]391worker1 # [ 0.032918] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])392client # [ 0.032252] ACPI: HPET id: 0x8086a201 base: 0xfed00000393lb # [ 0.032092] On node 0, zone DMA32: 40 pages in unavailable ranges394worker2 # [ 0.005687] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]395client # [ 0.032256] TSC deadline timer available396lb # [ 0.032561] ACPI: PM-Timer IO Port: 0x608397worker1 # [ 0.032947] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23398client # [ 0.032260] CPU topo: Max. logical packages: 1399worker2 # [ 0.005688] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]400lb # [ 0.032573] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])401worker1 # [ 0.032949] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)402client # [ 0.032260] CPU topo: Max. logical dies: 1403worker2 # [ 0.005708] On node 0, zone DMA: 1 pages in unavailable ranges404client # [ 0.032261] CPU topo: Max. dies per package: 1405lb # [ 0.032603] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23406worker1 # [ 0.032951] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)407worker2 # [ 0.005964] On node 0, zone DMA: 97 pages in unavailable ranges408client # [ 0.032264] CPU topo: Max. threads per core: 1409lb # [ 0.032605] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)410worker1 # [ 0.032952] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)411worker2 # [ 0.032237] On node 0, zone DMA32: 40 pages in unavailable ranges412client # [ 0.032265] CPU topo: Num. cores per package: 1413lb # [ 0.032607] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)414worker2 # [ 0.032701] ACPI: PM-Timer IO Port: 0x608415worker1 # [ 0.032953] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)416client # [ 0.032265] CPU topo: Num. threads per package: 1417worker2 # [ 0.032713] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])418lb # [ 0.032608] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)419client # [ 0.032265] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs420worker1 # [ 0.032954] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)421worker2 # [ 0.032738] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23422lb # [ 0.032610] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)423client # [ 0.032283] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()424worker1 # [ 0.032956] ACPI: Using ACPI (MADT) for SMP configuration information425lb # [ 0.032610] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)426worker2 # [ 0.032741] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)427worker1 # [ 0.032957] ACPI: HPET id: 0x8086a201 base: 0xfed00000428client # [ 0.032314] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]429worker1 # [ 0.032961] TSC deadline timer available430worker2 # [ 0.032742] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)431lb # [ 0.032613] ACPI: Using ACPI (MADT) for SMP configuration information432worker1 # [ 0.032965] CPU topo: Max. logical packages: 1433client # [ 0.032315] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]434lb # [ 0.032614] ACPI: HPET id: 0x8086a201 base: 0xfed00000435worker1 # [ 0.032966] CPU topo: Max. logical dies: 1436worker2 # [ 0.032744] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)437client # [ 0.032316] [mem 0x60000000-0xafffffff] available for PCI devices438lb # [ 0.032617] TSC deadline timer available439worker1 # [ 0.032966] CPU topo: Max. dies per package: 1440worker2 # [ 0.032745] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)441client # [ 0.032318] Booting paravirtualized kernel on KVM442lb # [ 0.032621] CPU topo: Max. logical packages: 1443worker1 # [ 0.032970] CPU topo: Max. threads per core: 1444lb # [ 0.032622] CPU topo: Max. logical dies: 1445worker2 # [ 0.032746] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)446worker1 # [ 0.032970] CPU topo: Num. cores per package: 1447lb # [ 0.032623] CPU topo: Max. dies per package: 1448client # [ 0.032321] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns449worker1 # [ 0.032971] CPU topo: Num. threads per package: 1450worker2 # [ 0.032748] ACPI: Using ACPI (MADT) for SMP configuration information451lb # [ 0.032626] CPU topo: Max. threads per core: 1452client # [ 0.036806] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1453worker2 # [ 0.032749] ACPI: HPET id: 0x8086a201 base: 0xfed00000454lb # [ 0.032627] CPU topo: Num. cores per package: 1455worker1 # [ 0.032971] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs456worker2 # [ 0.032752] TSC deadline timer available457lb # [ 0.032627] CPU topo: Num. threads per package: 1458client # [ 0.038963] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152459worker1 # [ 0.032989] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()460worker2 # [ 0.032757] CPU topo: Max. logical packages: 1461lb # [ 0.032627] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs462client # [ 0.039007] kvm-guest: PV spinlocks disabled, single CPU463worker2 # [ 0.032758] CPU topo: Max. logical dies: 1464worker1 # [ 0.033020] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]465worker2 # [ 0.032758] CPU topo: Max. dies per package: 1466lb # [ 0.032649] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()467worker1 # [ 0.033021] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]468worker2 # [ 0.032762] CPU topo: Max. threads per core: 1469lb # [ 0.032683] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]470worker2 # [ 0.032762] CPU topo: Num. cores per package: 1471worker1 # [ 0.033023] [mem 0x60000000-0xafffffff] available for PCI devices472worker2 # [ 0.032763] CPU topo: Num. threads per package: 1473worker1 # [ 0.033024] Booting paravirtualized kernel on KVM474lb # [ 0.032684] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]475worker2 # [ 0.032763] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs476lb # [ 0.032686] [mem 0x60000000-0xafffffff] available for PCI devices477client # [ 0.039009] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test/init regInfo=/nix/store/nm58dyaq8zm0vqgkfdrxs5bqws6f19cb-closure-info/registration console=ttyS0,115200n8 console=tty0478worker1 # [ 0.033027] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns479lb # [ 0.032687] Booting paravirtualized kernel on KVM480worker2 # [ 0.032783] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()481worker1 # [ 0.037465] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1482worker2 # [ 0.032815] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]483lb # [ 0.032690] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns484client # [ 0.039100] Unknown kernel command line parameters "regInfo=/nix/store/nm58dyaq8zm0vqgkfdrxs5bqws6f19cb-closure-info/registration", will be passed to user space.485worker1 # [ 0.039632] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152486worker2 # [ 0.032817] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]487client # [ 0.039112] random: crng init done488lb # [ 0.037138] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1489worker1 # [ 0.039676] kvm-guest: PV spinlocks disabled, single CPU490worker2 # [ 0.032818] [mem 0x60000000-0xafffffff] available for PCI devices491client # [ 0.039113] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes492lb # [ 0.039301] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152493worker2 # [ 0.032820] Booting paravirtualized kernel on KVM494lb # [ 0.039347] kvm-guest: PV spinlocks disabled, single CPU495client # [ 0.041512] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)496worker2 # [ 0.032822] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns497client # [ 0.042757] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)498client # [ 0.042787] Fallback order for Node 0: 0499worker2 # [ 0.037301] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1500worker1 # [ 0.039677] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lv03q5h92lsnc65incxxasgvq4y82d3q-nixos-system-worker1-test/init regInfo=/nix/store/k2lpqpf1yl5ndvc9v5w1mjf3dnbawjds-closure-info/registration console=ttyS0,115200n8 console=tty0501client # [ 0.042791] Built 1 zonelists, mobility grouping on. Total pages: 393078502worker2 # [ 0.039452] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152503client # [ 0.042792] Policy zone: DMA32504worker2 # [ 0.039492] kvm-guest: PV spinlocks disabled, single CPU505client # [ 0.045976] mem auto-init: stack:all(zero), heap alloc:on, heap free:off506lb # [ 0.039348] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8mvx6cn1l6frdr55gdj0p01an8nxnxzn-nixos-system-lb-test/init regInfo=/nix/store/0a2f35z8lm3xm6ffdv71jfscn9vyk5l1-closure-info/registration console=ttyS0,115200n8 console=tty0507worker1 # [ 0.039771] Unknown kernel command line parameters "regInfo=/nix/store/k2lpqpf1yl5ndvc9v5w1mjf3dnbawjds-closure-info/registration", will be passed to user space.508client # [ 0.048986] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1509worker1 # [ 0.039783] random: crng init done510client # [ 0.052596] allocated 3145728 bytes of page_ext511worker1 # [ 0.039784] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes512lb # [ 0.039447] Unknown kernel command line parameters "regInfo=/nix/store/0a2f35z8lm3xm6ffdv71jfscn9vyk5l1-closure-info/registration", will be passed to user space.513client # [ 0.062165] ftrace: allocating 48731 entries in 192 pages514lb # [ 0.039460] random: crng init done515client # [ 0.062167] ftrace: allocated 192 pages with 2 groups516worker1 # [ 0.041919] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)517client # [ 0.063325] Dynamic Preempt: lazy518lb # [ 0.039470] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes519worker2 # [ 0.039493] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m8qvh7i9ijf4y451014qsh100ca0dry7-nixos-system-worker2-test/init regInfo=/nix/store/2a07p09nzhh9mwi0id9lmsp9cpx2aiml-closure-info/registration console=ttyS0,115200n8 console=tty0520client # [ 0.063474] rcu: Preemptible hierarchical RCU implementation.521worker1 # [ 0.043237] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)522lb # [ 0.041625] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)523worker1 # [ 0.043269] Fallback order for Node 0: 0524client # [ 0.063475] rcu: RCU event tracing is enabled.525worker1 # [ 0.043272] Built 1 zonelists, mobility grouping on. Total pages: 393078526client # [ 0.063475] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.527worker2 # [ 0.039592] Unknown kernel command line parameters "regInfo=/nix/store/2a07p09nzhh9mwi0id9lmsp9cpx2aiml-closure-info/registration", will be passed to user space.528lb # [ 0.042856] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)529worker1 # [ 0.043273] Policy zone: DMA32530worker2 # [ 0.039612] random: crng init done531lb # [ 0.042889] Fallback order for Node 0: 0532client # [ 0.063477] Trampoline variant of Tasks RCU enabled.533client # [ 0.063477] Rude variant of Tasks RCU enabled.534worker1 # [ 0.045818] mem auto-init: stack:all(zero), heap alloc:on, heap free:off535lb # [ 0.042892] Built 1 zonelists, mobility grouping on. Total pages: 393078536worker2 # [ 0.039613] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes537client # [ 0.063477] Tracing variant of Tasks RCU enabled.538lb # [ 0.042893] Policy zone: DMA32539worker1 # [ 0.048937] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1540worker2 # [ 0.041724] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)541worker1 # [ 0.052244] allocated 3145728 bytes of page_ext542lb # [ 0.045363] mem auto-init: stack:all(zero), heap alloc:on, heap free:off543client # [ 0.063478] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.544worker1 # [ 0.062238] ftrace: allocating 48731 entries in 192 pages545worker2 # [ 0.042796] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)546lb # [ 0.048612] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1547client # [ 0.063479] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1548worker2 # [ 0.042830] Fallback order for Node 0: 0549worker1 # [ 0.062241] ftrace: allocated 192 pages with 2 groups550lb # [ 0.051936] allocated 3145728 bytes of page_ext551worker1 # [ 0.063099] Dynamic Preempt: lazy552client # [ 0.063538] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.553worker2 # [ 0.042834] Built 1 zonelists, mobility grouping on. Total pages: 393078554lb # [ 0.061911] ftrace: allocating 48731 entries in 192 pages555worker1 # [ 0.063263] rcu: Preemptible hierarchical RCU implementation.556worker2 # [ 0.042835] Policy zone: DMA32557lb # [ 0.061913] ftrace: allocated 192 pages with 2 groups558worker1 # [ 0.063264] rcu: RCU event tracing is enabled.559client # [ 0.063540] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.560lb # [ 0.062781] Dynamic Preempt: lazy561worker2 # [ 0.045518] mem auto-init: stack:all(zero), heap alloc:on, heap free:off562worker1 # [ 0.063264] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.563lb # [ 0.062958] rcu: Preemptible hierarchical RCU implementation.564worker2 # [ 0.048833] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1565client # [ 0.063541] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.566worker1 # [ 0.063266] Trampoline variant of Tasks RCU enabled.567lb # [ 0.062959] rcu: RCU event tracing is enabled.568worker2 # [ 0.052121] allocated 3145728 bytes of page_ext569worker1 # [ 0.063266] Rude variant of Tasks RCU enabled.570client # [ 0.067906] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16571lb # [ 0.062960] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.572worker2 # [ 0.062154] ftrace: allocating 48731 entries in 192 pages573worker1 # [ 0.063266] Tracing variant of Tasks RCU enabled.574lb # [ 0.062962] Trampoline variant of Tasks RCU enabled.575worker2 # [ 0.062157] ftrace: allocated 192 pages with 2 groups576client # [ 0.068171] rcu: srcu_init: Setting srcu_struct sizes based on contention.577worker2 # [ 0.063009] Dynamic Preempt: lazy578lb # [ 0.062962] Rude variant of Tasks RCU enabled.579worker1 # [ 0.063267] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.580lb # [ 0.062962] Tracing variant of Tasks RCU enabled.581worker2 # [ 0.063175] rcu: Preemptible hierarchical RCU implementation.582client # [ 0.068177] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns583worker1 # [ 0.063268] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1584worker2 # [ 0.063175] rcu: RCU event tracing is enabled.585lb # [ 0.062963] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.586client # [ 0.068276] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)587worker2 # [ 0.063176] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.588worker1 # [ 0.063284] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.589client # [ 0.071850] Console: colour VGA+ 80x25590lb # [ 0.062964] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1591worker2 # [ 0.063178] Trampoline variant of Tasks RCU enabled.592client # [ 0.071853] printk: legacy console [tty0] enabled593worker1 # [ 0.063286] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.594worker2 # [ 0.063178] Rude variant of Tasks RCU enabled.595lb # [ 0.062986] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.596client # [ 0.101315] printk: legacy console [ttyS0] enabled597worker2 # [ 0.063179] Tracing variant of Tasks RCU enabled.598worker1 # [ 0.063287] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.599client # [ 0.197293] ACPI: Core revision 20250807600lb # [ 0.062988] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.601worker2 # [ 0.063179] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.602worker1 # [ 0.067873] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16603worker2 # [ 0.063180] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1604lb # [ 0.062989] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.605worker1 # [ 0.068141] rcu: srcu_init: Setting srcu_struct sizes based on contention.606client # [ 0.198158] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns607lb # [ 0.067578] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16608worker2 # [ 0.063201] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.609client # [ 0.199650] APIC: Switch to symmetric I/O mode setup610worker1 # [ 0.068147] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns611lb # [ 0.067852] rcu: srcu_init: Setting srcu_struct sizes based on contention.612client # [ 0.200591] x2apic enabled613worker2 # [ 0.063203] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.614worker1 # [ 0.068252] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)615lb # [ 0.067860] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns616worker1 # [ 0.071796] Console: colour VGA+ 80x25617worker2 # [ 0.063204] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.618client # [ 0.201334] APIC: Switched APIC routing to: physical x2apic619worker1 # [ 0.071799] printk: legacy console [tty0] enabled620worker2 # [ 0.067582] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16621lb # [ 0.067969] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)622worker1 # [ 0.101201] printk: legacy console [ttyS0] enabled623lb # [ 0.071512] Console: colour VGA+ 80x25624worker2 # [ 0.067858] rcu: srcu_init: Setting srcu_struct sizes based on contention.625worker1 # [ 0.196803] ACPI: Core revision 20250807626lb # [ 0.071515] printk: legacy console [tty0] enabled627client # [ 0.203143] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1628lb # [ 0.101133] printk: legacy console [ttyS0] enabled629worker2 # [ 0.067864] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns630lb # [ 0.197109] ACPI: Core revision 20250807631worker1 # [ 0.197673] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns632client # [ 0.204090] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns633worker2 # [ 0.067969] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)634worker1 # [ 0.199168] APIC: Switch to symmetric I/O mode setup635worker2 # [ 0.071491] Console: colour VGA+ 80x25636client # [ 0.205703] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)637lb # [ 0.197961] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns638worker1 # [ 0.200103] x2apic enabled639worker2 # [ 0.071494] printk: legacy console [tty0] enabled640worker2 # [ 0.100954] printk: legacy console [ttyS0] enabled641lb # [ 0.199433] APIC: Switch to symmetric I/O mode setup642client # [ 0.206787] x86/cpu: User Mode Instruction Prevention (UMIP) activated643worker1 # [ 0.200842] APIC: Switched APIC routing to: physical x2apic644worker2 # [ 0.196217] ACPI: Core revision 20250807645lb # [ 0.200401] x2apic enabled646client # [ 0.207840] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127647lb # [ 0.201147] APIC: Switched APIC routing to: physical x2apic648worker2 # [ 0.197046] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns649worker1 # [ 0.202660] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1650client # [ 0.208700] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0651worker2 # [ 0.198551] APIC: Switch to symmetric I/O mode setup652worker2 # [ 0.199507] x2apic enabled653lb # [ 0.202935] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1654worker1 # [ 0.203635] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns655client # [ 0.209703] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto656worker2 # [ 0.200239] APIC: Switched APIC routing to: physical x2apic657worker1 # [ 0.205270] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)658lb # [ 0.203889] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns659client # [ 0.210700] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl660worker1 # [ 0.206360] x86/cpu: User Mode Instruction Prevention (UMIP) activated661client # [ 0.211700] Transient Scheduler Attacks: Vulnerable: No microcode662lb # [ 0.205513] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)663worker2 # [ 0.202043] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1664client # [ 0.212700] Spectre V2 : Mitigation: Enhanced / Automatic IBRS665worker1 # [ 0.207395] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127666lb # [ 0.206599] x86/cpu: User Mode Instruction Prevention (UMIP) activated667worker2 # [ 0.203004] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns668worker1 # [ 0.208267] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0669client # [ 0.213700] Speculative Return Stack Overflow: Mitigation: Safe RET670lb # [ 0.207654] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127671worker2 # [ 0.204652] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)672lb # [ 0.208511] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0673worker1 # [ 0.209270] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto674worker2 # [ 0.205734] x86/cpu: User Mode Instruction Prevention (UMIP) activated675lb # [ 0.209514] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto676worker1 # [ 0.210267] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl677worker2 # [ 0.206811] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127678client # [ 0.214700] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization679worker1 # [ 0.211267] Transient Scheduler Attacks: Vulnerable: No microcode680lb # [ 0.210511] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl681worker2 # [ 0.208322] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0682client # [ 0.216714] Spectre V2 : Enabling IBPB for BPF683worker1 # [ 0.212266] Spectre V2 : Mitigation: Enhanced / Automatic IBRS684lb # [ 0.211510] Transient Scheduler Attacks: Vulnerable: No microcode685worker1 # [ 0.213267] Speculative Return Stack Overflow: Mitigation: Safe RET686client # [ 0.217701] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier687worker2 # [ 0.208653] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto688lb # [ 0.212511] Spectre V2 : Mitigation: Enhanced / Automatic IBRS689client # [ 0.218701] active return thunk: srso_alias_return_thunk690lb # [ 0.213510] Speculative Return Stack Overflow: Mitigation: Safe RET691worker2 # [ 0.210649] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl692client # [ 0.219721] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'693worker2 # [ 0.211649] Transient Scheduler Attacks: Vulnerable: No microcode694client # [ 0.220700] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'695worker2 # [ 0.212649] Spectre V2 : Mitigation: Enhanced / Automatic IBRS696worker1 # [ 0.214266] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization697client # [ 0.221700] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'698worker1 # [ 0.216274] Spectre V2 : Enabling IBPB for BPF699worker2 # [ 0.213649] Speculative Return Stack Overflow: Mitigation: Safe RET700client # [ 0.222700] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'701lb # [ 0.214510] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization702worker1 # [ 0.216990] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier703lb # [ 0.216518] Spectre V2 : Enabling IBPB for BPF704client # [ 0.223700] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'705worker1 # [ 0.218267] active return thunk: srso_alias_return_thunk706client # [ 0.224700] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'707lb # [ 0.217511] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier708worker1 # [ 0.219287] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'709lb # [ 0.218511] active return thunk: srso_alias_return_thunk710worker2 # [ 0.214649] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization711client # [ 0.225700] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'712worker1 # [ 0.220267] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'713worker2 # [ 0.216656] Spectre V2 : Enabling IBPB for BPF714client # [ 0.227700] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256715lb # [ 0.219531] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'716worker1 # [ 0.221267] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'717client # [ 0.228610] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64718lb # [ 0.220510] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'719worker2 # [ 0.217650] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier720worker1 # [ 0.222266] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'721client # [ 0.229401] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512722worker2 # [ 0.218650] active return thunk: srso_alias_return_thunk723lb # [ 0.221510] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'724worker1 # [ 0.223266] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'725client # [ 0.230412] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024726lb # [ 0.222510] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'727worker2 # [ 0.219670] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'728worker1 # [ 0.224266] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'729client # [ 0.231411] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8730lb # [ 0.223510] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'731worker2 # [ 0.220649] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'732worker1 # [ 0.225266] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'733lb # [ 0.224510] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'734worker2 # [ 0.222385] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'735client # [ 0.231700] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.736worker1 # [ 0.226267] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256737worker2 # [ 0.223365] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'738lb # [ 0.225510] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'739worker1 # [ 0.227266] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64740lb # [ 0.226511] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256741worker2 # [ 0.223649] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'742worker1 # [ 0.228266] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512743lb # [ 0.227510] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64744worker2 # [ 0.224649] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'745worker1 # [ 0.229267] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024746lb # [ 0.228510] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512747worker1 # [ 0.230266] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8748worker2 # [ 0.226464] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'749lb # [ 0.229510] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024750worker2 # [ 0.227649] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256751worker1 # [ 0.231266] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.752lb # [ 0.230510] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8753worker2 # [ 0.228649] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64754worker2 # [ 0.229649] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512755lb # [ 0.231510] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.756worker2 # [ 0.230649] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757worker2 # [ 0.231649] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758worker2 # [ 0.232649] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759client # [ 0.266505] Freeing SMP alternatives memory: 44K760client # [ 0.267291] pid_max: default: 32768 minimum: 301761client # [ 0.267797] LSM: initializing lsm=capability,landlock,yama,bpf,ima762client # [ 0.268782] landlock: Up and running.763client # [ 0.269700] Yama: becoming mindful.764client # [ 0.270912] LSM support for eBPF active765client # [ 0.271784] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)766worker1 # [ 0.265929] Freeing SMP alternatives memory: 44K767worker1 # [ 0.266270] pid_max: default: 32768 minimum: 301768client # [ 0.272740] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)769lb # [ 0.266430] Freeing SMP alternatives memory: 44K770worker1 # [ 0.267352] LSM: initializing lsm=capability,landlock,yama,bpf,ima771lb # [ 0.267057] pid_max: default: 32768 minimum: 301772worker1 # [ 0.268371] landlock: Up and running.773worker1 # [ 0.269267] Yama: becoming mindful.774lb # [ 0.267605] LSM: initializing lsm=capability,landlock,yama,bpf,ima775client # [ 0.275319] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)776worker1 # [ 0.270061] LSM support for eBPF active777lb # [ 0.268608] landlock: Up and running.778lb # [ 0.269511] Yama: becoming mindful.779worker2 # [ 0.267039] Freeing SMP alternatives memory: 44K780client # [ 0.276283] Performance Events: Fam17h+ core perfctr, AMD PMU driver.781worker1 # [ 0.270857] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)782lb # [ 0.270682] LSM support for eBPF active783worker2 # [ 0.267652] pid_max: default: 32768 minimum: 301784client # [ 0.276705] ... version: 2785client # [ 0.277406] ... bit width: 48786worker1 # [ 0.271307] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)787worker2 # [ 0.268452] LSM: initializing lsm=capability,landlock,yama,bpf,ima788lb # [ 0.271409] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)789client # [ 0.277703] ... generic counters: 6790worker2 # [ 0.269703] landlock: Up and running.791worker2 # [ 0.270351] Yama: becoming mindful.792client # [ 0.278412] ... generic bitmap: 000000000000003f793lb # [ 0.272548] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)794client # [ 0.278713] ... fixed-purpose counters: 0795worker2 # [ 0.270863] LSM support for eBPF active796worker1 # [ 0.274409] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)797client # [ 0.279404] ... fixed-purpose bitmap: 0000000000000000798worker2 # [ 0.271775] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)799client # [ 0.279702] ... value mask: 0000ffffffffffff800lb # [ 0.274866] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)801worker1 # [ 0.275829] Performance Events: Fam17h+ core perfctr, AMD PMU driver.802worker1 # [ 0.276275] ... version: 2803client # [ 0.280554] ... max period: 00007fffffffffff804worker2 # [ 0.272692] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)805worker1 # [ 0.276987] ... bit width: 48806lb # [ 0.276092] Performance Events: Fam17h+ core perfctr, AMD PMU driver.807client # [ 0.281350] ... global_ctrl mask: 000000000000003f808worker1 # [ 0.277269] ... generic counters: 6809lb # [ 0.276519] ... version: 2810client # [ 0.281810] signal: max sigframe size: 3376811lb # [ 0.277252] ... bit width: 48812worker1 # [ 0.277975] ... generic bitmap: 000000000000003f813client # [ 0.282587] rcu: Hierarchical SRCU implementation.814worker2 # [ 0.275639] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)815worker1 # [ 0.278269] ... fixed-purpose counters: 0816lb # [ 0.277513] ... generic counters: 6817client # [ 0.283257] rcu: Max phase no-delay instances is 400.818worker1 # [ 0.278937] ... fixed-purpose bitmap: 0000000000000000819lb # [ 0.278205] ... generic bitmap: 000000000000003f820worker2 # [ 0.276192] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821lb # [ 0.278513] ... fixed-purpose counters: 0822worker1 # [ 0.279268] ... value mask: 0000ffffffffffff823worker2 # [ 0.276653] ... version: 2824lb # [ 0.279235] ... fixed-purpose bitmap: 0000000000000000825worker1 # [ 0.280165] ... max period: 00007fffffffffff826worker2 # [ 0.277350] ... bit width: 48827lb # [ 0.279512] ... value mask: 0000ffffffffffff828worker1 # [ 0.280961] ... global_ctrl mask: 000000000000003f829worker2 # [ 0.277652] ... generic counters: 6830lb # [ 0.280373] ... max period: 00007fffffffffff831worker1 # [ 0.281377] signal: max sigframe size: 3376832worker2 # [ 0.278338] ... generic bitmap: 000000000000003f833worker2 # [ 0.278663] ... fixed-purpose counters: 0834lb # [ 0.281196] ... global_ctrl mask: 000000000000003f835worker1 # [ 0.282196] rcu: Hierarchical SRCU implementation.836client # [ 0.288220] smp: Bringing up secondary CPUs ...837worker2 # [ 0.279348] ... fixed-purpose bitmap: 0000000000000000838lb # [ 0.281609] signal: max sigframe size: 3376839worker1 # [ 0.282858] rcu: Max phase no-delay instances is 400.840client # [ 0.288717] smp: Brought up 1 node, 1 CPU841worker2 # [ 0.279651] ... value mask: 0000ffffffffffff842lb # [ 0.282368] rcu: Hierarchical SRCU implementation.843client # [ 0.289373] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)844lb # [ 0.283101] rcu: Max phase no-delay instances is 400.845worker2 # [ 0.280508] ... max period: 00007fffffffffff846worker2 # [ 0.281312] ... global_ctrl mask: 000000000000003f847worker2 # [ 0.281772] signal: max sigframe size: 3376848client # [ 0.289939] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)849worker2 # [ 0.282558] rcu: Hierarchical SRCU implementation.850client # [ 0.290941] devtmpfs: initialized851worker2 # [ 0.283227] rcu: Max phase no-delay instances is 400.852worker1 # [ 0.287751] smp: Bringing up secondary CPUs ...853client # [ 0.291776] x86/mm: Memory block size: 128MB854worker1 # [ 0.288284] smp: Brought up 1 node, 1 CPU855lb # [ 0.288070] smp: Bringing up secondary CPUs ...856worker1 # [ 0.288972] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)857lb # [ 0.288527] smp: Brought up 1 node, 1 CPU858client # [ 0.293505] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)859lb # [ 0.289202] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)860client # [ 0.294637] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).861worker1 # [ 0.289486] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)862client # [ 0.295800] pinctrl core: initialized pinctrl subsystem863worker2 # [ 0.288131] smp: Bringing up secondary CPUs ...864worker1 # [ 0.290529] devtmpfs: initialized865worker2 # [ 0.288665] smp: Brought up 1 node, 1 CPU866lb # [ 0.289717] Memory: 1454568K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)867worker1 # [ 0.291313] x86/mm: Memory block size: 128MB868client # [ 0.296934] PM: RTC time: 10:48:24, date: 2026-09-22869lb # [ 0.290786] devtmpfs: initialized870worker2 # [ 0.289316] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)871lb # [ 0.291645] x86/mm: Memory block size: 128MB872worker1 # [ 0.293048] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)873worker2 # [ 0.289878] Memory: 1454548K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)874worker1 # [ 0.294205] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).875client # [ 0.300529] NET: Registered PF_NETLINK/PF_ROUTE protocol family876lb # [ 0.293397] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)877worker2 # [ 0.290964] devtmpfs: initialized878worker1 # [ 0.295359] pinctrl core: initialized pinctrl subsystem879worker2 # [ 0.291706] x86/mm: Memory block size: 128MB880lb # [ 0.294387] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).881client # [ 0.301906] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations882worker1 # [ 0.296478] PM: RTC time: 10:48:24, date: 2026-09-22883lb # [ 0.295612] pinctrl core: initialized pinctrl subsystem884client # [ 0.302724] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations885worker2 # [ 0.293465] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)886lb # [ 0.296794] PM: RTC time: 10:48:24, date: 2026-09-22887client # [ 0.303984] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations888worker2 # [ 0.294572] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).889client # [ 0.304719] audit: initializing netlink subsys (disabled)890worker2 # [ 0.295728] pinctrl core: initialized pinctrl subsystem891worker1 # [ 0.300007] NET: Registered PF_NETLINK/PF_ROUTE protocol family892client # [ 0.305820] thermal_sys: Registered thermal governor 'fair_share'893worker2 # [ 0.296849] PM: RTC time: 10:48:24, date: 2026-09-22894lb # [ 0.300166] NET: Registered PF_NETLINK/PF_ROUTE protocol family895worker1 # [ 0.300780] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations896client # [ 0.305822] thermal_sys: Registered thermal governor 'bang_bang'897client # [ 0.306703] thermal_sys: Registered thermal governor 'step_wise'898worker1 # [ 0.301293] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations899lb # [ 0.301028] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations900client # [ 0.307636] thermal_sys: Registered thermal governor 'user_space'901lb # [ 0.301536] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations902worker1 # [ 0.302544] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations903worker2 # [ 0.300355] NET: Registered PF_NETLINK/PF_ROUTE protocol family904client # [ 0.308435] audit: type=2000 audit(1790074105.374:1): state=initialized audit_enabled=0 res=1905worker1 # [ 0.303279] audit: initializing netlink subsys (disabled)906lb # [ 0.302783] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations907client # [ 0.309705] thermal_sys: Registered thermal governor 'power_allocator'908worker2 # [ 0.301154] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations909worker1 # [ 0.304362] thermal_sys: Registered thermal governor 'fair_share'910client # [ 0.309729] cpuidle: using governor menu911lb # [ 0.303523] audit: initializing netlink subsys (disabled)912worker1 # [ 0.304364] thermal_sys: Registered thermal governor 'bang_bang'913worker2 # [ 0.301674] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations914lb # [ 0.304653] thermal_sys: Registered thermal governor 'fair_share'915worker1 # [ 0.305270] thermal_sys: Registered thermal governor 'step_wise'916client # [ 0.311892] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5917lb # [ 0.304656] thermal_sys: Registered thermal governor 'bang_bang'918worker2 # [ 0.302930] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations919worker1 # [ 0.306192] thermal_sys: Registered thermal governor 'user_space'920client # [ 0.312932] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]921lb # [ 0.305514] thermal_sys: Registered thermal governor 'step_wise'922worker2 # [ 0.303661] audit: initializing netlink subsys (disabled)923worker1 # [ 0.306957] audit: type=2000 audit(1790074105.380:1): state=initialized audit_enabled=0 res=1924lb # [ 0.306459] thermal_sys: Registered thermal governor 'user_space'925client # [ 0.313705] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry926worker2 # [ 0.304707] thermal_sys: Registered thermal governor 'fair_share'927worker1 # [ 0.308271] thermal_sys: Registered thermal governor 'power_allocator'928client # [ 0.314714] PCI: Using configuration type 1 for base access929worker2 # [ 0.304709] thermal_sys: Registered thermal governor 'bang_bang'930lb # [ 0.307221] audit: type=2000 audit(1790074105.380:1): state=initialized audit_enabled=0 res=1931worker1 # [ 0.308285] cpuidle: using governor menu932worker2 # [ 0.305615] thermal_sys: Registered thermal governor 'step_wise'933lb # [ 0.308515] thermal_sys: Registered thermal governor 'power_allocator'934client # [ 0.315784] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.935worker2 # [ 0.306329] thermal_sys: Registered thermal governor 'user_space'936worker1 # [ 0.310478] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5937lb # [ 0.308530] cpuidle: using governor menu938worker2 # [ 0.306663] audit: type=2000 audit(1790074105.385:1): state=initialized audit_enabled=0 res=1939lb # [ 0.310724] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5940worker1 # [ 0.311558] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]941worker2 # [ 0.308654] thermal_sys: Registered thermal governor 'power_allocator'942worker1 # [ 0.312272] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry943worker2 # [ 0.308667] cpuidle: using governor menu944lb # [ 0.311754] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]945worker1 # [ 0.313281] PCI: Using configuration type 1 for base access946lb # [ 0.312516] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry947worker2 # [ 0.310848] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5948lb # [ 0.313525] PCI: Using configuration type 1 for base access949worker1 # [ 0.314414] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.950worker2 # [ 0.311886] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]951client # [ 0.323005] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages952lb # [ 0.314639] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.953worker2 # [ 0.312654] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry954client # [ 0.323704] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page955worker2 # [ 0.313663] PCI: Using configuration type 1 for base access956worker2 # [ 0.314739] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.957worker1 # [ 0.319516] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages958worker1 # [ 0.320271] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page959lb # [ 0.319793] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages960client # [ 0.328704] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages961lb # [ 0.320514] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page962client # [ 0.329703] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page963worker1 # [ 0.325271] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages964worker2 # [ 0.321932] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages965worker1 # [ 0.326269] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page966worker2 # [ 0.322653] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page967lb # [ 0.325514] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages968lb # [ 0.326513] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page969client # [ 0.338360] ACPI: Added _OSI(Module Device)970worker2 # [ 0.327653] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages971client # [ 0.338704] ACPI: Added _OSI(Processor Device)972client # [ 0.339430] ACPI: Added _OSI(Processor Aggregator Device)973worker2 # [ 0.328653] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page974worker1 # [ 0.336615] ACPI: Added _OSI(Module Device)975worker1 # [ 0.337271] ACPI: Added _OSI(Processor Device)976lb # [ 0.336901] ACPI: Added _OSI(Module Device)977worker1 # [ 0.338045] ACPI: Added _OSI(Processor Aggregator Device)978lb # [ 0.337514] ACPI: Added _OSI(Processor Device)979lb # [ 0.338284] ACPI: Added _OSI(Processor Aggregator Device)980client # [ 0.346280] ACPI: 1 ACPI AML tables successfully acquired and loaded981worker2 # [ 0.337246] ACPI: Added _OSI(Module Device)982worker2 # [ 0.337653] ACPI: Added _OSI(Processor Device)983worker2 # [ 0.338411] ACPI: Added _OSI(Processor Aggregator Device)984client # [ 0.349613] ACPI: Interpreter enabled985client # [ 0.350194] ACPI: PM: (supports S0 S3 S4 S5)986client # [ 0.350703] ACPI: Using IOAPIC for interrupt routing987client # [ 0.351601] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug988worker1 # [ 0.346834] ACPI: 1 ACPI AML tables successfully acquired and loaded989lb # [ 0.347055] ACPI: 1 ACPI AML tables successfully acquired and loaded990worker2 # [ 0.345202] ACPI: 1 ACPI AML tables successfully acquired and loaded991worker1 # [ 0.350631] ACPI: Interpreter enabled992client # [ 0.356703] PCI: Using E820 reservations for host bridge windows993worker1 # [ 0.351284] ACPI: PM: (supports S0 S3 S4 S5)994client # [ 0.357805] ACPI: Enabled 2 GPEs in block 00 to 3F995lb # [ 0.350926] ACPI: Interpreter enabled996worker1 # [ 0.351977] ACPI: Using IOAPIC for interrupt routing997worker2 # [ 0.348576] ACPI: Interpreter enabled998lb # [ 0.351528] ACPI: PM: (supports S0 S3 S4 S5)999worker2 # [ 0.349121] ACPI: PM: (supports S0 S3 S4 S5)1000lb # [ 0.352254] ACPI: Using IOAPIC for interrupt routing1001worker1 # [ 0.352343] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1002worker2 # [ 0.349653] ACPI: Using IOAPIC for interrupt routing1003worker1 # [ 0.353270] PCI: Using E820 reservations for host bridge windows1004lb # [ 0.352568] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1005worker2 # [ 0.350490] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1006lb # [ 0.353514] PCI: Using E820 reservations for host bridge windows1007worker1 # [ 0.356362] ACPI: Enabled 2 GPEs in block 00 to 3F1008lb # [ 0.356649] ACPI: Enabled 2 GPEs in block 00 to 3F1009client # [ 0.366455] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1010worker2 # [ 0.355652] PCI: Using E820 reservations for host bridge windows1011worker2 # [ 0.356709] ACPI: Enabled 2 GPEs in block 00 to 3F1012client # [ 0.367307] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1013client # [ 0.367781] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1014client # [ 0.368826] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1015client # [ 0.370057] PCI host bridge to bus 0000:001016worker1 # [ 0.364972] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1017client # [ 0.370707] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1018worker1 # [ 0.365275] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1019lb # [ 0.365339] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1020client # [ 0.371703] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1021client # [ 0.372702] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1022lb # [ 0.366243] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1023worker1 # [ 0.366881] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1024worker2 # [ 0.365329] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1025client # [ 0.373702] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1026worker1 # [ 0.367392] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1027lb # [ 0.367912] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1028worker1 # [ 0.368635] PCI host bridge to bus 0000:001029client # [ 0.374702] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1030worker2 # [ 0.366168] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1031lb # [ 0.368645] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1032worker1 # [ 0.369274] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1033client # [ 0.375702] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1034lb # [ 0.369848] PCI host bridge to bus 0000:001035worker2 # [ 0.366737] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1036client # [ 0.376703] pci_bus 0000:00: root bus resource [bus 00-ff]1037worker1 # [ 0.370270] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1038lb # [ 0.370518] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1039worker2 # [ 0.367773] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1040worker1 # [ 0.371269] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1041client # [ 0.377621] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1042lb # [ 0.371513] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1043worker2 # [ 0.368982] PCI host bridge to bus 0000:001044worker1 # [ 0.372269] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1045worker2 # [ 0.369645] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1046lb # [ 0.372513] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1047client # [ 0.379116] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1048worker2 # [ 0.370643] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1049worker1 # [ 0.373270] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1050lb # [ 0.373514] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1051worker2 # [ 0.371442] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1052worker1 # [ 0.374269] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1053lb # [ 0.374513] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1054worker1 # [ 0.375270] pci_bus 0000:00: root bus resource [bus 00-ff]1055worker2 # [ 0.372551] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1056lb # [ 0.375513] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1057worker2 # [ 0.373554] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1058lb # [ 0.376513] pci_bus 0000:00: root bus resource [bus 00-ff]1059worker1 # [ 0.376235] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1060worker2 # [ 0.374529] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1061lb # [ 0.377490] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1062worker1 # [ 0.377722] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1063worker2 # [ 0.375616] pci_bus 0000:00: root bus resource [bus 00-ff]1064lb # [ 0.379032] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1065worker2 # [ 0.376403] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1066client # [ 0.381817] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1067client # [ 0.382715] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1068worker2 # [ 0.378090] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1069client # [ 0.383707] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1070client # [ 0.384874] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1071client # [ 0.386490] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1072worker1 # [ 0.380358] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1073lb # [ 0.381593] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1074worker1 # [ 0.381287] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1075lb # [ 0.382526] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1076client # [ 0.388711] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1077worker2 # [ 0.380710] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1078worker1 # [ 0.382299] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1079lb # [ 0.383519] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1080client # [ 0.389543] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1081worker2 # [ 0.381664] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1082client # [ 0.390437] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1083worker1 # [ 0.383685] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1084worker2 # [ 0.382619] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1085lb # [ 0.384729] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1086client # [ 0.390708] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1087worker2 # [ 0.383515] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1088lb # [ 0.386225] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1089worker1 # [ 0.385227] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1090client # [ 0.392424] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1091worker2 # [ 0.385441] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1092lb # [ 0.388540] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1093client # [ 0.394711] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1094lb # [ 0.389400] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1095worker1 # [ 0.388302] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1096client # [ 0.395627] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1097worker2 # [ 0.387659] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1098lb # [ 0.390266] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1099client # [ 0.396425] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1100worker1 # [ 0.389281] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1101worker2 # [ 0.388474] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1102lb # [ 0.390519] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1103worker2 # [ 0.389337] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1104worker1 # [ 0.390302] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1105client # [ 0.397264] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1106worker2 # [ 0.389657] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1107worker1 # [ 0.391282] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1108lb # [ 0.392048] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1109worker2 # [ 0.391290] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1110worker1 # [ 0.393009] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1111lb # [ 0.393541] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1112client # [ 0.399722] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1113lb # [ 0.394415] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1114client # [ 0.400710] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1115worker2 # [ 0.392659] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1116lb # [ 0.395267] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1117worker2 # [ 0.393477] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1118client # [ 0.401663] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1119worker1 # [ 0.396281] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1120worker2 # [ 0.394374] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1121lb # [ 0.396089] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1122worker1 # [ 0.397280] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1123client # [ 0.403084] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1124worker2 # [ 0.395198] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1125worker1 # [ 0.398302] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1126client # [ 0.404720] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1127client # [ 0.405595] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1128lb # [ 0.398542] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1129worker1 # [ 0.399978] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1130client # [ 0.406476] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1131worker2 # [ 0.397660] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1132lb # [ 0.399520] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1133lb # [ 0.400468] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1134worker2 # [ 0.398659] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1135client # [ 0.408079] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1136worker2 # [ 0.399606] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1137worker1 # [ 0.403297] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1138lb # [ 0.401934] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1139worker1 # [ 0.404281] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1140worker2 # [ 0.401029] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1141client # [ 0.409710] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1142worker1 # [ 0.405300] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1143client # [ 0.410540] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1144lb # [ 0.403539] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1145worker2 # [ 0.402659] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1146lb # [ 0.404392] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1147client # [ 0.411433] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1148worker2 # [ 0.403474] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1149lb # [ 0.405254] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1150worker1 # [ 0.406982] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1151worker2 # [ 0.404383] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1152client # [ 0.412299] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1153lb # [ 0.406056] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1154worker2 # [ 0.405186] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1155client # [ 0.414714] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1156worker1 # [ 0.410295] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1157lb # [ 0.407548] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1158worker1 # [ 0.411281] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1159client # [ 0.415565] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1160lb # [ 0.408441] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1161worker2 # [ 0.407570] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1162worker1 # [ 0.412298] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1163worker2 # [ 0.408252] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1164client # [ 0.416725] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1165lb # [ 0.409231] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1166worker2 # [ 0.408673] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1167client # [ 0.418299] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1168lb # [ 0.410089] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1169worker1 # [ 0.414010] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1170worker2 # [ 0.410258] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1171client # [ 0.419712] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1172lb # [ 0.411543] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1173client # [ 0.420597] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1174worker2 # [ 0.411660] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1175client # [ 0.421470] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1176lb # [ 0.412397] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1177worker2 # [ 0.412491] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1178client # [ 0.421709] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1179worker1 # [ 0.417298] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1180lb # [ 0.413535] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1181worker2 # [ 0.413363] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1182worker1 # [ 0.418280] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1183worker1 # [ 0.419311] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1184lb # [ 0.415089] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1185client # [ 0.423187] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1186worker2 # [ 0.414725] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1187worker1 # [ 0.420888] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1188lb # [ 0.416542] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1189client # [ 0.424476] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1190worker2 # [ 0.416659] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1191lb # [ 0.417386] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1192client # [ 0.425413] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1193worker2 # [ 0.417475] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1194lb # [ 0.418251] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1195worker1 # [ 0.422302] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1196worker2 # [ 0.418384] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1197lb # [ 0.418519] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1198client # [ 0.426261] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1199worker1 # [ 0.423161] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1200worker2 # [ 0.418658] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1201client # [ 0.427711] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1202worker1 # [ 0.424275] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1203worker2 # [ 0.420162] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1204lb # [ 0.419994] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1205client # [ 0.428532] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1206client # [ 0.429461] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1207worker1 # [ 0.425856] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1208lb # [ 0.421384] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1209worker2 # [ 0.421623] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1210lb # [ 0.422225] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1211worker2 # [ 0.422674] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1212client # [ 0.430766] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1213worker1 # [ 0.427304] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1214lb # [ 0.423109] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1215worker1 # [ 0.428163] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1216worker2 # [ 0.424181] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1217client # [ 0.432703] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1218worker1 # [ 0.429053] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1219client # [ 0.433597] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1220lb # [ 0.424546] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1221worker2 # [ 0.425670] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1222worker1 # [ 0.430154] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1223lb # [ 0.425367] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1224client # [ 0.434434] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1225worker2 # [ 0.426476] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1226lb # [ 0.426223] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1227worker2 # [ 0.427352] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1228worker1 # [ 0.431584] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1229client # [ 0.435266] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1230worker2 # [ 0.428205] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1231client # [ 0.436259] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1232lb # [ 0.427086] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1233worker1 # [ 0.433306] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1234worker1 # [ 0.434292] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1235lb # [ 0.429542] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1236client # [ 0.436901] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1237worker2 # [ 0.429903] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1238lb # [ 0.430427] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1239client # [ 0.438240] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1240worker1 # [ 0.435830] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1241worker2 # [ 0.430659] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1242lb # [ 0.431329] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1243worker2 # [ 0.431574] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1244lb # [ 0.432949] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1245worker1 # [ 0.437277] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1246worker2 # [ 0.433087] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1247worker1 # [ 0.438151] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1248lb # [ 0.434091] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1249worker2 # [ 0.434250] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1250worker1 # [ 0.439039] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1251worker2 # [ 0.434856] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1252lb # [ 0.434748] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1253worker2 # [ 0.436182] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1254worker1 # [ 0.440840] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1255lb # [ 0.436090] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1256worker1 # [ 0.442300] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1257worker1 # [ 0.443178] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1258worker1 # [ 0.444061] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1259client # [ 0.438893] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1260client # [ 0.441240] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1261worker1 # [ 0.445754] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1262worker1 # [ 0.446933] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1263client # [ 0.441965] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1264client # [ 0.443217] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1265worker1 # [ 0.448260] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1266client # [ 0.443974] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1267worker1 # [ 0.449943] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1268worker2 # [ 0.436840] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1269lb # [ 0.436709] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1270client # [ 0.445316] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1271lb # [ 0.439067] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1272client # [ 0.445973] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1273lb # [ 0.439755] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1274client # [ 0.447481] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1275worker2 # [ 0.440172] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1276lb # [ 0.441009] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1277client # [ 0.447710] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1278worker2 # [ 0.440926] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1279lb # [ 0.441768] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1280client # [ 0.448957] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1281client # [ 0.450298] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1282worker2 # [ 0.442616] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1283lb # [ 0.443108] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1284worker2 # [ 0.443569] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1285lb # [ 0.443797] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1286worker2 # [ 0.444936] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1287lb # [ 0.445337] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1288client # [ 0.454046] ACPI: PCI: Interrupt link LNKA configured for IRQ 101289lb # [ 0.446199] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1290worker1 # [ 0.450477] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1291worker2 # [ 0.445907] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1292client # [ 0.454815] ACPI: PCI: Interrupt link LNKB configured for IRQ 101293worker1 # [ 0.452932] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1294lb # [ 0.446828] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1295client # [ 0.455804] ACPI: PCI: Interrupt link LNKC configured for IRQ 111296client # [ 0.456799] ACPI: PCI: Interrupt link LNKD configured for IRQ 111297worker2 # [ 0.448190] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1298lb # [ 0.448088] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1299worker1 # [ 0.453555] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1300worker2 # [ 0.448659] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1301client # [ 0.457798] ACPI: PCI: Interrupt link LNKE configured for IRQ 101302client # [ 0.458802] ACPI: PCI: Interrupt link LNKF configured for IRQ 101303worker1 # [ 0.454888] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1304worker2 # [ 0.449877] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1305client # [ 0.459801] ACPI: PCI: Interrupt link LNKG configured for IRQ 111306worker2 # [ 0.451373] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1307worker1 # [ 0.455496] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1308client # [ 0.460798] ACPI: PCI: Interrupt link LNKH configured for IRQ 111309lb # [ 0.451758] ACPI: PCI: Interrupt link LNKA configured for IRQ 101310client # [ 0.461901] ACPI: PCI: Interrupt link GSIA configured for IRQ 161311lb # [ 0.452621] ACPI: PCI: Interrupt link LNKB configured for IRQ 101312worker1 # [ 0.456563] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1313client # [ 0.462717] ACPI: PCI: Interrupt link GSIB configured for IRQ 171314lb # [ 0.453614] ACPI: PCI: Interrupt link LNKC configured for IRQ 111315worker1 # [ 0.457524] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1316client # [ 0.463691] ACPI: PCI: Interrupt link GSIC configured for IRQ 181317lb # [ 0.454622] ACPI: PCI: Interrupt link LNKD configured for IRQ 111318worker2 # [ 0.454510] ACPI: PCI: Interrupt link LNKA configured for IRQ 101319client # [ 0.464425] ACPI: PCI: Interrupt link GSID configured for IRQ 191320lb # [ 0.455613] ACPI: PCI: Interrupt link LNKE configured for IRQ 101321worker2 # [ 0.455443] ACPI: PCI: Interrupt link LNKB configured for IRQ 101322worker1 # [ 0.459291] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1323client # [ 0.464718] ACPI: PCI: Interrupt link GSIE configured for IRQ 201324lb # [ 0.456612] ACPI: PCI: Interrupt link LNKF configured for IRQ 101325worker1 # [ 0.460163] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1326worker2 # [ 0.456723] ACPI: PCI: Interrupt link LNKC configured for IRQ 111327client # [ 0.465661] ACPI: PCI: Interrupt link GSIF configured for IRQ 211328lb # [ 0.457613] ACPI: PCI: Interrupt link LNKG configured for IRQ 111329worker2 # [ 0.457734] ACPI: PCI: Interrupt link LNKD configured for IRQ 111330client # [ 0.466416] ACPI: PCI: Interrupt link GSIG configured for IRQ 221331worker1 # [ 0.461315] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1332lb # [ 0.458609] ACPI: PCI: Interrupt link LNKH configured for IRQ 111333client # [ 0.466714] ACPI: PCI: Interrupt link GSIH configured for IRQ 231334worker2 # [ 0.458724] ACPI: PCI: Interrupt link LNKE configured for IRQ 101335lb # [ 0.459546] ACPI: PCI: Interrupt link GSIA configured for IRQ 161336worker2 # [ 0.459719] ACPI: PCI: Interrupt link LNKF configured for IRQ 101337worker1 # [ 0.463322] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1338lb # [ 0.460526] ACPI: PCI: Interrupt link GSIB configured for IRQ 171339client # [ 0.468715] iommu: Default domain type: Translated1340worker2 # [ 0.460751] ACPI: PCI: Interrupt link LNKG configured for IRQ 111341lb # [ 0.461483] ACPI: PCI: Interrupt link GSIC configured for IRQ 181342client # [ 0.469501] iommu: DMA domain TLB invalidation policy: lazy mode1343worker2 # [ 0.461749] ACPI: PCI: Interrupt link LNKH configured for IRQ 111344lb # [ 0.462225] ACPI: PCI: Interrupt link GSID configured for IRQ 191345client # [ 0.470687] ACPI: bus type USB registered1346worker2 # [ 0.462685] ACPI: PCI: Interrupt link GSIA configured for IRQ 161347lb # [ 0.462525] ACPI: PCI: Interrupt link GSIE configured for IRQ 201348client # [ 0.471248] usbcore: registered new interface driver usbfs1349worker2 # [ 0.463628] ACPI: PCI: Interrupt link GSIB configured for IRQ 171350lb # [ 0.463473] ACPI: PCI: Interrupt link GSIF configured for IRQ 211351client # [ 0.471716] usbcore: registered new interface driver hub1352worker1 # [ 0.467178] ACPI: PCI: Interrupt link LNKA configured for IRQ 101353worker2 # [ 0.464378] ACPI: PCI: Interrupt link GSIC configured for IRQ 181354client # [ 0.472556] usbcore: registered new device driver usb1355lb # [ 0.464234] ACPI: PCI: Interrupt link GSIG configured for IRQ 221356worker1 # [ 0.468153] ACPI: PCI: Interrupt link LNKB configured for IRQ 101357worker2 # [ 0.464666] ACPI: PCI: Interrupt link GSID configured for IRQ 191358lb # [ 0.464525] ACPI: PCI: Interrupt link GSIH configured for IRQ 231359worker1 # [ 0.469086] ACPI: PCI: Interrupt link LNKC configured for IRQ 111360client # [ 0.474137] NetLabel: Initializing1361worker2 # [ 0.465601] ACPI: PCI: Interrupt link GSIE configured for IRQ 201362client # [ 0.474703] NetLabel: domain hash size = 1281363worker1 # [ 0.470110] ACPI: PCI: Interrupt link LNKD configured for IRQ 111364worker2 # [ 0.466386] ACPI: PCI: Interrupt link GSIF configured for IRQ 211365client # [ 0.475443] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1366lb # [ 0.466615] iommu: Default domain type: Translated1367worker1 # [ 0.471125] ACPI: PCI: Interrupt link LNKE configured for IRQ 101368worker2 # [ 0.466667] ACPI: PCI: Interrupt link GSIG configured for IRQ 221369client # [ 0.475744] NetLabel: unlabeled traffic allowed by default1370lb # [ 0.467427] iommu: DMA domain TLB invalidation policy: lazy mode1371worker2 # [ 0.467575] ACPI: PCI: Interrupt link GSIH configured for IRQ 231372client # [ 0.476595] PCI: Using ACPI for IRQ routing1373worker1 # [ 0.472106] ACPI: PCI: Interrupt link LNKF configured for IRQ 101374lb # [ 0.468480] ACPI: bus type USB registered1375lb # [ 0.469063] usbcore: registered new interface driver usbfs1376worker1 # [ 0.473355] ACPI: PCI: Interrupt link LNKG configured for IRQ 111377worker2 # [ 0.469374] iommu: Default domain type: Translated1378lb # [ 0.469526] usbcore: registered new interface driver hub1379worker1 # [ 0.474365] ACPI: PCI: Interrupt link LNKH configured for IRQ 111380worker2 # [ 0.469659] iommu: DMA domain TLB invalidation policy: lazy mode1381lb # [ 0.470408] usbcore: registered new device driver usb1382worker1 # [ 0.475305] ACPI: PCI: Interrupt link GSIA configured for IRQ 161383worker2 # [ 0.470853] ACPI: bus type USB registered1384worker1 # [ 0.476281] ACPI: PCI: Interrupt link GSIB configured for IRQ 171385worker2 # [ 0.471590] usbcore: registered new interface driver usbfs1386lb # [ 0.472061] NetLabel: Initializing1387lb # [ 0.472514] NetLabel: domain hash size = 1281388worker1 # [ 0.477272] ACPI: PCI: Interrupt link GSIC configured for IRQ 181389worker2 # [ 0.472355] usbcore: registered new interface driver hub1390worker2 # [ 0.472674] usbcore: registered new device driver usb1391lb # [ 0.473253] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1392worker1 # [ 0.478230] ACPI: PCI: Interrupt link GSID configured for IRQ 191393lb # [ 0.473552] NetLabel: unlabeled traffic allowed by default1394worker1 # [ 0.479016] ACPI: PCI: Interrupt link GSIE configured for IRQ 201395worker2 # [ 0.474353] NetLabel: Initializing1396lb # [ 0.474477] PCI: Using ACPI for IRQ routing1397worker1 # [ 0.479282] ACPI: PCI: Interrupt link GSIF configured for IRQ 211398worker2 # [ 0.474657] NetLabel: domain hash size = 1281399worker2 # [ 0.475373] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1400worker1 # [ 0.480250] ACPI: PCI: Interrupt link GSIG configured for IRQ 221401worker2 # [ 0.475710] NetLabel: unlabeled traffic allowed by default1402worker1 # [ 0.481036] ACPI: PCI: Interrupt link GSIH configured for IRQ 231403worker2 # [ 0.476617] PCI: Using ACPI for IRQ routing1404worker1 # [ 0.483377] iommu: Default domain type: Translated1405worker1 # [ 0.484243] iommu: DMA domain TLB invalidation policy: lazy mode1406worker1 # [ 0.485318] ACPI: bus type USB registered1407worker1 # [ 0.486108] usbcore: registered new interface driver usbfs1408worker1 # [ 0.487024] usbcore: registered new interface driver hub1409worker1 # [ 0.487280] usbcore: registered new device driver usb1410worker1 # [ 0.489097] NetLabel: Initializing1411worker1 # [ 0.489724] NetLabel: domain hash size = 1281412worker1 # [ 0.490269] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413worker1 # [ 0.491330] NetLabel: unlabeled traffic allowed by default1414worker1 # [ 0.492291] PCI: Using ACPI for IRQ routing1415client # [ 0.521585] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416client # [ 0.521699] pci 0000:00:01.0: vgaarb: bridge control possible1417client # [ 0.521699] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418client # [ 0.521708] vgaarb: loaded1419client # [ 0.522466] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420client # [ 0.522703] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421lb # [ 0.519776] pci 0000:00:01.0: vgaarb: setting as boot VGA device1422worker2 # [ 0.519686] pci 0000:00:01.0: vgaarb: setting as boot VGA device1423lb # [ 0.520509] pci 0000:00:01.0: vgaarb: bridge control possible1424client # [ 0.525800] clocksource: Switched to clocksource kvm-clock1425worker2 # [ 0.520648] pci 0000:00:01.0: vgaarb: bridge control possible1426lb # [ 0.520509] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1427worker2 # [ 0.520648] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1428lb # [ 0.520518] vgaarb: loaded1429worker2 # [ 0.520656] vgaarb: loaded1430lb # [ 0.521276] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01431worker2 # [ 0.521411] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01432client # [ 0.529260] VFS: Disk quotas dquot_6.6.01433lb # [ 0.521515] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1434worker2 # [ 0.521681] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1435client # [ 0.530096] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1436client # [ 0.531400] pnp: PnP ACPI init1437client # [ 0.532218] ACPI: IRQ 4 override to edge(!), high(!)1438client # [ 0.533218] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1439client # [ 0.534825] pnp: PnP ACPI: found 6 devices1440worker2 # [ 0.525838] clocksource: Switched to clocksource kvm-clock1441lb # [ 0.526604] clocksource: Switched to clocksource kvm-clock1442worker2 # [ 0.527616] VFS: Disk quotas dquot_6.6.01443lb # [ 0.530120] VFS: Disk quotas dquot_6.6.01444worker1 # [ 0.536943] pci 0000:00:01.0: vgaarb: setting as boot VGA device1445lb # [ 0.530880] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1446worker2 # [ 0.530447] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1447worker1 # [ 0.537265] pci 0000:00:01.0: vgaarb: bridge control possible1448lb # [ 0.532224] pnp: PnP ACPI init1449worker2 # [ 0.531854] pnp: PnP ACPI init1450lb # [ 0.533069] ACPI: IRQ 4 override to edge(!), high(!)1451worker1 # [ 0.537265] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1452worker2 # [ 0.532665] ACPI: IRQ 4 override to edge(!), high(!)1453worker1 # [ 0.537275] vgaarb: loaded1454client # [ 0.542473] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1455lb # [ 0.534064] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1456worker2 # [ 0.533655] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1457worker1 # [ 0.538026] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01458client # [ 0.543940] clocksource: Switched to clocksource acpi_pm1459lb # [ 0.535559] pnp: PnP ACPI: found 6 devices1460worker1 # [ 0.538271] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461worker2 # [ 0.535177] pnp: PnP ACPI: found 6 devices1462client # [ 0.544874] NET: Registered PF_INET protocol family1463client # [ 0.546024] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1464worker1 # [ 0.541361] clocksource: Switched to clocksource kvm-clock1465worker1 # [ 0.545133] VFS: Disk quotas dquot_6.6.01466lb # [ 0.543204] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1467worker2 # [ 0.542856] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1468worker1 # [ 0.545849] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1469lb # [ 0.544621] clocksource: Switched to clocksource acpi_pm1470worker2 # [ 0.544321] clocksource: Switched to clocksource acpi_pm1471worker1 # [ 0.547194] pnp: PnP ACPI init1472lb # [ 0.545559] NET: Registered PF_INET protocol family1473worker2 # [ 0.545305] NET: Registered PF_INET protocol family1474worker1 # [ 0.547970] ACPI: IRQ 4 override to edge(!), high(!)1475lb # [ 0.546796] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1476worker2 # [ 0.546479] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1477worker1 # [ 0.548980] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1478worker1 # [ 0.550533] pnp: PnP ACPI: found 6 devices1479client # [ 0.562927] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1480client # [ 0.564387] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1481client # [ 0.565656] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1482worker1 # [ 0.558202] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1483worker1 # [ 0.559745] clocksource: Switched to clocksource acpi_pm1484client # [ 0.566942] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1485worker1 # [ 0.560734] NET: Registered PF_INET protocol family1486client # [ 0.568135] TCP: Hash tables configured (established 16384 bind 16384)1487client # [ 0.569188] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1488worker1 # [ 0.561906] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1489client # [ 0.570399] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1490client # [ 0.571432] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1491client # [ 0.572636] NET: Registered PF_UNIX/PF_LOCAL protocol family1492lb # [ 0.563376] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1493client # [ 0.573563] NET: Registered PF_XDP protocol family1494worker2 # [ 0.563471] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1495lb # [ 0.564941] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1496client # [ 0.574345] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1497worker2 # [ 0.564985] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1498client # [ 0.575307] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1499lb # [ 0.566224] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1500worker2 # [ 0.566260] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1501client # [ 0.576263] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1502lb # [ 0.567537] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1503client # [ 0.577307] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1504worker2 # [ 0.567653] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1505lb # [ 0.568758] TCP: Hash tables configured (established 16384 bind 16384)1506client # [ 0.578343] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1507worker2 # [ 0.568924] TCP: Hash tables configured (established 16384 bind 16384)1508lb # [ 0.569860] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1509client # [ 0.579393] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1510worker2 # [ 0.570060] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1511lb # [ 0.571106] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1512client # [ 0.581068] ACPI: \_SB_.GSIA: Enabled at IRQ 161513worker2 # [ 0.571321] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1514lb # [ 0.572171] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1515worker2 # [ 0.572360] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1516lb # [ 0.573362] NET: Registered PF_UNIX/PF_LOCAL protocol family1517lb # [ 0.574320] NET: Registered PF_XDP protocol family1518worker2 # [ 0.573564] NET: Registered PF_UNIX/PF_LOCAL protocol family1519client # [ 0.583724] ACPI: \_SB_.GSIB: Enabled at IRQ 171520worker2 # [ 0.574547] NET: Registered PF_XDP protocol family1521lb # [ 0.575178] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1522worker2 # [ 0.575402] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1523lb # [ 0.576196] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1524client # [ 0.585633] ACPI: \_SB_.GSIC: Enabled at IRQ 181525worker2 # [ 0.576403] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1526worker1 # [ 0.578335] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1527lb # [ 0.577203] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1528worker2 # [ 0.577353] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1529lb # [ 0.578266] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1530worker1 # [ 0.579834] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1531client # [ 0.587725] ACPI: \_SB_.GSID: Enabled at IRQ 191532worker2 # [ 0.578405] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1533lb # [ 0.579307] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1534worker1 # [ 0.581140] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1535client # [ 0.589306] PCI: CLS 0 bytes, default 641536worker2 # [ 0.579458] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1537lb # [ 0.580388] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1538worker1 # [ 0.582473] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1539worker2 # [ 0.580538] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1540client # [ 0.590142] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1541lb # [ 0.582082] ACPI: \_SB_.GSIA: Enabled at IRQ 161542worker1 # [ 0.583705] TCP: Hash tables configured (established 16384 bind 16384)1543worker2 # [ 0.582293] ACPI: \_SB_.GSIA: Enabled at IRQ 161544client # [ 0.591792] Trying to unpack rootfs image as initramfs...1545worker1 # [ 0.584824] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1546lb # [ 0.584357] ACPI: \_SB_.GSIB: Enabled at IRQ 171547worker1 # [ 0.586207] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1548worker2 # [ 0.585357] ACPI: \_SB_.GSIB: Enabled at IRQ 171549worker1 # [ 0.587389] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1550lb # [ 0.586752] ACPI: \_SB_.GSIC: Enabled at IRQ 181551worker1 # [ 0.588669] NET: Registered PF_UNIX/PF_LOCAL protocol family1552worker2 # [ 0.587360] ACPI: \_SB_.GSIC: Enabled at IRQ 181553worker1 # [ 0.589687] NET: Registered PF_XDP protocol family1554lb # [ 0.588709] ACPI: \_SB_.GSID: Enabled at IRQ 191555worker1 # [ 0.590570] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1556worker2 # [ 0.589329] ACPI: \_SB_.GSID: Enabled at IRQ 191557worker1 # [ 0.591641] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1558lb # [ 0.590367] PCI: CLS 0 bytes, default 641559worker1 # [ 0.592669] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1560worker2 # [ 0.590929] PCI: CLS 0 bytes, default 641561lb # [ 0.591307] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1562worker1 # [ 0.593808] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1563lb # [ 0.593046] Trying to unpack rootfs image as initramfs...1564worker2 # [ 0.591765] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1565worker1 # [ 0.594943] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1566worker2 # [ 0.593447] Trying to unpack rootfs image as initramfs...1567worker1 # [ 0.596017] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1568worker1 # [ 0.597806] ACPI: \_SB_.GSIA: Enabled at IRQ 161569worker1 # [ 0.600475] ACPI: \_SB_.GSIB: Enabled at IRQ 171570worker1 # [ 0.602438] ACPI: \_SB_.GSIC: Enabled at IRQ 181571worker1 # [ 0.604405] ACPI: \_SB_.GSID: Enabled at IRQ 191572worker1 # [ 0.606168] PCI: CLS 0 bytes, default 641573worker1 # [ 0.607059] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574worker1 # [ 0.608895] Trying to unpack rootfs image as initramfs...1575client # [ 0.635623] Initialise system trusted keyrings1576client # [ 0.638591] workingset: timestamp_bits=40 max_order=19 bucket_order=01577worker2 # [ 0.638281] Initialise system trusted keyrings1578lb # [ 0.639082] Initialise system trusted keyrings1579worker2 # [ 0.639226] workingset: timestamp_bits=40 max_order=19 bucket_order=01580lb # [ 0.640050] workingset: timestamp_bits=40 max_order=19 bucket_order=01581client # [ 0.659816] Key type asymmetric registered1582client # [ 0.660574] Asymmetric key parser 'x509' registered1583worker1 # [ 0.653672] Initialise system trusted keyrings1584client # [ 0.661390] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1585worker1 # [ 0.654689] workingset: timestamp_bits=40 max_order=19 bucket_order=01586client # [ 0.666724] io scheduler mq-deadline registered1587client # [ 0.667458] io scheduler kyber registered1588client # [ 0.670108] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1589client # [ 0.671333] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1590worker2 # [ 0.663105] Key type asymmetric registered1591worker2 # [ 0.663825] Asymmetric key parser 'x509' registered1592lb # [ 0.664708] Key type asymmetric registered1593lb # [ 0.665405] Asymmetric key parser 'x509' registered1594worker2 # [ 0.664637] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1595lb # [ 0.666287] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1596client # [ 0.676953] Linux agpgart interface v0.1031597worker2 # [ 0.667978] io scheduler mq-deadline registered1598client # [ 0.677723] ACPI: bus type drm_connector registered1599lb # [ 0.668835] io scheduler mq-deadline registered1600worker2 # [ 0.668709] io scheduler kyber registered1601lb # [ 0.669554] io scheduler kyber registered1602client # [ 0.680964] usbcore: registered new interface driver usbserial_generic1603lb # [ 0.672312] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1604client # [ 0.682000] usbserial: USB Serial support registered for generic1605worker2 # [ 0.672842] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1606lb # [ 0.673578] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1607worker2 # [ 0.674077] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1608worker1 # [ 0.678241] Key type asymmetric registered1609client # [ 0.685554] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1610worker1 # [ 0.678967] Asymmetric key parser 'x509' registered1611client # [ 0.685554] Please enable it if your BIOS has the CPPC option.1612worker1 # [ 0.679818] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1613client # [ 0.687668] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1614lb # [ 0.679314] Linux agpgart interface v0.1031615lb # [ 0.680090] ACPI: bus type drm_connector registered1616client # [ 0.688967] drop_monitor: Initializing network drop monitor service1617worker2 # [ 0.679656] Linux agpgart interface v0.1031618worker2 # [ 0.680411] ACPI: bus type drm_connector registered1619client # [ 0.690063] NET: Registered PF_INET6 protocol family1620lb # [ 0.683115] usbcore: registered new interface driver usbserial_generic1621worker1 # [ 0.685267] io scheduler mq-deadline registered1622worker1 # [ 0.686081] io scheduler kyber registered1623lb # [ 0.684183] usbserial: USB Serial support registered for generic1624worker2 # [ 0.683515] usbcore: registered new interface driver usbserial_generic1625worker2 # [ 0.684585] usbserial: USB Serial support registered for generic1626client # [ 0.694892] Segment Routing with IPv61627client # [ 0.695589] In-situ OAM (IOAM) with IPv61628lb # [ 0.686688] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1629lb # [ 0.686688] Please enable it if your BIOS has the CPPC option.1630worker1 # [ 0.689713] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1631client # [ 0.697823] IPI shorthand broadcast: enabled1632worker2 # [ 0.687821] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1633lb # [ 0.688951] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1634worker2 # [ 0.687821] Please enable it if your BIOS has the CPPC option.1635worker1 # [ 0.691112] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1636worker2 # [ 0.689912] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1637lb # [ 0.692831] drop_monitor: Initializing network drop monitor service1638worker1 # [ 0.695074] Linux agpgart interface v0.1031639lb # [ 0.694013] NET: Registered PF_INET6 protocol family1640worker1 # [ 0.695861] ACPI: bus type drm_connector registered1641worker2 # [ 0.693974] drop_monitor: Initializing network drop monitor service1642worker2 # [ 0.695093] NET: Registered PF_INET6 protocol family1643lb # [ 0.697123] Segment Routing with IPv61644client # [ 0.706050] sched_clock: Marking stable (572014696, 133522563)->(780947035, -75409776)1645worker2 # [ 0.698354] Segment Routing with IPv61646worker1 # [ 0.700273] usbcore: registered new interface driver usbserial_generic1647lb # [ 0.699702] In-situ OAM (IOAM) with IPv61648worker1 # [ 0.701392] usbserial: USB Serial support registered for generic1649lb # [ 0.700763] IPI shorthand broadcast: enabled1650worker2 # [ 0.700837] In-situ OAM (IOAM) with IPv61651client # [ 0.710637] registered taskstats version 11652worker2 # [ 0.701912] IPI shorthand broadcast: enabled1653client # [ 0.711598] Loading compiled-in X.509 certificates1654worker1 # [ 0.705152] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1655worker1 # [ 0.705152] Please enable it if your BIOS has the CPPC option.1656worker1 # [ 0.707481] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1657lb # [ 0.709024] sched_clock: Marking stable (575014444, 133649282)->(785382933, -76719207)1658worker1 # [ 0.711279] drop_monitor: Initializing network drop monitor service1659worker1 # [ 0.712501] NET: Registered PF_INET6 protocol family1660worker2 # [ 0.710454] sched_clock: Marking stable (577014746, 132784556)->(783282614, -73483312)1661worker1 # [ 0.715856] Segment Routing with IPv61662lb # [ 0.714853] registered taskstats version 11663worker2 # [ 0.715003] registered taskstats version 11664lb # [ 0.715831] Loading compiled-in X.509 certificates1665worker1 # [ 0.718166] In-situ OAM (IOAM) with IPv61666worker2 # [ 0.716000] Loading compiled-in X.509 certificates1667worker1 # [ 0.720533] IPI shorthand broadcast: enabled1668client # [ 0.729553] Demotion targets for Node 0: null1669client # [ 0.730463] Key type .fscrypt registered1670client # [ 0.731169] Key type fscrypt-provisioning registered1671client # [ 0.732126] ima: No TPM chip found, activating TPM-bypass!1672client # [ 0.736559] ima: Allocated hash algorithm: sha11673worker1 # [ 0.728844] sched_clock: Marking stable (595014254, 133119405)->(804248189, -76114530)1674client # [ 0.737338] ima: No architecture policies found1675client # [ 0.738299] PM: Magic number: 10:519:8311676worker1 # [ 0.732539] registered taskstats version 11677worker1 # [ 0.733564] Loading compiled-in X.509 certificates1678lb # [ 0.732283] Demotion targets for Node 0: null1679client # [ 0.741932] RAS: Correctable Errors collector initialized.1680lb # [ 0.734730] Key type .fscrypt registered1681lb # [ 0.735410] Key type fscrypt-provisioning registered1682worker2 # [ 0.735817] Demotion targets for Node 0: null1683lb # [ 0.736369] ima: No TPM chip found, activating TPM-bypass!1684worker2 # [ 0.736687] Key type .fscrypt registered1685worker2 # [ 0.737413] Key type fscrypt-provisioning registered1686lb # [ 0.740680] ima: Allocated hash algorithm: sha11687lb # [ 0.741501] ima: No architecture policies found1688client # [ 0.750630] clk: Disabling unused clocks1689worker2 # [ 0.740937] ima: No TPM chip found, activating TPM-bypass!1690worker2 # [ 0.741826] ima: Allocated hash algorithm: sha11691lb # [ 0.742464] PM: Magic number: 10:519:8311692client # [ 0.751319] PM: genpd: Disabling unused power domains1693worker2 # [ 0.742571] ima: No architecture policies found1694worker2 # [ 0.745018] PM: Magic number: 10:519:8311695lb # [ 0.746085] RAS: Correctable Errors collector initialized.1696worker2 # [ 0.746752] RAS: Correctable Errors collector initialized.1697worker1 # [ 0.753154] Demotion targets for Node 0: null1698worker1 # [ 0.754089] Key type .fscrypt registered1699worker1 # [ 0.754770] Key type fscrypt-provisioning registered1700worker1 # [ 0.755771] ima: No TPM chip found, activating TPM-bypass!1701lb # [ 0.754787] clk: Disabling unused clocks1702lb # [ 0.755482] PM: genpd: Disabling unused power domains1703worker2 # [ 0.756548] clk: Disabling unused clocks1704worker1 # [ 0.760155] ima: Allocated hash algorithm: sha11705worker1 # [ 0.760952] ima: No architecture policies found1706worker2 # [ 0.758821] PM: genpd: Disabling unused power domains1707worker1 # [ 0.763149] PM: Magic number: 10:519:8311708worker1 # [ 0.764932] RAS: Correctable Errors collector initialized.1709worker1 # [ 0.774897] clk: Disabling unused clocks1710worker1 # [ 0.777152] PM: genpd: Disabling unused power domains1711client # [ 0.914696] Freeing initrd memory: 29096K1712client # [ 0.917894] Freeing unused decrypted memory: 2028K1713client # [ 0.920465] Freeing unused kernel image (initmem) memory: 3644K1714client # [ 0.921633] Write protecting the kernel read-only data: 32768k1715client # [ 0.923551] Freeing unused kernel image (text/rodata gap) memory: 1208K1716client # [ 0.925086] Freeing unused kernel image (rodata/data gap) memory: 752K1717lb # [ 0.919818] Freeing initrd memory: 29092K1718lb # [ 0.922958] Freeing unused decrypted memory: 2028K1719worker2 # [ 0.925153] Freeing initrd memory: 29092K1720lb # [ 0.925606] Freeing unused kernel image (initmem) memory: 3644K1721lb # [ 0.926690] Write protecting the kernel read-only data: 32768k1722lb # [ 0.928550] Freeing unused kernel image (text/rodata gap) memory: 1208K1723worker2 # [ 0.928457] Freeing unused decrypted memory: 2028K1724lb # [ 0.930077] Freeing unused kernel image (rodata/data gap) memory: 752K1725worker2 # [ 0.931297] Freeing unused kernel image (initmem) memory: 3644K1726worker2 # [ 0.932409] Write protecting the kernel read-only data: 32768k1727worker2 # [ 0.935269] Freeing unused kernel image (text/rodata gap) memory: 1208K1728worker2 # [ 0.937149] Freeing unused kernel image (rodata/data gap) memory: 752K1729worker1 # [ 0.956237] Freeing initrd memory: 29092K1730worker1 # [ 0.959574] Freeing unused decrypted memory: 2028K1731worker1 # [ 0.962420] Freeing unused kernel image (initmem) memory: 3644K1732worker1 # [ 0.963575] Write protecting the kernel read-only data: 32768k1733worker1 # [ 0.965581] Freeing unused kernel image (text/rodata gap) memory: 1208K1734worker1 # [ 0.967222] Freeing unused kernel image (rodata/data gap) memory: 752K1735client # [ 0.976019] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736client # [ 0.977115] Run /init as init process1737client # [ 0.986948] systemd[1]: Inserted module 'autofs4'1738lb # [ 0.981181] x86/mm: Checked W+X mappings: passed, no W+X pages found.1739lb # [ 0.982324] Run /init as init process1740worker2 # [ 0.988141] x86/mm: Checked W+X mappings: passed, no W+X pages found.1741worker2 # [ 0.989232] Run /init as init process1742lb # [ 0.992706] systemd[1]: Inserted module 'autofs4'1743client # [ 1.003172] fuse: init (API version 7.45)1744client # [ 1.009019] ACPI: \_SB_.GSIG: Enabled at IRQ 221745worker2 # [ 1.000213] systemd[1]: Inserted module 'autofs4'1746client # [ 1.011841] ACPI: \_SB_.GSIH: Enabled at IRQ 231747client # [ 1.015835] ACPI: \_SB_.GSIE: Enabled at IRQ 201748lb # [ 1.009014] fuse: init (API version 7.45)1749client # [ 1.018349] ACPI: \_SB_.GSIF: Enabled at IRQ 211750lb # [ 1.014834] ACPI: \_SB_.GSIG: Enabled at IRQ 221751worker1 # [ 1.018561] x86/mm: Checked W+X mappings: passed, no W+X pages found.1752lb # [ 1.017276] ACPI: \_SB_.GSIH: Enabled at IRQ 231753worker2 # [ 1.016727] fuse: init (API version 7.45)1754worker1 # [ 1.019691] Run /init as init process1755lb # [ 1.020241] ACPI: \_SB_.GSIE: Enabled at IRQ 201756lb # [ 1.022451] ACPI: \_SB_.GSIF: Enabled at IRQ 211757worker2 # [ 1.022557] ACPI: \_SB_.GSIG: Enabled at IRQ 221758worker2 # [ 1.025027] ACPI: \_SB_.GSIH: Enabled at IRQ 231759worker2 # [ 1.028047] ACPI: \_SB_.GSIE: Enabled at IRQ 201760worker1 # [ 1.031617] systemd[1]: Inserted module 'autofs4'1761worker2 # [ 1.030224] ACPI: \_SB_.GSIF: Enabled at IRQ 211762client # [ 1.045778] systemd[1]: Successfully made /usr/ read-only.1763worker1 # [ 1.048413] fuse: init (API version 7.45)1764lb # [ 1.049429] systemd[1]: Successfully made /usr/ read-only.1765worker1 # [ 1.054399] ACPI: \_SB_.GSIG: Enabled at IRQ 221766worker1 # [ 1.057401] ACPI: \_SB_.GSIH: Enabled at IRQ 231767worker2 # [ 1.057767] systemd[1]: Successfully made /usr/ read-only.1768worker1 # [ 1.061223] ACPI: \_SB_.GSIE: Enabled at IRQ 201769worker1 # [ 1.063792] ACPI: \_SB_.GSIF: Enabled at IRQ 211770worker1 # [ 1.092619] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.381790] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772client # [ 1.386928] systemd[1]: Detected virtualization kvm.1773client # [ 1.387826] systemd[1]: Detected architecture x86-64.1774client # [ 1.388722] systemd[1]: Running in initrd.1775client # [ 1.389962] systemd[1]: Initializing machine ID from random generator.1776client # [ 1.391196] systemd[1]: Hostname set to <client>.1777lb # [ 1.385922] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker2 # [ 1.393369] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1779worker2 # [ 1.398277] systemd[1]: Detected virtualization kvm.1780lb # [ 1.397895] systemd[1]: Detected virtualization kvm.1781worker2 # [ 1.399145] systemd[1]: Detected architecture x86-64.1782worker2 # [ 1.400023] systemd[1]: Running in initrd.1783lb # [ 1.400035] systemd[1]: Detected architecture x86-64.1784worker2 # [ 1.401271] systemd[1]: Initializing machine ID from random generator.1785worker2 # [ 1.402479] systemd[1]: Hostname set to <worker2>.1786lb # [ 1.402164] systemd[1]: Running in initrd.1787lb # [ 1.404634] systemd[1]: Initializing machine ID from random generator.1788lb # [ 1.407495] systemd[1]: Hostname set to <lb>.1789worker1 # [ 1.429070] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790worker1 # [ 1.441057] systemd[1]: Detected virtualization kvm.1791worker1 # [ 1.443202] systemd[1]: Detected architecture x86-64.1792worker1 # [ 1.445310] systemd[1]: Running in initrd.1793worker1 # [ 1.447853] systemd[1]: Initializing machine ID from random generator.1794worker1 # [ 1.450835] systemd[1]: Hostname set to <worker1>.1795client # [ 1.596585] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker2 # [ 1.595266] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797lb # [ 1.611765] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798worker2 # [ 1.634257] systemd[1]: Queued start job for default target Initrd Default Target.1799worker2 # [ 1.638183] systemd[1]: Created slice Slice /system/modprobe.1800worker2 # [ 1.639364] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1801worker2 # [ 1.640856] systemd[1]: Expecting device /dev/disk/by-label/nixos...1802worker2 # [ 1.641991] systemd[1]: Reached target Path Units.1803worker2 # [ 1.642873] systemd[1]: Reached target Slice Units.1804client # [ 1.652208] systemd[1]: Queued start job for default target Initrd Default Target.1805worker2 # [ 1.643720] systemd[1]: Reached target Swaps.1806worker2 # [ 1.644513] systemd[1]: Reached target Timer Units.1807worker2 # [ 1.645521] systemd[1]: Listening on D-Bus System Message Bus Socket.1808worker2 # [ 1.646816] systemd[1]: Listening on Journal Socket (/dev/log).1809client # [ 1.656966] systemd[1]: Created slice Slice /system/modprobe.1810worker2 # [ 1.647953] systemd[1]: Listening on Journal Sockets.1811worker2 # [ 1.648934] systemd[1]: Listening on udev Control Socket.1812client # [ 1.658147] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1813worker2 # [ 1.649936] systemd[1]: Listening on udev Kernel Socket.1814client # [ 1.659587] systemd[1]: Expecting device /dev/disk/by-label/nixos...1815lb # [ 1.650515] systemd[1]: Queued start job for default target Initrd Default Target.1816worker2 # [ 1.650850] systemd[1]: Reached target Socket Units.1817client # [ 1.660648] systemd[1]: Reached target Path Units.1818client # [ 1.661509] systemd[1]: Reached target Slice Units.1819worker2 # [ 1.652632] systemd[1]: Starting Create List of Static Device Nodes...1820client # [ 1.662565] systemd[1]: Reached target Swaps.1821client # [ 1.663358] systemd[1]: Reached target Timer Units.1822lb # [ 1.654967] systemd[1]: Created slice Slice /system/modprobe.1823client # [ 1.664383] systemd[1]: Listening on D-Bus System Message Bus Socket.1824lb # [ 1.656151] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1825client # [ 1.665647] systemd[1]: Listening on Journal Socket (/dev/log).1826lb # [ 1.657499] systemd[1]: Expecting device /dev/disk/by-label/nixos...1827worker2 # [ 1.656906] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1828client # [ 1.666829] systemd[1]: Listening on Journal Sockets.1829lb # [ 1.658551] systemd[1]: Reached target Path Units.1830client # [ 1.667802] systemd[1]: Listening on udev Control Socket.1831lb # [ 1.659375] systemd[1]: Reached target Slice Units.1832client # [ 1.668842] systemd[1]: Listening on udev Kernel Socket.1833lb # [ 1.660237] systemd[1]: Reached target Swaps.1834client # [ 1.669813] systemd[1]: Reached target Socket Units.1835lb # [ 1.661027] systemd[1]: Reached target Timer Units.1836client # [ 1.671605] systemd[1]: Starting Create List of Static Device Nodes...1837lb # [ 1.662837] systemd[1]: Listening on D-Bus System Message Bus Socket.1838lb # [ 1.664178] systemd[1]: Listening on Journal Socket (/dev/log).1839lb # [ 1.665436] systemd[1]: Listening on Journal Sockets.1840worker2 # [ 1.664865] systemd[1]: Starting Load Kernel Module configfs...1841lb # [ 1.666477] systemd[1]: Listening on udev Control Socket.1842client # [ 1.675630] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1843lb # [ 1.667591] systemd[1]: Listening on udev Kernel Socket.1844lb # [ 1.668568] systemd[1]: Reached target Socket Units.1845lb # [ 1.670747] systemd[1]: Starting Create List of Static Device Nodes...1846worker1 # [ 1.674015] systemd[1]: bpf-restrict-fs: LSM BPF program attached1847lb # [ 1.674561] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1848client # [ 1.684746] systemd[1]: Starting Load Kernel Module configfs...1849worker2 # [ 1.679869] systemd[1]: Starting Journal Service...1850lb # [ 1.682299] systemd[1]: Starting Load Kernel Module configfs...1851client # [ 1.700778] systemd[1]: Starting Journal Service...1852worker2 # [ 1.693871] systemd[1]: Starting Load Kernel Modules...1853lb # [ 1.699338] systemd[1]: Starting Journal Service...1854client # [ 1.715614] systemd[1]: Starting Load Kernel Modules...1855worker2 # [ 1.698882] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1856lb # [ 1.711762] systemd[1]: Starting Load Kernel Modules...1857worker2 # [ 1.714632] netfs: FS-Cache loaded1858worker2 # [ 1.719349] 9pnet: Installing 9P2000 support1859client # [ 1.720627] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1860client # [ 1.736753] netfs: FS-Cache loaded1861worker1 # [ 1.728807] systemd[1]: Queued start job for default target Initrd Default Target.1862worker2 # [ 1.728664] systemd[1]: Starting Coldplug All udev Devices...1863lb # [ 1.718765] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1864worker1 # [ 1.733480] systemd[1]: Created slice Slice /system/modprobe.1865client # [ 1.741780] 9pnet: Installing 9P2000 support1866worker1 # [ 1.734691] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1867worker1 # [ 1.736162] systemd[1]: Expecting device /dev/disk/by-label/nixos...1868lb # [ 1.735474] netfs: FS-Cache loaded1869worker1 # [ 1.737321] systemd[1]: Reached target Path Units.1870worker1 # [ 1.738225] systemd[1]: Reached target Slice Units.1871worker1 # [ 1.739161] systemd[1]: Reached target Swaps.1872worker1 # [ 1.739951] systemd[1]: Reached target Timer Units.1873worker1 # [ 1.740935] systemd[1]: Listening on D-Bus System Message Bus Socket.1874lb # [ 1.740532] 9pnet: Installing 9P2000 support1875worker1 # [ 1.742288] systemd[1]: Listening on Journal Socket (/dev/log).1876worker1 # [ 1.743476] systemd[1]: Listening on Journal Sockets.1877worker1 # [ 1.744522] systemd[1]: Listening on udev Control Socket.1878worker2 # [ 1.742893] systemd[1]: Finished Create List of Static Device Nodes.1879worker1 # [ 1.745592] systemd[1]: Listening on udev Kernel Socket.1880client # [ 1.753618] systemd[1]: Starting Coldplug All udev Devices...1881worker1 # [ 1.746564] systemd[1]: Reached target Socket Units.1882worker1 # [ 1.748477] systemd[1]: Starting Create List of Static Device Nodes...1883worker2 # [ 1.749414] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1884worker1 # [ 1.753199] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1885lb # [ 1.753751] systemd[1]: Starting Coldplug All udev Devices...1886worker2 # [ 1.757297] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1887client # [ 1.767655] systemd[1]: Finished Create List of Static Device Nodes.1888worker1 # [ 1.761209] systemd[1]: Starting Load Kernel Module configfs...1889worker2 # [ 1.761467] systemd-journald[67]: Collecting audit messages is disabled.1890worker2 # [ 1.762993] systemd[1]: modprobe@configfs.service: Deactivated successfully.1891client # [ 1.774147] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1892lb # [ 1.767801] systemd[1]: Finished Create List of Static Device Nodes.1893worker2 # [ 1.770167] systemd[1]: Finished Load Kernel Module configfs.1894worker1 # [ 1.774311] systemd[1]: Starting Journal Service...1895client # [ 1.781968] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1896lb # [ 1.774271] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897worker2 # [ 1.774310] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1898client # [ 1.786989] systemd[1]: modprobe@configfs.service: Deactivated successfully.1899client # [ 1.788424] systemd-journald[67]: Collecting audit messages is disabled.1900worker2 # [ 1.778473] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1901lb # [ 1.780722] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1902client # [ 1.793935] systemd[1]: Finished Load Kernel Module configfs.1903worker1 # [ 1.787430] systemd[1]: Starting Load Kernel Modules...1904lb # [ 1.786079] systemd[1]: modprobe@configfs.service: Deactivated successfully.1905worker2 # [ 1.787013] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1906worker2 # [ 1.788492] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1907lb # [ 1.789849] systemd-journald[67]: Collecting audit messages is disabled.1908client # [ 1.800019] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1909lb # [ 1.793054] systemd[1]: Finished Load Kernel Module configfs.1910client # [ 1.805659] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1911lb # [ 1.799170] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1912client # [ 1.811620] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1913client # [ 1.812925] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1914worker1 # [ 1.795231] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1915lb # [ 1.804791] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1916worker1 # [ 1.811287] netfs: FS-Cache loaded1917lb # [ 1.810742] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1918lb # [ 1.812025] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1919worker1 # [ 1.816009] 9pnet: Installing 9P2000 support1920worker2 # [ 1.818319] systemd[1]: Finished Load Kernel Modules.1921worker1 # [ 1.826216] systemd[1]: Starting Coldplug All udev Devices...1922worker2 # [ 1.824994] systemd[1]: Starting Apply Kernel Variables...1923client # [ 1.843160] systemd[1]: Finished Load Kernel Modules.1924worker2 # [ 1.836928] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1925worker1 # [ 1.840254] systemd[1]: Finished Create List of Static Device Nodes.1926client # [ 1.851742] systemd[1]: Starting Apply Kernel Variables...1927lb # [ 1.843594] systemd[1]: Finished Load Kernel Modules.1928worker1 # [ 1.847832] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1929worker2 # [ 1.849066] systemd[1]: Starting Create Static Device Nodes in /dev...1930lb # [ 1.849877] systemd[1]: Starting Apply Kernel Variables...1931worker1 # [ 1.853891] systemd-journald[67]: Collecting audit messages is disabled.1932client # [ 1.861673] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1933worker1 # [ 1.855525] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1934lb # [ 1.858872] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1935worker1 # [ 1.861551] systemd[1]: modprobe@configfs.service: Deactivated successfully.1936client # [ 1.870751] systemd[1]: Starting Create Static Device Nodes in /dev...1937worker2 # [ 1.861715] systemd[1]: Finished Apply Kernel Variables.1938worker1 # [ 1.867203] systemd[1]: Finished Load Kernel Module configfs.1939lb # [ 1.869857] systemd[1]: Starting Create Static Device Nodes in /dev...1940worker1 # [ 1.872591] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1941client # [ 1.885686] systemd[1]: Finished Apply Kernel Variables.1942worker1 # [ 1.877761] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1943worker2 # [ 1.746556] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.880480] systemd[1]: Started Journal Service.1944worker2 # 1945worker1 # [ 1.885333] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1946worker2 # [ 1.751203] systemd-modules-load[68]: Inserted module 'virtio_balloon'1947worker2 # [ 1.752240] systemd-modules-load[68]: Inserted module 'virtio_gpu'1948worker1 # [ 1.886808] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1949worker2 # [ 1.753259] systemd[1]: Finished Create Static Device Nodes in /dev.1950worker2 # [ 1.755081] systemd[1]: Reached target Preparation for Local File Systems.1951worker2 # [ 1.756193] systemd[1]: Reached target Local File Systems.1952lb # [ 1.889801] systemd[1]: Finished Apply Kernel Variables.1953worker2 # [ 1.759165] systemd[1]: Starting Create System Files and Directories...1954client # [ 1.903098] systemd[1]: Finished Create Static Device Nodes in /dev.1955client # [ 1.771400] systemd-modules-load[68]: Inserted module 'dm_mod'1956client # [ 1.773428] systemd-modules-load[68]: Inserted module 'virtio_balloon'1957worker2 # [ 1.765980] systemd[1]: Starting Rule-based Manager for Device Events and Files...1958client # [ 1.775273] systemd-modules-load[68]: Inserted module 'virtio_gpu'1959lb # [ 1.900753] systemd[1]: Started Journal Service.1960client # [ 1.909820] systemd[1]: Started Journal Service.1961lb # [ 1.770228] systemd-modules-load[68]: Inserted module 'dm_mod'1962lb # [ 1.771605] systemd-modules-load[68]: Inserted module 'virtio_balloon'1963lb # [ 1.774317] systemd-modules-load[68]: Inserted module 'virtio_gpu'1964lb # [ 1.775347] systemd[1]: Finished Create Static Device Nodes in /dev.1965client # [ 1.786122] systemd[1]: Reached target Preparation for Local File Systems.1966lb # [ 1.777349] systemd[1]: Reached target Preparation for Local File Systems.1967client # [ 1.787141] systemd[1]: Reached target Local File Systems.1968lb # [ 1.778662] systemd[1]: Reached target Local File Systems.1969client # [ 1.790075] systemd[1]: Starting Create System Files and Directories...1970worker1 # [ 1.916667] systemd[1]: Finished Load Kernel Modules.1971lb # [ 1.782077] systemd[1]: Starting Create System Files and Directories...1972client # [ 1.795070] systemd[1]: Starting Rule-based Manager for Device Events and Files...1973lb # [ 1.788073] systemd[1]: Starting Rule-based Manager for Device Events and Files...1974worker1 # [ 1.924319] systemd[1]: Starting Apply Kernel Variables...1975worker2 # [ 1.796588] systemd[1]: Finished Create System Files and Directories.1976worker1 # [ 1.938243] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1977worker1 # [ 1.945349] systemd[1]: Starting Create Static Device Nodes in /dev...1978client # [ 1.820426] systemd[1]: Finished Create System Files and Directories.1979lb # [ 1.812637] systemd[1]: Finished Create System Files and Directories.1980worker2 # [ 1.820453] systemd-udevd[76]: Using default interface naming scheme 'v261'.1981worker1 # [ 1.965735] systemd[1]: Finished Apply Kernel Variables.1982client # [ 1.843599] systemd-udevd[83]: Using default interface naming scheme 'v261'.1983worker1 # [ 1.974021] systemd[1]: Started Journal Service.1984lb # [ 1.841364] systemd-udevd[77]: Using default interface naming scheme 'v261'.1985worker1 # [ 1.844218] systemd-modules-load[68]: Inserted module 'dm_mod'1986worker2 # [ 1.843729] systemd[1]: Started Rule-based Manager for Device Events and Files.1987worker1 # [ 1.848142] systemd-modules-load[68]: Inserted module 'virtio_balloon'1988worker1 # [ 1.850234] systemd-modules-load[68]: Inserted module 'virtio_gpu'1989worker1 # [ 1.851277] systemd[1]: Finished Create Static Device Nodes in /dev.1990worker1 # [ 1.852495] systemd[1]: Reached target Preparation for Local File Systems.1991worker1 # [ 1.854319] systemd[1]: Reached target Local File Systems.1992worker1 # [ 1.857257] systemd[1]: Starting Create System Files and Directories...1993client # [ 1.867871] systemd[1]: Started Rule-based Manager for Device Events and Files.1994worker1 # [ 1.865984] systemd[1]: Starting Rule-based Manager for Device Events and Files...1995lb # [ 1.865249] systemd[1]: Started Rule-based Manager for Device Events and Files.1996worker2 # [ 1.892154] systemd[1]: Finished Coldplug All udev Devices.1997worker1 # [ 1.895209] systemd[1]: Finished Create System Files and Directories.1998worker2 # [ 1.894719] systemd[1]: Reached target System Initialization.1999worker2 # [ 1.895571] systemd[1]: Reached target Basic System.2000client # [ 1.908642] systemd[1]: Finished Coldplug All udev Devices.
loading 2,479 hidden lines…
4480worker2 # [ 33.489626] nix-grpc-daemon[787]: ts=2026-09-22T10:48:58Z level=info event=scheduler_connected addr=lb:500514481worker2 # [ 33.493883] nix-grpc-daemon[787]: ts=2026-09-22T10:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4482worker1 # [ 33.536930] nix-grpc-daemon[784]: ts=2026-09-22T10:48:58Z level=info event=scheduler_connected addr=lb:500514483worker1 # [ 33.540877] nix-grpc-daemon[784]: ts=2026-09-22T10:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4484lb: must succeed: curl -sf localhost:9901/clusters4485lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4486worker2 # [ 33.696324] nix-grpc-daemon[787]: ts=2026-09-22T10:48:58Z level=info event=scheduler_connected addr=lb:500514487worker2 # [ 33.699368] nix-grpc-daemon[787]: ts=2026-09-22T10:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4488worker1 # [ 33.741927] nix-grpc-daemon[784]: ts=2026-09-22T10:48:58Z level=info event=scheduler_connected addr=lb:500514489worker1 # [ 33.745888] nix-grpc-daemon[784]: ts=2026-09-22T10:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4490worker2 # [ 33.897126] nix-grpc-daemon[787]: ts=2026-09-22T10:48:58Z level=info event=scheduler_connected addr=lb:500514491worker2 # [ 33.898911] nix-grpc-daemon[787]: ts=2026-09-22T10:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4492worker1 # [ 33.946765] nix-grpc-daemon[784]: ts=2026-09-22T10:48:58Z level=info event=scheduler_connected addr=lb:500514493worker1 # [ 33.950983] nix-grpc-daemon[784]: ts=2026-09-22T10:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4494worker2 # [ 34.099640] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514495worker2 # [ 34.101530] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4496worker1 # [ 34.151879] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514497worker1 # [ 34.153507] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4498worker2 # [ 34.302152] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514499worker2 # [ 34.303911] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4500worker1 # [ 34.353903] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514501worker1 # [ 34.355687] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4502worker2 # [ 34.504471] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514503worker2 # [ 34.506227] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4504worker1 # [ 34.556183] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514505worker1 # [ 34.557944] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4506lb: must succeed: curl -sf localhost:9901/clusters4507lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4508worker2 # [ 34.708800] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514509worker2 # [ 34.711843] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4510worker1 # [ 34.759064] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514511worker1 # [ 34.762767] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4512worker2 # [ 34.909688] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514513worker2 # [ 34.913537] nix-grpc-daemon[787]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4514worker1 # [ 34.925471] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_take_over4515lb # [ 34.923266] niks3-server[1004]: 2026/09/22 10:48:59 INFO lead: acquired remote=192.168.1.3:498384516worker1 # [ 34.963526] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_connected addr=lb:500514517worker1 # [ 34.967287] nix-grpc-daemon[784]: ts=2026-09-22T10:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4518worker2 # [ 35.114151] nix-grpc-daemon[787]: ts=2026-09-22T10:49:00Z level=info event=scheduler_connected addr=lb:500514519worker2 # [ 35.115981] nix-grpc-daemon[787]: ts=2026-09-22T10:49:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4520worker1 # [ 35.168825] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=scheduler_connected addr=lb:500514521worker1 # [ 35.172575] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4522worker2 # [ 35.316772] nix-grpc-daemon[787]: ts=2026-09-22T10:49:00Z level=info event=scheduler_connected addr=lb:500514523worker2 # [ 35.318473] nix-grpc-daemon[787]: ts=2026-09-22T10:49:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4524worker1 # [ 35.373420] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=scheduler_connected addr=lb:500514525worker1 # [ 35.377247] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4526worker2 # [ 35.519143] nix-grpc-daemon[787]: ts=2026-09-22T10:49:00Z level=info event=scheduler_connected addr=lb:500514527worker1 # [ 35.553305] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=oidc_jwks provider=mock keys=14528worker1 # [ 35.557989] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:422284529worker1 # [ 35.562383] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04530worker1 # [ 35.577862] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=scheduler_connected addr=lb:500514531worker1 # [ 35.582174] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:422284532worker1 # [ 35.585768] nix-grpc-daemon[784]: ts=2026-09-22T10:49:00Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04533lb: must succeed: curl -sf localhost:9901/clusters4534lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4535worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4536worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4537lb: must succeed: curl -sf localhost:9901/clusters4538lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4539lb: must succeed: curl -sf localhost:9901/clusters4540lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4541lb: must succeed: curl -sf localhost:9901/clusters4542lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4543lb: must succeed: curl -sf localhost:9901/clusters4544lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4545worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4546worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.03 seconds)4547worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4548worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4549lb: must succeed: curl -sf localhost:9901/clusters4550lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4551lb: must succeed: curl -sf localhost:9901/clusters4552lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4553worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4554worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4555worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4556worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4557worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4558worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4559worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4560worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4561(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.19 seconds)4562subtest: balancer auth: client cert, bearer token, nothing, unknown CN4563worker1 # [ 36.632298] nix-grpc-daemon[784]: ts=2026-09-22T10:49:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:422424564worker1 # [ 36.639603] systemd[1]: Started Nix Daemon.4565worker1 # [ 36.808087] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4566worker1 # [ 36.831239] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 1/5)4567worker1 # [ 36.922818] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 178 ms (attempt 2/5)4568worker1 # [ 37.101278] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 228 ms (attempt 3/5)4569worker1 # [ 37.334845] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 565 ms (attempt 4/5)4570worker1 # [ 37.906303] nix-grpc-daemon[784]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4571worker1 # [ 37.908345] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4572worker2 # [ 37.934181] nix-grpc-daemon[787]: ts=2026-09-22T10:49:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:556244573worker2 # [ 37.940365] systemd[1]: Started Nix Daemon.4574worker2 # [ 38.113159] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4575worker2 # [ 38.136183] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 1/5)4576worker2 # [ 38.214749] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 2/5)4577worker2 # [ 38.292333] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 279 ms (attempt 3/5)4578worker2 # [ 38.571981] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 446 ms (attempt 4/5)4579worker2 # [ 39.027993] nix-grpc-daemon[787]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4580worker2 # [ 39.032176] nix-grpc-daemon[787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4581worker2 # [ 39.045599] nix-grpc-daemon[787]: ts=2026-09-22T10:49:04Z level=debug event=path_info_miss path=7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test error="error: path '/nix/store/7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test' is required, but there is no substituter that can build it"4582worker1 # [ 39.059924] nix-grpc-daemon[784]: ts=2026-09-22T10:49:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:422424583worker1 # [ 39.079517] nix-grpc-daemon[784]: ts=2026-09-22T10:49:04Z level=debug event=path_info_miss path=7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test error="error: path '/nix/store/7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test' is required, but there is no substituter that can build it"4584client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4585client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.06 seconds)4586client: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4587client: (finished: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.03 seconds)4588worker1 # [ 39.381916] nix-grpc-daemon[784]: ts=2026-09-22T10:49:04Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:422424589worker2 # [ 39.402478] nix-grpc-daemon[787]: ts=2026-09-22T10:49:04Z level=info event=oidc_jwks provider=mock keys=14590worker2 # [ 39.404673] nix-grpc-daemon[787]: ts=2026-09-22T10:49:04Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:556244591worker2 # [ 39.410035] nix-grpc-daemon[787]: ts=2026-09-22T10:49:04Z level=debug event=path_info_miss path=7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test error="error: path '/nix/store/7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test' is required, but there is no substituter that can build it"4592worker2 # [ 39.413736] nix-grpc-daemon[787]: ts=2026-09-22T10:49:04Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:556244593worker2 # [ 39.418770] nix-grpc-daemon[787]: ts=2026-09-22T10:49:04Z level=debug event=path_info_miss path=7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test error="error: path '/nix/store/7490adp5dzw5d66wqqx7bxllyl143sr2-nixos-system-client-test' is required, but there is no substituter that can build it"4594client: must succeed: rm -r /var/lib/nix-grpc-store4595client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4596worker1 # [ 40.687252] nix-grpc-daemon[784]: ts=2026-09-22T10:49:05Z level=info event=denied method=QueryMissing cn=- role=none4597worker1 # [ 40.935648] nix-grpc-daemon[784]: ts=2026-09-22T10:49:05Z level=info event=denied method=QueryMissing cn=stranger role=none4598(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.68 seconds)4599subtest: DAG build is scheduled across workers and lands in the cache4600client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24601worker1 # [ 41.205401] nix-grpc-daemon[784]: ts=2026-09-22T10:49:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:422424602client # don't know how to build these paths:4603client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4604worker1 # [ 41.219213] nix-grpc-daemon[784]: ts=2026-09-22T10:49:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422424605worker1 # [ 41.256831] nix-grpc-daemon[784]: ts=2026-09-22T10:49:06Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04606worker1 # [ 41.261825] nix-grpc-daemon[784]: ts=2026-09-22T10:49:06Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14607worker2 # [ 41.309217] nix-grpc-daemon[787]: ts=2026-09-22T10:49:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556244608worker2 # [ 41.324604] nix-grpc-daemon[787]: ts=2026-09-22T10:49:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556244609client # copying 1 paths to 'lb:50051'...4610worker2 # [ 41.332987] nix-grpc-daemon[787]: ts=2026-09-22T10:49:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556244611client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4612worker2 # [ 41.347744] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4613worker2 # [ 41.364614] nix-grpc-daemon[787]: ts=2026-09-22T10:49:06Z level=info event=niks3_push_started pid=8694614worker2 # [ 41.635250] nix-grpc-daemon[869]: time=2026-09-22T10:49:06.590Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""4615worker2 # [ 41.824694] nix-daemon[856]: accepted connection from pid 882, user nix-grpc-daemon (trusted)4616worker2 # [ 41.843123] nix-daemon[856]: reaped child process 885, status = succeeded4617worker2 # [ 42.013642] nix-daemon[856]: accepted connection from pid 887, user nix-grpc-daemon (trusted)4618worker2 # [ 42.026558] nix-daemon[856]: reaped child process 890, status = succeeded4619lb # [ 42.043894] niks3-server[1004]: 2026/09/22 10:49:06 INFO Received uploads request method=POST path=/api/pending_closures4620worker2 # [ 42.067907] nix-grpc-daemon[869]: time=2026-09-22T10:49:07.026Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4621worker2 # [ 42.070050] nix-grpc-daemon[869]: time=2026-09-22T10:49:07.028Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4622lb # [ 42.117322] niks3-server[1004]: 2026/09/22 10:49:06 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4623lb # [ 42.120473] niks3-server[1004]: 2026/09/22 10:49:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4624lb # [ 42.128854] niks3-server[1004]: 2026/09/22 10:49:07 INFO Signed narinfos id=1 count=14625worker2 # [ 42.129468] nix-grpc-daemon[869]: time=2026-09-22T10:49:07.087Z level=INFO msg="Uploading 1 narinfos"4626lb # [ 42.160320] niks3-server[1004]: 2026/09/22 10:49:07 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4627lb # [ 42.169486] niks3-server[1004]: 2026/09/22 10:49:07 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4628lb # [ 42.180548] niks3-server[1004]: 2026/09/22 10:49:07 INFO Completed upload id=14629worker2 # [ 42.181409] nix-grpc-daemon[869]: time=2026-09-22T10:49:07.139Z level=INFO msg="Upload complete. (533ms)"4630lb # [ 42.183542] niks3-server[1004]: 2026/09/22 10:49:07 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4631worker2 # [ 42.183411] nix-grpc-daemon[787]: ts=2026-09-22T10:49:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=0 paths=1 nar_bytes_in=5044632worker2 # [ 42.187835] nix-grpc-daemon[787]: ts=2026-09-22T10:49:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556244633worker2 # [ 42.194794] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4634worker2 # [ 42.204423] nix-daemon[856]: reaped child process 867, status = succeeded4635client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4636client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4637worker2 # [ 42.214094] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4638client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4639client # farm-a> LOG-farm-a-t14640worker2 # [ 43.443370] nix-daemon[856]: reaped child process 894, status = succeeded4641worker2 # [ 43.581807] nix-daemon[856]: accepted connection from pid 911, user nix-grpc-daemon (trusted)4642worker2 # [ 43.594689] nix-daemon[856]: reaped child process 914, status = succeeded4643lb # [ 43.609239] niks3-server[1004]: 2026/09/22 10:49:08 INFO Received uploads request method=POST path=/api/pending_closures4644worker2 # [ 43.619721] nix-grpc-daemon[869]: time=2026-09-22T10:49:08.577Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4645worker2 # [ 43.621369] nix-grpc-daemon[869]: time=2026-09-22T10:49:08.579Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4646lb # [ 43.645401] niks3-server[1004]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4647lb # [ 43.659992] niks3-server[1004]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4648lb # [ 43.673132] niks3-server[1004]: 2026/09/22 10:49:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4649worker2 # [ 43.679174] nix-grpc-daemon[869]: time=2026-09-22T10:49:08.637Z level=INFO msg="Uploading 1 narinfos"4650lb # [ 43.678844] niks3-server[1004]: 2026/09/22 10:49:08 INFO Signed narinfos id=2 count=14651lb # [ 43.683199] niks3-server[1004]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4652lb # [ 43.693552] niks3-server[1004]: 2026/09/22 10:49:08 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4653lb # [ 43.696457] niks3-server[1004]: 2026/09/22 10:49:08 INFO Completed upload id=24654worker2 # [ 43.696916] nix-grpc-daemon[869]: time=2026-09-22T10:49:08.655Z level=INFO msg="Upload complete. (271ms)"4655lb # [ 43.698367] niks3-server[1004]: 2026/09/22 10:49:08 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4656worker2 # [ 43.701522] nix-grpc-daemon[787]: ts=2026-09-22T10:49:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14657worker2 # [ 43.712986] nix-daemon[856]: reaped child process 892, status = succeeded4658worker1 # [ 43.722096] nix-grpc-daemon[784]: ts=2026-09-22T10:49:08Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04659worker1 # [ 43.726200] nix-grpc-daemon[784]: ts=2026-09-22T10:49:08Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14660worker2 # [ 43.764463] nix-grpc-daemon[787]: ts=2026-09-22T10:49:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556244661worker2 # [ 43.777254] nix-grpc-daemon[787]: ts=2026-09-22T10:49:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556244662worker2 # [ 43.790283] nix-grpc-daemon[869]: time=2026-09-22T10:49:08.748Z level=INFO msg="All 1 paths already cached"4663client # copying 1 paths to 'lb:50051'...4664worker2 # [ 43.798301] nix-grpc-daemon[787]: ts=2026-09-22T10:49:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556244665client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4666worker2 # [ 43.818161] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4667worker2 # [ 43.979093] nix-daemon[856]: accepted connection from pid 918, user nix-grpc-daemon (trusted)4668worker2 # [ 43.993667] nix-daemon[856]: reaped child process 921, status = succeeded4669worker2 # [ 44.115572] nix-daemon[856]: accepted connection from pid 923, user nix-grpc-daemon (trusted)4670worker2 # [ 44.128942] nix-daemon[856]: reaped child process 926, status = succeeded4671lb # [ 44.137208] niks3-server[1004]: 2026/09/22 10:49:09 INFO Received uploads request method=POST path=/api/pending_closures4672worker2 # [ 44.147433] nix-grpc-daemon[869]: time=2026-09-22T10:49:09.105Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4673worker2 # [ 44.148888] nix-grpc-daemon[869]: time=2026-09-22T10:49:09.105Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4674lb # [ 44.166497] niks3-server[1004]: 2026/09/22 10:49:09 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4675lb # [ 44.174741] niks3-server[1004]: 2026/09/22 10:49:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4676worker2 # [ 44.180077] nix-grpc-daemon[869]: time=2026-09-22T10:49:09.138Z level=INFO msg="Uploading 1 narinfos"4677lb # [ 44.180140] niks3-server[1004]: 2026/09/22 10:49:09 INFO Signed narinfos id=3 count=14678lb # [ 44.186710] niks3-server[1004]: 2026/09/22 10:49:09 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4679lb # [ 44.197495] niks3-server[1004]: 2026/09/22 10:49:09 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4680lb # [ 44.205079] niks3-server[1004]: 2026/09/22 10:49:09 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4681lb # [ 44.210488] niks3-server[1004]: 2026/09/22 10:49:09 INFO Completed upload id=34682worker2 # [ 44.210854] nix-grpc-daemon[869]: time=2026-09-22T10:49:09.169Z level=INFO msg="Upload complete. (378ms)"4683worker2 # [ 44.214488] nix-grpc-daemon[787]: ts=2026-09-22T10:49:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=0 paths=1 nar_bytes_in=6244684worker2 # [ 44.220273] nix-grpc-daemon[787]: ts=2026-09-22T10:49:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556244685worker2 # [ 44.228784] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4686worker2 # [ 44.237449] nix-daemon[856]: reaped child process 916, status = succeeded4687client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4688worker2 # [ 44.241687] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4689client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4690client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4691client # farm-b> LOG-farm-b-t14692worker2 # [ 45.457240] nix-daemon[856]: reaped child process 930, status = succeeded4693worker2 # [ 45.581986] nix-daemon[856]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4694worker2 # [ 45.594918] nix-daemon[856]: reaped child process 949, status = succeeded4695lb # [ 45.610767] niks3-server[1004]: 2026/09/22 10:49:10 INFO Received uploads request method=POST path=/api/pending_closures4696worker2 # [ 45.622627] nix-grpc-daemon[869]: time=2026-09-22T10:49:10.580Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4697worker2 # [ 45.624083] nix-grpc-daemon[869]: time=2026-09-22T10:49:10.582Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4698lb # [ 45.646381] niks3-server[1004]: 2026/09/22 10:49:10 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4699lb # [ 45.655143] niks3-server[1004]: 2026/09/22 10:49:10 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4700lb # [ 45.666569] niks3-server[1004]: 2026/09/22 10:49:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4701worker2 # [ 45.670802] nix-grpc-daemon[869]: time=2026-09-22T10:49:10.629Z level=INFO msg="Uploading 1 narinfos"4702lb # [ 45.670577] niks3-server[1004]: 2026/09/22 10:49:10 INFO Signed narinfos id=4 count=14703lb # [ 45.674809] niks3-server[1004]: 2026/09/22 10:49:10 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4704lb # [ 45.682152] niks3-server[1004]: 2026/09/22 10:49:10 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4705lb # [ 45.686347] niks3-server[1004]: 2026/09/22 10:49:10 INFO Completed upload id=44706worker2 # [ 45.686712] nix-grpc-daemon[869]: time=2026-09-22T10:49:10.645Z level=INFO msg="Upload complete. (238ms)"4707lb # [ 45.687829] niks3-server[1004]: 2026/09/22 10:49:10 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4708worker2 # [ 45.692890] nix-grpc-daemon[787]: ts=2026-09-22T10:49:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14709worker1 # [ 45.700842] nix-grpc-daemon[784]: ts=2026-09-22T10:49:10Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04710worker2 # [ 45.705244] nix-daemon[856]: reaped child process 928, status = succeeded4711worker1 # [ 45.705182] nix-grpc-daemon[784]: ts=2026-09-22T10:49:10Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14712worker2 # [ 45.748511] nix-grpc-daemon[787]: ts=2026-09-22T10:49:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556244713worker2 # [ 45.760362] nix-grpc-daemon[787]: ts=2026-09-22T10:49:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556244714worker2 # [ 45.773496] nix-grpc-daemon[869]: time=2026-09-22T10:49:10.731Z level=INFO msg="All 2 paths already cached"4715client # copying 1 paths to 'lb:50051'...4716worker2 # [ 45.781528] nix-grpc-daemon[787]: ts=2026-09-22T10:49:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556244717client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4718worker2 # [ 45.800797] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4719worker2 # [ 45.969554] nix-daemon[856]: accepted connection from pid 953, user nix-grpc-daemon (trusted)4720worker2 # [ 45.982478] nix-daemon[856]: reaped child process 956, status = succeeded4721worker2 # [ 46.137064] nix-daemon[856]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4722worker2 # [ 46.149621] nix-daemon[856]: reaped child process 961, status = succeeded4723lb # [ 46.163729] niks3-server[1004]: 2026/09/22 10:49:11 INFO Received uploads request method=POST path=/api/pending_closures4724worker2 # [ 46.173553] nix-grpc-daemon[869]: time=2026-09-22T10:49:11.131Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4725worker2 # [ 46.175429] nix-grpc-daemon[869]: time=2026-09-22T10:49:11.133Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4726lb # [ 46.192306] niks3-server[1004]: 2026/09/22 10:49:11 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4727lb # [ 46.203807] niks3-server[1004]: 2026/09/22 10:49:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4728lb # [ 46.209478] niks3-server[1004]: 2026/09/22 10:49:11 INFO Signed narinfos id=5 count=14729worker2 # [ 46.212931] nix-grpc-daemon[869]: time=2026-09-22T10:49:11.171Z level=INFO msg="Uploading 1 narinfos"4730lb # [ 46.216416] niks3-server[1004]: 2026/09/22 10:49:11 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4731lb # [ 46.226402] niks3-server[1004]: 2026/09/22 10:49:11 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4732lb # [ 46.231363] niks3-server[1004]: 2026/09/22 10:49:11 INFO Completed upload id=54733worker2 # [ 46.234059] nix-grpc-daemon[869]: time=2026-09-22T10:49:11.192Z level=INFO msg="Upload complete. (418ms)"4734lb # [ 46.235477] niks3-server[1004]: 2026/09/22 10:49:11 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4735worker2 # [ 46.238123] nix-grpc-daemon[787]: ts=2026-09-22T10:49:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=0 paths=1 nar_bytes_in=7524736worker2 # [ 46.243222] nix-grpc-daemon[787]: ts=2026-09-22T10:49:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556244737worker2 # [ 46.253693] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4738worker2 # [ 46.259118] nix-daemon[856]: reaped child process 951, status = succeeded4739client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4740client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4741worker2 # [ 46.262313] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4742client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4743client # farm-top> LOG-farm-top-t14744worker2 # [ 47.278060] nix-daemon[856]: reaped child process 965, status = succeeded4745worker2 # [ 47.422570] nix-daemon[856]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4746worker2 # [ 47.435510] nix-daemon[856]: reaped child process 984, status = succeeded4747lb # [ 47.450825] niks3-server[1004]: 2026/09/22 10:49:12 INFO Received uploads request method=POST path=/api/pending_closures4748worker2 # [ 47.459211] nix-grpc-daemon[869]: time=2026-09-22T10:49:12.417Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4749worker2 # [ 47.460676] nix-grpc-daemon[869]: time=2026-09-22T10:49:12.419Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4750lb # [ 47.488735] niks3-server[1004]: 2026/09/22 10:49:12 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4751lb # [ 47.492979] niks3-server[1004]: 2026/09/22 10:49:12 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4752lb # [ 47.496034] niks3-server[1004]: 2026/09/22 10:49:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4753lb # [ 47.498434] niks3-server[1004]: 2026/09/22 10:49:12 INFO Signed narinfos id=6 count=14754worker2 # [ 47.500441] nix-grpc-daemon[869]: time=2026-09-22T10:49:12.458Z level=INFO msg="Uploading 1 narinfos"4755lb # [ 47.503054] niks3-server[1004]: 2026/09/22 10:49:12 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4756lb # [ 47.507244] niks3-server[1004]: 2026/09/22 10:49:12 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4757lb # [ 47.510655] niks3-server[1004]: 2026/09/22 10:49:12 INFO Completed upload id=64758worker2 # [ 47.512148] nix-grpc-daemon[869]: time=2026-09-22T10:49:12.470Z level=INFO msg="Upload complete. (241ms)"4759lb # [ 47.512603] niks3-server[1004]: 2026/09/22 10:49:12 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4760worker2 # [ 47.514556] nix-grpc-daemon[787]: ts=2026-09-22T10:49:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14761worker2 # [ 47.521907] nix-daemon[856]: reaped child process 963, status = succeeded4762client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.58 seconds)4763client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4764client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.19 seconds)4765client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4766client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4767client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4768client # copying 3 paths...4769client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4770client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4771client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4772client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.28 seconds)4773(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.14 seconds)4774subtest: repeat is answered Cached by the scheduler without building4775worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4776lb # [ 48.138731] envoy[920]: conn peer=[2001:db8:1::1]:55180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=24944777lb # [ 48.143466] envoy[920]: conn peer=[2001:db8:1::1]:55184 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=363 ms=504778lb # [ 48.147496] envoy[920]: conn peer=[2001:db8:1::1]:55186 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114779lb # [ 48.151472] envoy[920]: conn peer=[2001:db8:1::1]:55190 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=84780lb # [ 48.155698] envoy[920]: conn peer=[2001:db8:1::1]:55192 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9524 tx=4096 ms=63584781worker1 # finding garbage collector roots...4782worker1 # deleting specified paths...4783worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4784worker1 # deleting unused links...4785worker1 # note: hard linking is currently saving 0.0 KiB4786worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.27 seconds)4787worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4788worker1 # [ 48.428409] nix-grpc-daemon[784]: E0000 00:00:1790074153.433958 823 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4789worker1 # [ 48.440060] nix-grpc-daemon[784]: E0000 00:00:1790074153.446520 823 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4790worker1 # [ 48.442254] nix-grpc-daemon[784]: E0000 00:00:1790074153.446956 823 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4791worker1 # [ 48.453052] nix-grpc-daemon[784]: E0000 00:00:1790074153.460192 823 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4792worker2 # finding garbage collector roots...4793worker2 # deleting specified paths...4794worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4795worker2 # deleting unused links...4796worker2 # note: hard linking is currently saving 0.0 KiB4797worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4798client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24799client # don't know how to build these paths:4800client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4801worker1 # [ 48.859086] nix-grpc-daemon[784]: ts=2026-09-22T10:49:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:422424802worker1 # [ 48.864135] nix-grpc-daemon[784]: ts=2026-09-22T10:49:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422424803worker1 # [ 48.892146] nix-grpc-daemon[784]: ts=2026-09-22T10:49:13Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14804client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.29 seconds)4805client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4806client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.18 seconds)4807lb: must succeed: curl -sf localhost:9901/clusters4808lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4809lb: must succeed: curl -sf localhost:9901/clusters4810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4811worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4812worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4813(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.23 seconds)4814subtest: inputs already in the cache are not sent to the scheduler4815lb: must succeed: curl -sf localhost:9901/clusters4816lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4817lb: must succeed: curl -sf localhost:9901/clusters4818lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4819worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4820worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4821client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24822client # don't know how to build these paths:4823worker2 # [ 49.693105] nix-grpc-daemon[787]: ts=2026-09-22T10:49:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:556244824client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4825worker2 # [ 49.697060] nix-grpc-daemon[787]: ts=2026-09-22T10:49:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556244826worker2 # [ 49.702821] nix-grpc-daemon[869]: time=2026-09-22T10:49:14.661Z level=INFO msg="All 2 paths already cached"4827worker1 # [ 49.713462] nix-grpc-daemon[784]: ts=2026-09-22T10:49:14Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04828worker1 # [ 49.715509] nix-grpc-daemon[784]: ts=2026-09-22T10:49:14Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14829worker2 # [ 49.754915] nix-grpc-daemon[787]: ts=2026-09-22T10:49:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556244830worker2 # [ 49.766235] nix-grpc-daemon[787]: ts=2026-09-22T10:49:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556244831worker2 # [ 49.778102] nix-grpc-daemon[869]: time=2026-09-22T10:49:14.736Z level=INFO msg="All 2 paths already cached"4832client # copying 1 paths to 'lb:50051'...4833client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4834worker2 # [ 49.786158] nix-grpc-daemon[787]: ts=2026-09-22T10:49:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556244835worker2 # [ 49.799191] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4836worker2 # [ 49.957689] nix-daemon[856]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4837worker2 # [ 49.971407] nix-daemon[856]: reaped child process 1004, status = succeeded4838worker2 # [ 50.108314] nix-daemon[856]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4839worker2 # [ 50.121036] nix-daemon[856]: reaped child process 1009, status = succeeded4840lb # [ 50.136875] niks3-server[1004]: 2026/09/22 10:49:15 INFO Received uploads request method=POST path=/api/pending_closures4841worker2 # [ 50.144760] nix-grpc-daemon[869]: time=2026-09-22T10:49:15.103Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4842worker2 # [ 50.146266] nix-grpc-daemon[869]: time=2026-09-22T10:49:15.104Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4843lb # [ 50.163408] niks3-server[1004]: 2026/09/22 10:49:15 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4844lb # [ 50.174874] niks3-server[1004]: 2026/09/22 10:49:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4845lb # [ 50.180113] niks3-server[1004]: 2026/09/22 10:49:15 INFO Signed narinfos id=7 count=14846worker2 # [ 50.183407] nix-grpc-daemon[869]: time=2026-09-22T10:49:15.141Z level=INFO msg="Uploading 1 narinfos"4847lb # [ 50.186225] niks3-server[1004]: 2026/09/22 10:49:15 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4848lb # [ 50.197282] niks3-server[1004]: 2026/09/22 10:49:15 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4849worker2 # [ 50.203440] nix-grpc-daemon[869]: time=2026-09-22T10:49:15.161Z level=INFO msg="Upload complete. (392ms)"4850lb # [ 50.202967] niks3-server[1004]: 2026/09/22 10:49:15 INFO Completed upload id=74851worker2 # [ 50.205582] nix-grpc-daemon[787]: ts=2026-09-22T10:49:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=0 paths=1 nar_bytes_in=7524852lb # [ 50.208225] niks3-server[1004]: 2026/09/22 10:49:15 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4853worker2 # [ 50.210417] nix-grpc-daemon[787]: ts=2026-09-22T10:49:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556244854worker2 # [ 50.220060] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4855worker2 # [ 50.224291] nix-daemon[856]: reaped child process 999, status = succeeded4856client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4857client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4858worker2 # [ 50.229267] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4859client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4860client # farm-top> LOG-farm-top-t1b4861worker2 # [ 51.242746] nix-daemon[856]: reaped child process 1013, status = succeeded4862worker2 # [ 51.370839] nix-daemon[856]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)4863worker2 # [ 51.384257] nix-daemon[856]: reaped child process 1032, status = succeeded4864lb # [ 51.394505] niks3-server[1004]: 2026/09/22 10:49:16 INFO Received uploads request method=POST path=/api/pending_closures4865worker2 # [ 51.399284] nix-grpc-daemon[869]: time=2026-09-22T10:49:16.357Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4866worker2 # [ 51.400771] nix-grpc-daemon[869]: time=2026-09-22T10:49:16.359Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4867lb # [ 51.414070] niks3-server[1004]: 2026/09/22 10:49:16 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4868lb # [ 51.418796] niks3-server[1004]: 2026/09/22 10:49:16 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4869lb # [ 51.422660] niks3-server[1004]: 2026/09/22 10:49:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4870lb # [ 51.424603] niks3-server[1004]: 2026/09/22 10:49:16 INFO Signed narinfos id=8 count=14871worker2 # [ 51.426094] nix-grpc-daemon[869]: time=2026-09-22T10:49:16.384Z level=INFO msg="Uploading 1 narinfos"4872lb # [ 51.428502] niks3-server[1004]: 2026/09/22 10:49:16 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4873lb # [ 51.432508] niks3-server[1004]: 2026/09/22 10:49:16 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4874lb # [ 51.436364] niks3-server[1004]: 2026/09/22 10:49:16 INFO Completed upload id=84875worker2 # [ 51.437921] nix-grpc-daemon[869]: time=2026-09-22T10:49:16.396Z level=INFO msg="Upload complete. (202ms)"4876lb # [ 51.438235] niks3-server[1004]: 2026/09/22 10:49:16 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4877worker2 # [ 51.441171] nix-grpc-daemon[787]: ts=2026-09-22T10:49:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14878worker2 # [ 51.447211] nix-daemon[856]: reaped child process 1011, status = succeeded4879client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.00 seconds)4880client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4881client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.16 seconds)4882lb: must succeed: curl -sf localhost:9901/clusters4883lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4884lb: must succeed: curl -sf localhost:9901/clusters4885lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4886worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4887worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4888(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.42 seconds)4889subtest: two clients wanting the same drv build it once4890client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4891client # [ 51.813631] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4892client # Running as unit: dedup1.service; invocation ID: a9582bdb1e934b72aa1aa341fae3bd324893client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.06 seconds)4894client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4895client # [ 51.994330] nix[896]: don't know how to build these paths:4896client # [ 51.995241] nix[896]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4897worker1 # [ 51.986573] nix-grpc-daemon[784]: ts=2026-09-22T10:49:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:422424898worker1 # [ 51.998281] nix-grpc-daemon[784]: ts=2026-09-22T10:49:17Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04899worker1 # [ 52.004929] nix-grpc-daemon[784]: ts=2026-09-22T10:49:17Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14900worker1 # [ 52.009705] nix-grpc-daemon[784]: ts=2026-09-22T10:49:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422424901worker1 # [ 52.023426] nix-grpc-daemon[784]: ts=2026-09-22T10:49:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422424902client # [ 52.041132] nix[896]: copying 1 paths to 'lb:50051'...4903client # [ 52.041927] nix[896]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4904worker1 # [ 52.034929] nix-grpc-daemon[784]: ts=2026-09-22T10:49:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422424905worker1 # [ 52.046347] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4906worker1 # [ 52.067360] nix-grpc-daemon[784]: ts=2026-09-22T10:49:17Z level=info event=niks3_push_started pid=9194907worker1 # [ 52.296226] nix-grpc-daemon[919]: time=2026-09-22T10:49:17.300Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""4908worker1 # [ 52.455849] nix-daemon[867]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4909worker1 # [ 52.470140] nix-daemon[867]: reaped child process 935, status = succeeded4910worker1 # [ 52.629633] nix-daemon[867]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4911worker1 # [ 52.641736] nix-daemon[867]: reaped child process 940, status = succeeded4912lb # [ 52.655481] niks3-server[1004]: 2026/09/22 10:49:17 INFO Received uploads request method=POST path=/api/pending_closures4913worker1 # [ 52.666259] nix-grpc-daemon[919]: time=2026-09-22T10:49:17.673Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4914worker1 # [ 52.668356] nix-grpc-daemon[919]: time=2026-09-22T10:49:17.675Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4915lb # [ 52.688870] niks3-server[1004]: 2026/09/22 10:49:17 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4916lb # [ 52.700934] niks3-server[1004]: 2026/09/22 10:49:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4917lb # [ 52.705846] niks3-server[1004]: 2026/09/22 10:49:17 INFO Signed narinfos id=9 count=14918worker1 # [ 52.711897] nix-grpc-daemon[919]: time=2026-09-22T10:49:17.719Z level=INFO msg="Uploading 1 narinfos"4919lb # [ 52.719551] niks3-server[1004]: 2026/09/22 10:49:17 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4920lb # [ 52.726846] niks3-server[1004]: 2026/09/22 10:49:17 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4921worker1 # [ 52.736339] nix-grpc-daemon[919]: time=2026-09-22T10:49:17.743Z level=INFO msg="Upload complete. (430ms)"4922lb # [ 52.733961] niks3-server[1004]: 2026/09/22 10:49:17 INFO Completed upload id=94923worker1 # [ 52.738728] nix-grpc-daemon[784]: ts=2026-09-22T10:49:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=0 paths=1 nar_bytes_in=4324924worker1 # [ 52.741751] nix-grpc-daemon[784]: ts=2026-09-22T10:49:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422424925worker1 # [ 52.746788] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4926worker1 # [ 52.751659] nix-daemon[867]: reaped child process 917, status = succeeded4927worker1 # [ 52.757104] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4928client # [ 52.763171] nix[896]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4929client # [ 52.885605] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4930client # Running as unit: dedup2.service; invocation ID: dad201a721fe4a34890856d4684eed314931client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)4932worker2 # [ 53.133537] nix-grpc-daemon[787]: ts=2026-09-22T10:49:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:556244933client # [ 53.148337] nix[912]: don't know how to build these paths:4934client # [ 53.149205] nix[912]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4935worker1 # [ 53.150057] nix-grpc-daemon[784]: ts=2026-09-22T10:49:18Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04936worker1 # [ 53.153938] nix-grpc-daemon[784]: ts=2026-09-22T10:49:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422424937client # [ 53.163300] nix[912]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4938client: must succeed: sleep 24939client: (finished: must succeed: sleep 2, in 2.05 seconds)4940client: waiting for success: ! systemctl is-active dedup1 dedup24941worker1 # [ 56.384350] nix-daemon[867]: reaped child process 944, status = succeeded4942worker1 # [ 56.520078] nix-daemon[867]: accepted connection from pid 976, user nix-grpc-daemon (trusted)4943worker1 # [ 56.532418] nix-daemon[867]: reaped child process 979, status = succeeded4944lb # [ 56.540208] niks3-server[1004]: 2026/09/22 10:49:21 INFO Received uploads request method=POST path=/api/pending_closures4945worker1 # [ 56.556522] nix-grpc-daemon[919]: time=2026-09-22T10:49:21.563Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4946worker1 # [ 56.558094] nix-grpc-daemon[919]: time=2026-09-22T10:49:21.565Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4947lb # [ 56.600633] niks3-server[1004]: 2026/09/22 10:49:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4948lb # [ 56.606390] niks3-server[1004]: 2026/09/22 10:49:21 INFO Signed narinfos id=10 count=14949worker1 # [ 56.612106] nix-grpc-daemon[919]: time=2026-09-22T10:49:21.618Z level=INFO msg="Uploading 1 narinfos"4950lb # [ 56.618298] niks3-server[1004]: 2026/09/22 10:49:21 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4951lb # [ 56.670602] niks3-server[1004]: 2026/09/22 10:49:21 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4952lb # [ 56.676606] niks3-server[1004]: 2026/09/22 10:49:21 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4953lb # [ 56.681861] niks3-server[1004]: 2026/09/22 10:49:21 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4954lb # [ 56.695771] niks3-server[1004]: 2026/09/22 10:49:21 INFO Completed upload id=104955worker1 # [ 56.698439] nix-grpc-daemon[919]: time=2026-09-22T10:49:21.705Z level=INFO msg="Upload complete. (329ms)"4956lb # [ 56.698892] niks3-server[1004]: 2026/09/22 10:49:21 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4957worker1 # [ 56.707611] nix-grpc-daemon[784]: ts=2026-09-22T10:49:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14958worker1 # [ 56.714507] nix-grpc-daemon[784]: ts=2026-09-22T10:49:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14959worker1 # [ 56.727140] nix-daemon[867]: reaped child process 942, status = succeeded4960client # [ 56.752582] systemd[1]: dedup2.service: Deactivated successfully.4961client # [ 56.757337] systemd[1]: dedup2.service: Consumed 143ms CPU time over 3.870s wall clock time, 33.1M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4962client # [ 56.799111] systemd[1]: dedup1.service: Deactivated successfully.4963client # [ 56.800631] systemd[1]: dedup1.service: Consumed 122ms CPU time over 4.986s wall clock time, 43.5M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.4964client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.11 seconds)4965client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4966client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4967worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4968worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4969worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4970worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4971worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4972worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4973worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4974worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4975(finished: subtest: two clients wanting the same drv build it once, in 5.91 seconds)4976subtest: the first client leaving does not fail the second4977client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share4978client # [ 57.727888] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share.4979client # Running as unit: share1.service; invocation ID: 7d126db04325441ab8e764729426a4744980client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share, in 0.06 seconds)4981client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share4982worker1 # [ 57.904838] nix-grpc-daemon[784]: ts=2026-09-22T10:49:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:422424983client # [ 57.917714] nix[951]: don't know how to build these paths:4984client # [ 57.918562] nix[951]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv4985worker1 # [ 57.920985] nix-grpc-daemon[784]: ts=2026-09-22T10:49:22Z level=debug event=want client=6 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=04986worker1 # [ 57.925191] nix-grpc-daemon[784]: ts=2026-09-22T10:49:22Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=14987worker2 # [ 57.963612] nix-grpc-daemon[787]: ts=2026-09-22T10:49:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556244988worker2 # [ 57.976392] nix-grpc-daemon[787]: ts=2026-09-22T10:49:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556244989client # [ 57.995296] nix[951]: copying 1 paths to 'lb:50051'...4990worker2 # [ 57.990062] nix-grpc-daemon[787]: ts=2026-09-22T10:49:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556244991client # [ 58.004151] nix[951]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...4992worker2 # [ 58.003152] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)4993lb # [ 58.138711] envoy[920]: conn peer=[2001:db8:1::1]:45286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=524994lb # [ 58.142480] envoy[920]: conn peer=[2001:db8:1::1]:45292 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4195 tx=1807 ms=17584995lb # [ 58.147249] envoy[920]: conn peer=[2001:db8:1::1]:37372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=35944996lb # [ 58.150850] envoy[920]: conn peer=[2001:db8:1::1]:45302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3011 tx=1448 ms=47814997worker2 # [ 58.169168] nix-daemon[856]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)4998worker2 # [ 58.184621] nix-daemon[856]: reaped child process 1060, status = succeeded4999worker2 # [ 58.330447] nix-daemon[856]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)5000worker2 # [ 58.343382] nix-daemon[856]: reaped child process 1065, status = succeeded5001lb # [ 58.359382] niks3-server[1004]: 2026/09/22 10:49:23 INFO Received uploads request method=POST path=/api/pending_closures5002worker2 # [ 58.367915] nix-grpc-daemon[869]: time=2026-09-22T10:49:23.325Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5003worker2 # [ 58.369409] nix-grpc-daemon[869]: time=2026-09-22T10:49:23.325Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5004lb # [ 58.383477] niks3-server[1004]: 2026/09/22 10:49:23 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5005lb # [ 58.396535] niks3-server[1004]: 2026/09/22 10:49:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign5006lb # [ 58.403824] niks3-server[1004]: 2026/09/22 10:49:23 INFO Signed narinfos id=11 count=15007worker2 # [ 58.407316] nix-grpc-daemon[869]: time=2026-09-22T10:49:23.365Z level=INFO msg="Uploading 1 narinfos"5008lb # [ 58.409053] niks3-server[1004]: 2026/09/22 10:49:23 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5009lb # [ 58.421703] niks3-server[1004]: 2026/09/22 10:49:23 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete5010lb # [ 58.428780] niks3-server[1004]: 2026/09/22 10:49:23 INFO Completed upload id=115011worker2 # [ 58.429296] nix-grpc-daemon[869]: time=2026-09-22T10:49:23.387Z level=INFO msg="Upload complete. (414ms)"5012worker2 # [ 58.432068] nix-grpc-daemon[787]: ts=2026-09-22T10:49:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=0 paths=1 nar_bytes_in=4325013lb # [ 58.432565] niks3-server[1004]: 2026/09/22 10:49:23 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5014worker2 # [ 58.436784] nix-grpc-daemon[787]: ts=2026-09-22T10:49:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556245015worker2 # [ 58.442929] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5016worker2 # [ 58.445467] nix-daemon[856]: reaped child process 1055, status = succeeded5017worker2 # [ 58.450572] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5018client # [ 58.459252] nix[951]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5019client # [ 58.802719] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share.5020client # Running as unit: share2.service; invocation ID: b5ee4fd8bed6431d99bb5a90b047d4165021client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag share, in 1.08 seconds)5022worker1 # [ 59.015274] nix-grpc-daemon[784]: ts=2026-09-22T10:49:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:422425023client # [ 59.027749] nix[967]: don't know how to build these paths:5024client # [ 59.028634] nix[967]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5025worker1 # [ 59.030151] nix-grpc-daemon[784]: ts=2026-09-22T10:49:24Z level=debug event=want client=7 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05026client # [ 59.042217] nix[967]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5027worker2 # [ 59.032634] nix-grpc-daemon[787]: ts=2026-09-22T10:49:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556245028client: must succeed: sleep 2; systemctl kill -s INT share15029client # [ 62.136608] systemd[1]: share1.service: Sent signal SIGINT to main process 951 (nix) on client request.5030client # [ 62.142686] nix[951]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5031client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.07 seconds)5032client: waiting for success: ! systemctl is-active share15033client # [ 62.151706] nix[951]: error: interrupted by the user5034client # [ 62.197328] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5035client # [ 62.198845] systemd[1]: share1.service: Failed with result 'exit-code'.5036client # [ 62.201619] systemd[1]: share1.service: Consumed 126ms CPU time over 4.472s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8.3K outgoing IP traffic.5037client: (finished: waiting for success: ! systemctl is-active share1, in 1.11 seconds)5038client: waiting for success: ! systemctl is-active share25039worker2 # [ 63.669141] nix-daemon[856]: reaped child process 1069, status = succeeded5040worker2 # [ 63.811552] nix-daemon[856]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)5041worker2 # [ 63.824385] nix-daemon[856]: reaped child process 1106, status = succeeded5042lb # [ 63.841367] niks3-server[1004]: 2026/09/22 10:49:28 INFO Received uploads request method=POST path=/api/pending_closures5043worker2 # [ 63.850219] nix-grpc-daemon[869]: time=2026-09-22T10:49:28.808Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5044worker2 # [ 63.851759] nix-grpc-daemon[869]: time=2026-09-22T10:49:28.810Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5045lb # [ 63.881168] niks3-server[1004]: 2026/09/22 10:49:28 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5046lb # [ 63.894677] niks3-server[1004]: 2026/09/22 10:49:28 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5047lb # [ 63.906314] niks3-server[1004]: 2026/09/22 10:49:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5048lb # [ 63.911110] niks3-server[1004]: 2026/09/22 10:49:28 INFO Signed narinfos id=12 count=15049worker2 # [ 63.911119] nix-grpc-daemon[869]: time=2026-09-22T10:49:28.869Z level=INFO msg="Uploading 1 narinfos"5050lb # [ 63.916109] niks3-server[1004]: 2026/09/22 10:49:28 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5051lb # [ 63.926226] niks3-server[1004]: 2026/09/22 10:49:28 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5052lb # [ 63.929580] niks3-server[1004]: 2026/09/22 10:49:28 INFO Completed upload id=125053worker2 # [ 63.930232] nix-grpc-daemon[869]: time=2026-09-22T10:49:28.888Z level=INFO msg="Upload complete. (269ms)"5054lb # [ 63.931943] niks3-server[1004]: 2026/09/22 10:49:28 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5055worker2 # [ 63.937135] nix-grpc-daemon[787]: ts=2026-09-22T10:49:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=6 outputs=15056worker2 # [ 63.942815] nix-grpc-daemon[787]: ts=2026-09-22T10:49:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=6 outputs=15057worker2 # [ 63.954688] nix-daemon[856]: reaped child process 1067, status = succeeded5058client # [ 63.982739] systemd[1]: share2.service: Deactivated successfully.5059client # [ 63.986463] systemd[1]: share2.service: Consumed 124ms CPU time over 5.183s wall clock time, 35.4M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.5060client: (finished: waiting for success: ! systemctl is-active share2, in 1.09 seconds)5061client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5062client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5063(finished: subtest: the first client leaving does not fail the second, in 6.94 seconds)5064subtest: build hook: nix-daemon with builders = grpc://lb5065client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log5066client # [ 64.784857] systemd[1]: Started Nix Daemon.5067client # [ 64.960236] nix-daemon[1019]: accepted connection from pid 1016, user root (trusted)5068worker2 # [ 65.164354] nix-grpc-daemon[787]: ts=2026-09-22T10:49:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:556245069worker2 # [ 65.169550] nix-grpc-daemon[787]: ts=2026-09-22T10:49:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556245070worker1 # [ 65.183818] nix-grpc-daemon[784]: ts=2026-09-22T10:49:30Z level=debug event=want client=8 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05071worker1 # [ 65.189592] nix-grpc-daemon[784]: ts=2026-09-22T10:49:30Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15072worker1 # [ 65.196254] nix-grpc-daemon[784]: ts=2026-09-22T10:49:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422425073worker1 # [ 65.206893] nix-grpc-daemon[784]: ts=2026-09-22T10:49:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422425074worker1 # [ 65.217706] nix-grpc-daemon[784]: ts=2026-09-22T10:49:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422425075worker1 # [ 65.228734] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5076worker1 # [ 65.401668] nix-daemon[867]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)5077worker1 # [ 65.414759] nix-daemon[867]: reaped child process 1017, status = succeeded5078worker1 # [ 65.577294] nix-daemon[867]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)5079worker1 # [ 65.588114] nix-daemon[867]: reaped child process 1022, status = succeeded5080lb # [ 65.600173] niks3-server[1004]: 2026/09/22 10:49:30 INFO Received uploads request method=POST path=/api/pending_closures5081worker1 # [ 65.610488] nix-grpc-daemon[919]: time=2026-09-22T10:49:30.617Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5082worker1 # [ 65.612655] nix-grpc-daemon[919]: time=2026-09-22T10:49:30.619Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5083lb # [ 65.627148] niks3-server[1004]: 2026/09/22 10:49:30 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5084lb # [ 65.639306] niks3-server[1004]: 2026/09/22 10:49:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5085worker1 # [ 65.646310] nix-grpc-daemon[919]: time=2026-09-22T10:49:30.653Z level=INFO msg="Uploading 1 narinfos"5086lb # [ 65.644493] niks3-server[1004]: 2026/09/22 10:49:30 INFO Signed narinfos id=13 count=15087lb # [ 65.649683] niks3-server[1004]: 2026/09/22 10:49:30 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5088lb # [ 65.662450] niks3-server[1004]: 2026/09/22 10:49:30 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5089worker1 # [ 65.671564] nix-grpc-daemon[919]: time=2026-09-22T10:49:30.678Z level=INFO msg="Upload complete. (431ms)"5090lb # [ 65.669238] niks3-server[1004]: 2026/09/22 10:49:30 INFO Completed upload id=135091lb # [ 65.672327] niks3-server[1004]: 2026/09/22 10:49:30 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5092worker1 # [ 65.674630] nix-grpc-daemon[784]: ts=2026-09-22T10:49:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=0 paths=1 nar_bytes_in=5125093worker1 # [ 65.680289] nix-grpc-daemon[784]: ts=2026-09-22T10:49:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422425094worker1 # [ 65.688141] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5095worker1 # [ 65.697132] nix-daemon[867]: reaped child process 1012, status = succeeded5096worker1 # [ 65.701185] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5097worker1 # [ 66.527894] nix-daemon[867]: reaped child process 1026, status = succeeded5098worker1 # [ 66.685094] nix-daemon[867]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)5099worker1 # [ 66.697651] nix-daemon[867]: reaped child process 1045, status = succeeded5100lb # [ 66.707575] niks3-server[1004]: 2026/09/22 10:49:31 INFO Received uploads request method=POST path=/api/pending_closures5101worker1 # [ 66.718148] nix-grpc-daemon[919]: time=2026-09-22T10:49:31.725Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5102worker1 # [ 66.719603] nix-grpc-daemon[919]: time=2026-09-22T10:49:31.726Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5103lb # [ 66.741715] niks3-server[1004]: 2026/09/22 10:49:31 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5104lb # [ 66.752748] niks3-server[1004]: 2026/09/22 10:49:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5105lb # [ 66.760415] niks3-server[1004]: 2026/09/22 10:49:31 INFO Signed narinfos id=14 count=15106worker1 # [ 66.766376] nix-grpc-daemon[919]: time=2026-09-22T10:49:31.773Z level=INFO msg="Uploading 1 narinfos"5107lb # [ 66.767316] niks3-server[1004]: 2026/09/22 10:49:31 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5108lb # [ 66.772058] niks3-server[1004]: 2026/09/22 10:49:31 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5109lb # [ 66.780153] niks3-server[1004]: 2026/09/22 10:49:31 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5110lb # [ 66.787131] niks3-server[1004]: 2026/09/22 10:49:31 INFO Completed upload id=145111worker1 # [ 66.789776] nix-grpc-daemon[919]: time=2026-09-22T10:49:31.796Z level=INFO msg="Upload complete. (280ms)"5112lb # [ 66.789583] niks3-server[1004]: 2026/09/22 10:49:31 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5113worker1 # [ 66.796670] nix-grpc-daemon[784]: ts=2026-09-22T10:49:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=7 outputs=15114worker1 # [ 66.809482] nix-daemon[867]: reaped child process 1024, status = succeeded5115worker2 # [ 66.840890] nix-grpc-daemon[787]: ts=2026-09-22T10:49:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:556245116worker2 # [ 66.860144] nix-grpc-daemon[787]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5117worker2 # [ 66.893620] nix-grpc-daemon[787]: ts=2026-09-22T10:49:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:556245118worker2 # [ 66.899384] nix-grpc-daemon[787]: ts=2026-09-22T10:49:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=0 paths=1 nar_bytes_out=1285119worker2 # [ 67.183216] nix-grpc-daemon[787]: ts=2026-09-22T10:49:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:556245120worker1 # [ 67.188877] nix-grpc-daemon[784]: ts=2026-09-22T10:49:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422425121worker1 # [ 67.199476] nix-grpc-daemon[919]: time=2026-09-22T10:49:32.206Z level=INFO msg="All 1 paths already cached"5122worker1 # [ 67.214134] nix-grpc-daemon[784]: ts=2026-09-22T10:49:32Z level=debug event=want client=9 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05123worker1 # [ 67.219369] nix-grpc-daemon[784]: ts=2026-09-22T10:49:32Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15124worker1 # [ 67.224961] nix-grpc-daemon[784]: ts=2026-09-22T10:49:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422425125worker1 # [ 67.236504] nix-grpc-daemon[784]: ts=2026-09-22T10:49:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422425126worker1 # [ 67.248559] nix-grpc-daemon[919]: time=2026-09-22T10:49:32.255Z level=INFO msg="All 1 paths already cached"5127worker1 # [ 67.255771] nix-grpc-daemon[784]: ts=2026-09-22T10:49:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422425128worker1 # [ 67.264284] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5129worker1 # [ 67.434619] nix-daemon[867]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)5130worker1 # [ 67.447215] nix-daemon[867]: reaped child process 1054, status = succeeded5131worker1 # [ 67.612357] nix-daemon[867]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)5132worker1 # [ 67.624921] nix-daemon[867]: reaped child process 1059, status = succeeded5133lb # [ 67.636738] niks3-server[1004]: 2026/09/22 10:49:32 INFO Received uploads request method=POST path=/api/pending_closures5134worker1 # [ 67.647137] nix-grpc-daemon[919]: time=2026-09-22T10:49:32.654Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5135worker1 # [ 67.648928] nix-grpc-daemon[919]: time=2026-09-22T10:49:32.655Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5136lb # [ 67.663844] niks3-server[1004]: 2026/09/22 10:49:32 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5137lb # [ 67.674919] niks3-server[1004]: 2026/09/22 10:49:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5138lb # [ 67.681439] niks3-server[1004]: 2026/09/22 10:49:32 INFO Signed narinfos id=15 count=15139worker1 # [ 67.686979] nix-grpc-daemon[919]: time=2026-09-22T10:49:32.694Z level=INFO msg="Uploading 1 narinfos"5140lb # [ 67.684811] niks3-server[1004]: 2026/09/22 10:49:32 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5141lb # [ 67.699552] niks3-server[1004]: 2026/09/22 10:49:32 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5142lb # [ 67.703959] niks3-server[1004]: 2026/09/22 10:49:32 INFO Completed upload id=155143worker1 # [ 67.706221] nix-grpc-daemon[919]: time=2026-09-22T10:49:32.713Z level=INFO msg="Upload complete. (427ms)"5144lb # [ 67.706553] niks3-server[1004]: 2026/09/22 10:49:32 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5145worker1 # [ 67.709722] nix-grpc-daemon[784]: ts=2026-09-22T10:49:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=0 paths=1 nar_bytes_in=6245146worker1 # [ 67.715542] nix-grpc-daemon[784]: ts=2026-09-22T10:49:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422425147worker1 # [ 67.724353] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5148worker1 # [ 67.728564] nix-daemon[867]: reaped child process 1049, status = succeeded5149worker1 # [ 67.733725] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5150lb # [ 68.138832] envoy[920]: conn peer=[2001:db8:1::1]:37378 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2982 tx=814 ms=42525151lb # [ 68.142416] envoy[920]: conn peer=[2001:db8:1::1]:37384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1085 ms=49365152lb # [ 68.145980] envoy[920]: conn peer=[2001:db8:1::1]:36450 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3395 tx=1851 ms=17885153lb # [ 68.149489] envoy[920]: conn peer=[2001:db8:1::1]:36462 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=595154worker1 # [ 69.159839] nix-daemon[867]: reaped child process 1063, status = succeeded5155worker1 # [ 69.316427] nix-daemon[867]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)5156worker1 # [ 69.328357] nix-daemon[867]: reaped child process 1082, status = succeeded5157lb # [ 69.343578] niks3-server[1004]: 2026/09/22 10:49:34 INFO Received uploads request method=POST path=/api/pending_closures5158worker1 # [ 69.354246] nix-grpc-daemon[919]: time=2026-09-22T10:49:34.361Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5159worker1 # [ 69.355731] nix-grpc-daemon[919]: time=2026-09-22T10:49:34.362Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5160lb # [ 69.382446] niks3-server[1004]: 2026/09/22 10:49:34 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5161lb # [ 69.392954] niks3-server[1004]: 2026/09/22 10:49:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5162worker1 # [ 69.401874] nix-grpc-daemon[919]: time=2026-09-22T10:49:34.408Z level=INFO msg="Uploading 1 narinfos"5163lb # [ 69.399046] niks3-server[1004]: 2026/09/22 10:49:34 INFO Signed narinfos id=16 count=15164lb # [ 69.406687] niks3-server[1004]: 2026/09/22 10:49:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5165lb # [ 69.410969] niks3-server[1004]: 2026/09/22 10:49:34 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5166lb # [ 69.422278] niks3-server[1004]: 2026/09/22 10:49:34 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5167lb # [ 69.428071] niks3-server[1004]: 2026/09/22 10:49:34 INFO Completed upload id=165168lb # [ 69.429925] niks3-server[1004]: 2026/09/22 10:49:34 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5169worker1 # [ 69.431914] nix-grpc-daemon[919]: time=2026-09-22T10:49:34.438Z level=INFO msg="Upload complete. (289ms)"5170worker1 # [ 69.440181] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=8 outputs=15171worker1 # [ 69.446575] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:422425172worker1 # [ 69.456268] nix-daemon[867]: reaped child process 1061, status = succeeded5173worker1 # [ 69.464203] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:422425174worker1 # [ 69.468577] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=0 paths=1 nar_bytes_out=1765175worker1 # [ 69.747259] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:422425176worker1 # [ 69.753479] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422425177worker1 # [ 69.757981] nix-grpc-daemon[919]: time=2026-09-22T10:49:34.765Z level=INFO msg="All 2 paths already cached"5178worker1 # [ 69.772923] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=debug event=want client=10 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05179worker1 # [ 69.779281] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15180worker1 # [ 69.784276] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422425181worker1 # [ 69.794928] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422425182worker1 # [ 69.807293] nix-grpc-daemon[919]: time=2026-09-22T10:49:34.814Z level=INFO msg="All 2 paths already cached"5183worker1 # [ 69.814350] nix-grpc-daemon[784]: ts=2026-09-22T10:49:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422425184worker1 # [ 69.825161] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5185worker1 # [ 69.997705] nix-daemon[867]: accepted connection from pid 1088, user nix-grpc-daemon (trusted)5186worker1 # [ 70.010584] nix-daemon[867]: reaped child process 1091, status = succeeded5187worker1 # [ 70.169123] nix-daemon[867]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)5188worker1 # [ 70.180748] nix-daemon[867]: reaped child process 1096, status = succeeded5189lb # [ 70.194152] niks3-server[1004]: 2026/09/22 10:49:35 INFO Received uploads request method=POST path=/api/pending_closures5190worker1 # [ 70.204288] nix-grpc-daemon[919]: time=2026-09-22T10:49:35.211Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5191worker1 # [ 70.205953] nix-grpc-daemon[919]: time=2026-09-22T10:49:35.213Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5192lb # [ 70.226139] niks3-server[1004]: 2026/09/22 10:49:35 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5193lb # [ 70.233168] niks3-server[1004]: 2026/09/22 10:49:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5194worker1 # [ 70.240600] nix-grpc-daemon[919]: time=2026-09-22T10:49:35.247Z level=INFO msg="Uploading 1 narinfos"5195lb # [ 70.238181] niks3-server[1004]: 2026/09/22 10:49:35 INFO Signed narinfos id=17 count=15196lb # [ 70.244383] niks3-server[1004]: 2026/09/22 10:49:35 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5197lb # [ 70.255316] niks3-server[1004]: 2026/09/22 10:49:35 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5198worker1 # [ 70.264090] nix-grpc-daemon[919]: time=2026-09-22T10:49:35.271Z level=INFO msg="Upload complete. (427ms)"5199lb # [ 70.261630] niks3-server[1004]: 2026/09/22 10:49:35 INFO Completed upload id=175200worker1 # [ 70.266385] nix-grpc-daemon[784]: ts=2026-09-22T10:49:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=0 paths=1 nar_bytes_in=7525201lb # [ 70.265617] niks3-server[1004]: 2026/09/22 10:49:35 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5202worker1 # [ 70.271962] nix-grpc-daemon[784]: ts=2026-09-22T10:49:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422425203worker1 # [ 70.282473] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5204worker1 # [ 70.285072] nix-daemon[867]: reaped child process 1086, status = succeeded5205worker1 # [ 70.290210] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5206worker1 # [ 71.310735] nix-daemon[867]: reaped child process 1100, status = succeeded5207worker1 # [ 71.471586] nix-daemon[867]: accepted connection from pid 1116, user nix-grpc-daemon (trusted)5208worker1 # [ 71.483540] nix-daemon[867]: reaped child process 1119, status = succeeded5209lb # [ 71.499802] niks3-server[1004]: 2026/09/22 10:49:36 INFO Received uploads request method=POST path=/api/pending_closures5210worker1 # [ 71.513385] nix-grpc-daemon[919]: time=2026-09-22T10:49:36.520Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5211worker1 # [ 71.515482] nix-grpc-daemon[919]: time=2026-09-22T10:49:36.522Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5212lb # [ 71.536610] niks3-server[1004]: 2026/09/22 10:49:36 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5213lb # [ 71.552421] niks3-server[1004]: 2026/09/22 10:49:36 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5214lb # [ 71.560996] niks3-server[1004]: 2026/09/22 10:49:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5215worker1 # [ 71.571297] nix-grpc-daemon[919]: time=2026-09-22T10:49:36.578Z level=INFO msg="Uploading 1 narinfos"5216lb # [ 71.568651] niks3-server[1004]: 2026/09/22 10:49:36 INFO Signed narinfos id=18 count=15217lb # [ 71.571960] niks3-server[1004]: 2026/09/22 10:49:36 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5218lb # [ 71.582210] niks3-server[1004]: 2026/09/22 10:49:36 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5219lb # [ 71.587255] niks3-server[1004]: 2026/09/22 10:49:36 INFO Completed upload id=185220lb # [ 71.589744] niks3-server[1004]: 2026/09/22 10:49:36 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5221worker1 # [ 71.591156] nix-grpc-daemon[919]: time=2026-09-22T10:49:36.598Z level=INFO msg="Upload complete. (294ms)"5222worker1 # [ 71.597849] nix-grpc-daemon[784]: ts=2026-09-22T10:49:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=9 outputs=15223worker1 # [ 71.610734] nix-daemon[867]: reaped child process 1098, status = succeeded5224worker1 # [ 71.642933] nix-grpc-daemon[784]: ts=2026-09-22T10:49:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:422425225worker1 # [ 71.658156] nix-grpc-daemon[784]: ts=2026-09-22T10:49:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:422425226worker1 # [ 71.662382] nix-grpc-daemon[784]: ts=2026-09-22T10:49:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=0 paths=1 nar_bytes_out=2325227client # [ 71.779240] nix-daemon[1019]: reaped child process 1021, status = succeeded5228client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.17 seconds)5229client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5230client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5231client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5232client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)5233client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5234client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5235(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.27 seconds)5236subtest: an input only one worker has still reaches the builder5237client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5238client # this derivation will be built:5239client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5240client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5241client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.46 seconds)5242client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5243client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.19 seconds)5244worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5245worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)5246worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5247worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5248client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25249client # [ 72.978773] nix-daemon[1019]: accepted connection from pid 1101, user root (trusted)5250client # this derivation will be built:5251client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5252worker2 # [ 73.178732] nix-grpc-daemon[787]: ts=2026-09-22T10:49:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:556245253worker2 # [ 73.181141] nix-grpc-daemon[787]: ts=2026-09-22T10:49:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556245254client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5255client # copying 1 paths...5256client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5257worker1 # [ 73.192841] nix-grpc-daemon[784]: ts=2026-09-22T10:49:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422425258worker1 # [ 73.204291] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5259worker1 # [ 73.389220] nix-daemon[867]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)5260worker1 # [ 73.401178] nix-daemon[867]: reaped child process 1131, status = succeeded5261lb # [ 73.414587] niks3-server[1004]: 2026/09/22 10:49:38 INFO Received uploads request method=POST path=/api/pending_closures5262worker1 # [ 73.421585] nix-grpc-daemon[919]: time=2026-09-22T10:49:38.428Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5263worker1 # [ 73.423426] nix-grpc-daemon[919]: time=2026-09-22T10:49:38.430Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5264lb # [ 73.429587] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5265lb # [ 73.436273] niks3-server[1004]: 2026/09/22 10:49:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5266lb # [ 73.441356] niks3-server[1004]: 2026/09/22 10:49:38 INFO Signed narinfos id=19 count=15267worker1 # [ 73.447716] nix-grpc-daemon[919]: time=2026-09-22T10:49:38.454Z level=INFO msg="Uploading 1 narinfos"5268lb # [ 73.447080] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5269lb # [ 73.458160] niks3-server[1004]: 2026/09/22 10:49:38 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5270worker1 # [ 73.465556] nix-grpc-daemon[919]: time=2026-09-22T10:49:38.472Z level=INFO msg="Upload complete. (251ms)"5271lb # [ 73.463226] niks3-server[1004]: 2026/09/22 10:49:38 INFO Completed upload id=195272worker1 # [ 73.467530] nix-grpc-daemon[784]: ts=2026-09-22T10:49:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=0 paths=1 nar_bytes_in=1205273lb # [ 73.467172] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5274worker1 # [ 73.473704] nix-daemon[867]: reaped child process 1126, status = succeeded5275worker1 # [ 73.481205] nix-grpc-daemon[784]: ts=2026-09-22T10:49:38Z level=debug event=want client=11 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05276worker1 # [ 73.483130] nix-grpc-daemon[784]: ts=2026-09-22T10:49:38Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15277worker2 # [ 73.523250] nix-grpc-daemon[787]: ts=2026-09-22T10:49:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556245278worker2 # [ 73.535717] nix-grpc-daemon[787]: ts=2026-09-22T10:49:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556245279client # copying 2 paths to 'lb:50051'...5280client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5281worker2 # [ 73.551625] nix-grpc-daemon[787]: ts=2026-09-22T10:49:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556245282client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5283worker2 # [ 73.561075] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5284worker2 # [ 73.735161] nix-daemon[856]: accepted connection from pid 1114, user nix-grpc-daemon (trusted)5285worker2 # [ 73.750453] nix-daemon[856]: reaped child process 1117, status = succeeded5286worker2 # [ 73.907538] nix-daemon[856]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5287worker2 # [ 73.920555] nix-daemon[856]: reaped child process 1122, status = succeeded5288lb # [ 73.934587] niks3-server[1004]: 2026/09/22 10:49:38 INFO Received uploads request method=POST path=/api/pending_closures5289lb # [ 73.942618] niks3-server[1004]: 2026/09/22 10:49:38 INFO Received uploads request method=POST path=/api/pending_closures5290worker2 # [ 73.948456] nix-grpc-daemon[869]: time=2026-09-22T10:49:38.906Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5291worker2 # [ 73.949903] nix-grpc-daemon[869]: time=2026-09-22T10:49:38.906Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5292worker2 # [ 73.951453] nix-grpc-daemon[869]: time=2026-09-22T10:49:38.906Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5293lb # [ 73.958974] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5294lb # [ 73.963056] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5295lb # [ 73.968115] niks3-server[1004]: 2026/09/22 10:49:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5296lb # [ 73.970078] niks3-server[1004]: 2026/09/22 10:49:38 INFO Signed narinfos id=21 count=25297lb # [ 73.972462] niks3-server[1004]: 2026/09/22 10:49:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5298lb # [ 73.974126] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5299lb # [ 73.975869] niks3-server[1004]: 2026/09/22 10:49:38 INFO Signed narinfos id=20 count=15300worker2 # [ 73.977806] nix-grpc-daemon[869]: time=2026-09-22T10:49:38.935Z level=INFO msg="Uploading 3 narinfos"5301lb # [ 73.979365] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5302lb # [ 73.990458] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5303lb # [ 73.994673] niks3-server[1004]: 2026/09/22 10:49:38 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5304lb # [ 73.996470] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5305lb # [ 73.999647] niks3-server[1004]: 2026/09/22 10:49:38 INFO Completed upload id=205306lb # [ 74.000917] niks3-server[1004]: 2026/09/22 10:49:38 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5307lb # [ 74.003240] niks3-server[1004]: 2026/09/22 10:49:38 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5308lb # [ 74.005469] niks3-server[1004]: 2026/09/22 10:49:38 INFO Completed upload id=215309worker2 # [ 74.007216] nix-grpc-daemon[869]: time=2026-09-22T10:49:38.965Z level=INFO msg="Upload complete. (430ms)"5310worker2 # [ 74.010075] nix-grpc-daemon[787]: ts=2026-09-22T10:49:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=0 paths=2 nar_bytes_in=9765311worker2 # [ 74.012874] nix-grpc-daemon[787]: ts=2026-09-22T10:49:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556245312worker2 # [ 74.019540] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5313worker2 # [ 74.022132] nix-daemon[856]: reaped child process 1112, status = succeeded5314worker2 # [ 74.028672] nix-grpc-daemon[787]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5315client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5316worker2 # [ 74.036373] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5317client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5318client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5319worker2 # [ 75.661941] nix-daemon[856]: reaped child process 1127, status = succeeded5320worker2 # [ 75.793073] nix-daemon[856]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)5321worker2 # [ 75.806765] nix-daemon[856]: reaped child process 1146, status = succeeded5322lb # [ 75.816073] niks3-server[1004]: 2026/09/22 10:49:40 INFO Received uploads request method=POST path=/api/pending_closures5323worker2 # [ 75.820671] nix-grpc-daemon[869]: time=2026-09-22T10:49:40.778Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5324lb # [ 75.830221] niks3-server[1004]: 2026/09/22 10:49:40 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5325lb # [ 75.835919] niks3-server[1004]: 2026/09/22 10:49:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5326worker2 # [ 75.844738] nix-grpc-daemon[869]: time=2026-09-22T10:49:40.803Z level=INFO msg="Uploading 1 narinfos"5327lb # [ 75.844344] niks3-server[1004]: 2026/09/22 10:49:40 INFO Signed narinfos id=22 count=15328lb # [ 75.853122] niks3-server[1004]: 2026/09/22 10:49:40 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5329lb # [ 75.863294] niks3-server[1004]: 2026/09/22 10:49:40 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5330lb # [ 75.867159] niks3-server[1004]: 2026/09/22 10:49:40 INFO Completed upload id=225331worker2 # [ 75.868966] nix-grpc-daemon[869]: time=2026-09-22T10:49:40.827Z level=INFO msg="Upload complete. (223ms)"5332lb # [ 75.869163] niks3-server[1004]: 2026/09/22 10:49:40 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5333worker2 # [ 75.871675] nix-grpc-daemon[787]: ts=2026-09-22T10:49:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=10 outputs=15334client # copying 1 paths...5335worker2 # [ 75.878705] nix-daemon[856]: reaped child process 1124, status = succeeded5336worker1 # [ 75.882123] nix-grpc-daemon[784]: ts=2026-09-22T10:49:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:422425337worker1 # [ 75.891856] nix-grpc-daemon[784]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5338client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5339worker1 # [ 75.928904] nix-grpc-daemon[784]: ts=2026-09-22T10:49:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:422425340worker1 # [ 75.933763] nix-grpc-daemon[784]: ts=2026-09-22T10:49:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=0 paths=1 nar_bytes_out=1205341client # [ 76.046825] nix-daemon[1019]: reaped child process 1104, status = succeeded5342client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.30 seconds)5343client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25344client # [ 76.232745] nix-daemon[1019]: accepted connection from pid 1123, user root (trusted)5345client # this derivation will be built:5346client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5347worker1 # [ 76.428036] nix-grpc-daemon[784]: ts=2026-09-22T10:49:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:422425348worker1 # [ 76.430803] nix-grpc-daemon[784]: ts=2026-09-22T10:49:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422425349client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5350worker1 # [ 76.436517] nix-grpc-daemon[919]: time=2026-09-22T10:49:41.443Z level=INFO msg="All 1 paths already cached"5351worker1 # [ 76.446354] nix-grpc-daemon[784]: ts=2026-09-22T10:49:41Z level=debug event=want client=12 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05352worker1 # [ 76.448518] nix-grpc-daemon[784]: ts=2026-09-22T10:49:41Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15353worker1 # [ 76.451118] nix-grpc-daemon[784]: ts=2026-09-22T10:49:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422425354worker1 # [ 76.458386] nix-grpc-daemon[784]: ts=2026-09-22T10:49:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422425355client # copying 1 paths to 'lb:50051'...5356client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5357worker1 # [ 76.470942] nix-grpc-daemon[784]: ts=2026-09-22T10:49:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422425358worker1 # [ 76.476134] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5359worker1 # [ 76.483905] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 1/5)5360worker1 # [ 76.586652] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 2/5)5361worker1 # [ 76.717341] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 245 ms (attempt 3/5)5362worker1 # [ 76.963452] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 239 ms (attempt 4/5)5363worker1 # [ 77.208411] nix-grpc-daemon[784]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5364worker1 # [ 77.212091] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5365worker1 # [ 77.217561] nix-grpc-daemon[784]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5366worker1 # [ 77.401298] nix-daemon[867]: accepted connection from pid 1145, user nix-grpc-daemon (trusted)5367worker1 # [ 77.414279] nix-daemon[867]: reaped child process 1148, status = succeeded5368worker1 # [ 77.545351] nix-daemon[867]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)5369worker1 # [ 77.556916] nix-daemon[867]: reaped child process 1153, status = succeeded5370lb # [ 77.567282] niks3-server[1004]: 2026/09/22 10:49:42 INFO Received uploads request method=POST path=/api/pending_closures5371worker1 # [ 77.577786] nix-grpc-daemon[919]: time=2026-09-22T10:49:42.584Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5372worker1 # [ 77.579230] nix-grpc-daemon[919]: time=2026-09-22T10:49:42.584Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5373lb # [ 77.593752] niks3-server[1004]: 2026/09/22 10:49:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5374lb # [ 77.598490] niks3-server[1004]: 2026/09/22 10:49:42 INFO Signed narinfos id=23 count=15375worker1 # [ 77.605593] nix-grpc-daemon[919]: time=2026-09-22T10:49:42.612Z level=INFO msg="Uploading 1 narinfos"5376lb # [ 77.606945] niks3-server[1004]: 2026/09/22 10:49:42 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5377lb # [ 77.611728] niks3-server[1004]: 2026/09/22 10:49:42 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5378lb # [ 77.624303] niks3-server[1004]: 2026/09/22 10:49:42 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5379lb # [ 77.631716] niks3-server[1004]: 2026/09/22 10:49:42 INFO Completed upload id=235380worker1 # [ 77.633972] nix-grpc-daemon[919]: time=2026-09-22T10:49:42.641Z level=INFO msg="Upload complete. (396ms)"5381lb # [ 77.634489] niks3-server[1004]: 2026/09/22 10:49:42 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5382worker1 # [ 77.637442] nix-grpc-daemon[784]: ts=2026-09-22T10:49:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=1 paths=1 nar_bytes_in=5605383worker1 # [ 77.643588] nix-grpc-daemon[784]: ts=2026-09-22T10:49:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422425384worker1 # [ 77.651559] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5385worker1 # [ 77.657235] nix-daemon[867]: reaped child process 1138, status = succeeded5386worker1 # [ 77.660372] nix-daemon[867]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5387client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5388client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5389lb # [ 78.139625] envoy[920]: conn peer=[2001:db8:1::1]:36466 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3891 tx=1956 ms=23365390lb # [ 78.143393] envoy[920]: conn peer=[2001:db8:1::1]:36472 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=565391lb # [ 78.146903] envoy[920]: conn peer=[2001:db8:1::1]:36478 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4326 tx=2179 ms=19645392lb # [ 78.150475] envoy[920]: conn peer=[2001:db8:1::1]:36492 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=545393lb # [ 78.153961] envoy[920]: conn peer=[2001:db8:1::1]:45188 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4219 tx=1962 ms=28055394lb # [ 78.157513] envoy[920]: conn peer=[2001:db8:1::1]:45198 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=625395client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5396worker1 # [ 79.074709] nix-daemon[867]: reaped child process 1157, status = succeeded5397worker1 # [ 79.213586] nix-daemon[867]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5398worker1 # [ 79.225844] nix-daemon[867]: reaped child process 1176, status = succeeded5399lb # [ 79.239560] niks3-server[1004]: 2026/09/22 10:49:44 INFO Received uploads request method=POST path=/api/pending_closures5400worker1 # [ 79.248632] nix-grpc-daemon[919]: time=2026-09-22T10:49:44.255Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5401lb # [ 79.271866] niks3-server[1004]: 2026/09/22 10:49:44 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5402lb # [ 79.275922] niks3-server[1004]: 2026/09/22 10:49:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5403lb # [ 79.280621] niks3-server[1004]: 2026/09/22 10:49:44 INFO Signed narinfos id=24 count=15404worker1 # [ 79.286041] nix-grpc-daemon[919]: time=2026-09-22T10:49:44.293Z level=INFO msg="Uploading 1 narinfos"5405lb # [ 79.292586] niks3-server[1004]: 2026/09/22 10:49:44 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5406lb # [ 79.300737] niks3-server[1004]: 2026/09/22 10:49:44 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5407lb # [ 79.307136] niks3-server[1004]: 2026/09/22 10:49:44 INFO Completed upload id=245408worker1 # [ 79.309438] nix-grpc-daemon[919]: time=2026-09-22T10:49:44.316Z level=INFO msg="Upload complete. (241ms)"5409lb # [ 79.309645] niks3-server[1004]: 2026/09/22 10:49:44 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5410worker1 # [ 79.313845] nix-grpc-daemon[784]: ts=2026-09-22T10:49:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=11 outputs=15411client # copying 1 paths...5412client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5413worker1 # [ 79.322795] nix-grpc-daemon[784]: ts=2026-09-22T10:49:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:422425414worker1 # [ 79.330579] nix-daemon[867]: reaped child process 1155, status = succeeded5415worker1 # [ 79.333080] nix-grpc-daemon[784]: ts=2026-09-22T10:49:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:422425416worker1 # [ 79.334845] nix-grpc-daemon[784]: ts=2026-09-22T10:49:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42242 duration_s=0 paths=1 nar_bytes_out=1205417client # [ 79.449473] nix-daemon[1019]: reaped child process 1126, status = succeeded5418client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.40 seconds)5419client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25420client # [ 79.613123] nix-daemon[1019]: accepted connection from pid 1145, user root (trusted)5421client # this derivation will be built:5422client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5423worker1 # [ 79.765264] nix-grpc-daemon[784]: ts=2026-09-22T10:49:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:422425424client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5425worker2 # [ 79.769066] nix-grpc-daemon[787]: ts=2026-09-22T10:49:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556245426worker2 # [ 79.774549] nix-grpc-daemon[869]: time=2026-09-22T10:49:44.731Z level=INFO msg="All 1 paths already cached"5427worker1 # [ 79.784539] nix-grpc-daemon[784]: ts=2026-09-22T10:49:44Z level=debug event=want client=13 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05428worker1 # [ 79.788973] nix-grpc-daemon[784]: ts=2026-09-22T10:49:44Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15429worker2 # [ 79.826832] nix-grpc-daemon[787]: ts=2026-09-22T10:49:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556245430worker2 # [ 79.836830] nix-grpc-daemon[787]: ts=2026-09-22T10:49:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556245431worker2 # [ 79.847121] nix-grpc-daemon[869]: time=2026-09-22T10:49:44.805Z level=INFO msg="All 1 paths already cached"5432client # copying 1 paths to 'lb:50051'...5433client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5434worker2 # [ 79.854860] nix-grpc-daemon[787]: ts=2026-09-22T10:49:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556245435worker2 # [ 79.866534] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5436worker2 # [ 80.023121] nix-daemon[856]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)5437worker2 # [ 80.037060] nix-daemon[856]: reaped child process 1153, status = succeeded5438worker2 # [ 80.184309] nix-daemon[856]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5439worker2 # [ 80.197482] nix-daemon[856]: reaped child process 1158, status = succeeded5440lb # [ 80.210383] niks3-server[1004]: 2026/09/22 10:49:45 INFO Received uploads request method=POST path=/api/pending_closures5441worker2 # [ 80.218920] nix-grpc-daemon[869]: time=2026-09-22T10:49:45.177Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5442worker2 # [ 80.220466] nix-grpc-daemon[869]: time=2026-09-22T10:49:45.178Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5443lb # [ 80.239988] niks3-server[1004]: 2026/09/22 10:49:45 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5444lb # [ 80.251692] niks3-server[1004]: 2026/09/22 10:49:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5445lb # [ 80.256842] niks3-server[1004]: 2026/09/22 10:49:45 INFO Signed narinfos id=25 count=15446worker2 # [ 80.260731] nix-grpc-daemon[869]: time=2026-09-22T10:49:45.219Z level=INFO msg="Uploading 1 narinfos"5447lb # [ 80.263654] niks3-server[1004]: 2026/09/22 10:49:45 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5448lb # [ 80.273578] niks3-server[1004]: 2026/09/22 10:49:45 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5449worker2 # [ 80.280902] nix-grpc-daemon[869]: time=2026-09-22T10:49:45.239Z level=INFO msg="Upload complete. (403ms)"5450lb # [ 80.280548] niks3-server[1004]: 2026/09/22 10:49:45 INFO Completed upload id=255451worker2 # [ 80.282459] nix-grpc-daemon[787]: ts=2026-09-22T10:49:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=0 paths=1 nar_bytes_in=5605452lb # [ 80.285532] niks3-server[1004]: 2026/09/22 10:49:45 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5453worker2 # [ 80.289637] nix-daemon[856]: reaped child process 1148, status = succeeded5454worker2 # [ 80.291358] nix-grpc-daemon[787]: ts=2026-09-22T10:49:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556245455worker2 # [ 80.295886] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5456worker2 # [ 80.301529] nix-daemon[856]: accepted connection from pid 787, user nix-grpc-daemon (trusted)5457client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5458client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5459client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5460worker2 # [ 81.127591] nix-daemon[856]: reaped child process 1162, status = succeeded5461worker2 # [ 81.275930] nix-daemon[856]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5462worker2 # [ 81.288842] nix-daemon[856]: reaped child process 1181, status = succeeded5463lb # [ 81.301658] niks3-server[1004]: 2026/09/22 10:49:46 INFO Received uploads request method=POST path=/api/pending_closures5464worker2 # [ 81.311318] nix-grpc-daemon[869]: time=2026-09-22T10:49:46.269Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5465lb # [ 81.333160] niks3-server[1004]: 2026/09/22 10:49:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5466worker2 # [ 81.339606] nix-grpc-daemon[869]: time=2026-09-22T10:49:46.298Z level=INFO msg="Uploading 1 narinfos"5467lb # [ 81.339084] niks3-server[1004]: 2026/09/22 10:49:46 INFO Signed narinfos id=26 count=15468lb # [ 81.348767] niks3-server[1004]: 2026/09/22 10:49:46 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5469lb # [ 81.352960] niks3-server[1004]: 2026/09/22 10:49:46 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5470lb # [ 81.363574] niks3-server[1004]: 2026/09/22 10:49:46 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5471lb # [ 81.368601] niks3-server[1004]: 2026/09/22 10:49:46 INFO Completed upload id=265472worker2 # [ 81.371076] nix-grpc-daemon[869]: time=2026-09-22T10:49:46.329Z level=INFO msg="Upload complete. (259ms)"5473lb # [ 81.374136] niks3-server[1004]: 2026/09/22 10:49:46 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5474worker2 # [ 81.375375] nix-grpc-daemon[787]: ts=2026-09-22T10:49:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=12 outputs=15475worker2 # [ 81.386613] nix-daemon[856]: reaped child process 1160, status = succeeded5476client # copying 1 paths...5477worker2 # [ 81.421582] nix-grpc-daemon[787]: ts=2026-09-22T10:49:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:556245478client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5479worker2 # [ 81.444172] nix-grpc-daemon[787]: ts=2026-09-22T10:49:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:556245480worker2 # [ 81.448481] nix-grpc-daemon[787]: ts=2026-09-22T10:49:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55624 duration_s=0 paths=1 nar_bytes_out=1205481client # [ 81.562584] nix-daemon[1019]: reaped child process 1148, status = succeeded5482client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.11 seconds)5483(finished: subtest: an input only one worker has still reaches the builder, in 9.68 seconds)5484subtest: upload whose reference the worker lacks is completed from the cache5485client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5486client # this derivation will be built:5487client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5488client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5489client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.41 seconds)5490client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5491client # this derivation will be built:5492client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5493client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5494client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.39 seconds)5495client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5496client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.17 seconds)5497worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5498worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.17 seconds)5499worker2 # [ 82.756594] systemd[1]: Stopping Nix worker-protocol over gRPC...5500worker2 # [ 83.513946] nix-grpc-daemon[787]: ts=2026-09-22T10:49:48Z level=info event=signal_exit5501worker2 # [ 84.786909] nix-grpc-daemon[787]: ts=2026-09-22T10:49:49Z level=info event=scheduler_restarting5502worker1 # [ 84.794564] nix-grpc-daemon[784]: ts=2026-09-22T10:49:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:422285503worker1 # [ 84.796062] nix-grpc-daemon[784]: ts=2026-09-22T10:49:49Z level=info event=worker_gone addr=192.168.1.4:500515504worker2 # [ 84.807701] nix-grpc-daemon[787]: ts=2026-09-22T10:49:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5505worker2 # [ 84.826445] nix-daemon[856]: reaped child process 858, status = succeeded5506worker2 # [ 85.644480] nix-grpc-daemon[787]: E0000 00:00:1790074190.602638 832 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5507worker2 # [ 85.649448] nix-grpc-daemon[787]: E0000 00:00:1790074190.602784 832 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5508worker2 # [ 85.729676] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5509worker2 # [ 85.733635] systemd[1]: Stopped Nix worker-protocol over gRPC.5510worker2 # [ 85.736652] systemd[1]: nix-grpc-daemon.service: Consumed 2.342s CPU time over 1min 7.471s wall clock time, 60.9M memory peak, 86.6K incoming IP traffic, 212.5K outgoing IP traffic.5511worker2 # [ 85.747661] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5512worker2 # [ 85.751072] systemd[1]: Closed Nix worker-protocol over gRPC.5513worker2 # [ 85.753628] systemd[1]: nix-grpc-daemon.socket: Consumed 322us CPU time over 1min 16.266s wall clock time, 256K memory peak, 76.8K incoming IP traffic, 62.6K outgoing IP traffic.5514lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5515lb: must succeed: curl -sf localhost:9901/clusters5516lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5517lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5518client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25519worker1 # [ 86.079688] nix-grpc-daemon[784]: ts=2026-09-22T10:49:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:422425520client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5521worker1 # [ 86.085075] nix-grpc-daemon[784]: ts=2026-09-22T10:49:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:422425522client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.28 seconds)5523worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5524worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5525worker2 # [ 86.199826] systemd[1]: Listening on Nix worker-protocol over gRPC.5526worker1 # [ 86.256865] systemd[1]: Stopping Nix worker-protocol over gRPC...5527worker1 # [ 86.421918] nix-grpc-daemon[784]: ts=2026-09-22T10:49:51Z level=info event=signal_exit5528lb # [ 86.714158] niks3-server[1004]: 2026/09/22 10:49:51 INFO lead: released remote=192.168.1.3:498385529worker2 # [ 86.718140] systemd[1]: Starting scheduler-token.service...5530worker2 # [ 86.809463] systemd[1]: scheduler-token.service: Deactivated successfully.5531worker2 # [ 86.811282] systemd[1]: Finished scheduler-token.service.5532worker2 # [ 86.812834] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 90ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5533worker2 # [ 86.817939] systemd[1]: Starting Nix worker-protocol over gRPC...5534worker2 # [ 86.968869] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5535worker2 # [ 86.971236] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5536worker2 # [ 86.974863] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15537worker2 # [ 86.982065] nix-grpc-daemon[1205]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5538worker2 # [ 86.983402] nix-grpc-daemon[1205]: E0000 00:00:1790074191.939889 1205 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5539worker2 # [ 86.988379] nix-grpc-daemon[1205]: E0000 00:00:1790074191.944912 1205 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5540worker2 # [ 86.992704] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5541worker2 # [ 86.995674] systemd[1]: Started Nix worker-protocol over gRPC.5542lb # [ 87.001529] niks3-server[1004]: 2026/09/22 10:49:51 INFO lead: acquired remote=192.168.1.4:555165543worker2 # [ 87.003070] nix-grpc-daemon[1205]: E0000 00:00:1790074191.960708 1247 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5544worker2 # [ 87.005993] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:51Z level=info event=scheduler_take_over5545worker2 # [ 87.018039] nix-grpc-daemon[1205]: E0000 00:00:1790074191.976399 1247 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5546worker2 # [ 87.022154] nix-grpc-daemon[1205]: E0000 00:00:1790074191.978619 1247 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5547worker2 # [ 87.033063] nix-grpc-daemon[1205]: E0000 00:00:1790074191.991029 1247 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5548worker2 # [ 87.041664] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_connected addr=lb:500515549worker2 # [ 87.043242] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5550worker2 # [ 87.244883] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_connected addr=lb:500515551worker2 # [ 87.248795] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5552worker2 # [ 87.449881] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_connected addr=lb:500515553worker2 # [ 87.453884] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5554worker2 # [ 87.654939] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_connected addr=lb:500515555worker2 # [ 87.658856] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5556worker1 # [ 87.718970] nix-grpc-daemon[784]: ts=2026-09-22T10:49:52Z level=info event=scheduler_restarting5557worker1 # [ 87.724890] nix-grpc-daemon[784]: ts=2026-09-22T10:49:52Z level=info event=scheduler_restarting addr=lb:500515558worker1 # [ 87.728657] nix-grpc-daemon[784]: ts=2026-09-22T10:49:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:422285559worker1 # [ 87.732250] nix-grpc-daemon[784]: ts=2026-09-22T10:49:52Z level=info event=worker_gone addr=192.168.1.3:500515560worker1 # [ 87.752883] nix-daemon[867]: reaped child process 869, status = succeeded5561worker2 # [ 87.859577] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_connected addr=lb:500515562worker2 # [ 87.861496] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5563worker1 # [ 87.931357] nix-grpc-daemon[784]: ts=2026-09-22T10:49:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5564worker2 # [ 88.062507] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:53Z level=info event=scheduler_connected addr=lb:500515565worker2 # [ 88.069883] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:53Z level=info event=oidc_jwks provider=mock keys=15566worker2 # [ 88.071298] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:553205567worker2 # [ 88.073039] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05568lb # [ 88.140125] envoy[920]: conn peer=[2001:db8:1::1]:45204 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3752 tx=1896 ms=29595569lb # [ 88.143766] envoy[920]: conn peer=[2001:db8:1::1]:45216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=515570lb # [ 88.147351] envoy[920]: conn peer=[2001:db8:1::1]:45218 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3755 tx=1900 ms=17315571lb # [ 88.150885] envoy[920]: conn peer=[2001:db8:1::1]:45232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545572lb # [ 88.154453] envoy[920]: conn peer=[2001:db8:1::4]:56084 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=71010 tx=3523 ms=662245573lb # [ 88.158064] envoy[920]: conn peer=[2001:db8:1::1]:51646 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165574lb # [ 88.161445] envoy[920]: conn peer=[2001:db8:1::3]:44914 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=16865 tx=3189 ms=684045575worker1 # [ 88.577860] nix-grpc-daemon[784]: E0000 00:00:1790074193.584900 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5576worker1 # [ 88.579981] nix-grpc-daemon[784]: E0000 00:00:1790074193.585013 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5577worker1 # [ 89.673969] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5578worker1 # [ 89.675340] systemd[1]: Stopped Nix worker-protocol over gRPC.5579worker1 # [ 89.676878] systemd[1]: nix-grpc-daemon.service: Consumed 2.083s CPU time over 1min 11.545s wall clock time, 57.7M memory peak, 78.5K incoming IP traffic, 176.5K outgoing IP traffic.5580worker1 # [ 89.680500] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5581worker1 # [ 89.682212] systemd[1]: Closed Nix worker-protocol over gRPC.5582worker1 # [ 89.683573] systemd[1]: nix-grpc-daemon.socket: Consumed 299us CPU time over 1min 20.197s wall clock time, 256K memory peak, 93.4K incoming IP traffic, 80K outgoing IP traffic.5583lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5584lb: must succeed: curl -sf localhost:9901/clusters5585lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5586lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5587client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25588worker2 # [ 89.988753] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:553345589worker2 # [ 90.001245] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5590worker2 # [ 90.021894] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 1/5)5591worker2 # [ 90.033426] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 2/5)5592worker2 # [ 90.063377] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 238 ms (attempt 3/5)5593worker2 # [ 90.304582] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 773 ms (attempt 4/5)5594worker2 # [ 91.080065] nix-grpc-daemon[1205]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5595worker2 # [ 91.083608] nix-grpc-daemon[1205]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5596client # copying 2 paths...5597client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5598worker2 # [ 91.105133] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:553345599client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5600worker2 # [ 91.119399] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5601worker2 # [ 91.138575] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:56Z level=info event=niks3_push_started pid=12655602worker2 # [ 91.320077] nix-grpc-daemon[1265]: time=2026-09-22T10:49:56.277Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5603worker2 # [ 91.486089] nix-daemon[856]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5604worker2 # [ 91.499282] nix-daemon[856]: reaped child process 1281, status = succeeded5605lb # [ 91.515477] niks3-server[1004]: 2026/09/22 10:49:56 INFO Received uploads request method=POST path=/api/pending_closures5606lb # [ 91.525409] niks3-server[1004]: 2026/09/22 10:49:56 INFO Received uploads request method=POST path=/api/pending_closures5607worker2 # [ 91.532267] nix-grpc-daemon[1265]: time=2026-09-22T10:49:56.490Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5608worker2 # [ 91.535656] nix-grpc-daemon[1265]: time=2026-09-22T10:49:56.490Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5609worker2 # [ 91.538400] nix-grpc-daemon[1265]: time=2026-09-22T10:49:56.492Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5610lb # [ 91.564753] niks3-server[1004]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5611lb # [ 91.579595] niks3-server[1004]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5612lb # [ 91.596504] niks3-server[1004]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5613lb # [ 91.599259] niks3-server[1004]: 2026/09/22 10:49:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5614lb # [ 91.601821] niks3-server[1004]: 2026/09/22 10:49:56 INFO Signed narinfos id=27 count=15615lb # [ 91.603140] niks3-server[1004]: 2026/09/22 10:49:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5616lb # [ 91.605248] niks3-server[1004]: 2026/09/22 10:49:56 INFO Signed narinfos id=28 count=25617lb # [ 91.608680] niks3-server[1004]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5618worker2 # [ 91.608159] nix-grpc-daemon[1265]: time=2026-09-22T10:49:56.565Z level=INFO msg="Uploading 3 narinfos"5619lb # [ 91.621962] niks3-server[1004]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5620lb # [ 91.625646] niks3-server[1004]: 2026/09/22 10:49:56 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5621lb # [ 91.627975] niks3-server[1004]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5622lb # [ 91.630469] niks3-server[1004]: 2026/09/22 10:49:56 INFO Completed upload id=275623lb # [ 91.631914] niks3-server[1004]: 2026/09/22 10:49:56 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5624lb # [ 91.633804] niks3-server[1004]: 2026/09/22 10:49:56 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5625lb # [ 91.636344] niks3-server[1004]: 2026/09/22 10:49:56 INFO Completed upload id=285626worker2 # [ 91.638191] nix-grpc-daemon[1265]: time=2026-09-22T10:49:56.596Z level=INFO msg="Upload complete. (314ms)"5627worker2 # [ 91.643144] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55334 duration_s=0 paths=2 nar_bytes_in=3125628worker2 # [ 91.655680] nix-daemon[856]: reaped child process 1263, status = succeeded5629client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.92 seconds)5630worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5631worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5632worker1 # [ 91.766637] systemd[1]: Listening on Nix worker-protocol over gRPC.5633worker1 # [ 91.773241] systemd[1]: Starting Nix worker-protocol over gRPC...5634worker1 # [ 91.921250] nix-grpc-daemon[1201]: ts=2026-09-22T10:49:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5635worker1 # [ 91.923147] nix-grpc-daemon[1201]: ts=2026-09-22T10:49:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5636worker1 # [ 91.926688] nix-grpc-daemon[1201]: ts=2026-09-22T10:49:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15637worker1 # [ 91.933159] nix-grpc-daemon[1201]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5638worker1 # [ 91.934454] nix-grpc-daemon[1201]: E0000 00:00:1790074196.939936 1201 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5639worker1 # [ 91.941073] nix-grpc-daemon[1201]: E0000 00:00:1790074196.945114 1201 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5640worker1 # [ 91.943119] nix-grpc-daemon[1201]: ts=2026-09-22T10:49:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5641worker1 # [ 91.944630] systemd[1]: Started Nix worker-protocol over gRPC.5642worker1 # [ 91.951935] nix-grpc-daemon[1201]: E0000 00:00:1790074196.958255 1242 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5643lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5644lb: must succeed: curl -sf localhost:9901/clusters5645worker1 # [ 91.963436] nix-grpc-daemon[1201]: E0000 00:00:1790074196.970587 1242 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5646worker1 # [ 91.968055] nix-grpc-daemon[1201]: E0000 00:00:1790074196.972922 1242 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5647worker1 # [ 91.977732] nix-grpc-daemon[1201]: E0000 00:00:1790074196.984875 1242 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5648worker1 # [ 91.987771] nix-grpc-daemon[1201]: ts=2026-09-22T10:49:56Z level=info event=scheduler_connected addr=lb:500515649worker2 # [ 91.987858] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:553205650worker2 # [ 91.992294] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05651lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5652lb: must succeed: curl -sf localhost:9901/clusters5653lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5654lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)5655(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.53 seconds)5656subtest: a niks3 restart does not cost the leader its role5657lb: must succeed: curl -sf localhost:9901/clusters5658lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5659lb: must succeed: curl -sf localhost:9901/clusters5660lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5661lb # [ 93.224572] niks3-server[1004]: 2026/09/22 10:49:58 INFO Shutdown signal received, draining in-flight requests timeout=10s5662lb # [ 93.227481] systemd[1]: Stopping niks3 server...5663worker1 # [ 93.230370] nix-grpc-daemon[1201]: ts=2026-09-22T10:49:58Z level=info event=scheduler_lock_lost status=2005664lb # [ 93.231241] niks3-server[1004]: 2026/09/22 10:49:58 INFO lead: released remote=192.168.1.4:555165665worker2 # [ 93.232738] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:58Z level=info event=scheduler_lock_lost status=2005666lb # [ 93.239719] systemd[1]: niks3.service: Deactivated successfully.5667lb # [ 93.241209] systemd[1]: Stopped niks3 server.5668lb # [ 93.242141] systemd[1]: niks3.service: Consumed 600ms CPU time over 1min 8.763s wall clock time, 20.5M memory peak, 69.4K incoming IP traffic, 101.4K outgoing IP traffic.5669lb # [ 93.249990] systemd[1]: Starting niks3 server...5670lb # [ 93.316382] niks3-server[1143]: 2026/09/22 10:49:58 goose: no migrations to run. current version: 202609200000005671lb # [ 93.322511] niks3-server[1143]: 2026/09/22 10:49:58 OK 1_commit_pending_closure.sql (4.58ms)5672lb # [ 93.326121] niks3-server[1143]: 2026/09/22 10:49:58 OK 2_object_stats_trigger.sql (2.01ms)5673lb # [ 93.327202] niks3-server[1143]: 2026/09/22 10:49:58 goose: up to current file version: 25674lb # [ 93.328849] niks3-server[1143]: 2026/09/22 10:49:58 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key5675lb # [ 93.340127] niks3-server[1143]: 2026/09/22 10:49:58 INFO Read proxy enabled — serving cache objects from S35676lb # [ 93.341352] niks3-server[1143]: 2026/09/22 10:49:58 INFO Using socket-activated listener address=0.0.0.0:57515677lb # [ 93.344313] systemd[1]: Started niks3 server.5678lb # [ 93.344944] niks3-server[1143]: 2026/09/22 10:49:58 INFO systemd watchdog enabled interval=15s5679lb # [ 93.346234] niks3-server[1143]: 2026/09/22 10:49:58 INFO Starting HTTP server address=0.0.0.0:57515680lb: waiting for unit niks3.service5681lb: (finished: waiting for unit niks3.service, in 0.04 seconds)5682lb: must succeed: curl -sf localhost:9901/clusters5683lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5684worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5685worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5686lb: must succeed: curl -sf localhost:9901/clusters5687lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5688lb: must succeed: curl -sf localhost:9901/clusters5689lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5690worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5691worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5692client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart >&25693worker1 # [ 93.884921] nix-grpc-daemon[1201]: ts=2026-09-22T10:49:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505485694worker1 # [ 93.897119] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5695client # don't know how to build these paths:5696client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5697worker2 # [ 93.910573] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:553345698worker2 # [ 93.931451] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:58Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05699worker2 # [ 93.936604] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:58Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15700worker2 # [ 93.976060] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:553345701worker2 # [ 93.986917] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:553345702client # copying 1 paths to 'lb:50051'...5703client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5704worker2 # [ 93.997595] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:553345705worker2 # [ 94.006788] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5706worker2 # [ 94.172449] nix-daemon[856]: accepted connection from pid 1298, user nix-grpc-daemon (trusted)5707worker2 # [ 94.185712] nix-daemon[856]: reaped child process 1301, status = succeeded5708lb # [ 94.236128] niks3-server[1143]: 2026/09/22 10:49:59 INFO lead: acquired remote=192.168.1.4:436265709worker2 # [ 94.346301] nix-daemon[856]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5710worker2 # [ 94.362981] nix-daemon[856]: reaped child process 1308, status = succeeded5711lb # [ 94.377599] niks3-server[1143]: 2026/09/22 10:49:59 INFO Received uploads request method=POST path=/api/pending_closures5712worker2 # [ 94.400125] nix-grpc-daemon[1265]: time=2026-09-22T10:49:59.357Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5713worker2 # [ 94.401880] nix-grpc-daemon[1265]: time=2026-09-22T10:49:59.357Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5714lb # [ 94.424043] niks3-server[1143]: 2026/09/22 10:49:59 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5715lb # [ 94.433585] niks3-server[1143]: 2026/09/22 10:49:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5716lb # [ 94.446138] niks3-server[1143]: 2026/09/22 10:49:59 INFO Signed narinfos id=29 count=15717worker2 # [ 94.447116] nix-grpc-daemon[1265]: time=2026-09-22T10:49:59.405Z level=INFO msg="Uploading 1 narinfos"5718lb # [ 94.453959] niks3-server[1143]: 2026/09/22 10:49:59 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5719lb # [ 94.459910] niks3-server[1143]: 2026/09/22 10:49:59 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5720lb # [ 94.464058] niks3-server[1143]: 2026/09/22 10:49:59 INFO Completed upload id=295721worker2 # [ 94.464633] nix-grpc-daemon[1265]: time=2026-09-22T10:49:59.422Z level=INFO msg="Upload complete. (441ms)"5722worker2 # [ 94.469912] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55334 duration_s=0 paths=1 nar_bytes_in=5365723worker2 # [ 94.477359] nix-grpc-daemon[1205]: ts=2026-09-22T10:49:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:553345724worker2 # [ 94.485550] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5725worker2 # [ 94.497482] nix-daemon[856]: reaped child process 1296, status = succeeded5726worker2 # [ 94.500764] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5727client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5728client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5729client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5730client # farm-a> LOG-farm-a-after-niks3-restart5731worker2 # [ 95.114427] nix-daemon[856]: reaped child process 1312, status = succeeded5732worker2 # [ 95.256848] nix-daemon[856]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5733worker2 # [ 95.270135] nix-daemon[856]: reaped child process 1331, status = succeeded5734lb # [ 95.283872] niks3-server[1143]: 2026/09/22 10:50:00 INFO Received uploads request method=POST path=/api/pending_closures5735worker2 # [ 95.296082] nix-grpc-daemon[1265]: time=2026-09-22T10:50:00.253Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5736worker2 # [ 95.297512] nix-grpc-daemon[1265]: time=2026-09-22T10:50:00.253Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5737lb # [ 95.334433] niks3-server[1143]: 2026/09/22 10:50:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5738worker2 # [ 95.338869] nix-grpc-daemon[1265]: time=2026-09-22T10:50:00.296Z level=INFO msg="Uploading 1 narinfos"5739lb # [ 95.338807] niks3-server[1143]: 2026/09/22 10:50:00 INFO Signed narinfos id=30 count=15740lb # [ 95.346577] niks3-server[1143]: 2026/09/22 10:50:00 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5741lb # [ 95.350806] niks3-server[1143]: 2026/09/22 10:50:00 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5742lb # [ 95.360104] niks3-server[1143]: 2026/09/22 10:50:00 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5743lb # [ 95.363137] niks3-server[1143]: 2026/09/22 10:50:00 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5744lb # [ 95.367048] niks3-server[1143]: 2026/09/22 10:50:00 INFO Completed upload id=305745worker2 # [ 95.367740] nix-grpc-daemon[1265]: time=2026-09-22T10:50:00.326Z level=INFO msg="Upload complete. (260ms)"5746lb # [ 95.368825] niks3-server[1143]: 2026/09/22 10:50:00 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5747worker2 # [ 95.376676] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55334 duration_s=0 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15748worker2 # [ 95.392114] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:00Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05749worker2 # [ 95.393850] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:00Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15750worker2 # [ 95.398300] nix-daemon[856]: reaped child process 1310, status = succeeded5751worker2 # [ 95.436687] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:553345752worker2 # [ 95.448600] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:553345753worker2 # [ 95.464095] nix-grpc-daemon[1265]: time=2026-09-22T10:50:00.421Z level=INFO msg="All 1 paths already cached"5754client # copying 1 paths to 'lb:50051'...5755client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5756worker2 # [ 95.472962] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:553345757worker2 # [ 95.482174] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5758worker2 # [ 95.641560] nix-daemon[856]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5759worker2 # [ 95.654731] nix-daemon[856]: reaped child process 1340, status = succeeded5760worker2 # [ 95.808593] nix-daemon[856]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5761worker2 # [ 95.822307] nix-daemon[856]: reaped child process 1345, status = succeeded5762lb # [ 95.836235] niks3-server[1143]: 2026/09/22 10:50:00 INFO Received uploads request method=POST path=/api/pending_closures5763worker2 # [ 95.874148] nix-grpc-daemon[1265]: time=2026-09-22T10:50:00.832Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5764worker2 # [ 95.876260] nix-grpc-daemon[1265]: time=2026-09-22T10:50:00.834Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5765lb # [ 95.892601] niks3-server[1143]: 2026/09/22 10:50:00 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5766lb # [ 95.902515] niks3-server[1143]: 2026/09/22 10:50:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5767lb # [ 95.910604] niks3-server[1143]: 2026/09/22 10:50:00 INFO Signed narinfos id=31 count=15768worker2 # [ 95.911675] nix-grpc-daemon[1265]: time=2026-09-22T10:50:00.869Z level=INFO msg="Uploading 1 narinfos"5769lb # [ 95.914944] niks3-server[1143]: 2026/09/22 10:50:00 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5770lb # [ 95.927538] niks3-server[1143]: 2026/09/22 10:50:00 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5771lb # [ 95.933777] niks3-server[1143]: 2026/09/22 10:50:00 INFO Completed upload id=315772worker2 # [ 95.934432] nix-grpc-daemon[1265]: time=2026-09-22T10:50:00.892Z level=INFO msg="Upload complete. (441ms)"5773lb # [ 95.937336] niks3-server[1143]: 2026/09/22 10:50:00 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5774worker2 # [ 95.939147] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55334 duration_s=0 paths=1 nar_bytes_in=6565775worker2 # [ 95.945333] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:553345776worker2 # [ 95.956648] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5777worker2 # [ 95.965985] nix-daemon[856]: reaped child process 1335, status = succeeded5778client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5779client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5780worker2 # [ 95.975589] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5781client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5782client # farm-b> LOG-farm-b-after-niks3-restart5783worker2 # [ 97.406663] nix-daemon[856]: reaped child process 1349, status = succeeded5784worker2 # [ 97.564114] nix-daemon[856]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5785worker2 # [ 97.576470] nix-daemon[856]: reaped child process 1368, status = succeeded5786lb # [ 97.591376] niks3-server[1143]: 2026/09/22 10:50:02 INFO Received uploads request method=POST path=/api/pending_closures5787worker2 # [ 97.599779] nix-grpc-daemon[1265]: time=2026-09-22T10:50:02.557Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5788worker2 # [ 97.602153] nix-grpc-daemon[1265]: time=2026-09-22T10:50:02.559Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5789lb # [ 97.627937] niks3-server[1143]: 2026/09/22 10:50:02 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5790lb # [ 97.641703] niks3-server[1143]: 2026/09/22 10:50:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5791worker2 # [ 97.647581] nix-grpc-daemon[1265]: time=2026-09-22T10:50:02.605Z level=INFO msg="Uploading 1 narinfos"5792lb # [ 97.646692] niks3-server[1143]: 2026/09/22 10:50:02 INFO Signed narinfos id=32 count=15793lb # [ 97.653184] niks3-server[1143]: 2026/09/22 10:50:02 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5794lb # [ 97.656646] niks3-server[1143]: 2026/09/22 10:50:02 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5795lb # [ 97.661056] niks3-server[1143]: 2026/09/22 10:50:02 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5796lb # [ 97.666968] niks3-server[1143]: 2026/09/22 10:50:02 INFO Completed upload id=325797worker2 # [ 97.667673] nix-grpc-daemon[1265]: time=2026-09-22T10:50:02.625Z level=INFO msg="Upload complete. (282ms)"5798lb # [ 97.668731] niks3-server[1143]: 2026/09/22 10:50:02 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5799worker2 # [ 97.675154] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55334 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15800worker2 # [ 97.682799] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:02Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05801worker2 # [ 97.687256] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:02Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15802worker2 # [ 97.694780] nix-daemon[856]: reaped child process 1347, status = succeeded5803worker2 # [ 97.727346] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:553345804worker2 # [ 97.737767] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:553345805worker2 # [ 97.751634] nix-grpc-daemon[1265]: time=2026-09-22T10:50:02.709Z level=INFO msg="All 2 paths already cached"5806client # copying 1 paths to 'lb:50051'...5807client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5808worker2 # [ 97.759794] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:553345809worker2 # [ 97.768149] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5810worker2 # [ 97.921657] nix-daemon[856]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)5811worker2 # [ 97.935294] nix-daemon[856]: reaped child process 1377, status = succeeded5812worker2 # [ 98.057251] nix-daemon[856]: accepted connection from pid 1379, user nix-grpc-daemon (trusted)5813worker2 # [ 98.070494] nix-daemon[856]: reaped child process 1382, status = succeeded5814lb # [ 98.077880] niks3-server[1143]: 2026/09/22 10:50:02 INFO Received uploads request method=POST path=/api/pending_closures5815worker2 # [ 98.088128] nix-grpc-daemon[1265]: time=2026-09-22T10:50:03.046Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5816worker2 # [ 98.089463] nix-grpc-daemon[1265]: time=2026-09-22T10:50:03.046Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5817lb # [ 98.105484] niks3-server[1143]: 2026/09/22 10:50:02 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5818lb # [ 98.113729] niks3-server[1143]: 2026/09/22 10:50:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5819lb # [ 98.118493] niks3-server[1143]: 2026/09/22 10:50:02 INFO Signed narinfos id=33 count=15820worker2 # [ 98.121521] nix-grpc-daemon[1265]: time=2026-09-22T10:50:03.079Z level=INFO msg="Uploading 1 narinfos"5821lb # [ 98.126170] niks3-server[1143]: 2026/09/22 10:50:03 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5822lb # [ 98.135575] niks3-server[1143]: 2026/09/22 10:50:03 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5823lb # [ 98.140937] envoy[920]: conn peer=[2001:db8:1::1]:51662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=305 ms=16685824lb # [ 98.143660] niks3-server[1143]: 2026/09/22 10:50:03 INFO Completed upload id=335825worker2 # [ 98.143980] nix-grpc-daemon[1265]: time=2026-09-22T10:50:03.102Z level=INFO msg="Upload complete. (363ms)"5826lb # [ 98.145359] niks3-server[1143]: 2026/09/22 10:50:03 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5827worker2 # [ 98.146074] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55334 duration_s=0 paths=1 nar_bytes_in=7845828worker2 # [ 98.149878] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:553345829worker2 # [ 98.156286] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5830worker2 # [ 98.158656] nix-daemon[856]: reaped child process 1372, status = succeeded5831worker2 # [ 98.164084] nix-daemon[856]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5832client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5833client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5834client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5835client # farm-top> LOG-farm-top-after-niks3-restart5836worker2 # [ 98.776620] nix-daemon[856]: reaped child process 1386, status = succeeded5837worker2 # [ 98.913285] nix-daemon[856]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)5838worker2 # [ 98.926572] nix-daemon[856]: reaped child process 1405, status = succeeded5839lb # [ 98.939638] niks3-server[1143]: 2026/09/22 10:50:03 INFO Received uploads request method=POST path=/api/pending_closures5840worker2 # [ 98.948455] nix-grpc-daemon[1265]: time=2026-09-22T10:50:03.906Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5841worker2 # [ 98.950023] nix-grpc-daemon[1265]: time=2026-09-22T10:50:03.908Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5842lb # [ 98.974772] niks3-server[1143]: 2026/09/22 10:50:03 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5843lb # [ 98.985154] niks3-server[1143]: 2026/09/22 10:50:03 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5844lb # [ 98.993352] niks3-server[1143]: 2026/09/22 10:50:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5845worker2 # [ 98.998566] nix-grpc-daemon[1265]: time=2026-09-22T10:50:03.956Z level=INFO msg="Uploading 1 narinfos"5846lb # [ 98.998100] niks3-server[1143]: 2026/09/22 10:50:03 INFO Signed narinfos id=34 count=15847lb # [ 99.003726] niks3-server[1143]: 2026/09/22 10:50:03 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5848lb # [ 99.012809] niks3-server[1143]: 2026/09/22 10:50:03 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5849lb # [ 99.018577] niks3-server[1143]: 2026/09/22 10:50:03 INFO Completed upload id=345850worker2 # [ 99.018853] nix-grpc-daemon[1265]: time=2026-09-22T10:50:03.977Z level=INFO msg="Upload complete. (249ms)"5851lb # [ 99.021294] niks3-server[1143]: 2026/09/22 10:50:03 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5852worker2 # [ 99.028182] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55334 duration_s=0 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15853worker2 # [ 99.038880] nix-daemon[856]: reaped child process 1384, status = succeeded5854client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart >&2, in 5.41 seconds)5855client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart outPath5856client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag after-niks3-restart outPath, in 0.20 seconds)5857(finished: subtest: a niks3 restart does not cost the leader its role, in 6.16 seconds)5858subtest: scheduler down: the other node takes the lock and keeps it5859lb: must succeed: curl -sf localhost:9901/clusters5860lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5861lb: must succeed: curl -sf localhost:9901/clusters5862lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5863lb: must succeed: curl -sf localhost:9901/clusters5864lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5865lb: must succeed: curl -sf localhost:9901/clusters5866lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5867worker2 # [ 99.510579] systemd[1]: Stopping Nix worker-protocol over gRPC...5868worker2 # [ 100.003604] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:04Z level=info event=signal_exit5869lb # [ 100.442781] niks3-server[1143]: 2026/09/22 10:50:05 INFO lead: released remote=192.168.1.4:436265870worker2 # [ 101.444870] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:06Z level=info event=scheduler_restarting5871worker1 # [ 101.446593] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:06Z level=info event=scheduler_restarting addr=lb:500515872worker2 # [ 101.450943] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:06Z level=info event=scheduler_restarting addr=lb:500515873worker2 # [ 101.454644] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:553205874worker2 # [ 101.458195] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:06Z level=info event=worker_gone addr=192.168.1.3:500515875worker2 # [ 101.461544] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:553205876worker2 # [ 101.465316] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:06Z level=info event=worker_gone addr=192.168.1.4:500515877worker2 # [ 101.479404] nix-daemon[856]: reaped child process 1257, status = succeeded5878worker1 # [ 101.646438] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:06Z level=info event=scheduler_connected addr=lb:500515879worker1 # [ 101.650760] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5880worker2 # [ 101.657872] nix-grpc-daemon[1205]: ts=2026-09-22T10:50:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5881worker1 # [ 101.851651] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:06Z level=info event=scheduler_connected addr=lb:500515882worker1 # [ 101.855761] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5883worker2 # [ 102.031407] nix-grpc-daemon[1205]: E0000 00:00:1790074206.989348 1250 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5884worker2 # [ 102.036213] nix-grpc-daemon[1205]: E0000 00:00:1790074206.989754 1250 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5885worker1 # [ 102.056539] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_connected addr=lb:500515886worker1 # [ 102.060487] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5887worker1 # [ 102.261512] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_connected addr=lb:500515888worker1 # [ 102.265529] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5889worker1 # [ 102.467105] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_connected addr=lb:500515890worker1 # [ 102.470843] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5891worker1 # [ 102.671772] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_connected addr=lb:500515892worker1 # [ 102.675679] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5893worker1 # [ 102.876120] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_connected addr=lb:500515894worker1 # [ 102.877894] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5895worker1 # [ 103.078403] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_connected addr=lb:500515896worker1 # [ 103.080110] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5897worker1 # [ 103.281099] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_connected addr=lb:500515898worker1 # [ 103.284953] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5899worker1 # [ 103.487208] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_connected addr=lb:500515900worker1 # [ 103.490314] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5901worker2 # [ 103.530492] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5902worker2 # [ 103.533602] systemd[1]: Stopped Nix worker-protocol over gRPC.5903worker2 # [ 103.536960] systemd[1]: nix-grpc-daemon.service: Consumed 1.109s CPU time over 16.711s wall clock time, 35.7M memory peak, 46.3K incoming IP traffic, 83.4K outgoing IP traffic.5904worker2 # [ 103.549393] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5905worker2 # [ 103.553168] systemd[1]: Closed Nix worker-protocol over gRPC.5906worker2 # [ 103.556487] systemd[1]: nix-grpc-daemon.socket: Consumed 309us CPU time over 17.356s wall clock time, 256K memory peak, 38K incoming IP traffic, 28K outgoing IP traffic.5907worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5908worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5909worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5910worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5911worker1 # [ 103.687685] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_connected addr=lb:500515912worker1 # [ 103.689401] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5913worker1 # [ 103.890320] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_connected addr=lb:500515914worker1 # [ 103.894208] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5915worker1 # [ 104.095391] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=scheduler_connected addr=lb:500515916worker1 # [ 104.099266] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5917lb # [ 104.232856] niks3-server[1143]: 2026/09/22 10:50:09 INFO lead: acquired remote=192.168.1.3:351185918worker1 # [ 104.235631] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=scheduler_take_over5919worker1 # [ 104.299959] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=scheduler_connected addr=lb:500515920worker1 # [ 104.303894] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5921worker1 # [ 104.506207] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=scheduler_connected addr=lb:500515922worker1 # [ 104.509366] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5923worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5924worker1 # [ 104.706867] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=scheduler_connected addr=lb:500515925worker1 # [ 104.714448] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:492145926worker1 # [ 104.718319] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05927worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5928client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25929worker1 # [ 104.943943] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505485930client # don't know how to build these paths:5931client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5932worker1 # [ 104.953176] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505485933worker1 # [ 104.964467] nix-grpc-daemon[1201]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5)5934worker1 # [ 104.987980] nix-grpc-daemon[1201]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 150 ms (attempt 2/5)5935worker1 # [ 105.140827] nix-grpc-daemon[1201]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 371 ms (attempt 3/5)5936worker1 # [ 105.514930] nix-grpc-daemon[1201]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 355 ms (attempt 4/5)5937worker1 # [ 105.873188] nix-grpc-daemon[1201]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5938worker1 # [ 105.876099] nix-grpc-daemon[1201]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5939worker1 # [ 105.897750] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:10Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05940worker1 # [ 105.903295] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:10Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15941worker1 # [ 105.946769] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505485942worker1 # [ 105.959374] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505485943client # copying 1 paths to 'lb:50051'...5944client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5945worker1 # [ 105.971813] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505485946worker1 # [ 105.983394] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5947worker1 # [ 106.000912] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:11Z level=info event=niks3_push_started pid=12775948worker1 # [ 106.153235] nix-grpc-daemon[1277]: time=2026-09-22T10:50:11.159Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5949worker1 # [ 106.279708] nix-daemon[867]: accepted connection from pid 1290, user nix-grpc-daemon (trusted)5950worker1 # [ 106.290643] nix-daemon[867]: reaped child process 1293, status = succeeded5951worker1 # [ 106.429442] nix-daemon[867]: accepted connection from pid 1295, user nix-grpc-daemon (trusted)5952worker1 # [ 106.441171] nix-daemon[867]: reaped child process 1298, status = succeeded5953lb # [ 106.449713] niks3-server[1143]: 2026/09/22 10:50:11 INFO Received uploads request method=POST path=/api/pending_closures5954worker1 # [ 106.461350] nix-grpc-daemon[1277]: time=2026-09-22T10:50:11.468Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5955worker1 # [ 106.463287] nix-grpc-daemon[1277]: time=2026-09-22T10:50:11.470Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5956lb # [ 106.483418] niks3-server[1143]: 2026/09/22 10:50:11 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5957lb # [ 106.495975] niks3-server[1143]: 2026/09/22 10:50:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5958worker1 # [ 106.506346] nix-grpc-daemon[1277]: time=2026-09-22T10:50:11.513Z level=INFO msg="Uploading 1 narinfos"5959lb # [ 106.504145] niks3-server[1143]: 2026/09/22 10:50:11 INFO Signed narinfos id=35 count=15960lb # [ 106.511709] niks3-server[1143]: 2026/09/22 10:50:11 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5961lb # [ 106.522470] niks3-server[1143]: 2026/09/22 10:50:11 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5962lb # [ 106.529663] niks3-server[1143]: 2026/09/22 10:50:11 INFO Completed upload id=355963worker1 # [ 106.532165] nix-grpc-daemon[1277]: time=2026-09-22T10:50:11.539Z level=INFO msg="Upload complete. (375ms)"5964lb # [ 106.531470] niks3-server[1143]: 2026/09/22 10:50:11 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5965worker1 # [ 106.538168] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50548 duration_s=0 paths=1 nar_bytes_in=5205966worker1 # [ 106.542394] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505485967worker1 # [ 106.551932] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5968worker1 # [ 106.557992] nix-daemon[867]: reaped child process 1275, status = succeeded5969client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5970client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5971worker1 # [ 106.561653] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5972client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5973client # farm-a> LOG-farm-a-on-standby5974worker1 # [ 107.773894] nix-daemon[867]: reaped child process 1302, status = succeeded5975worker1 # [ 107.908217] nix-daemon[867]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5976worker1 # [ 107.920545] nix-daemon[867]: reaped child process 1321, status = succeeded5977lb # [ 107.932241] niks3-server[1143]: 2026/09/22 10:50:12 INFO Received uploads request method=POST path=/api/pending_closures5978worker1 # [ 107.943085] nix-grpc-daemon[1277]: time=2026-09-22T10:50:12.950Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5979worker1 # [ 107.944610] nix-grpc-daemon[1277]: time=2026-09-22T10:50:12.951Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5980lb # [ 107.963844] niks3-server[1143]: 2026/09/22 10:50:12 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5981lb # [ 107.977325] niks3-server[1143]: 2026/09/22 10:50:12 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5982lb # [ 107.988925] niks3-server[1143]: 2026/09/22 10:50:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5983worker1 # [ 107.996532] nix-grpc-daemon[1277]: time=2026-09-22T10:50:13.003Z level=INFO msg="Uploading 1 narinfos"5984lb # [ 107.993793] niks3-server[1143]: 2026/09/22 10:50:12 INFO Signed narinfos id=36 count=15985lb # [ 107.999697] niks3-server[1143]: 2026/09/22 10:50:12 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5986lb # [ 108.006809] niks3-server[1143]: 2026/09/22 10:50:12 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5987lb # [ 108.012145] niks3-server[1143]: 2026/09/22 10:50:12 INFO Completed upload id=365988lb # [ 108.014238] niks3-server[1143]: 2026/09/22 10:50:12 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5989worker1 # [ 108.016067] nix-grpc-daemon[1277]: time=2026-09-22T10:50:13.023Z level=INFO msg="Upload complete. (247ms)"5990worker1 # [ 108.023835] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50548 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15991worker1 # [ 108.033306] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:13Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05992worker1 # [ 108.037449] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:13Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15993worker1 # [ 108.045370] nix-daemon[867]: reaped child process 1300, status = succeeded5994worker1 # [ 108.077247] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505485995worker1 # [ 108.085364] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505485996worker1 # [ 108.092681] nix-grpc-daemon[1277]: time=2026-09-22T10:50:13.099Z level=INFO msg="All 1 paths already cached"5997client # copying 1 paths to 'lb:50051'...5998client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5999worker1 # [ 108.098713] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505486000worker1 # [ 108.103912] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)6001lb # [ 108.141192] envoy[920]: conn peer=[2001:db8:1::1]:48704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9666 tx=4065 ms=51576002lb # [ 108.144856] envoy[920]: conn peer=[2001:db8:1::4]:57626 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5880 tx=648 ms=146096003worker1 # [ 108.262128] nix-daemon[867]: accepted connection from pid 1327, user nix-grpc-daemon (trusted)6004worker1 # [ 108.275083] nix-daemon[867]: reaped child process 1330, status = succeeded6005worker1 # [ 108.396786] nix-daemon[867]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)6006worker1 # [ 108.409088] nix-daemon[867]: reaped child process 1335, status = succeeded6007lb # [ 108.420398] niks3-server[1143]: 2026/09/22 10:50:13 INFO Received uploads request method=POST path=/api/pending_closures6008worker1 # [ 108.431897] nix-grpc-daemon[1277]: time=2026-09-22T10:50:13.439Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6009worker1 # [ 108.433431] nix-grpc-daemon[1277]: time=2026-09-22T10:50:13.440Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6010lb # [ 108.450075] niks3-server[1143]: 2026/09/22 10:50:13 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6011lb # [ 108.460221] niks3-server[1143]: 2026/09/22 10:50:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6012lb # [ 108.464784] niks3-server[1143]: 2026/09/22 10:50:13 INFO Signed narinfos id=37 count=16013worker1 # [ 108.470146] nix-grpc-daemon[1277]: time=2026-09-22T10:50:13.477Z level=INFO msg="Uploading 1 narinfos"6014lb # [ 108.474698] niks3-server[1143]: 2026/09/22 10:50:13 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6015lb # [ 108.483402] niks3-server[1143]: 2026/09/22 10:50:13 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6016worker1 # [ 108.492053] nix-grpc-daemon[1277]: time=2026-09-22T10:50:13.499Z level=INFO msg="Upload complete. (381ms)"6017lb # [ 108.489883] niks3-server[1143]: 2026/09/22 10:50:13 INFO Completed upload id=376018worker1 # [ 108.494146] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50548 duration_s=0 paths=1 nar_bytes_in=6406019lb # [ 108.493080] niks3-server[1143]: 2026/09/22 10:50:13 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6020worker1 # [ 108.499419] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505486021worker1 # [ 108.505485] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)6022worker1 # [ 108.506800] nix-daemon[867]: reaped child process 1325, status = succeeded6023worker1 # [ 108.512042] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)6024client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6025client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6026client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6027client # farm-b> LOG-farm-b-on-standby6028worker1 # [ 109.337725] nix-daemon[867]: reaped child process 1339, status = succeeded6029worker1 # [ 109.481098] nix-daemon[867]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)6030worker1 # [ 109.492891] nix-daemon[867]: reaped child process 1358, status = succeeded6031lb # [ 109.504073] niks3-server[1143]: 2026/09/22 10:50:14 INFO Received uploads request method=POST path=/api/pending_closures6032worker1 # [ 109.512412] nix-grpc-daemon[1277]: time=2026-09-22T10:50:14.519Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6033worker1 # [ 109.514483] nix-grpc-daemon[1277]: time=2026-09-22T10:50:14.521Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6034lb # [ 109.523592] niks3-server[1143]: 2026/09/22 10:50:14 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6035lb # [ 109.529642] niks3-server[1143]: 2026/09/22 10:50:14 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6036lb # [ 109.533408] niks3-server[1143]: 2026/09/22 10:50:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6037lb # [ 109.535057] niks3-server[1143]: 2026/09/22 10:50:14 INFO Signed narinfos id=38 count=16038worker1 # [ 109.538020] nix-grpc-daemon[1277]: time=2026-09-22T10:50:14.544Z level=INFO msg="Uploading 1 narinfos"6039lb # [ 109.539546] niks3-server[1143]: 2026/09/22 10:50:14 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6040lb # [ 109.543043] niks3-server[1143]: 2026/09/22 10:50:14 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6041lb # [ 109.545579] niks3-server[1143]: 2026/09/22 10:50:14 INFO Completed upload id=386042worker1 # [ 109.549248] nix-grpc-daemon[1277]: time=2026-09-22T10:50:14.556Z level=INFO msg="Upload complete. (230ms)"6043lb # [ 109.547876] niks3-server[1143]: 2026/09/22 10:50:14 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6044worker1 # [ 109.551846] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50548 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16045worker1 # [ 109.558369] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:14Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06046worker1 # [ 109.560279] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:14Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16047worker1 # [ 109.564555] nix-daemon[867]: reaped child process 1337, status = succeeded6048worker1 # [ 109.602228] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505486049worker1 # [ 109.608677] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505486050worker1 # [ 109.617358] nix-grpc-daemon[1277]: time=2026-09-22T10:50:14.624Z level=INFO msg="All 2 paths already cached"6051client # copying 1 paths to 'lb:50051'...6052client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6053worker1 # [ 109.624067] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505486054worker1 # [ 109.630893] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)6055worker1 # [ 109.783516] nix-daemon[867]: accepted connection from pid 1364, user nix-grpc-daemon (trusted)6056worker1 # [ 109.795853] nix-daemon[867]: reaped child process 1367, status = succeeded6057worker1 # [ 109.944927] nix-daemon[867]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6058worker1 # [ 109.957601] nix-daemon[867]: reaped child process 1372, status = succeeded6059lb # [ 109.969273] niks3-server[1143]: 2026/09/22 10:50:14 INFO Received uploads request method=POST path=/api/pending_closures6060worker1 # [ 109.976397] nix-grpc-daemon[1277]: time=2026-09-22T10:50:14.983Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6061worker1 # [ 109.977844] nix-grpc-daemon[1277]: time=2026-09-22T10:50:14.983Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6062lb # [ 109.982448] niks3-server[1143]: 2026/09/22 10:50:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6063lb # [ 109.985790] niks3-server[1143]: 2026/09/22 10:50:14 INFO Signed narinfos id=39 count=16064worker1 # [ 109.992113] nix-grpc-daemon[1277]: time=2026-09-22T10:50:14.999Z level=INFO msg="Uploading 1 narinfos"6065lb # [ 109.991841] niks3-server[1143]: 2026/09/22 10:50:14 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6066lb # [ 110.004514] niks3-server[1143]: 2026/09/22 10:50:14 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6067lb # [ 110.012377] niks3-server[1143]: 2026/09/22 10:50:14 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6068lb # [ 110.019121] niks3-server[1143]: 2026/09/22 10:50:14 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6069lb # [ 110.027337] niks3-server[1143]: 2026/09/22 10:50:14 INFO Completed upload id=396070worker1 # [ 110.029767] nix-grpc-daemon[1277]: time=2026-09-22T10:50:15.036Z level=INFO msg="Upload complete. (390ms)"6071worker1 # [ 110.033078] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50548 duration_s=0 paths=1 nar_bytes_in=7686072worker1 # [ 110.035518] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505486073worker1 # [ 110.041262] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)6074worker1 # [ 110.044211] nix-daemon[867]: reaped child process 1362, status = succeeded6075worker1 # [ 110.048994] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)6076client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6077client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6078client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6079client # farm-top> LOG-farm-top-on-standby6080worker1 # [ 111.272867] nix-daemon[867]: reaped child process 1376, status = succeeded6081worker1 # [ 111.435353] nix-daemon[867]: accepted connection from pid 1392, user nix-grpc-daemon (trusted)6082worker1 # [ 111.447622] nix-daemon[867]: reaped child process 1395, status = succeeded6083lb # [ 111.456411] niks3-server[1143]: 2026/09/22 10:50:16 INFO Received uploads request method=POST path=/api/pending_closures6084worker1 # [ 111.468678] nix-grpc-daemon[1277]: time=2026-09-22T10:50:16.475Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6085worker1 # [ 111.470148] nix-grpc-daemon[1277]: time=2026-09-22T10:50:16.477Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6086lb # [ 111.497069] niks3-server[1143]: 2026/09/22 10:50:16 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6087lb # [ 111.505367] niks3-server[1143]: 2026/09/22 10:50:16 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6088lb # [ 111.516569] niks3-server[1143]: 2026/09/22 10:50:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6089worker1 # [ 111.525387] nix-grpc-daemon[1277]: time=2026-09-22T10:50:16.532Z level=INFO msg="Uploading 1 narinfos"6090lb # [ 111.523659] niks3-server[1143]: 2026/09/22 10:50:16 INFO Signed narinfos id=40 count=16091lb # [ 111.527977] niks3-server[1143]: 2026/09/22 10:50:16 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6092lb # [ 111.537546] niks3-server[1143]: 2026/09/22 10:50:16 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6093lb # [ 111.541969] niks3-server[1143]: 2026/09/22 10:50:16 INFO Completed upload id=406094worker1 # [ 111.545703] nix-grpc-daemon[1277]: time=2026-09-22T10:50:16.552Z level=INFO msg="Upload complete. (290ms)"6095lb # [ 111.544426] niks3-server[1143]: 2026/09/22 10:50:16 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6096worker1 # [ 111.552511] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50548 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16097worker1 # [ 111.565845] nix-daemon[867]: reaped child process 1374, status = succeeded6098client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.83 seconds)6099client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath6100client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.19 seconds)6101worker2 # [ 111.838561] systemd[1]: Listening on Nix worker-protocol over gRPC.6102worker2 # [ 111.845861] systemd[1]: Starting scheduler-token.service...6103worker2 # [ 111.916072] systemd[1]: scheduler-token.service: Deactivated successfully.6104worker2 # [ 111.917550] systemd[1]: Finished scheduler-token.service.6105worker2 # [ 111.920175] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 71ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6106worker2 # [ 111.924420] systemd[1]: Starting Nix worker-protocol over gRPC...6107worker2 # [ 112.083826] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:17Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6108worker2 # [ 112.086268] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6109worker2 # [ 112.090098] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16110worker2 # [ 112.098068] nix-grpc-daemon[1426]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6111worker2 # [ 112.099556] nix-grpc-daemon[1426]: E0000 00:00:1790074217.055577 1426 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6112worker2 # [ 112.105120] nix-grpc-daemon[1426]: E0000 00:00:1790074217.060807 1426 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6113worker2 # [ 112.108729] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6114worker2 # [ 112.111465] systemd[1]: Started Nix worker-protocol over gRPC.6115worker2 # [ 112.117054] nix-grpc-daemon[1426]: E0000 00:00:1790074217.074334 1468 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6116lb: must succeed: curl -sf localhost:9901/clusters6117worker2 # [ 112.128895] nix-grpc-daemon[1426]: E0000 00:00:1790074217.087274 1468 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6118worker2 # [ 112.132134] nix-grpc-daemon[1426]: E0000 00:00:1790074217.089447 1468 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6119worker2 # [ 112.143035] nix-grpc-daemon[1426]: E0000 00:00:1790074217.101399 1468 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6120worker2 # [ 112.154928] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:17Z level=info event=scheduler_connected addr=lb:500516121worker1 # [ 112.165233] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:17Z level=info event=oidc_jwks provider=mock keys=16122worker1 # [ 112.169418] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:492146123worker1 # [ 112.173321] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06124lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6125worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6126worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6127lb: must succeed: curl -sf localhost:9901/clusters6128lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6129lb: must succeed: curl -sf localhost:9901/clusters6130lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6131worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6132worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6133(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.19 seconds)6134subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6135client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs6136client # [ 112.502662] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.6137client # Running as unit: rs.service; invocation ID: 0f49b805a76e4593bb69b645422a021f6138client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)6139worker2 # [ 112.685902] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:576666140worker2 # [ 112.695985] nix-daemon[856]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6141client # [ 112.719280] nix[1271]: don't know how to build these paths:6142client # [ 112.720210] nix[1271]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6143worker1 # [ 112.724592] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:17Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06144worker1 # [ 112.728600] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:17Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16145worker1 # [ 112.767289] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505486146worker1 # [ 112.780385] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505486147client # [ 112.797436] nix[1271]: copying 1 paths to 'lb:50051'...6148worker1 # [ 112.791530] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505486149client # [ 112.800139] nix[1271]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6150worker1 # [ 112.801674] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)6151worker1 # [ 112.954610] nix-daemon[867]: accepted connection from pid 1408, user nix-grpc-daemon (trusted)6152worker1 # [ 112.966641] nix-daemon[867]: reaped child process 1411, status = succeeded6153worker1 # [ 113.094961] nix-daemon[867]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6154worker1 # [ 113.106860] nix-daemon[867]: reaped child process 1416, status = succeeded6155lb # [ 113.115526] niks3-server[1143]: 2026/09/22 10:50:17 INFO Received uploads request method=POST path=/api/pending_closures6156worker1 # [ 113.126512] nix-grpc-daemon[1277]: time=2026-09-22T10:50:18.133Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6157worker1 # [ 113.127932] nix-grpc-daemon[1277]: time=2026-09-22T10:50:18.133Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6158lb # [ 113.147288] niks3-server[1143]: 2026/09/22 10:50:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6159lb # [ 113.152544] niks3-server[1143]: 2026/09/22 10:50:18 INFO Signed narinfos id=41 count=16160worker1 # [ 113.157446] nix-grpc-daemon[1277]: time=2026-09-22T10:50:18.164Z level=INFO msg="Uploading 1 narinfos"6161lb # [ 113.155667] niks3-server[1143]: 2026/09/22 10:50:18 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6162lb # [ 113.166613] niks3-server[1143]: 2026/09/22 10:50:18 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6163lb # [ 113.173115] niks3-server[1143]: 2026/09/22 10:50:18 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6164lb # [ 113.180161] niks3-server[1143]: 2026/09/22 10:50:18 INFO Completed upload id=416165worker1 # [ 113.182580] nix-grpc-daemon[1277]: time=2026-09-22T10:50:18.189Z level=INFO msg="Upload complete. (367ms)"6166lb # [ 113.181955] niks3-server[1143]: 2026/09/22 10:50:18 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6167worker1 # [ 113.184707] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50548 duration_s=0 paths=1 nar_bytes_in=4166168worker1 # [ 113.188552] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505486169worker1 # [ 113.196282] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)6170worker1 # [ 113.199336] nix-daemon[867]: reaped child process 1406, status = succeeded6171worker1 # [ 113.204891] nix-daemon[867]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)6172client # [ 113.210475] nix[1271]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6173lb: must succeed: curl -sf localhost:9901/clusters6174lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6175lb: must succeed: curl -sf localhost:9901/clusters6176lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6177lb: must succeed: curl -sf localhost:9901/clusters6178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6179lb: must succeed: curl -sf localhost:9901/clusters6180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6181worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6182worker2 # [ 115.369006] systemd-journald[287]: Received client request to rotate journal, rotating.6183worker2 # [ 115.384986] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/f72413b90a6a4542a90a86403397559b.6184worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)6185worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6186worker1 # [ 115.307978] systemd[1]: Stopping Nix worker-protocol over gRPC...6187worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)6188client: must succeed: sleep 5.06189worker1 # [ 115.949825] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:20Z level=info event=draining6190lb # [ 118.141843] envoy[920]: conn peer=[2001:db8:1::1]:50416 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9564 tx=4100 ms=66206191client: (finished: must succeed: sleep 5.0, in 5.06 seconds)6192worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6193worker1 # [ 120.624813] nix-daemon[867]: reaped child process 1420, status = succeeded6194worker1 # [ 120.776097] nix-daemon[867]: accepted connection from pid 1461, user nix-grpc-daemon (trusted)6195worker1 # [ 120.787647] nix-daemon[867]: reaped child process 1464, status = succeeded6196lb # [ 120.801428] niks3-server[1143]: 2026/09/22 10:50:25 INFO Received uploads request method=POST path=/api/pending_closures6197worker1 # [ 120.812406] nix-grpc-daemon[1277]: time=2026-09-22T10:50:25.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6198worker1 # [ 120.815059] nix-grpc-daemon[1277]: time=2026-09-22T10:50:25.821Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6199lb # [ 120.842411] niks3-server[1143]: 2026/09/22 10:50:25 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6200lb # [ 120.846838] niks3-server[1143]: 2026/09/22 10:50:25 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6201lb # [ 120.856696] niks3-server[1143]: 2026/09/22 10:50:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6202lb # [ 120.863928] niks3-server[1143]: 2026/09/22 10:50:25 INFO Signed narinfos id=42 count=16203worker1 # [ 120.869534] nix-grpc-daemon[1277]: time=2026-09-22T10:50:25.876Z level=INFO msg="Uploading 1 narinfos"6204lb # [ 120.870086] niks3-server[1143]: 2026/09/22 10:50:25 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6205lb # [ 120.880728] niks3-server[1143]: 2026/09/22 10:50:25 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6206lb # [ 120.885588] niks3-server[1143]: 2026/09/22 10:50:25 INFO Completed upload id=426207worker1 # [ 120.888318] nix-grpc-daemon[1277]: time=2026-09-22T10:50:25.895Z level=INFO msg="Upload complete. (271ms)"6208lb # [ 120.888109] niks3-server[1143]: 2026/09/22 10:50:25 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6209worker1 # [ 120.894926] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50548 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16210worker1 # [ 120.906254] nix-daemon[867]: reaped child process 1418, status = succeeded6211client # [ 120.933340] systemd[1]: rs.service: Deactivated successfully.6212client # [ 120.935500] systemd[1]: rs.service: Consumed 125ms CPU time over 8.432s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.6213worker1 # [ 120.950772] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:25Z level=info event=signal_exit6214lb # [ 121.464177] niks3-server[1143]: 2026/09/22 10:50:26 INFO lead: released remote=192.168.1.3:351186215lb # [ 122.115333] niks3-server[1143]: 2026/09/22 10:50:26 INFO lead: acquired remote=192.168.1.4:472286216worker2 # [ 122.119465] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:27Z level=info event=scheduler_take_over6217worker1 # [ 122.465546] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:27Z level=info event=scheduler_restarting6218worker1 # [ 122.468508] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:27Z level=info event=scheduler_restarting addr=lb:500516219worker2 # [ 122.465843] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:27Z level=info event=scheduler_restarting addr=lb:500516220worker1 # [ 122.471241] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:492146221worker1 # [ 122.472579] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:27Z level=info event=worker_gone addr=192.168.1.4:500516222worker1 # [ 122.474687] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:492146223worker1 # [ 122.476163] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:27Z level=info event=worker_gone addr=192.168.1.3:500516224worker1 # [ 122.484548] nix-daemon[867]: reaped child process 1253, status = succeeded6225worker2 # [ 122.667560] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:27Z level=info event=scheduler_connected addr=lb:500516226worker1 # [ 122.674599] nix-grpc-daemon[1201]: ts=2026-09-22T10:50:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6227worker2 # [ 122.671611] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6228worker1 # [ 122.721858] nix-grpc-daemon[1201]: E0000 00:00:1790074227.728675 1246 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6229worker1 # [ 122.726598] nix-grpc-daemon[1201]: E0000 00:00:1790074227.728845 1246 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6230worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.13 seconds)6231client: waiting for success: ! systemctl is-active rs6232client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)6233client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6234client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)6235client: must fail: journalctl -u rs | grep -q 'reconnecting'6236worker2 # [ 122.872393] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:27Z level=info event=scheduler_connected addr=lb:500516237worker2 # [ 122.874141] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6238client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6239lb: must succeed: curl -sf localhost:9901/clusters6240lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6241worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6242worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6243worker2 # [ 123.075379] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=scheduler_connected addr=lb:500516244worker2 # [ 123.079116] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6245worker2 # [ 123.281236] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=scheduler_connected addr=lb:500516246worker2 # [ 123.284797] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6247worker2 # [ 123.485854] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=scheduler_connected addr=lb:500516248worker2 # [ 123.489865] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6249worker2 # [ 123.690820] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=scheduler_connected addr=lb:500516250worker2 # [ 123.703134] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=oidc_jwks provider=mock keys=16251worker2 # [ 123.707174] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:570346252worker2 # [ 123.711085] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06253lb: must succeed: curl -sf localhost:9901/clusters6254lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6255worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6256worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6257worker1 # [ 124.300460] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6258worker1 # [ 124.303923] systemd[1]: Stopped Nix worker-protocol over gRPC.6259worker1 # [ 124.306202] systemd[1]: nix-grpc-daemon.service: Consumed 1.278s CPU time over 32.526s wall clock time, 38.5M memory peak, 48.7K incoming IP traffic, 95K outgoing IP traffic.6260worker1 # [ 124.322638] systemd[1]: Starting Nix worker-protocol over gRPC...6261worker1 # [ 124.479318] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6262worker1 # [ 124.481185] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6263worker1 # [ 124.484622] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16264worker1 # [ 124.491067] nix-grpc-daemon[1471]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6265worker1 # [ 124.492401] nix-grpc-daemon[1471]: E0000 00:00:1790074229.497729 1471 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6266worker1 # [ 124.497660] nix-grpc-daemon[1471]: E0000 00:00:1790074229.502794 1471 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6267worker1 # [ 124.500696] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:29Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6268worker1 # [ 124.503382] systemd[1]: Started Nix worker-protocol over gRPC.6269worker1 # [ 124.509067] nix-grpc-daemon[1471]: E0000 00:00:1790074229.514870 1513 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6270worker1 # [ 124.520235] nix-grpc-daemon[1471]: E0000 00:00:1790074229.527393 1513 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6271worker1 # [ 124.525066] nix-grpc-daemon[1471]: E0000 00:00:1790074229.529761 1513 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6272worker1 # [ 124.534626] nix-grpc-daemon[1471]: E0000 00:00:1790074229.541799 1513 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6273worker1 # [ 124.544097] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:29Z level=info event=scheduler_connected addr=lb:500516274worker2 # [ 124.544436] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:570346275worker2 # [ 124.548637] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06276lb: must succeed: curl -sf localhost:9901/clusters6277lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6278worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6279worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6280(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.79 seconds)6281subtest: scheduler restart mid-build: build finishes, no double build6282client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid6283client # [ 125.295983] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.6284client # Running as unit: mid.service; invocation ID: 52c371f2b8c548359183961e85a5e9946285client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)6286worker1 # [ 125.530183] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590746287worker1 # [ 125.542225] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6288client # [ 125.561433] nix[1307]: don't know how to build these paths:6289client # [ 125.562331] nix[1307]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6290worker2 # [ 125.567498] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:30Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06291worker2 # [ 125.571518] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:30Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16292worker2 # [ 125.610119] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576666293worker2 # [ 125.622250] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)6294worker2 # [ 125.723126] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 2/5)6295worker2 # [ 125.895132] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 3/5)6296worker2 # [ 125.975187] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 301 ms (attempt 4/5)6297worker2 # [ 126.276888] nix-grpc-daemon[1426]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6298worker2 # [ 126.280803] nix-grpc-daemon[1426]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6299worker2 # [ 126.299402] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:576666300client # [ 126.317918] nix[1307]: copying 1 paths to 'lb:50051'...6301client # [ 126.322254] nix[1307]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6302worker2 # [ 126.313865] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:576666303worker2 # [ 126.325714] nix-daemon[856]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6304worker2 # [ 126.346101] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:31Z level=info event=niks3_push_started pid=15516305worker2 # [ 126.559648] nix-grpc-daemon[1551]: time=2026-09-22T10:50:31.517Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6306worker2 # [ 126.723103] nix-daemon[856]: accepted connection from pid 1568, user nix-grpc-daemon (trusted)6307worker2 # [ 126.735794] nix-daemon[856]: reaped child process 1571, status = succeeded6308worker2 # [ 126.911599] nix-daemon[856]: accepted connection from pid 1573, user nix-grpc-daemon (trusted)6309worker2 # [ 126.922359] nix-daemon[856]: reaped child process 1576, status = succeeded6310lb # [ 126.938327] niks3-server[1143]: 2026/09/22 10:50:31 INFO Received uploads request method=POST path=/api/pending_closures6311worker2 # [ 126.949412] nix-grpc-daemon[1551]: time=2026-09-22T10:50:31.907Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6312worker2 # [ 126.950976] nix-grpc-daemon[1551]: time=2026-09-22T10:50:31.909Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6313lb # [ 126.978252] niks3-server[1143]: 2026/09/22 10:50:31 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6314lb # [ 126.982418] niks3-server[1143]: 2026/09/22 10:50:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6315lb # [ 126.987774] niks3-server[1143]: 2026/09/22 10:50:31 INFO Signed narinfos id=43 count=16316worker2 # [ 126.990534] nix-grpc-daemon[1551]: time=2026-09-22T10:50:31.948Z level=INFO msg="Uploading 1 narinfos"6317lb # [ 126.995634] niks3-server[1143]: 2026/09/22 10:50:31 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6318lb # [ 127.007154] niks3-server[1143]: 2026/09/22 10:50:31 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6319lb # [ 127.009947] niks3-server[1143]: 2026/09/22 10:50:31 INFO Completed upload id=436320lb # [ 127.011432] niks3-server[1143]: 2026/09/22 10:50:31 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6321worker2 # [ 127.011510] nix-grpc-daemon[1551]: time=2026-09-22T10:50:31.969Z level=INFO msg="Upload complete. (448ms)"6322worker2 # [ 127.015392] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57666 duration_s=0 paths=1 nar_bytes_in=4246323worker2 # [ 127.021278] nix-grpc-daemon[1426]: ts=2026-09-22T10:50:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:576666324worker2 # [ 127.029278] nix-daemon[856]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6325worker2 # [ 127.034358] nix-daemon[856]: reaped child process 1549, status = succeeded6326worker2 # [ 127.039395] nix-daemon[856]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6327client # [ 127.048569] nix[1307]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6328lb # [ 128.142597] envoy[920]: conn peer=[2001:db8:1::1]:50428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1404 ms=82196329lb # [ 128.146441] envoy[920]: conn peer=[2001:db8:1::3]:49774 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4581 tx=1106 ms=306886330lb: must succeed: curl -sf localhost:9901/clusters6331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6332lb: must succeed: curl -sf localhost:9901/clusters6333lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6334worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6335worker2 # [ 129.093330] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1426 (nix-grpc-daemon) on client request.6336worker2 # [ 129.094949] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6337lb # [ 129.100708] niks3-server[1143]: 2026/09/22 10:50:33 INFO lead: released remote=192.168.1.4:472286338worker2 # [ 129.112567] systemd[1]: Starting scheduler-token.service...6339worker1 # [ 129.110393] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:34Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6340client # [ 129.118307] nix[1307]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6341client # [ 129.123426] nix[1307]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6342worker2 # [ 129.122410] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6343worker2 # [ 129.124151] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6344worker2 # [ 129.127397] systemd[1]: nix-grpc-daemon.service: Consumed 458ms CPU time over 17.197s wall clock time, 33.4M memory peak, 16.7K incoming IP traffic, 35.2K outgoing IP traffic.6345worker2 # [ 129.138888] nix-daemon[856]: reaped child process 1578, status = succeeded6346worker2 # [ 129.140934] nix-daemon[856]: reaped child process 1485, status = succeeded6347worker2 # [ 129.152272] nix-daemon[1580]: unexpected Nix daemon error: error: interrupted by the user6348worker2 # [ 129.156703] nix-daemon[856]: reaped child process 1580, status = failed with exit code 16349worker2 # [ 129.201075] systemd[1]: scheduler-token.service: Deactivated successfully.6350worker2 # [ 129.203224] systemd[1]: Finished scheduler-token.service.6351worker2 # [ 129.203955] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 88ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6352worker2 # [ 129.206185] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6353worker2 # [ 129.210791] systemd[1]: Starting scheduler-token.service...6354worker2 # [ 129.273715] systemd[1]: scheduler-token.service: Deactivated successfully.6355worker2 # [ 129.274784] systemd[1]: Finished scheduler-token.service.6356worker2 # [ 129.276197] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 62ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6357worker2 # [ 129.281557] systemd[1]: Starting Nix worker-protocol over gRPC...6358worker1 # [ 129.310045] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:34Z level=info event=scheduler_connected addr=lb:500516359worker2 # [ 129.409039] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6360worker2 # [ 129.411465] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6361worker2 # [ 129.414462] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16362worker2 # [ 129.420067] nix-grpc-daemon[1623]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6363worker2 # [ 129.422096] nix-grpc-daemon[1623]: E0000 00:00:1790074234.377888 1623 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6364worker2 # [ 129.426650] nix-grpc-daemon[1623]: E0000 00:00:1790074234.383046 1623 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6365worker2 # [ 129.430609] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6366worker2 # [ 129.433168] systemd[1]: Started Nix worker-protocol over gRPC.6367worker2 # [ 129.439807] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=scheduler_take_over6368worker1 # [ 129.441377] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6369worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.39 seconds)6370lb # [ 129.439414] niks3-server[1143]: 2026/09/22 10:50:34 INFO lead: acquired remote=192.168.1.4:367666371client: must succeed: sleep 5.06372worker2 # [ 129.441884] nix-grpc-daemon[1623]: E0000 00:00:1790074234.400323 1662 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6373worker2 # [ 129.453692] nix-grpc-daemon[1623]: E0000 00:00:1790074234.412086 1662 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6374worker2 # [ 129.457121] nix-grpc-daemon[1623]: E0000 00:00:1790074234.414327 1662 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6375worker2 # [ 129.467655] nix-grpc-daemon[1623]: E0000 00:00:1790074234.426048 1662 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6376worker2 # [ 129.476815] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=scheduler_connected addr=lb:500516377worker2 # [ 129.481835] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=oidc_jwks provider=mock keys=16378worker2 # [ 129.483276] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:570406379worker2 # [ 129.484817] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06380worker2 # [ 129.621230] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06381worker2 # [ 129.623382] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16382worker1 # [ 129.641976] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:34Z level=info event=scheduler_connected addr=lb:500516383worker2 # [ 129.641291] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:570406384worker2 # [ 129.642733] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06385worker2 # [ 129.647867] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570666386worker2 # [ 129.653071] nix-daemon[856]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)6387worker2 # [ 129.658942] nix-daemon[856]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)6388worker2 # [ 129.665745] nix-daemon[856]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)6389client # [ 129.673993] nix[1307]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6390client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6391client: waiting for success: ! systemctl is-active mid6392worker2 # [ 134.879643] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:39Z level=info event=niks3_push_started pid=17066393worker2 # [ 134.891581] nix-daemon[856]: reaped child process 1681, status = succeeded6394worker2 # [ 135.056557] nix-grpc-daemon[1706]: time=2026-09-22T10:50:40.014Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6395worker2 # [ 135.203944] nix-daemon[856]: accepted connection from pid 1719, user nix-grpc-daemon (trusted)6396worker2 # [ 135.215636] nix-daemon[856]: reaped child process 1722, status = succeeded6397lb # [ 135.233520] niks3-server[1143]: 2026/09/22 10:50:40 INFO Received uploads request method=POST path=/api/pending_closures6398worker2 # [ 135.243296] nix-grpc-daemon[1706]: time=2026-09-22T10:50:40.201Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6399worker2 # [ 135.244802] nix-grpc-daemon[1706]: time=2026-09-22T10:50:40.203Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6400lb # [ 135.281755] niks3-server[1143]: 2026/09/22 10:50:40 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6401lb # [ 135.296175] niks3-server[1143]: 2026/09/22 10:50:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6402worker2 # [ 135.300130] nix-grpc-daemon[1706]: time=2026-09-22T10:50:40.258Z level=INFO msg="Uploading 1 narinfos"6403lb # [ 135.299652] niks3-server[1143]: 2026/09/22 10:50:40 INFO Signed narinfos id=44 count=16404lb # [ 135.311851] niks3-server[1143]: 2026/09/22 10:50:40 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6405lb # [ 135.315737] niks3-server[1143]: 2026/09/22 10:50:40 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6406lb # [ 135.318950] niks3-server[1143]: 2026/09/22 10:50:40 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6407lb # [ 135.324283] niks3-server[1143]: 2026/09/22 10:50:40 INFO Completed upload id=446408worker2 # [ 135.324761] nix-grpc-daemon[1706]: time=2026-09-22T10:50:40.283Z level=INFO msg="Upload complete. (265ms)"6409lb # [ 135.325573] niks3-server[1143]: 2026/09/22 10:50:40 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6410worker2 # [ 135.331877] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57066 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16411worker2 # [ 135.343589] nix-daemon[856]: reaped child process 1679, status = succeeded6412client # [ 135.381651] systemd[1]: mid.service: Deactivated successfully.6413client # [ 135.385808] systemd[1]: mid.service: Consumed 153ms CPU time over 10.088s wall clock time, 41.4M memory peak, 8.4K incoming IP traffic, 10.4K outgoing IP traffic.6414client: (finished: waiting for success: ! systemctl is-active mid, in 1.11 seconds)6415client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6416client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6417(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.61 seconds)6418subtest: interrupting the client stops the build on the worker6419client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6420client # [ 135.877407] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6421client # Running as unit: intr.service; invocation ID: 7600f97e4f4e4c1aaaa4d95ba3b995a86422client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6423worker1 # [ 136.081624] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590746424client # [ 136.094817] nix[1343]: don't know how to build these paths:6425client # [ 136.095716] nix[1343]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6426worker2 # [ 136.094513] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:41Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06427worker2 # [ 136.098946] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:41Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16428worker1 # [ 136.140244] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746429worker1 # [ 136.153281] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 1/5)6430worker1 # [ 136.235294] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 2/5)6431worker1 # [ 136.295254] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 3/5)6432worker1 # [ 136.492052] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 677 ms (attempt 4/5)6433worker1 # [ 137.172505] nix-grpc-daemon[1471]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6434worker1 # [ 137.174143] nix-grpc-daemon[1471]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6435worker1 # [ 137.187911] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746436client # [ 137.204367] nix[1343]: copying 1 paths to 'lb:50051'...6437client # [ 137.207910] nix[1343]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6438worker1 # [ 137.201163] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590746439worker1 # [ 137.210908] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6440worker1 # [ 137.231611] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:42Z level=info event=niks3_push_started pid=15676441worker1 # [ 137.395432] nix-grpc-daemon[1567]: time=2026-09-22T10:50:42.401Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6442worker1 # [ 137.533629] nix-daemon[867]: accepted connection from pid 1580, user nix-grpc-daemon (trusted)6443worker1 # [ 137.547062] nix-daemon[867]: reaped child process 1583, status = succeeded6444worker1 # [ 137.685089] nix-daemon[867]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)6445worker1 # [ 137.698818] nix-daemon[867]: reaped child process 1588, status = succeeded6446lb # [ 137.706235] niks3-server[1143]: 2026/09/22 10:50:42 INFO Received uploads request method=POST path=/api/pending_closures6447worker1 # [ 137.720144] nix-grpc-daemon[1567]: time=2026-09-22T10:50:42.727Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6448worker1 # [ 137.722986] nix-grpc-daemon[1567]: time=2026-09-22T10:50:42.729Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6449lb # [ 137.741123] niks3-server[1143]: 2026/09/22 10:50:42 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6450lb # [ 137.750390] niks3-server[1143]: 2026/09/22 10:50:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6451lb # [ 137.755544] niks3-server[1143]: 2026/09/22 10:50:42 INFO Signed narinfos id=45 count=16452worker1 # [ 137.760536] nix-grpc-daemon[1567]: time=2026-09-22T10:50:42.767Z level=INFO msg="Uploading 1 narinfos"6453lb # [ 137.766750] niks3-server[1143]: 2026/09/22 10:50:42 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6454lb # [ 137.774368] niks3-server[1143]: 2026/09/22 10:50:42 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6455lb # [ 137.777168] niks3-server[1143]: 2026/09/22 10:50:42 INFO Completed upload id=456456worker1 # [ 137.779682] nix-grpc-daemon[1567]: time=2026-09-22T10:50:42.786Z level=INFO msg="Upload complete. (381ms)"6457lb # [ 137.779226] niks3-server[1143]: 2026/09/22 10:50:42 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6458worker1 # [ 137.781951] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=4246459worker1 # [ 137.786431] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746460worker1 # [ 137.792836] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6461worker1 # [ 137.795381] nix-daemon[867]: reaped child process 1565, status = succeeded6462worker1 # [ 137.800975] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6463client # [ 137.807087] nix[1343]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6464lb # [ 138.142779] envoy[920]: conn peer=[2001:db8:1::4]:60064 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6171 tx=511 ms=169596465lb # [ 138.144603] envoy[920]: conn peer=[2001:db8:1::1]:45084 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3934 tx=1830 ms=98226466client: must succeed: systemctl kill -s INT intr6467client # [ 139.356167] systemd[1]: intr.service: Sent signal SIGINT to main process 1343 (nix) on client request.6468worker1 # [ 139.355581] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:44Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6469client # [ 139.364303] nix[1343]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6470client # [ 139.369942] nix[1343]: error: interrupted by the user6471client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6472client # [ 139.406031] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6473client # [ 139.407475] systemd[1]: intr.service: Failed with result 'exit-code'.6474client # [ 139.409730] systemd[1]: intr.service: Consumed 124ms CPU time over 3.531s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6475worker1 # [ 139.425482] nix-daemon[867]: reaped child process 1590, status = succeeded6476worker1 # [ 139.429279] nix-daemon[1592]: unexpected Nix daemon error: error: interrupted by the user6477worker1 # [ 139.433125] nix-daemon[867]: reaped child process 1592, status = failed with exit code 16478(finished: subtest: interrupting the client stops the build on the worker, in 4.75 seconds)6479subtest: a deploy mid-build drains: the build finishes, the new generation takes over6480client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6481client # [ 140.656823] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6482client # Running as unit: sw.service; invocation ID: 16ae3ff61a58428982794b79112165736483client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)6484worker2 # [ 140.864821] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:570666485client # [ 140.880230] nix[1364]: don't know how to build these paths:6486client # [ 140.881204] nix[1364]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6487worker2 # [ 140.878829] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:45Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06488worker2 # [ 140.883886] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:45Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16489worker2 # [ 140.922957] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570666490worker2 # [ 140.934093] nix-grpc-daemon[1623]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5)6491worker2 # [ 141.027942] nix-grpc-daemon[1623]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 2/5)6492worker2 # [ 141.132098] nix-grpc-daemon[1623]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 3/5)6493worker2 # [ 141.229069] nix-grpc-daemon[1623]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 327 ms (attempt 4/5)6494worker2 # [ 141.562307] nix-grpc-daemon[1623]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6495worker2 # [ 141.566332] nix-grpc-daemon[1623]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6496worker2 # [ 141.585939] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:570666497client # [ 141.606682] nix[1364]: copying 1 paths to 'lb:50051'...6498client # [ 141.610376] nix[1364]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6499worker2 # [ 141.602420] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:570666500worker2 # [ 141.614199] nix-daemon[856]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)6501worker2 # [ 141.789470] nix-daemon[856]: accepted connection from pid 1762, user nix-grpc-daemon (trusted)6502worker2 # [ 141.802298] nix-daemon[856]: reaped child process 1765, status = succeeded6503worker2 # [ 141.956095] nix-daemon[856]: accepted connection from pid 1767, user nix-grpc-daemon (trusted)6504worker2 # [ 141.966982] nix-daemon[856]: reaped child process 1774, status = succeeded6505lb # [ 141.980412] niks3-server[1143]: 2026/09/22 10:50:46 INFO Received uploads request method=POST path=/api/pending_closures6506worker2 # [ 141.990064] nix-grpc-daemon[1706]: time=2026-09-22T10:50:46.947Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6507worker2 # [ 141.991408] nix-grpc-daemon[1706]: time=2026-09-22T10:50:46.947Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6508lb # [ 142.006287] niks3-server[1143]: 2026/09/22 10:50:46 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6509lb # [ 142.016171] niks3-server[1143]: 2026/09/22 10:50:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6510worker2 # [ 142.021651] nix-grpc-daemon[1706]: time=2026-09-22T10:50:46.979Z level=INFO msg="Uploading 1 narinfos"6511lb # [ 142.021137] niks3-server[1143]: 2026/09/22 10:50:46 INFO Signed narinfos id=46 count=16512lb # [ 142.027642] niks3-server[1143]: 2026/09/22 10:50:46 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6513lb # [ 142.036697] niks3-server[1143]: 2026/09/22 10:50:46 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6514worker2 # [ 142.041454] nix-grpc-daemon[1706]: time=2026-09-22T10:50:46.999Z level=INFO msg="Upload complete. (415ms)"6515lb # [ 142.041151] niks3-server[1143]: 2026/09/22 10:50:46 INFO Completed upload id=466516worker2 # [ 142.043691] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57066 duration_s=0 paths=1 nar_bytes_in=4166517worker2 # [ 142.047153] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:570666518worker2 # [ 142.053870] nix-daemon[856]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)6519worker2 # [ 142.057033] nix-daemon[856]: reaped child process 1760, status = succeeded6520client # [ 142.070254] nix[1364]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6521worker2 # [ 142.061783] nix-daemon[856]: accepted connection from pid 1623, user nix-grpc-daemon (trusted)6522worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26523worker2 # Checking switch inhibitors... done6524worker2 # [ 144.861510] nixos[1810]: switching to system configuration /nix/store/xh6qbb2748pf5i997dfarf5fpg1abbid-nixos-system-worker2-test6525worker2 # activating the configuration...6526worker2 # [ 144.865448] systemd[1]: Stopped target Remote File Systems.6527worker2 # [ 144.867058] systemd[1]: Stopped target Local File Systems.6528worker2 # setting up /etc...6529worker2 # [ 145.472643] systemd[1]: Reload requested from client PID 1810 ('.switch-to-conf') (unit backdoor.service)...6530worker2 # [ 145.474296] systemd[1]: Reloading...6531worker2 # [ 146.106187] systemd[1]: Reloading finished in 629 ms.6532worker2 # restarting sysinit-reactivation.target6533worker2 # [ 146.162410] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6534worker2 # [ 146.292700] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6535worker2 # [ 146.293992] systemd[1]: Reached target Reactivate sysinit units.6536worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6537worker2 # [ 146.298617] systemd[1]: Reloading D-Bus System Message Bus...6538worker2 # [ 146.300831] dbus-broker-launch[534]: Caught SIGHUP, trigger reload.6539worker2 # [ 146.304641] systemd[1]: Reloading Nix worker-protocol over gRPC...6540worker2 # [ 146.335642] dbus-broker-launch[534]: Looking up NSS user entry for 'systemd-timesync'...6541worker2 # [ 146.337603] dbus-broker-launch[534]: NSS returned no entry for 'systemd-timesync'6542worker2 # [ 146.338714] dbus-broker-launch[534]: Invalid user-name in /nix/store/s93qka0z548yjbxqnmj4c4b32apww6jv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6543worker2 # [ 146.349689] systemd[1]: Reloaded Nix worker-protocol over gRPC.6544worker2 # [ 146.367399] systemd[1]: Reloaded D-Bus System Message Bus.6545worker2 # [ 146.371181] systemd[1]: Reached target Remote File Systems.6546worker2 # [ 146.374918] systemd[1]: Starting linger-users.service...6547worker2 # [ 146.377734] systemd[1]: Reached target Local File Systems.6548worker2 # [ 146.387072] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6549worker2 # [ 146.390305] systemd[1]: Starting Create SUID/SGID Wrappers...6550worker2 # [ 146.391501] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6551worker2 # [ 146.392655] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676552worker2 # [ 146.397130] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6553worker2 # [ 146.398549] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6554worker2 # [ 146.399682] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6555worker2 # [ 146.407228] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6556worker2 # [ 146.410192] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6557worker2 # [ 146.430808] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:51Z level=info event=draining6558worker2 # [ 146.451142] systemd[1]: linger-users.service: Deactivated successfully.6559worker2 # [ 146.452318] systemd[1]: Finished linger-users.service.6560worker2 # [ 146.818544] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6561worker2 # [ 146.822909] systemd[1]: Finished Create SUID/SGID Wrappers.6562worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6563worker2 # [ 147.076889] nixos[1810]: finished switching to system configuration /nix/store/xh6qbb2748pf5i997dfarf5fpg1abbid-nixos-system-worker2-test6564worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.87 seconds)6565worker2: must succeed: pgrep -f 'read -t [9]0 x'6566worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6567client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26568client # don't know how to build these paths:6569client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6570worker1 # [ 147.377970] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590746571worker1 # [ 147.384129] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746572worker2 # [ 147.397823] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:52Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06573worker2 # [ 147.399789] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:52Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16574worker1 # [ 147.447898] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746575worker1 # [ 147.460091] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746576client # copying 1 paths to 'lb:50051'...6577worker1 # [ 147.472465] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590746578client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6579worker1 # [ 147.500613] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6580worker1 # [ 147.674434] nix-daemon[867]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)6581worker1 # [ 147.686645] nix-daemon[867]: reaped child process 1650, status = succeeded6582worker1 # [ 147.839177] nix-daemon[867]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6583worker1 # [ 147.851179] nix-daemon[867]: reaped child process 1655, status = succeeded6584lb # [ 147.864785] niks3-server[1143]: 2026/09/22 10:50:52 INFO Received uploads request method=POST path=/api/pending_closures6585worker1 # [ 147.875713] nix-grpc-daemon[1567]: time=2026-09-22T10:50:52.882Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6586worker1 # [ 147.877266] nix-grpc-daemon[1567]: time=2026-09-22T10:50:52.884Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6587lb # [ 147.893690] niks3-server[1143]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6588lb # [ 147.902982] niks3-server[1143]: 2026/09/22 10:50:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6589lb # [ 147.907706] niks3-server[1143]: 2026/09/22 10:50:52 INFO Signed narinfos id=47 count=16590worker1 # [ 147.913145] nix-grpc-daemon[1567]: time=2026-09-22T10:50:52.920Z level=INFO msg="Uploading 1 narinfos"6591lb # [ 147.920370] niks3-server[1143]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6592lb # [ 147.927511] niks3-server[1143]: 2026/09/22 10:50:52 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6593lb # [ 147.932531] niks3-server[1143]: 2026/09/22 10:50:52 INFO Completed upload id=476594worker1 # [ 147.937219] nix-grpc-daemon[1567]: time=2026-09-22T10:50:52.944Z level=INFO msg="Upload complete. (424ms)"6595lb # [ 147.936564] niks3-server[1143]: 2026/09/22 10:50:52 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6596worker1 # [ 147.941417] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=5286597worker1 # [ 147.946376] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746598worker1 # [ 147.954459] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6599worker1 # [ 147.964125] nix-daemon[867]: reaped child process 1645, status = succeeded6600worker1 # [ 147.967734] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6601client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6602client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6603lb # [ 148.143845] envoy[920]: conn peer=[2001:db8:1::1]:32946 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=850 ms=32966604client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6605client # farm-a> LOG-farm-a-during-drain6606worker1 # [ 148.995629] nix-daemon[867]: reaped child process 1659, status = succeeded6607worker1 # [ 149.144486] nix-daemon[867]: accepted connection from pid 1675, user nix-grpc-daemon (trusted)6608worker1 # [ 149.156606] nix-daemon[867]: reaped child process 1678, status = succeeded6609lb # [ 149.168581] niks3-server[1143]: 2026/09/22 10:50:54 INFO Received uploads request method=POST path=/api/pending_closures6610worker1 # [ 149.180789] nix-grpc-daemon[1567]: time=2026-09-22T10:50:54.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6611worker1 # [ 149.182319] nix-grpc-daemon[1567]: time=2026-09-22T10:50:54.189Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6612lb # [ 149.206447] niks3-server[1143]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6613lb # [ 149.222210] niks3-server[1143]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6614lb # [ 149.225346] niks3-server[1143]: 2026/09/22 10:50:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6615worker1 # [ 149.234570] nix-grpc-daemon[1567]: time=2026-09-22T10:50:54.241Z level=INFO msg="Uploading 1 narinfos"6616lb # [ 149.232129] niks3-server[1143]: 2026/09/22 10:50:54 INFO Signed narinfos id=48 count=16617lb # [ 149.237167] niks3-server[1143]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6618lb # [ 149.246640] niks3-server[1143]: 2026/09/22 10:50:54 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6619lb # [ 149.249335] niks3-server[1143]: 2026/09/22 10:50:54 INFO Completed upload id=486620worker1 # [ 149.251964] nix-grpc-daemon[1567]: time=2026-09-22T10:50:54.259Z level=INFO msg="Upload complete. (276ms)"6621lb # [ 149.251630] niks3-server[1143]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6622worker1 # [ 149.257519] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16623worker2 # [ 149.259748] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:54Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06624worker2 # [ 149.261628] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:54Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16625worker1 # [ 149.268822] nix-daemon[867]: reaped child process 1657, status = succeeded6626worker1 # [ 149.306168] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746627worker1 # [ 149.318312] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746628worker1 # [ 149.329324] nix-grpc-daemon[1567]: time=2026-09-22T10:50:54.336Z level=INFO msg="All 1 paths already cached"6629client # copying 1 paths to 'lb:50051'...6630worker1 # [ 149.338764] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590746631client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6632worker1 # [ 149.353134] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6633worker1 # [ 149.510603] nix-daemon[867]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6634worker1 # [ 149.523322] nix-daemon[867]: reaped child process 1685, status = succeeded6635worker1 # [ 149.669285] nix-daemon[867]: accepted connection from pid 1687, user nix-grpc-daemon (trusted)6636worker1 # [ 149.681496] nix-daemon[867]: reaped child process 1690, status = succeeded6637lb # [ 149.691585] niks3-server[1143]: 2026/09/22 10:50:54 INFO Received uploads request method=POST path=/api/pending_closures6638worker1 # [ 149.701640] nix-grpc-daemon[1567]: time=2026-09-22T10:50:54.708Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6639worker1 # [ 149.703168] nix-grpc-daemon[1567]: time=2026-09-22T10:50:54.710Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6640lb # [ 149.720856] niks3-server[1143]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6641lb # [ 149.730763] niks3-server[1143]: 2026/09/22 10:50:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6642lb # [ 149.737668] niks3-server[1143]: 2026/09/22 10:50:54 INFO Signed narinfos id=49 count=16643worker1 # [ 149.742943] nix-grpc-daemon[1567]: time=2026-09-22T10:50:54.750Z level=INFO msg="Uploading 1 narinfos"6644lb # [ 149.740988] niks3-server[1143]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6645lb # [ 149.754923] niks3-server[1143]: 2026/09/22 10:50:54 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6646worker1 # [ 149.765466] nix-grpc-daemon[1567]: time=2026-09-22T10:50:54.772Z level=INFO msg="Upload complete. (402ms)"6647lb # [ 149.762943] niks3-server[1143]: 2026/09/22 10:50:54 INFO Completed upload id=496648lb # [ 149.765450] niks3-server[1143]: 2026/09/22 10:50:54 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6649worker1 # [ 149.769142] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=6406650worker1 # [ 149.774364] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746651worker1 # [ 149.782237] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6652worker1 # [ 149.789106] nix-daemon[867]: reaped child process 1680, status = succeeded6653client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6654worker1 # [ 149.795401] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6655client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6656client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6657client # farm-b> LOG-farm-b-during-drain6658worker1 # [ 151.010512] nix-daemon[867]: reaped child process 1694, status = succeeded6659worker1 # [ 151.144212] nix-daemon[867]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6660worker1 # [ 151.156164] nix-daemon[867]: reaped child process 1713, status = succeeded6661lb # [ 151.162431] niks3-server[1143]: 2026/09/22 10:50:56 INFO Received uploads request method=POST path=/api/pending_closures6662worker1 # [ 151.173030] nix-grpc-daemon[1567]: time=2026-09-22T10:50:56.179Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6663worker1 # [ 151.174923] nix-grpc-daemon[1567]: time=2026-09-22T10:50:56.181Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6664lb # [ 151.203748] niks3-server[1143]: 2026/09/22 10:50:56 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6665lb # [ 151.210076] niks3-server[1143]: 2026/09/22 10:50:56 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6666lb # [ 151.215294] niks3-server[1143]: 2026/09/22 10:50:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6667lb # [ 151.219652] niks3-server[1143]: 2026/09/22 10:50:56 INFO Signed narinfos id=50 count=16668worker1 # [ 151.225278] nix-grpc-daemon[1567]: time=2026-09-22T10:50:56.232Z level=INFO msg="Uploading 1 narinfos"6669lb # [ 151.231616] niks3-server[1143]: 2026/09/22 10:50:56 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6670lb # [ 151.235299] niks3-server[1143]: 2026/09/22 10:50:56 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6671lb # [ 151.240932] niks3-server[1143]: 2026/09/22 10:50:56 INFO Completed upload id=506672worker1 # [ 151.243429] nix-grpc-daemon[1567]: time=2026-09-22T10:50:56.250Z level=INFO msg="Upload complete. (241ms)"6673lb # [ 151.245469] niks3-server[1143]: 2026/09/22 10:50:56 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6674worker1 # [ 151.249284] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16675worker2 # [ 151.253052] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:56Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06676worker2 # [ 151.254962] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:56Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16677worker1 # [ 151.260692] nix-daemon[867]: reaped child process 1692, status = succeeded6678worker1 # [ 151.299466] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746679worker1 # [ 151.311371] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746680worker1 # [ 151.323598] nix-grpc-daemon[1567]: time=2026-09-22T10:50:56.330Z level=INFO msg="All 2 paths already cached"6681client # copying 1 paths to 'lb:50051'...6682worker1 # [ 151.331504] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590746683client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6684worker1 # [ 151.345162] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6685worker1 # [ 151.503496] nix-daemon[867]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)6686worker1 # [ 151.516458] nix-daemon[867]: reaped child process 1720, status = succeeded6687worker1 # [ 151.663948] nix-daemon[867]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6688worker1 # [ 151.676158] nix-daemon[867]: reaped child process 1725, status = succeeded6689lb # [ 151.686985] niks3-server[1143]: 2026/09/22 10:50:56 INFO Received uploads request method=POST path=/api/pending_closures6690worker1 # [ 151.697278] nix-grpc-daemon[1567]: time=2026-09-22T10:50:56.704Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6691worker1 # [ 151.698682] nix-grpc-daemon[1567]: time=2026-09-22T10:50:56.704Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6692lb # [ 151.713859] niks3-server[1143]: 2026/09/22 10:50:56 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6693lb # [ 151.721355] niks3-server[1143]: 2026/09/22 10:50:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6694lb # [ 151.727125] niks3-server[1143]: 2026/09/22 10:50:56 INFO Signed narinfos id=51 count=16695worker1 # [ 151.732724] nix-grpc-daemon[1567]: time=2026-09-22T10:50:56.739Z level=INFO msg="Uploading 1 narinfos"6696lb # [ 151.734411] niks3-server[1143]: 2026/09/22 10:50:56 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6697lb # [ 151.744286] niks3-server[1143]: 2026/09/22 10:50:56 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6698lb # [ 151.749446] niks3-server[1143]: 2026/09/22 10:50:56 INFO Completed upload id=516699worker1 # [ 151.754218] nix-grpc-daemon[1567]: time=2026-09-22T10:50:56.761Z level=INFO msg="Upload complete. (395ms)"6700lb # [ 151.753875] niks3-server[1143]: 2026/09/22 10:50:56 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6701worker1 # [ 151.756365] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=7686702worker1 # [ 151.762685] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746703worker1 # [ 151.768077] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6704worker1 # [ 151.770211] nix-daemon[867]: reaped child process 1715, status = succeeded6705worker1 # [ 151.775603] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6706client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6707client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6708client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6709client # farm-top> LOG-farm-top-during-drain6710worker1 # [ 152.789371] nix-daemon[867]: reaped child process 1729, status = succeeded6711worker1 # [ 152.923468] nix-daemon[867]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6712worker1 # [ 152.934262] nix-daemon[867]: reaped child process 1748, status = succeeded6713lb # [ 152.945728] niks3-server[1143]: 2026/09/22 10:50:57 INFO Received uploads request method=POST path=/api/pending_closures6714worker1 # [ 152.955727] nix-grpc-daemon[1567]: time=2026-09-22T10:50:57.962Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6715worker1 # [ 152.957242] nix-grpc-daemon[1567]: time=2026-09-22T10:50:57.964Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6716lb # [ 152.985399] niks3-server[1143]: 2026/09/22 10:50:57 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6717lb # [ 152.991182] niks3-server[1143]: 2026/09/22 10:50:57 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6718lb # [ 153.001680] niks3-server[1143]: 2026/09/22 10:50:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6719lb # [ 153.005936] niks3-server[1143]: 2026/09/22 10:50:57 INFO Signed narinfos id=52 count=16720worker1 # [ 153.011313] nix-grpc-daemon[1567]: time=2026-09-22T10:50:58.018Z level=INFO msg="Uploading 1 narinfos"6721lb # [ 153.012299] niks3-server[1143]: 2026/09/22 10:50:57 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6722lb # [ 153.021115] niks3-server[1143]: 2026/09/22 10:50:57 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6723lb # [ 153.026570] niks3-server[1143]: 2026/09/22 10:50:57 INFO Completed upload id=526724worker1 # [ 153.029092] nix-grpc-daemon[1567]: time=2026-09-22T10:50:58.036Z level=INFO msg="Upload complete. (246ms)"6725lb # [ 153.029560] niks3-server[1143]: 2026/09/22 10:50:57 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6726worker1 # [ 153.035551] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16727worker1 # [ 153.048200] nix-daemon[867]: reaped child process 1727, status = succeeded6728client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 5.94 seconds)6729client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6730client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6731worker2: must succeed: pkill -f 'read -t [9]0 x'6732worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6733client: waiting for success: ! systemctl is-active sw6734worker2 # [ 153.498780] nix-daemon[856]: reaped child process 1778, status = succeeded6735worker2 # [ 153.655330] nix-daemon[856]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)6736worker2 # [ 153.668146] nix-daemon[856]: reaped child process 1959, status = succeeded6737lb # [ 153.684964] niks3-server[1143]: 2026/09/22 10:50:58 INFO Received uploads request method=POST path=/api/pending_closures6738worker2 # [ 153.695262] nix-grpc-daemon[1706]: time=2026-09-22T10:50:58.653Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6739worker2 # [ 153.696750] nix-grpc-daemon[1706]: time=2026-09-22T10:50:58.655Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6740lb # [ 153.725117] niks3-server[1143]: 2026/09/22 10:50:58 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6741lb # [ 153.733081] niks3-server[1143]: 2026/09/22 10:50:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6742worker2 # [ 153.738272] nix-grpc-daemon[1706]: time=2026-09-22T10:50:58.696Z level=INFO msg="Uploading 1 narinfos"6743lb # [ 153.738041] niks3-server[1143]: 2026/09/22 10:50:58 INFO Signed narinfos id=53 count=16744lb # [ 153.745515] niks3-server[1143]: 2026/09/22 10:50:58 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6745lb # [ 153.750488] niks3-server[1143]: 2026/09/22 10:50:58 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6746lb # [ 153.756495] niks3-server[1143]: 2026/09/22 10:50:58 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6747lb # [ 153.759769] niks3-server[1143]: 2026/09/22 10:50:58 INFO Completed upload id=536748lb # [ 153.761366] niks3-server[1143]: 2026/09/22 10:50:58 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6749worker2 # [ 153.761578] nix-grpc-daemon[1706]: time=2026-09-22T10:50:58.719Z level=INFO msg="Upload complete. (282ms)"6750worker2 # [ 153.769263] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57066 duration_s=11 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16751worker2 # [ 153.780279] nix-daemon[856]: reaped child process 1776, status = succeeded6752client # [ 153.801953] systemd[1]: sw.service: Deactivated successfully.6753client # [ 153.803983] systemd[1]: sw.service: Consumed 128ms CPU time over 13.146s wall clock time, 41.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.6754client: (finished: waiting for success: ! systemctl is-active sw, in 1.08 seconds)6755client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6756worker2 # [ 154.434102] nix-grpc-daemon[1623]: ts=2026-09-22T10:50:59Z level=info event=signal_exit6757client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6758worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6759lb # [ 154.441944] niks3-server[1143]: 2026/09/22 10:50:59 INFO lead: released remote=192.168.1.4:367666760worker2 # [ 154.479155] systemd[1]: Starting scheduler-token.service...6761worker1 # [ 154.509100] nix-grpc-daemon[1471]: ts=2026-09-22T10:50:59Z level=info event=scheduler_take_over6762lb # [ 154.506212] niks3-server[1143]: 2026/09/22 10:50:59 INFO lead: acquired remote=192.168.1.3:329846763worker2 # [ 154.539657] systemd[1]: scheduler-token.service: Deactivated successfully.6764worker2 # [ 154.541250] systemd[1]: Finished scheduler-token.service.6765worker2 # [ 154.542523] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 62ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6766worker2 # [ 155.442228] nix-grpc-daemon[1623]: ts=2026-09-22T10:51:00Z level=info event=scheduler_restarting6767worker1 # [ 155.444512] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:00Z level=info event=scheduler_restarting addr=lb:500516768worker2 # [ 155.446278] nix-grpc-daemon[1623]: ts=2026-09-22T10:51:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:570406769worker2 # [ 155.448432] nix-grpc-daemon[1623]: ts=2026-09-22T10:51:00Z level=info event=worker_gone addr=192.168.1.3:500516770worker2 # [ 155.449952] nix-grpc-daemon[1623]: ts=2026-09-22T10:51:00Z level=info event=scheduler_restarting addr=lb:500516771worker2 # [ 155.451730] nix-grpc-daemon[1623]: ts=2026-09-22T10:51:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:570406772worker2 # [ 155.453572] nix-grpc-daemon[1623]: ts=2026-09-22T10:51:00Z level=info event=worker_gone addr=192.168.1.4:500516773worker2 # [ 155.465821] nix-daemon[856]: reaped child process 1677, status = succeeded6774worker2 # [ 155.610884] nix-grpc-daemon[1623]: E0000 00:00:1790074260.569054 1668 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6775worker2 # [ 155.615804] nix-grpc-daemon[1623]: E0000 00:00:1790074260.569200 1668 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6776worker1 # [ 155.647094] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:00Z level=info event=scheduler_connected addr=lb:500516777worker1 # [ 155.650259] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6778worker2 # [ 155.657233] nix-grpc-daemon[1623]: ts=2026-09-22T10:51:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6779worker1 # [ 155.847682] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:00Z level=info event=scheduler_connected addr=lb:500516780worker1 # [ 155.851554] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6781worker1 # [ 156.051993] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:01Z level=info event=scheduler_connected addr=lb:500516782worker1 # [ 156.053791] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6783worker1 # [ 156.254816] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:01Z level=info event=scheduler_connected addr=lb:500516784worker1 # [ 156.258697] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6785worker1 # [ 156.459410] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:01Z level=info event=scheduler_connected addr=lb:500516786worker1 # [ 156.469954] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:370506787worker1 # [ 156.473575] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06788worker2 # [ 157.475171] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6789worker2 # [ 157.479415] systemd[1]: nix-grpc-daemon.service: Consumed 655ms CPU time over 28.347s wall clock time, 33.3M memory peak, 25.2K incoming IP traffic, 32.4K outgoing IP traffic.6790worker2 # [ 157.496327] systemd[1]: Starting Nix worker-protocol over gRPC...6791worker2 # [ 157.664395] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:02Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6792worker2 # [ 157.666867] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:02Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6793worker2 # [ 157.670726] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:02Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16794worker2 # [ 157.678066] nix-grpc-daemon[1976]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6795worker2 # [ 157.680097] nix-grpc-daemon[1976]: E0000 00:00:1790074262.635941 1976 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6796worker2 # [ 157.686448] nix-grpc-daemon[1976]: E0000 00:00:1790074262.641667 1976 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6797worker2 # [ 157.689888] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:02Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6798worker2 # [ 157.693278] systemd[1]: Started Nix worker-protocol over gRPC.6799worker2 # [ 157.700089] nix-grpc-daemon[1976]: E0000 00:00:1790074262.656439 2017 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6800worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.27 seconds)6801lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6802lb: must succeed: curl -sf localhost:9901/clusters6803worker2 # [ 157.711362] nix-grpc-daemon[1976]: E0000 00:00:1790074262.669790 2017 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6804worker2 # [ 157.714152] nix-grpc-daemon[1976]: E0000 00:00:1790074262.672089 2017 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6805worker2 # [ 157.725971] nix-grpc-daemon[1976]: E0000 00:00:1790074262.684320 2017 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6806worker2 # [ 157.738212] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:02Z level=info event=scheduler_connected addr=lb:500516807worker1 # [ 157.748820] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:02Z level=info event=oidc_jwks provider=mock keys=16808worker1 # [ 157.753119] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:370506809worker1 # [ 157.756902] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6811lb # [ 158.144967] envoy[920]: conn peer=[2001:db8:1::1]:46862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9585 tx=4032 ms=56746812lb # [ 158.148717] envoy[920]: conn peer=[2001:db8:1::1]:32948 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1449 ms=129116813lb # [ 158.152321] envoy[920]: conn peer=[2001:db8:1::4]:37816 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1584 tx=344 ms=261746814lb: must succeed: curl -sf localhost:9901/clusters6815lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6816lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.14 seconds)6817lb: must succeed: curl -sf localhost:9901/clusters6818lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6819worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6820worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6821(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 18.35 seconds)6822subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6823client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16824worker2 # [ 159.145246] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:516206825worker2 # [ 159.150874] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6826worker1 # [ 159.161937] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746827worker1 # [ 159.199425] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06828worker1 # [ 159.201447] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16829worker1 # [ 159.203471] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06830worker1 # [ 159.205786] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06831worker1 # [ 159.244726] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746832worker1 # [ 159.256891] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746833worker1 # [ 159.269179] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590746834worker1 # [ 159.277844] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6835worker1 # [ 159.440276] nix-daemon[867]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6836worker1 # [ 159.452646] nix-daemon[867]: reaped child process 1760, status = succeeded6837worker1 # [ 159.603652] nix-daemon[867]: accepted connection from pid 1762, user nix-grpc-daemon (trusted)6838worker1 # [ 159.615746] nix-daemon[867]: reaped child process 1765, status = succeeded6839lb # [ 159.628864] niks3-server[1143]: 2026/09/22 10:51:04 INFO Received uploads request method=POST path=/api/pending_closures6840worker1 # [ 159.640278] nix-grpc-daemon[1567]: time=2026-09-22T10:51:04.647Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6841worker1 # [ 159.641858] nix-grpc-daemon[1567]: time=2026-09-22T10:51:04.649Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6842lb # [ 159.661356] niks3-server[1143]: 2026/09/22 10:51:04 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6843lb # [ 159.669601] niks3-server[1143]: 2026/09/22 10:51:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6844worker1 # [ 159.676675] nix-grpc-daemon[1567]: time=2026-09-22T10:51:04.683Z level=INFO msg="Uploading 1 narinfos"6845lb # [ 159.674064] niks3-server[1143]: 2026/09/22 10:51:04 INFO Signed narinfos id=54 count=16846lb # [ 159.679481] niks3-server[1143]: 2026/09/22 10:51:04 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6847lb # [ 159.688463] niks3-server[1143]: 2026/09/22 10:51:04 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6848lb # [ 159.691517] niks3-server[1143]: 2026/09/22 10:51:04 INFO Completed upload id=546849worker1 # [ 159.694264] nix-grpc-daemon[1567]: time=2026-09-22T10:51:04.701Z level=INFO msg="Upload complete. (402ms)"6850worker1 # [ 159.696395] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=5046851worker1 # [ 159.699251] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746852worker1 # [ 159.704274] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6853worker1 # [ 159.709385] nix-daemon[867]: reaped child process 1755, status = succeeded6854worker1 # [ 159.714366] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6855worker1 # [ 161.141491] nix-daemon[867]: reaped child process 1769, status = succeeded6856worker1 # [ 161.292793] nix-daemon[867]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6857worker1 # [ 161.303906] nix-daemon[867]: reaped child process 1788, status = succeeded6858lb # [ 161.314798] niks3-server[1143]: 2026/09/22 10:51:06 INFO Received uploads request method=POST path=/api/pending_closures6859worker1 # [ 161.328761] nix-grpc-daemon[1567]: time=2026-09-22T10:51:06.335Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6860worker1 # [ 161.330241] nix-grpc-daemon[1567]: time=2026-09-22T10:51:06.337Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6861lb # [ 161.363949] niks3-server[1143]: 2026/09/22 10:51:06 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6862lb # [ 161.375848] niks3-server[1143]: 2026/09/22 10:51:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6863lb # [ 161.383236] niks3-server[1143]: 2026/09/22 10:51:06 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6864lb # [ 161.386928] niks3-server[1143]: 2026/09/22 10:51:06 INFO Signed narinfos id=55 count=16865worker1 # [ 161.389932] nix-grpc-daemon[1567]: time=2026-09-22T10:51:06.396Z level=INFO msg="Uploading 1 narinfos"6866lb # [ 161.401770] niks3-server[1143]: 2026/09/22 10:51:06 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6867lb # [ 161.405328] niks3-server[1143]: 2026/09/22 10:51:06 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6868lb # [ 161.407360] niks3-server[1143]: 2026/09/22 10:51:06 INFO Completed upload id=556869lb # [ 161.408822] niks3-server[1143]: 2026/09/22 10:51:06 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6870worker1 # [ 161.409937] nix-grpc-daemon[1567]: time=2026-09-22T10:51:06.417Z level=INFO msg="Upload complete. (286ms)"6871worker1 # [ 161.417722] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16872worker1 # [ 161.428776] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:06Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16873worker1 # [ 161.434239] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:06Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06874worker1 # [ 161.442166] nix-daemon[867]: reaped child process 1767, status = succeeded6875worker1 # [ 161.468485] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746876worker1 # [ 161.480366] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746877worker1 # [ 161.491855] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590746878worker1 # [ 161.502181] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6879worker1 # [ 161.655253] nix-daemon[867]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6880worker1 # [ 161.667703] nix-daemon[867]: reaped child process 1797, status = succeeded6881worker1 # [ 161.808289] nix-daemon[867]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6882worker1 # [ 161.820213] nix-daemon[867]: reaped child process 1802, status = succeeded6883lb # [ 161.830748] niks3-server[1143]: 2026/09/22 10:51:06 INFO Received uploads request method=POST path=/api/pending_closures6884worker1 # [ 161.841040] nix-grpc-daemon[1567]: time=2026-09-22T10:51:06.848Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6885worker1 # [ 161.842373] nix-grpc-daemon[1567]: time=2026-09-22T10:51:06.848Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6886lb # [ 161.857383] niks3-server[1143]: 2026/09/22 10:51:06 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6887lb # [ 161.869825] niks3-server[1143]: 2026/09/22 10:51:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6888worker1 # [ 161.877656] nix-grpc-daemon[1567]: time=2026-09-22T10:51:06.884Z level=INFO msg="Uploading 1 narinfos"6889lb # [ 161.875157] niks3-server[1143]: 2026/09/22 10:51:06 INFO Signed narinfos id=56 count=16890lb # [ 161.882233] niks3-server[1143]: 2026/09/22 10:51:06 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6891lb # [ 161.893178] niks3-server[1143]: 2026/09/22 10:51:06 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6892worker1 # [ 161.903777] nix-grpc-daemon[1567]: time=2026-09-22T10:51:06.910Z level=INFO msg="Upload complete. (389ms)"6893lb # [ 161.901527] niks3-server[1143]: 2026/09/22 10:51:06 INFO Completed upload id=566894lb # [ 161.905189] niks3-server[1143]: 2026/09/22 10:51:06 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6895worker1 # [ 161.907218] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=5046896worker1 # [ 161.912115] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746897worker1 # [ 161.922137] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6898worker1 # [ 161.926450] nix-daemon[867]: reaped child process 1792, status = succeeded6899worker1 # [ 161.931435] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6900worker1 # [ 163.346665] nix-daemon[867]: reaped child process 1806, status = succeeded6901worker1 # [ 163.483097] nix-daemon[867]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)6902worker1 # [ 163.495315] nix-daemon[867]: reaped child process 1825, status = succeeded6903lb # [ 163.507761] niks3-server[1143]: 2026/09/22 10:51:08 INFO Received uploads request method=POST path=/api/pending_closures6904worker1 # [ 163.518221] nix-grpc-daemon[1567]: time=2026-09-22T10:51:08.525Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6905worker1 # [ 163.519790] nix-grpc-daemon[1567]: time=2026-09-22T10:51:08.526Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6906lb # [ 163.549672] niks3-server[1143]: 2026/09/22 10:51:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6907worker1 # [ 163.556659] nix-grpc-daemon[1567]: time=2026-09-22T10:51:08.563Z level=INFO msg="Uploading 1 narinfos"6908lb # [ 163.554254] niks3-server[1143]: 2026/09/22 10:51:08 INFO Signed narinfos id=57 count=16909lb # [ 163.566860] niks3-server[1143]: 2026/09/22 10:51:08 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6910lb # [ 163.571109] niks3-server[1143]: 2026/09/22 10:51:08 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6911lb # [ 163.577675] niks3-server[1143]: 2026/09/22 10:51:08 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6912lb # [ 163.586154] niks3-server[1143]: 2026/09/22 10:51:08 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6913lb # [ 163.589891] niks3-server[1143]: 2026/09/22 10:51:08 INFO Completed upload id=576914worker1 # [ 163.592612] nix-grpc-daemon[1567]: time=2026-09-22T10:51:08.599Z level=INFO msg="Upload complete. (253ms)"6915lb # [ 163.592052] niks3-server[1143]: 2026/09/22 10:51:08 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6916worker1 # [ 163.603365] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16917worker1 # [ 163.609244] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:08Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16918worker1 # [ 163.612732] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746919worker1 # [ 163.615571] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:08Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06920worker1 # [ 163.621296] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746921worker1 # [ 163.624525] nix-daemon[867]: reaped child process 1804, status = succeeded6922worker1 # [ 163.628085] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590746923worker1 # [ 163.632118] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6924worker1 # [ 163.792232] nix-daemon[867]: accepted connection from pid 1831, user nix-grpc-daemon (trusted)6925worker1 # [ 163.805861] nix-daemon[867]: reaped child process 1834, status = succeeded6926worker1 # [ 163.956056] nix-daemon[867]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)6927worker1 # [ 163.968241] nix-daemon[867]: reaped child process 1839, status = succeeded6928lb # [ 163.979461] niks3-server[1143]: 2026/09/22 10:51:08 INFO Received uploads request method=POST path=/api/pending_closures6929worker1 # [ 163.993528] nix-grpc-daemon[1567]: time=2026-09-22T10:51:09.000Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6930worker1 # [ 163.994935] nix-grpc-daemon[1567]: time=2026-09-22T10:51:09.000Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6931lb # [ 164.006989] niks3-server[1143]: 2026/09/22 10:51:08 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6932lb # [ 164.019889] niks3-server[1143]: 2026/09/22 10:51:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6933lb # [ 164.025402] niks3-server[1143]: 2026/09/22 10:51:08 INFO Signed narinfos id=58 count=16934worker1 # [ 164.030682] nix-grpc-daemon[1567]: time=2026-09-22T10:51:09.037Z level=INFO msg="Uploading 1 narinfos"6935lb # [ 164.032575] niks3-server[1143]: 2026/09/22 10:51:08 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6936lb # [ 164.042703] niks3-server[1143]: 2026/09/22 10:51:08 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6937lb # [ 164.050495] niks3-server[1143]: 2026/09/22 10:51:08 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6938worker1 # [ 164.058057] nix-grpc-daemon[1567]: time=2026-09-22T10:51:09.065Z level=INFO msg="Upload complete. (420ms)"6939lb # [ 164.055642] niks3-server[1143]: 2026/09/22 10:51:08 INFO Completed upload id=586940worker1 # [ 164.060635] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=5046941worker1 # [ 164.066367] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746942worker1 # [ 164.077114] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6943worker1 # [ 164.085490] nix-daemon[867]: reaped child process 1829, status = succeeded6944worker1 # [ 164.090547] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6945worker1 # [ 165.514111] nix-daemon[867]: reaped child process 1843, status = succeeded6946worker1 # [ 165.652630] nix-daemon[867]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)6947worker1 # [ 165.664606] nix-daemon[867]: reaped child process 1862, status = succeeded6948lb # [ 165.675770] niks3-server[1143]: 2026/09/22 10:51:10 INFO Received uploads request method=POST path=/api/pending_closures6949worker1 # [ 165.686173] nix-grpc-daemon[1567]: time=2026-09-22T10:51:10.693Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6950worker1 # [ 165.687646] nix-grpc-daemon[1567]: time=2026-09-22T10:51:10.694Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6951lb # [ 165.704177] niks3-server[1143]: 2026/09/22 10:51:10 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6952lb # [ 165.720692] niks3-server[1143]: 2026/09/22 10:51:10 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6953lb # [ 165.730256] niks3-server[1143]: 2026/09/22 10:51:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6954worker1 # [ 165.737737] nix-grpc-daemon[1567]: time=2026-09-22T10:51:10.744Z level=INFO msg="Uploading 1 narinfos"6955lb # [ 165.735323] niks3-server[1143]: 2026/09/22 10:51:10 INFO Signed narinfos id=59 count=16956lb # [ 165.740367] niks3-server[1143]: 2026/09/22 10:51:10 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6957lb # [ 165.749593] niks3-server[1143]: 2026/09/22 10:51:10 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6958lb # [ 165.756159] niks3-server[1143]: 2026/09/22 10:51:10 INFO Completed upload id=596959lb # [ 165.757901] niks3-server[1143]: 2026/09/22 10:51:10 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6960worker1 # [ 165.759900] nix-grpc-daemon[1567]: time=2026-09-22T10:51:10.767Z level=INFO msg="Upload complete. (262ms)"6961worker1 # [ 165.770511] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16962worker1 # [ 165.776907] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:10Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16963worker1 # [ 165.780366] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:10Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06964worker1 # [ 165.789133] nix-daemon[867]: reaped child process 1841, status = succeeded6965worker1 # [ 165.810268] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746966worker1 # [ 165.820076] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590746967worker1 # [ 165.832869] nix-grpc-daemon[1567]: time=2026-09-22T10:51:10.840Z level=INFO msg="All 1 paths already cached"6968worker1 # [ 165.838868] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590746969worker1 # [ 165.845847] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6970worker1 # [ 165.999683] nix-daemon[867]: accepted connection from pid 1868, user nix-grpc-daemon (trusted)6971worker1 # [ 166.012132] nix-daemon[867]: reaped child process 1871, status = succeeded6972worker1 # [ 166.159577] nix-daemon[867]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6973worker1 # [ 166.171953] nix-daemon[867]: reaped child process 1876, status = succeeded6974lb # [ 166.181613] niks3-server[1143]: 2026/09/22 10:51:11 INFO Received uploads request method=POST path=/api/pending_closures6975worker1 # [ 166.188294] nix-grpc-daemon[1567]: time=2026-09-22T10:51:11.195Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6976worker1 # [ 166.189850] nix-grpc-daemon[1567]: time=2026-09-22T10:51:11.197Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6977lb # [ 166.197730] niks3-server[1143]: 2026/09/22 10:51:11 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6978lb # [ 166.200530] niks3-server[1143]: 2026/09/22 10:51:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6979lb # [ 166.202239] niks3-server[1143]: 2026/09/22 10:51:11 INFO Signed narinfos id=60 count=16980worker1 # [ 166.204708] nix-grpc-daemon[1567]: time=2026-09-22T10:51:11.211Z level=INFO msg="Uploading 1 narinfos"6981lb # [ 166.205841] niks3-server[1143]: 2026/09/22 10:51:11 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6982lb # [ 166.210368] niks3-server[1143]: 2026/09/22 10:51:11 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6983lb # [ 166.212912] niks3-server[1143]: 2026/09/22 10:51:11 INFO Completed upload id=606984worker1 # [ 166.215318] nix-grpc-daemon[1567]: time=2026-09-22T10:51:11.222Z level=INFO msg="Upload complete. (358ms)"6985worker1 # [ 166.216772] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=6246986lb # [ 166.215225] niks3-server[1143]: 2026/09/22 10:51:11 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6987worker1 # [ 166.222378] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590746988worker1 # [ 166.227098] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6989worker1 # [ 166.229354] nix-daemon[867]: reaped child process 1866, status = succeeded6990worker1 # [ 166.234542] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)6991worker1 # [ 167.462528] nix-daemon[867]: reaped child process 1880, status = succeeded6992worker1 # [ 167.615294] nix-daemon[867]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)6993worker1 # [ 167.627325] nix-daemon[867]: reaped child process 1899, status = succeeded6994lb # [ 167.639290] niks3-server[1143]: 2026/09/22 10:51:12 INFO Received uploads request method=POST path=/api/pending_closures6995worker1 # [ 167.649642] nix-grpc-daemon[1567]: time=2026-09-22T10:51:12.656Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6996worker1 # [ 167.651098] nix-grpc-daemon[1567]: time=2026-09-22T10:51:12.658Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6997lb # [ 167.679239] niks3-server[1143]: 2026/09/22 10:51:12 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6998lb # [ 167.686125] niks3-server[1143]: 2026/09/22 10:51:12 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6999lb # [ 167.699440] niks3-server[1143]: 2026/09/22 10:51:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign7000worker1 # [ 167.706325] nix-grpc-daemon[1567]: time=2026-09-22T10:51:12.713Z level=INFO msg="Uploading 1 narinfos"7001lb # [ 167.704128] niks3-server[1143]: 2026/09/22 10:51:12 INFO Signed narinfos id=61 count=17002lb # [ 167.710321] niks3-server[1143]: 2026/09/22 10:51:12 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7003lb # [ 167.718604] niks3-server[1143]: 2026/09/22 10:51:12 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete7004lb # [ 167.725694] niks3-server[1143]: 2026/09/22 10:51:12 INFO Completed upload id=617005lb # [ 167.727374] niks3-server[1143]: 2026/09/22 10:51:12 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7006worker1 # [ 167.728435] nix-grpc-daemon[1567]: time=2026-09-22T10:51:12.735Z level=INFO msg="Upload complete. (286ms)"7007worker1 # [ 167.741147] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=17008worker1 # [ 167.746467] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:12Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17009worker1 # [ 167.751307] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747010worker1 # [ 167.754550] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:12Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07011worker1 # [ 167.764563] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590747012worker1 # [ 167.767759] nix-daemon[867]: reaped child process 1878, status = succeeded7013worker1 # [ 167.772348] nix-grpc-daemon[1567]: time=2026-09-22T10:51:12.779Z level=INFO msg="All 1 paths already cached"7014worker1 # [ 167.776880] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590747015worker1 # [ 167.781833] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7016worker1 # [ 167.928258] nix-daemon[867]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)7017worker1 # [ 167.940862] nix-daemon[867]: reaped child process 1908, status = succeeded7018worker1 # [ 168.092384] nix-daemon[867]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)7019worker1 # [ 168.104651] nix-daemon[867]: reaped child process 1913, status = succeeded7020lb # [ 168.115725] niks3-server[1143]: 2026/09/22 10:51:12 INFO Received uploads request method=POST path=/api/pending_closures7021worker1 # [ 168.126021] nix-grpc-daemon[1567]: time=2026-09-22T10:51:13.133Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7022worker1 # [ 168.127467] nix-grpc-daemon[1567]: time=2026-09-22T10:51:13.134Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7023lb # [ 168.144490] niks3-server[1143]: 2026/09/22 10:51:13 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7024lb # [ 168.155195] niks3-server[1143]: 2026/09/22 10:51:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7025worker1 # [ 168.162223] nix-grpc-daemon[1567]: time=2026-09-22T10:51:13.169Z level=INFO msg="Uploading 1 narinfos"7026lb # [ 168.159896] niks3-server[1143]: 2026/09/22 10:51:13 INFO Signed narinfos id=62 count=17027lb # [ 168.166826] niks3-server[1143]: 2026/09/22 10:51:13 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7028lb # [ 168.177854] niks3-server[1143]: 2026/09/22 10:51:13 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7029worker1 # [ 168.186425] nix-grpc-daemon[1567]: time=2026-09-22T10:51:13.193Z level=INFO msg="Upload complete. (399ms)"7030lb # [ 168.184404] niks3-server[1143]: 2026/09/22 10:51:13 INFO Completed upload id=627031lb # [ 168.187872] niks3-server[1143]: 2026/09/22 10:51:13 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7032worker1 # [ 168.189513] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=6247033worker1 # [ 168.194889] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747034worker1 # [ 168.205117] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7035worker1 # [ 168.209713] nix-daemon[867]: reaped child process 1903, status = succeeded7036worker1 # [ 168.214569] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7037worker1 # [ 169.642084] nix-daemon[867]: reaped child process 1917, status = succeeded7038worker1 # [ 169.766814] nix-daemon[867]: accepted connection from pid 1933, user nix-grpc-daemon (trusted)7039worker1 # [ 169.779154] nix-daemon[867]: reaped child process 1936, status = succeeded7040lb # [ 169.787490] niks3-server[1143]: 2026/09/22 10:51:14 INFO Received uploads request method=POST path=/api/pending_closures7041worker1 # [ 169.799514] nix-grpc-daemon[1567]: time=2026-09-22T10:51:14.806Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7042worker1 # [ 169.802155] nix-grpc-daemon[1567]: time=2026-09-22T10:51:14.808Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7043lb # [ 169.829895] niks3-server[1143]: 2026/09/22 10:51:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7044lb # [ 169.834941] niks3-server[1143]: 2026/09/22 10:51:14 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7045lb # [ 169.841073] niks3-server[1143]: 2026/09/22 10:51:14 INFO Signed narinfos id=63 count=17046worker1 # [ 169.848281] nix-grpc-daemon[1567]: time=2026-09-22T10:51:14.855Z level=INFO msg="Uploading 1 narinfos"7047lb # [ 169.847140] niks3-server[1143]: 2026/09/22 10:51:14 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7048lb # [ 169.851802] niks3-server[1143]: 2026/09/22 10:51:14 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7049lb # [ 169.862602] niks3-server[1143]: 2026/09/22 10:51:14 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7050lb # [ 169.867991] niks3-server[1143]: 2026/09/22 10:51:14 INFO Completed upload id=637051lb # [ 169.871299] niks3-server[1143]: 2026/09/22 10:51:14 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7052worker1 # [ 169.873192] nix-grpc-daemon[1567]: time=2026-09-22T10:51:14.880Z level=INFO msg="Upload complete. (249ms)"7053worker1 # [ 169.885925] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=17054worker1 # [ 169.891239] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:14Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17055worker1 # [ 169.894640] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747056worker1 # [ 169.898226] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:14Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07057worker1 # [ 169.907508] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590747058worker1 # [ 169.911546] nix-daemon[867]: reaped child process 1915, status = succeeded7059worker1 # [ 169.913415] nix-grpc-daemon[1567]: time=2026-09-22T10:51:14.920Z level=INFO msg="All 1 paths already cached"7060worker1 # [ 169.917463] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590747061worker1 # [ 169.922097] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7062worker1 # [ 170.045461] nix-daemon[867]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)7063worker1 # [ 170.058474] nix-daemon[867]: reaped child process 1945, status = succeeded7064worker1 # [ 170.191289] nix-daemon[867]: accepted connection from pid 1947, user nix-grpc-daemon (trusted)7065worker1 # [ 170.205904] nix-daemon[867]: reaped child process 1950, status = succeeded7066lb # [ 170.212116] niks3-server[1143]: 2026/09/22 10:51:15 INFO Received uploads request method=POST path=/api/pending_closures7067worker1 # [ 170.218436] nix-grpc-daemon[1567]: time=2026-09-22T10:51:15.225Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7068worker1 # [ 170.219946] nix-grpc-daemon[1567]: time=2026-09-22T10:51:15.227Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7069lb # [ 170.235196] niks3-server[1143]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7070lb # [ 170.244062] niks3-server[1143]: 2026/09/22 10:51:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7071worker1 # [ 170.250948] nix-grpc-daemon[1567]: time=2026-09-22T10:51:15.258Z level=INFO msg="Uploading 1 narinfos"7072lb # [ 170.248662] niks3-server[1143]: 2026/09/22 10:51:15 INFO Signed narinfos id=64 count=17073lb # [ 170.255439] niks3-server[1143]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7074lb # [ 170.265865] niks3-server[1143]: 2026/09/22 10:51:15 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7075worker1 # [ 170.274429] nix-grpc-daemon[1567]: time=2026-09-22T10:51:15.281Z level=INFO msg="Upload complete. (347ms)"7076lb # [ 170.272052] niks3-server[1143]: 2026/09/22 10:51:15 INFO Completed upload id=647077worker1 # [ 170.276429] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=6247078lb # [ 170.275885] niks3-server[1143]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7079worker1 # [ 170.282543] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747080worker1 # [ 170.286934] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7081worker1 # [ 170.289258] nix-daemon[867]: reaped child process 1940, status = succeeded7082worker1 # [ 170.294833] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7083worker1 # [ 170.908739] nix-daemon[867]: reaped child process 1954, status = succeeded7084worker1 # [ 171.035417] nix-daemon[867]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)7085worker1 # [ 171.046313] nix-daemon[867]: reaped child process 1973, status = succeeded7086lb # [ 171.057327] niks3-server[1143]: 2026/09/22 10:51:15 INFO Received uploads request method=POST path=/api/pending_closures7087worker1 # [ 171.067430] nix-grpc-daemon[1567]: time=2026-09-22T10:51:16.074Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7088worker1 # [ 171.068901] nix-grpc-daemon[1567]: time=2026-09-22T10:51:16.076Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7089lb # [ 171.088626] niks3-server[1143]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7090lb # [ 171.103468] niks3-server[1143]: 2026/09/22 10:51:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7091lb # [ 171.109189] niks3-server[1143]: 2026/09/22 10:51:15 INFO Signed narinfos id=65 count=17092worker1 # [ 171.114566] nix-grpc-daemon[1567]: time=2026-09-22T10:51:16.121Z level=INFO msg="Uploading 1 narinfos"7093lb # [ 171.117191] niks3-server[1143]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7094lb # [ 171.121389] niks3-server[1143]: 2026/09/22 10:51:15 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7095lb # [ 171.130259] niks3-server[1143]: 2026/09/22 10:51:16 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7096lb # [ 171.134823] niks3-server[1143]: 2026/09/22 10:51:16 INFO Completed upload id=657097worker1 # [ 171.137379] nix-grpc-daemon[1567]: time=2026-09-22T10:51:16.144Z level=INFO msg="Upload complete. (236ms)"7098lb # [ 171.139073] niks3-server[1143]: 2026/09/22 10:51:16 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7099worker1 # [ 171.147801] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=17100worker1 # [ 171.152885] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:16Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17101worker1 # [ 171.156836] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747102worker1 # [ 171.160171] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:16Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07103worker1 # [ 171.169048] nix-daemon[867]: reaped child process 1952, status = succeeded7104worker1 # [ 171.171229] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590747105worker1 # [ 171.177907] nix-grpc-daemon[1567]: time=2026-09-22T10:51:16.185Z level=INFO msg="All 2 paths already cached"7106worker1 # [ 171.182153] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590747107worker1 # [ 171.185998] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7108worker1 # [ 171.330416] nix-daemon[867]: accepted connection from pid 1979, user nix-grpc-daemon (trusted)7109worker1 # [ 171.343278] nix-daemon[867]: reaped child process 1982, status = succeeded7110worker1 # [ 171.491093] nix-daemon[867]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)7111worker1 # [ 171.503246] nix-daemon[867]: reaped child process 1987, status = succeeded7112lb # [ 171.514236] niks3-server[1143]: 2026/09/22 10:51:16 INFO Received uploads request method=POST path=/api/pending_closures7113worker1 # [ 171.524287] nix-grpc-daemon[1567]: time=2026-09-22T10:51:16.531Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7114worker1 # [ 171.525675] nix-grpc-daemon[1567]: time=2026-09-22T10:51:16.531Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7115lb # [ 171.539759] niks3-server[1143]: 2026/09/22 10:51:16 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7116lb # [ 171.551139] niks3-server[1143]: 2026/09/22 10:51:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7117lb # [ 171.555860] niks3-server[1143]: 2026/09/22 10:51:16 INFO Signed narinfos id=66 count=17118worker1 # [ 171.561313] nix-grpc-daemon[1567]: time=2026-09-22T10:51:16.568Z level=INFO msg="Uploading 1 narinfos"7119lb # [ 171.562736] niks3-server[1143]: 2026/09/22 10:51:16 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7120lb # [ 171.572804] niks3-server[1143]: 2026/09/22 10:51:16 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7121worker1 # [ 171.580048] nix-grpc-daemon[1567]: time=2026-09-22T10:51:16.587Z level=INFO msg="Upload complete. (388ms)"7122lb # [ 171.577770] niks3-server[1143]: 2026/09/22 10:51:16 INFO Completed upload id=667123worker1 # [ 171.582069] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=7527124lb # [ 171.580832] niks3-server[1143]: 2026/09/22 10:51:16 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7125worker1 # [ 171.587108] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747126worker1 # [ 171.593455] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7127worker1 # [ 171.594766] nix-daemon[867]: reaped child process 1977, status = succeeded7128worker1 # [ 171.600052] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7129worker1 # [ 173.026605] nix-daemon[867]: reaped child process 1991, status = succeeded7130worker1 # [ 173.154940] nix-daemon[867]: accepted connection from pid 2007, user nix-grpc-daemon (trusted)7131worker1 # [ 173.166718] nix-daemon[867]: reaped child process 2010, status = succeeded7132lb # [ 173.177848] niks3-server[1143]: 2026/09/22 10:51:18 INFO Received uploads request method=POST path=/api/pending_closures7133worker1 # [ 173.188241] nix-grpc-daemon[1567]: time=2026-09-22T10:51:18.195Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7134worker1 # [ 173.189814] nix-grpc-daemon[1567]: time=2026-09-22T10:51:18.197Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7135lb # [ 173.218406] niks3-server[1143]: 2026/09/22 10:51:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7136lb # [ 173.222427] niks3-server[1143]: 2026/09/22 10:51:18 INFO Signed narinfos id=67 count=17137worker1 # [ 173.227948] nix-grpc-daemon[1567]: time=2026-09-22T10:51:18.235Z level=INFO msg="Uploading 1 narinfos"7138lb # [ 173.232582] niks3-server[1143]: 2026/09/22 10:51:18 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7139lb # [ 173.238240] niks3-server[1143]: 2026/09/22 10:51:18 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7140lb # [ 173.246448] niks3-server[1143]: 2026/09/22 10:51:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7141lb # [ 173.253894] niks3-server[1143]: 2026/09/22 10:51:18 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7142lb # [ 173.259101] niks3-server[1143]: 2026/09/22 10:51:18 INFO Completed upload id=677143worker1 # [ 173.261340] nix-grpc-daemon[1567]: time=2026-09-22T10:51:18.268Z level=INFO msg="Upload complete. (253ms)"7144lb # [ 173.262040] niks3-server[1143]: 2026/09/22 10:51:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7145worker1 # [ 173.268845] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=17146worker1 # [ 173.275762] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:18Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17147worker1 # [ 173.285557] nix-daemon[867]: reaped child process 1989, status = succeeded7148worker1 # [ 173.311240] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747149worker1 # [ 173.323362] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590747150worker1 # [ 173.335675] nix-grpc-daemon[1567]: time=2026-09-22T10:51:18.342Z level=INFO msg="All 2 paths already cached"7151worker1 # [ 173.342832] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590747152worker1 # [ 173.350958] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7153worker1 # [ 173.508128] nix-daemon[867]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)7154worker1 # [ 173.520828] nix-daemon[867]: reaped child process 2017, status = succeeded7155worker1 # [ 173.669428] nix-daemon[867]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)7156worker1 # [ 173.681640] nix-daemon[867]: reaped child process 2022, status = succeeded7157lb # [ 173.692900] niks3-server[1143]: 2026/09/22 10:51:18 INFO Received uploads request method=POST path=/api/pending_closures7158worker1 # [ 173.702624] nix-grpc-daemon[1567]: time=2026-09-22T10:51:18.709Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7159worker1 # [ 173.704136] nix-grpc-daemon[1567]: time=2026-09-22T10:51:18.711Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7160lb # [ 173.719730] niks3-server[1143]: 2026/09/22 10:51:18 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7161lb # [ 173.730839] niks3-server[1143]: 2026/09/22 10:51:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7162worker1 # [ 173.737474] nix-grpc-daemon[1567]: time=2026-09-22T10:51:18.744Z level=INFO msg="Uploading 1 narinfos"7163lb # [ 173.735594] niks3-server[1143]: 2026/09/22 10:51:18 INFO Signed narinfos id=68 count=17164lb # [ 173.742073] niks3-server[1143]: 2026/09/22 10:51:18 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7165lb # [ 173.753108] niks3-server[1143]: 2026/09/22 10:51:18 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7166lb # [ 173.759825] niks3-server[1143]: 2026/09/22 10:51:18 INFO Completed upload id=687167worker1 # [ 173.762156] nix-grpc-daemon[1567]: time=2026-09-22T10:51:18.769Z level=INFO msg="Upload complete. (398ms)"7168lb # [ 173.762230] niks3-server[1143]: 2026/09/22 10:51:18 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7169worker1 # [ 173.766127] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=7527170worker1 # [ 173.770701] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747171worker1 # [ 173.780637] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7172worker1 # [ 173.783888] nix-daemon[867]: reaped child process 2012, status = succeeded7173worker1 # [ 173.788934] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7174worker1 # [ 175.016563] nix-daemon[867]: reaped child process 2026, status = succeeded7175worker1 # [ 175.165382] nix-daemon[867]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)7176worker1 # [ 175.176344] nix-daemon[867]: reaped child process 2045, status = succeeded7177lb # [ 175.187469] niks3-server[1143]: 2026/09/22 10:51:20 INFO Received uploads request method=POST path=/api/pending_closures7178worker1 # [ 175.193937] nix-grpc-daemon[1567]: time=2026-09-22T10:51:20.200Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7179worker1 # [ 175.195449] nix-grpc-daemon[1567]: time=2026-09-22T10:51:20.202Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7180lb # [ 175.204859] niks3-server[1143]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7181lb # [ 175.209634] niks3-server[1143]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7182lb # [ 175.212241] niks3-server[1143]: 2026/09/22 10:51:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7183lb # [ 175.214321] niks3-server[1143]: 2026/09/22 10:51:20 INFO Signed narinfos id=69 count=17184worker1 # [ 175.216633] nix-grpc-daemon[1567]: time=2026-09-22T10:51:20.223Z level=INFO msg="Uploading 1 narinfos"7185lb # [ 175.217021] niks3-server[1143]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7186lb # [ 175.221151] niks3-server[1143]: 2026/09/22 10:51:20 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7187lb # [ 175.224191] niks3-server[1143]: 2026/09/22 10:51:20 INFO Completed upload id=697188worker1 # [ 175.226413] nix-grpc-daemon[1567]: time=2026-09-22T10:51:20.233Z level=INFO msg="Upload complete. (228ms)"7189lb # [ 175.226792] niks3-server[1143]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7190worker1 # [ 175.230605] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=17191worker1 # [ 175.233273] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:20Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17192worker1 # [ 175.238145] nix-daemon[867]: reaped child process 2024, status = succeeded7193worker1 # [ 175.274511] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747194worker1 # [ 175.283489] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590747195worker1 # [ 175.291896] nix-grpc-daemon[1567]: time=2026-09-22T10:51:20.299Z level=INFO msg="All 2 paths already cached"7196worker1 # [ 175.297417] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:590747197worker1 # [ 175.305937] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7198worker1 # [ 175.448864] nix-daemon[867]: accepted connection from pid 2049, user nix-grpc-daemon (trusted)7199worker1 # [ 175.461119] nix-daemon[867]: reaped child process 2052, status = succeeded7200worker1 # [ 175.584071] nix-daemon[867]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)7201worker1 # [ 175.596075] nix-daemon[867]: reaped child process 2057, status = succeeded7202lb # [ 175.603212] niks3-server[1143]: 2026/09/22 10:51:20 INFO Received uploads request method=POST path=/api/pending_closures7203worker1 # [ 175.613078] nix-grpc-daemon[1567]: time=2026-09-22T10:51:20.620Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7204worker1 # [ 175.614841] nix-grpc-daemon[1567]: time=2026-09-22T10:51:20.621Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7205lb # [ 175.632164] niks3-server[1143]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7206lb # [ 175.640551] niks3-server[1143]: 2026/09/22 10:51:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7207worker1 # [ 175.649129] nix-grpc-daemon[1567]: time=2026-09-22T10:51:20.656Z level=INFO msg="Uploading 1 narinfos"7208lb # [ 175.646642] niks3-server[1143]: 2026/09/22 10:51:20 INFO Signed narinfos id=70 count=17209lb # [ 175.651300] niks3-server[1143]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7210lb # [ 175.663962] niks3-server[1143]: 2026/09/22 10:51:20 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7211lb # [ 175.670415] niks3-server[1143]: 2026/09/22 10:51:20 INFO Completed upload id=707212worker1 # [ 175.672784] nix-grpc-daemon[1567]: time=2026-09-22T10:51:20.679Z level=INFO msg="Upload complete. (361ms)"7213lb # [ 175.672770] niks3-server[1143]: 2026/09/22 10:51:20 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7214worker1 # [ 175.676497] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=0 paths=1 nar_bytes_in=7527215worker1 # [ 175.681415] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:590747216worker1 # [ 175.690621] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7217worker1 # [ 175.698723] nix-daemon[867]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)7218worker1 # [ 175.700822] nix-daemon[867]: reaped child process 2047, status = succeeded7219worker1 # [ 177.113459] nix-daemon[867]: reaped child process 2061, status = succeeded7220worker1 # [ 177.227769] nix-daemon[867]: accepted connection from pid 2077, user nix-grpc-daemon (trusted)7221worker1 # [ 177.240740] nix-daemon[867]: reaped child process 2080, status = succeeded7222lb # [ 177.246974] niks3-server[1143]: 2026/09/22 10:51:22 INFO Received uploads request method=POST path=/api/pending_closures7223worker1 # [ 177.257684] nix-grpc-daemon[1567]: time=2026-09-22T10:51:22.264Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7224worker1 # [ 177.259375] nix-grpc-daemon[1567]: time=2026-09-22T10:51:22.266Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7225lb # [ 177.277375] niks3-server[1143]: 2026/09/22 10:51:22 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7226lb # [ 177.293644] niks3-server[1143]: 2026/09/22 10:51:22 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7227lb # [ 177.302934] niks3-server[1143]: 2026/09/22 10:51:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7228lb # [ 177.307960] niks3-server[1143]: 2026/09/22 10:51:22 INFO Signed narinfos id=71 count=17229worker1 # [ 177.313604] nix-grpc-daemon[1567]: time=2026-09-22T10:51:22.320Z level=INFO msg="Uploading 1 narinfos"7230lb # [ 177.313794] niks3-server[1143]: 2026/09/22 10:51:22 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7231lb # [ 177.324726] niks3-server[1143]: 2026/09/22 10:51:22 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7232lb # [ 177.329570] niks3-server[1143]: 2026/09/22 10:51:22 INFO Completed upload id=717233lb # [ 177.331265] niks3-server[1143]: 2026/09/22 10:51:22 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7234worker1 # [ 177.332220] nix-grpc-daemon[1567]: time=2026-09-22T10:51:22.339Z level=INFO msg="Upload complete. (225ms)"7235worker1 # [ 177.340318] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59074 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=17236worker1 # [ 177.352524] nix-daemon[867]: reaped child process 2059, status = succeeded7237client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.43 seconds)7238worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7239worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7240client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'7241client # [ 177.479041] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7242client # Running as unit: gpu.service; invocation ID: f13db03db86d4fbf8ff612af807655227243client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.04 seconds)7244lb: must succeed: curl -sf localhost:9901/clusters7245lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7246lb: must succeed: curl -sf localhost:9901/clusters7247lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7248worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7249worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7250client # [ 177.690319] nix[1444]: don't know how to build these paths:7251client # [ 177.691272] nix[1444]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7252worker1 # [ 177.681858] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590747253worker1 # [ 177.687070] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590747254client # [ 177.714631] nix[1444]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker7255worker1 # [ 177.707063] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:22Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07256lb # [ 178.150500] envoy[920]: conn peer=[2001:db8:1::1]:41290 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27124 tx=11326 ms=182097257lb: must succeed: curl -sf localhost:9901/clusters7258lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7259lb: must succeed: curl -sf localhost:9901/clusters7260lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7261worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7262worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7263client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7264client # [ 207.715899] nix[1444]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}7265client # [ 207.724790] nix[1444]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7266client # [ 207.763613] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7267client # [ 207.768427] systemd[1]: gpu.service: Failed with result 'exit-code'.7268client # [ 207.771635] systemd[1]: gpu.service: Consumed 149ms CPU time over 30.288s wall clock time, 37.4M memory peak, 5.1K incoming IP traffic, 5.1K outgoing IP traffic.7269lb # [ 208.151695] envoy[920]: conn peer=[2001:db8:1::1]:45726 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300507270client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.62 seconds)7271client: must succeed: journalctl -u gpu -o cat7272client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7273lb: must succeed: curl -sf localhost:9901/clusters7274lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7275lb: must succeed: curl -sf localhost:9901/clusters7276lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7277worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7278worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7279(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.67 seconds)7280subtest: low disk drains a worker and builds go to the other7281worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7282worker1 # [ 209.526963] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:54Z level=info event=unhealthy reason=min_free7283worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.91 seconds)7284worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7285worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.04 seconds)7286client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&27287worker1 # [ 209.768939] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:590747288client # don't know how to build these paths:7289client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7290worker1 # [ 209.777748] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:590747291worker1 # [ 209.797594] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:54Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07292worker1 # [ 209.801795] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:54Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17293worker2 # [ 209.842701] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:516207294worker2 # [ 209.855212] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 1/5)7295worker2 # [ 209.892906] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5)7296worker2 # [ 210.091841] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 235 ms (attempt 3/5)7297worker2 # [ 210.333057] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 346 ms (attempt 4/5)7298worker2 # [ 210.685263] nix-grpc-daemon[1976]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7299worker2 # [ 210.688863] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7300worker2 # [ 210.703490] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:516207301client # copying 1 paths to 'lb:50051'...7302client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7303worker2 # [ 210.715112] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:516207304worker2 # [ 210.723827] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7305worker2 # [ 210.744918] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:55Z level=info event=niks3_push_started pid=20367306worker2 # [ 210.911605] nix-grpc-daemon[2036]: time=2026-09-22T10:51:55.869Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""7307worker2 # [ 211.057795] nix-daemon[856]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)7308worker2 # [ 211.069123] nix-daemon[856]: reaped child process 2057, status = succeeded7309worker2 # [ 211.216780] nix-daemon[856]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)7310worker2 # [ 211.229963] nix-daemon[856]: reaped child process 2062, status = succeeded7311lb # [ 211.242451] niks3-server[1143]: 2026/09/22 10:51:56 INFO Received uploads request method=POST path=/api/pending_closures7312worker2 # [ 211.247761] nix-grpc-daemon[2036]: time=2026-09-22T10:51:56.206Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7313worker2 # [ 211.249782] nix-grpc-daemon[2036]: time=2026-09-22T10:51:56.207Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7314lb # [ 211.265334] niks3-server[1143]: 2026/09/22 10:51:56 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7315lb # [ 211.270712] niks3-server[1143]: 2026/09/22 10:51:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7316lb # [ 211.272955] niks3-server[1143]: 2026/09/22 10:51:56 INFO Signed narinfos id=72 count=17317worker2 # [ 211.275113] nix-grpc-daemon[2036]: time=2026-09-22T10:51:56.233Z level=INFO msg="Uploading 1 narinfos"7318lb # [ 211.278453] niks3-server[1143]: 2026/09/22 10:51:56 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7319lb # [ 211.282913] niks3-server[1143]: 2026/09/22 10:51:56 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7320lb # [ 211.287319] niks3-server[1143]: 2026/09/22 10:51:56 INFO Completed upload id=727321worker2 # [ 211.287756] nix-grpc-daemon[2036]: time=2026-09-22T10:51:56.246Z level=INFO msg="Upload complete. (373ms)"7322lb # [ 211.289496] niks3-server[1143]: 2026/09/22 10:51:56 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7323worker2 # [ 211.290225] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51620 duration_s=0 paths=1 nar_bytes_in=5127324worker2 # [ 211.293850] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:516207325worker2 # [ 211.300334] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7326worker2 # [ 211.303984] nix-daemon[856]: reaped child process 2034, status = succeeded7327client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7328client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7329worker2 # [ 211.309161] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7330client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7331client # farm-a> LOG-farm-a-drain7332worker2 # [ 212.525216] nix-daemon[856]: reaped child process 2066, status = succeeded7333worker2 # [ 212.660595] nix-daemon[856]: accepted connection from pid 2082, user nix-grpc-daemon (trusted)7334worker2 # [ 212.673419] nix-daemon[856]: reaped child process 2085, status = succeeded7335lb # [ 212.683169] niks3-server[1143]: 2026/09/22 10:51:57 INFO Received uploads request method=POST path=/api/pending_closures7336worker2 # [ 212.689256] nix-grpc-daemon[2036]: time=2026-09-22T10:51:57.647Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7337worker2 # [ 212.690730] nix-grpc-daemon[2036]: time=2026-09-22T10:51:57.649Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7338lb # [ 212.703988] niks3-server[1143]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7339lb # [ 212.710649] niks3-server[1143]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7340lb # [ 212.715314] niks3-server[1143]: 2026/09/22 10:51:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7341lb # [ 212.717723] niks3-server[1143]: 2026/09/22 10:51:57 INFO Signed narinfos id=73 count=17342worker2 # [ 212.719947] nix-grpc-daemon[2036]: time=2026-09-22T10:51:57.677Z level=INFO msg="Uploading 1 narinfos"7343lb # [ 212.722024] niks3-server[1143]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7344lb # [ 212.725972] niks3-server[1143]: 2026/09/22 10:51:57 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7345lb # [ 212.729459] niks3-server[1143]: 2026/09/22 10:51:57 INFO Completed upload id=737346worker2 # [ 212.729723] nix-grpc-daemon[2036]: time=2026-09-22T10:51:57.688Z level=INFO msg="Upload complete. (213ms)"7347lb # [ 212.731788] niks3-server[1143]: 2026/09/22 10:51:57 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7348worker2 # [ 212.732248] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51620 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17349worker2 # [ 212.738862] nix-daemon[856]: reaped child process 2064, status = succeeded7350worker1 # [ 212.742653] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:57Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07351worker1 # [ 212.746672] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:57Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17352worker2 # [ 212.783720] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:516207353worker2 # [ 212.793703] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:516207354worker2 # [ 212.803258] nix-grpc-daemon[2036]: time=2026-09-22T10:51:57.761Z level=INFO msg="All 1 paths already cached"7355client # copying 1 paths to 'lb:50051'...7356client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7357worker2 # [ 212.810917] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:516207358worker2 # [ 212.819256] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7359worker2 # [ 212.963807] nix-daemon[856]: accepted connection from pid 2089, user nix-grpc-daemon (trusted)7360worker2 # [ 212.976624] nix-daemon[856]: reaped child process 2092, status = succeeded7361worker2 # [ 213.115360] nix-daemon[856]: accepted connection from pid 2094, user nix-grpc-daemon (trusted)7362worker2 # [ 213.128176] nix-daemon[856]: reaped child process 2097, status = succeeded7363lb # [ 213.141397] niks3-server[1143]: 2026/09/22 10:51:58 INFO Received uploads request method=POST path=/api/pending_closures7364worker2 # [ 213.149557] nix-grpc-daemon[2036]: time=2026-09-22T10:51:58.107Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7365worker2 # [ 213.150968] nix-grpc-daemon[2036]: time=2026-09-22T10:51:58.107Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7366lb # [ 213.166278] niks3-server[1143]: 2026/09/22 10:51:58 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7367lb # [ 213.177487] niks3-server[1143]: 2026/09/22 10:51:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7368lb # [ 213.183545] niks3-server[1143]: 2026/09/22 10:51:58 INFO Signed narinfos id=74 count=17369worker2 # [ 213.187080] nix-grpc-daemon[2036]: time=2026-09-22T10:51:58.145Z level=INFO msg="Uploading 1 narinfos"7370lb # [ 213.186774] niks3-server[1143]: 2026/09/22 10:51:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7371lb # [ 213.201206] niks3-server[1143]: 2026/09/22 10:51:58 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7372lb # [ 213.206731] niks3-server[1143]: 2026/09/22 10:51:58 INFO Completed upload id=747373worker2 # [ 213.207104] nix-grpc-daemon[2036]: time=2026-09-22T10:51:58.165Z level=INFO msg="Upload complete. (377ms)"7374worker2 # [ 213.209187] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51620 duration_s=0 paths=1 nar_bytes_in=6247375lb # [ 213.209861] niks3-server[1143]: 2026/09/22 10:51:58 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7376worker2 # [ 213.214726] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:516207377worker2 # [ 213.222993] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7378worker2 # [ 213.225394] nix-daemon[856]: reaped child process 2087, status = succeeded7379worker2 # [ 213.231124] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7380client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7381client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7382client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7383client # farm-b> LOG-farm-b-drain7384worker2 # [ 214.657829] nix-daemon[856]: reaped child process 2101, status = succeeded7385worker2 # [ 214.824698] nix-daemon[856]: accepted connection from pid 2117, user nix-grpc-daemon (trusted)7386worker2 # [ 214.839685] nix-daemon[856]: reaped child process 2120, status = succeeded7387lb # [ 214.855783] niks3-server[1143]: 2026/09/22 10:51:59 INFO Received uploads request method=POST path=/api/pending_closures7388worker2 # [ 214.871872] nix-grpc-daemon[2036]: time=2026-09-22T10:51:59.830Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7389worker2 # [ 214.874297] nix-grpc-daemon[2036]: time=2026-09-22T10:51:59.831Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7390lb # [ 214.896806] niks3-server[1143]: 2026/09/22 10:51:59 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7391lb # [ 214.908943] niks3-server[1143]: 2026/09/22 10:51:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7392lb # [ 214.914417] niks3-server[1143]: 2026/09/22 10:51:59 INFO Signed narinfos id=75 count=17393worker2 # [ 214.918052] nix-grpc-daemon[2036]: time=2026-09-22T10:51:59.876Z level=INFO msg="Uploading 1 narinfos"7394lb # [ 214.922658] niks3-server[1143]: 2026/09/22 10:51:59 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7395lb # [ 214.930711] niks3-server[1143]: 2026/09/22 10:51:59 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7396lb # [ 214.937568] niks3-server[1143]: 2026/09/22 10:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7397lb # [ 214.943831] niks3-server[1143]: 2026/09/22 10:51:59 INFO Completed upload id=757398worker2 # [ 214.944235] nix-grpc-daemon[2036]: time=2026-09-22T10:51:59.902Z level=INFO msg="Upload complete. (303ms)"7399lb # [ 214.945367] niks3-server[1143]: 2026/09/22 10:51:59 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7400worker2 # [ 214.949618] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51620 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17401worker1 # [ 214.957844] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:59Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07402worker1 # [ 214.962394] nix-grpc-daemon[1471]: ts=2026-09-22T10:51:59Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17403worker2 # [ 214.964779] nix-daemon[856]: reaped child process 2099, status = succeeded7404worker2 # [ 214.999822] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:516207405worker2 # [ 215.011465] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:516207406worker2 # [ 215.024970] nix-grpc-daemon[2036]: time=2026-09-22T10:51:59.983Z level=INFO msg="All 2 paths already cached"7407client # copying 1 paths to 'lb:50051'...7408client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7409worker2 # [ 215.031793] nix-grpc-daemon[1976]: ts=2026-09-22T10:51:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:516207410worker2 # [ 215.042124] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7411worker2 # [ 215.168760] nix-daemon[856]: accepted connection from pid 2124, user nix-grpc-daemon (trusted)7412worker2 # [ 215.183259] nix-daemon[856]: reaped child process 2127, status = succeeded7413worker2 # [ 215.306661] nix-daemon[856]: accepted connection from pid 2129, user nix-grpc-daemon (trusted)7414worker2 # [ 215.320860] nix-daemon[856]: reaped child process 2132, status = succeeded7415lb # [ 215.335808] niks3-server[1143]: 2026/09/22 10:52:00 INFO Received uploads request method=POST path=/api/pending_closures7416worker2 # [ 215.344522] nix-grpc-daemon[2036]: time=2026-09-22T10:52:00.302Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7417worker2 # [ 215.346482] nix-grpc-daemon[2036]: time=2026-09-22T10:52:00.304Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7418lb # [ 215.362388] niks3-server[1143]: 2026/09/22 10:52:00 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7419lb # [ 215.374584] niks3-server[1143]: 2026/09/22 10:52:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7420worker2 # [ 215.379766] nix-grpc-daemon[2036]: time=2026-09-22T10:52:00.337Z level=INFO msg="Uploading 1 narinfos"7421lb # [ 215.379340] niks3-server[1143]: 2026/09/22 10:52:00 INFO Signed narinfos id=76 count=17422lb # [ 215.387132] niks3-server[1143]: 2026/09/22 10:52:00 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7423lb # [ 215.397633] niks3-server[1143]: 2026/09/22 10:52:00 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7424worker2 # [ 215.404305] nix-grpc-daemon[2036]: time=2026-09-22T10:52:00.362Z level=INFO msg="Upload complete. (351ms)"7425lb # [ 215.403918] niks3-server[1143]: 2026/09/22 10:52:00 INFO Completed upload id=767426worker2 # [ 215.406433] nix-grpc-daemon[1976]: ts=2026-09-22T10:52:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51620 duration_s=0 paths=1 nar_bytes_in=7527427lb # [ 215.408720] niks3-server[1143]: 2026/09/22 10:52:00 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7428worker2 # [ 215.409613] nix-grpc-daemon[1976]: ts=2026-09-22T10:52:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:516207429worker2 # [ 215.419104] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7430worker2 # [ 215.423390] nix-daemon[856]: reaped child process 2122, status = succeeded7431worker2 # [ 215.428829] nix-daemon[856]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7432client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7433client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7434client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7435client # farm-top> LOG-farm-top-drain7436worker2 # [ 216.654765] nix-daemon[856]: reaped child process 2136, status = succeeded7437worker2 # [ 216.793896] nix-daemon[856]: accepted connection from pid 2152, user nix-grpc-daemon (trusted)7438worker2 # [ 216.807365] nix-daemon[856]: reaped child process 2155, status = succeeded7439lb # [ 216.821781] niks3-server[1143]: 2026/09/22 10:52:01 INFO Received uploads request method=POST path=/api/pending_closures7440worker2 # [ 216.831865] nix-grpc-daemon[2036]: time=2026-09-22T10:52:01.790Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7441worker2 # [ 216.833512] nix-grpc-daemon[2036]: time=2026-09-22T10:52:01.791Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7442lb # [ 216.866156] niks3-server[1143]: 2026/09/22 10:52:01 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7443lb # [ 216.871502] niks3-server[1143]: 2026/09/22 10:52:01 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7444lb # [ 216.878598] niks3-server[1143]: 2026/09/22 10:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7445lb # [ 216.883466] niks3-server[1143]: 2026/09/22 10:52:01 INFO Signed narinfos id=77 count=17446worker2 # [ 216.886818] nix-grpc-daemon[2036]: time=2026-09-22T10:52:01.845Z level=INFO msg="Uploading 1 narinfos"7447lb # [ 216.890722] niks3-server[1143]: 2026/09/22 10:52:01 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7448lb # [ 216.899574] niks3-server[1143]: 2026/09/22 10:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7449lb # [ 216.905100] niks3-server[1143]: 2026/09/22 10:52:01 INFO Completed upload id=777450lb # [ 216.908598] niks3-server[1143]: 2026/09/22 10:52:01 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7451worker2 # [ 216.907689] nix-grpc-daemon[2036]: time=2026-09-22T10:52:01.865Z level=INFO msg="Upload complete. (268ms)"7452worker2 # [ 216.914651] nix-grpc-daemon[1976]: ts=2026-09-22T10:52:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51620 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17453worker2 # [ 216.927968] nix-daemon[856]: reaped child process 2134, status = succeeded7454client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 7.39 seconds)7455client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7456client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.20 seconds)7457worker1: must succeed: rm /nix/.rw-store/fill7458worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7459worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7460worker1 # [ 217.529603] nix-grpc-daemon[1471]: ts=2026-09-22T10:52:02Z level=info event=healthy reason=min_free7461lb # [ 218.151923] envoy[920]: conn peer=[2001:db8:1::1]:41940 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9576 tx=4130 ms=71567462worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)7463(finished: subtest: low disk drains a worker and builds go to the other, in 9.82 seconds)7464(finished: run the VM test script, in 219.33 seconds)7465test script finished in 219.55s7466cleanup7467kill QemuMachine (pid 49)7468client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7469kill QemuMachine (pid 50)7470lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7471kill QemuMachine (pid 52)7472worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7473kill QemuMachine (pid 51)7474worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7475(finished: cleanup, in 1.01 seconds)7476additionally exposed symbols:7477 client, lb, worker1, worker2,7478 vlan1,7479 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh