vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #224
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1: QEMU running (pid 51)14worker1 # Formatting '/build/vm-state-worker1/tmp.mnH68HXAQx', fmt=raw size=107374182415worker1 # mke2fs 1.47.4 (6-Mar-2025)16worker1 # Discarding device blocks: 0/262144 done17worker2 # Disk image does not exist, creating the virtualisation disk image...18worker2: QEMU running (pid 52)19client # Disk image does not exist, creating the virtualisation disk image...20worker2 # Formatting '/build/vm-state-worker2/tmp.6Hdpt1pgkW', fmt=raw size=107374182421client: QEMU running (pid 49)22worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes23worker1 # Filesystem UUID: 993d37ea-af05-4292-afe0-1d56808ff99d24worker2 # mke2fs 1.47.4 (6-Mar-2025)25worker1 # Superblock backups stored on blocks:26worker2 # Discarding device blocks: 0/262144 done27client # Formatting '/build/vm-state-client/tmp.zgDcZjG98S', fmt=raw size=107374182428worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes29worker2 # Filesystem UUID: 22c198e2-fee1-49c8-b9a8-ff8d1808d9b830client # mke2fs 1.47.4 (6-Mar-2025)31worker1 # 32768, 98304, 163840, 22937632client # Discarding device blocks: 0/262144 done33client # Creating filesystem with 262144 4k blocks and 65536 inodes34client # Filesystem UUID: 0b56ecb1-5382-434f-bdf1-1bc0964985b835worker1 # 36client # Superblock backups stored on blocks:37lb: QEMU running (pid 50)38lb # Disk image does not exist, creating the virtualisation disk image...39worker2 # Superblock backups stored on blocks:40worker2 # 32768, 98304, 163840, 22937641client # 32768, 98304, 163840, 22937642lb # Formatting '/build/vm-state-lb/tmp.Ig4xsdROxo', fmt=raw size=107374182443client # 44worker1 # Allocating group tables: 0/8 done45(finished: start all VMs, in 0.30 seconds)46client # Allocating group tables: 0/8 done47worker1 # Writing inode tables: 0/8 done48worker1 # Creating journal (8192 blocks): done49client # Writing inode tables: 0/8 done50client # Creating journal (8192 blocks): done51worker1 # Writing superblocks and filesystem accounting information: 0/8 done52lb # mke2fs 1.47.4 (6-Mar-2025)53client # Writing superblocks and filesystem accounting information: 0/8 done54worker2 # 55worker1 # 56client # 57worker2 # Allocating group tables: 0/8 done58worker1 # Virtualisation disk image created.59client # Virtualisation disk image created.60worker2 # Writing inode tables: 0/8 done61client: waiting for unit network-addresses-eth1.service62worker2 # Creating journal (8192 blocks): done63worker2 # Writing superblocks and filesystem accounting information: 0/8 done64worker2 # 65client: waiting for the VM to finish booting66lb # Discarding device blocks: 0/262144 done67lb # Creating filesystem with 262144 4k blocks and 65536 inodes68worker2 # Virtualisation disk image created.69lb # Filesystem UUID: 6ed14835-2e51-4b69-844d-96a8d7f40b3a70lb # Superblock backups stored on blocks:71lb # 32768, 98304, 163840, 22937672lb # 73lb # Allocating group tables: 0/8 done74lb # Writing inode tables: 0/8 done75lb # Creating journal (8192 blocks): done76lb # Writing superblocks and filesystem accounting information: 0/8 done77lb # 78lb # Virtualisation disk image created.79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83client # 84client # 85client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker2 # 87worker2 # 88worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90client # 91client # 92client # 93client # 94client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0095lb # 96lb # 97lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0098worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...99worker2 # 100worker2 # 101worker2 # 102worker2 # 103worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00104client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105client # 106client # 107lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...108lb # 109lb # 110lb # 111lb # 112lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00113worker1 # 114worker1 # 115worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00116worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117worker2 # 118worker2 # 119lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...120lb # 121lb # 122worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123worker1 # 124worker1 # 125worker1 # 126worker1 # 127worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129worker1 # 130worker1 # 131client # Booting from ROM...132lb # Booting from ROM...133worker1 # Booting from ROM...134worker2 # Booting from ROM...135client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test/init regInfo=/nix/store/6kgfl2iqfvz961zp6fi4ckdwfp6lx06q-closure-info/registration console=ttyS0,115200n8 console=tty0137client # [ 0.000000] BIOS-provided physical RAM map:138client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable139client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved140client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved141client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable142client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved143client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved144client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved145client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved146worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026147client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved148client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved149client # [ 0.000000] NX (Execute Disable) protection: active150client # [ 0.000000] APIC: Static calls initialized151client # [ 0.000000] SMBIOS 2.8 present.152worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0m0synxj7cnm2qgsfvjzm9pmmycgj9s6-nixos-system-worker1-test/init regInfo=/nix/store/drn0bn3cd0p6bqw1v1r0ramnh3x7qb3w-closure-info/registration console=ttyS0,115200n8 console=tty0153lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026154client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014155worker1 # [ 0.000000] BIOS-provided physical RAM map:156client # [ 0.000000] DMI: Memory slots populated: 1/1157client # [ 0.000000] Hypervisor detected: KVM158worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable159client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000160worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved161client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00162worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved163client # [ 0.000001] kvm-clock: using sched offset of 486402668 cycles164worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable165lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8y2rmlwiyr2iliam1yhl8i23qgaaknj8-nixos-system-lb-test/init regInfo=/nix/store/9qjmxix17wyd2infdr8gh5ii9mr60zqd-closure-info/registration console=ttyS0,115200n8 console=tty0166client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns167lb # [ 0.000000] BIOS-provided physical RAM map:168worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved169client # [ 0.000006] tsc: Detected 2400.012 MHz processor170lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable171worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved172client # [ 0.000833] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000173lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved174worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved175client # [ 0.000862] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs176lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved177worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved178client # [ 0.000865] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT179lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable180client # [ 0.002808] found SMP MP-table at [mem 0x000f5450-0x000f545f]181worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved182client # [ 0.002822] Using GB pages for direct mapping183lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved184worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved185client # [ 0.002905] RAMDISK: [mem 0x5e366000-0x5ffcffff]186lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved187worker1 # [ 0.000000] NX (Execute Disable) protection: active188client # [ 0.002916] ACPI: Early table checksum verification disabled189worker1 # [ 0.000000] APIC: Static calls initialized190lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved191worker1 # [ 0.000000] SMBIOS 2.8 present.192client # [ 0.002920] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )193lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved194client # [ 0.002925] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)195worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014196lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved197worker1 # [ 0.000000] DMI: Memory slots populated: 1/1198client # [ 0.002930] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)199worker1 # [ 0.000000] Hypervisor detected: KVM200lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved201worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000202lb # [ 0.000000] NX (Execute Disable) protection: active203client # [ 0.002939] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)204worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00205lb # [ 0.000000] APIC: Static calls initialized206client # [ 0.002941] ACPI: FACS 0x000000005FFE0000 000040207lb # [ 0.000000] SMBIOS 2.8 present.208worker1 # [ 0.000001] kvm-clock: using sched offset of 503134845 cycles209client # [ 0.002943] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)210lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014211worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns212lb # [ 0.000000] DMI: Memory slots populated: 1/1213client # [ 0.002944] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)214worker1 # [ 0.000006] tsc: Detected 2400.012 MHz processor215lb # [ 0.000000] Hypervisor detected: KVM216worker1 # [ 0.000832] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000217client # [ 0.002946] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)218lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000219lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00220worker1 # [ 0.000860] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs221client # [ 0.002948] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)222lb # [ 0.000000] kvm-clock: using sched offset of 502153840 cycles223worker1 # [ 0.000863] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT224client # [ 0.002949] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]225worker1 # [ 0.002802] found SMP MP-table at [mem 0x000f5450-0x000f545f]226lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns227client # [ 0.002950] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]228worker1 # [ 0.002815] Using GB pages for direct mapping229lb # [ 0.000006] tsc: Detected 2400.012 MHz processor230worker1 # [ 0.002948] RAMDISK: [mem 0x5e367000-0x5ffcffff]231client # [ 0.002951] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]232lb # [ 0.000841] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000233worker1 # [ 0.002956] ACPI: Early table checksum verification disabled234client # [ 0.002951] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]235lb # [ 0.000870] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs236worker1 # [ 0.002961] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )237client # [ 0.002952] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]238lb # [ 0.000873] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT239worker1 # [ 0.002965] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)240client # [ 0.002952] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]241lb # [ 0.002818] found SMP MP-table at [mem 0x000f5450-0x000f545f]242client # [ 0.002953] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]243lb # [ 0.002833] Using GB pages for direct mapping244worker1 # [ 0.002970] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)245client # [ 0.003217] No NUMA configuration found246lb # [ 0.002916] RAMDISK: [mem 0x5e367000-0x5ffcffff]247worker1 # [ 0.002977] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)248lb # [ 0.002924] ACPI: Early table checksum verification disabled249client # [ 0.003218] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]250worker1 # [ 0.002979] ACPI: FACS 0x000000005FFE0000 000040251client # [ 0.003221] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]252lb # [ 0.002928] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )253client # [ 0.005518] Zone ranges:254worker1 # [ 0.002981] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)255client # [ 0.005519] DMA [mem 0x0000000000001000-0x0000000000ffffff]256lb # [ 0.002932] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)257worker1 # [ 0.002983] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)258client # [ 0.005521] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]259lb # [ 0.002937] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)260client # [ 0.005522] Normal empty261client # [ 0.005523] Device empty262worker1 # [ 0.002984] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)263client # [ 0.005523] Movable zone start for each node264lb # [ 0.002945] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)265client # [ 0.005524] Early memory node ranges266lb # [ 0.002947] ACPI: FACS 0x000000005FFE0000 000040267worker1 # [ 0.002986] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)268client # [ 0.005525] node 0: [mem 0x0000000000001000-0x000000000009efff]269worker1 # [ 0.002987] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]270lb # [ 0.002949] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)271client # [ 0.005526] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]272worker1 # [ 0.002988] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]273lb # [ 0.002950] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)274client # [ 0.005527] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]275worker1 # [ 0.002989] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]276client # [ 0.005550] On node 0, zone DMA: 1 pages in unavailable ranges277lb # [ 0.002952] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)278worker1 # [ 0.002989] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]279client # [ 0.005814] On node 0, zone DMA: 97 pages in unavailable ranges280lb # [ 0.002954] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)281client # [ 0.032286] On node 0, zone DMA32: 40 pages in unavailable ranges282worker1 # [ 0.002990] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]283client # [ 0.032779] ACPI: PM-Timer IO Port: 0x608284lb # [ 0.002955] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]285worker1 # [ 0.002990] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]286client # [ 0.032796] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])287lb # [ 0.002956] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]288worker1 # [ 0.002991] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]289client # [ 0.032835] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23290worker1 # [ 0.003228] No NUMA configuration found291lb # [ 0.002957] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]292client # [ 0.032837] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)293worker1 # [ 0.003228] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]294lb # [ 0.002957] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]295client # [ 0.032839] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)296worker1 # [ 0.003231] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]297lb # [ 0.002958] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]298worker1 # [ 0.005349] Zone ranges:299client # [ 0.032840] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)300worker1 # [ 0.005350] DMA [mem 0x0000000000001000-0x0000000000ffffff]301lb # [ 0.002958] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]302client # [ 0.032842] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)303worker1 # [ 0.005352] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]304lb # [ 0.002959] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]305worker1 # [ 0.005353] Normal empty306client # [ 0.032842] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)307lb # [ 0.003205] No NUMA configuration found308worker1 # [ 0.005354] Device empty309worker1 # [ 0.005354] Movable zone start for each node310client # [ 0.032845] ACPI: Using ACPI (MADT) for SMP configuration information311lb # [ 0.003205] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]312worker1 # [ 0.005355] Early memory node ranges313client # [ 0.032846] ACPI: HPET id: 0x8086a201 base: 0xfed00000314lb # [ 0.003208] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]315client # [ 0.032852] TSC deadline timer available316worker1 # [ 0.005356] node 0: [mem 0x0000000000001000-0x000000000009efff]317lb # [ 0.005370] Zone ranges:318client # [ 0.032857] CPU topo: Max. logical packages: 1319lb # [ 0.005372] DMA [mem 0x0000000000001000-0x0000000000ffffff]320worker1 # [ 0.005357] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]321client # [ 0.032858] CPU topo: Max. logical dies: 1322lb # [ 0.005373] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]323worker1 # [ 0.005358] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]324client # [ 0.032858] CPU topo: Max. dies per package: 1325lb # [ 0.005374] Normal empty326client # [ 0.032862] CPU topo: Max. threads per core: 1327lb # [ 0.005375] Device empty328worker1 # [ 0.005378] On node 0, zone DMA: 1 pages in unavailable ranges329lb # [ 0.005376] Movable zone start for each node330client # [ 0.032863] CPU topo: Num. cores per package: 1331worker1 # [ 0.005630] On node 0, zone DMA: 97 pages in unavailable ranges332lb # [ 0.005377] Early memory node ranges333client # [ 0.032863] CPU topo: Num. threads per package: 1334worker1 # [ 0.031314] On node 0, zone DMA32: 40 pages in unavailable ranges335lb # [ 0.005377] node 0: [mem 0x0000000000001000-0x000000000009efff]336client # [ 0.032863] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs337worker1 # [ 0.031816] ACPI: PM-Timer IO Port: 0x608338lb # [ 0.005378] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]339worker1 # [ 0.031831] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])340client # [ 0.032890] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()341lb # [ 0.005379] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]342worker1 # [ 0.031862] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23343client # [ 0.032930] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]344lb # [ 0.005399] On node 0, zone DMA: 1 pages in unavailable ranges345worker1 # [ 0.031865] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)346client # [ 0.032931] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]347lb # [ 0.005650] On node 0, zone DMA: 97 pages in unavailable ranges348worker1 # [ 0.031867] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)349client # [ 0.032933] [mem 0x60000000-0xafffffff] available for PCI devices350lb # [ 0.031974] On node 0, zone DMA32: 40 pages in unavailable ranges351worker1 # [ 0.031869] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)352client # [ 0.032935] Booting paravirtualized kernel on KVM353lb # [ 0.032475] ACPI: PM-Timer IO Port: 0x608354worker1 # [ 0.031870] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)355lb # [ 0.032489] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])356client # [ 0.032939] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns357worker1 # [ 0.031870] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)358lb # [ 0.032520] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23359client # [ 0.037457] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1360worker1 # [ 0.031873] ACPI: Using ACPI (MADT) for SMP configuration information361lb # [ 0.032523] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)362client # [ 0.039642] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152363worker1 # [ 0.031874] ACPI: HPET id: 0x8086a201 base: 0xfed00000364lb # [ 0.032525] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)365worker1 # [ 0.031880] TSC deadline timer available366client # [ 0.039695] kvm-guest: PV spinlocks disabled, single CPU367worker1 # [ 0.031884] CPU topo: Max. logical packages: 1368lb # [ 0.032527] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)369worker1 # [ 0.031885] CPU topo: Max. logical dies: 1370lb # [ 0.032528] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)371worker1 # [ 0.031886] CPU topo: Max. dies per package: 1372worker1 # [ 0.031890] CPU topo: Max. threads per core: 1373lb # [ 0.032529] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)374worker1 # [ 0.031891] CPU topo: Num. cores per package: 1375lb # [ 0.032531] ACPI: Using ACPI (MADT) for SMP configuration information376worker1 # [ 0.031891] CPU topo: Num. threads per package: 1377lb # [ 0.032532] ACPI: HPET id: 0x8086a201 base: 0xfed00000378client # [ 0.039697] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test/init regInfo=/nix/store/6kgfl2iqfvz961zp6fi4ckdwfp6lx06q-closure-info/registration console=ttyS0,115200n8 console=tty0379lb # [ 0.032538] TSC deadline timer available380worker1 # [ 0.031892] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs381lb # [ 0.032542] CPU topo: Max. logical packages: 1382worker1 # [ 0.031915] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()383lb # [ 0.032543] CPU topo: Max. logical dies: 1384client # [ 0.039801] Unknown kernel command line parameters "regInfo=/nix/store/6kgfl2iqfvz961zp6fi4ckdwfp6lx06q-closure-info/registration", will be passed to user space.385lb # [ 0.032543] CPU topo: Max. dies per package: 1386worker1 # [ 0.031952] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]387client # [ 0.039815] random: crng init done388lb # [ 0.032547] CPU topo: Max. threads per core: 1389worker1 # [ 0.031953] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]390client # [ 0.039816] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes391lb # [ 0.032548] CPU topo: Num. cores per package: 1392worker1 # [ 0.031955] [mem 0x60000000-0xafffffff] available for PCI devices393lb # [ 0.032548] CPU topo: Num. threads per package: 1394client # [ 0.041996] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)395worker1 # [ 0.031957] Booting paravirtualized kernel on KVM396lb # [ 0.032549] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs397client # [ 0.043237] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)398lb # [ 0.032572] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()399client # [ 0.043280] Fallback order for Node 0: 0400worker1 # [ 0.031960] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns401client # [ 0.043285] Built 1 zonelists, mobility grouping on. Total pages: 393078402lb # [ 0.032610] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]403worker1 # [ 0.036650] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1404client # [ 0.043286] Policy zone: DMA32405lb # [ 0.032611] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]406worker1 # [ 0.038868] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152407client # [ 0.045818] mem auto-init: stack:all(zero), heap alloc:on, heap free:off408lb # [ 0.032613] [mem 0x60000000-0xafffffff] available for PCI devices409worker1 # [ 0.038918] kvm-guest: PV spinlocks disabled, single CPU410client # [ 0.049305] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1411lb # [ 0.032614] Booting paravirtualized kernel on KVM412client # [ 0.052778] allocated 3145728 bytes of page_ext413client # [ 0.063973] ftrace: allocating 48731 entries in 192 pages414lb # [ 0.032618] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns415client # [ 0.063978] ftrace: allocated 192 pages with 2 groups416client # [ 0.065036] Dynamic Preempt: lazy417lb # [ 0.037395] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1418client # [ 0.065236] rcu: Preemptible hierarchical RCU implementation.419lb # [ 0.039675] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152420client # [ 0.065237] rcu: RCU event tracing is enabled.421worker1 # [ 0.038920] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0m0synxj7cnm2qgsfvjzm9pmmycgj9s6-nixos-system-worker1-test/init regInfo=/nix/store/drn0bn3cd0p6bqw1v1r0ramnh3x7qb3w-closure-info/registration console=ttyS0,115200n8 console=tty0422lb # [ 0.039724] kvm-guest: PV spinlocks disabled, single CPU423client # [ 0.065238] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.424client # [ 0.065239] Trampoline variant of Tasks RCU enabled.425worker1 # [ 0.039024] Unknown kernel command line parameters "regInfo=/nix/store/drn0bn3cd0p6bqw1v1r0ramnh3x7qb3w-closure-info/registration", will be passed to user space.426client # [ 0.065240] Rude variant of Tasks RCU enabled.427worker1 # [ 0.039039] random: crng init done428client # [ 0.065240] Tracing variant of Tasks RCU enabled.429worker1 # [ 0.039040] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes430client # [ 0.065241] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.431worker1 # [ 0.041226] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)432lb # [ 0.039726] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8y2rmlwiyr2iliam1yhl8i23qgaaknj8-nixos-system-lb-test/init regInfo=/nix/store/9qjmxix17wyd2infdr8gh5ii9mr60zqd-closure-info/registration console=ttyS0,115200n8 console=tty0433client # [ 0.065242] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1434worker1 # [ 0.042452] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)435client # [ 0.065261] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.436worker1 # [ 0.042493] Fallback order for Node 0: 0437lb # [ 0.039827] Unknown kernel command line parameters "regInfo=/nix/store/9qjmxix17wyd2infdr8gh5ii9mr60zqd-closure-info/registration", will be passed to user space.438worker1 # [ 0.042497] Built 1 zonelists, mobility grouping on. Total pages: 393078439lb # [ 0.039842] random: crng init done440client # [ 0.065263] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.441worker1 # [ 0.042498] Policy zone: DMA32442lb # [ 0.039843] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes443worker1 # [ 0.045045] mem auto-init: stack:all(zero), heap alloc:on, heap free:off444client # [ 0.065264] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.445lb # [ 0.042010] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)446client # [ 0.070013] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16447worker1 # [ 0.048417] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1448worker1 # [ 0.051791] allocated 3145728 bytes of page_ext449client # [ 0.070341] rcu: srcu_init: Setting srcu_struct sizes based on contention.450lb # [ 0.043093] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)451worker1 # [ 0.062347] ftrace: allocating 48731 entries in 192 pages452lb # [ 0.043137] Fallback order for Node 0: 0453worker1 # [ 0.062351] ftrace: allocated 192 pages with 2 groups454client # [ 0.070352] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns455lb # [ 0.043140] Built 1 zonelists, mobility grouping on. Total pages: 393078456worker1 # [ 0.063294] Dynamic Preempt: lazy457lb # [ 0.043142] Policy zone: DMA32458worker1 # [ 0.063459] rcu: Preemptible hierarchical RCU implementation.459client # [ 0.070474] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)460lb # [ 0.045820] mem auto-init: stack:all(zero), heap alloc:on, heap free:off461worker1 # [ 0.063460] rcu: RCU event tracing is enabled.462client # [ 0.074572] Console: colour VGA+ 80x25463lb # [ 0.049169] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1464client # [ 0.074576] printk: legacy console [tty0] enabled465worker1 # [ 0.063461] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.466lb # [ 0.052556] allocated 3145728 bytes of page_ext467client # [ 0.106021] printk: legacy console [ttyS0] enabled468worker1 # [ 0.063462] Trampoline variant of Tasks RCU enabled.469lb # [ 0.063123] ftrace: allocating 48731 entries in 192 pages470worker1 # [ 0.063463] Rude variant of Tasks RCU enabled.471client # [ 0.208375] ACPI: Core revision 20250807472lb # [ 0.063126] ftrace: allocated 192 pages with 2 groups473worker1 # [ 0.063463] Tracing variant of Tasks RCU enabled.474lb # [ 0.064079] Dynamic Preempt: lazy475worker1 # [ 0.063464] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.476lb # [ 0.064214] rcu: Preemptible hierarchical RCU implementation.477client # [ 0.209309] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns478lb # [ 0.064215] rcu: RCU event tracing is enabled.479worker1 # [ 0.063465] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1480client # [ 0.210937] APIC: Switch to symmetric I/O mode setup481lb # [ 0.064216] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.482worker1 # [ 0.063487] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.483client # [ 0.211912] x2apic enabled484lb # [ 0.064217] Trampoline variant of Tasks RCU enabled.485lb # [ 0.064218] Rude variant of Tasks RCU enabled.486worker1 # [ 0.063489] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.487client # [ 0.212687] APIC: Switched APIC routing to: physical x2apic488lb # [ 0.064218] Tracing variant of Tasks RCU enabled.489worker1 # [ 0.063490] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.490lb # [ 0.064219] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.491worker1 # [ 0.068278] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16492client # [ 0.214518] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1493lb # [ 0.064220] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1494worker1 # [ 0.068571] rcu: srcu_init: Setting srcu_struct sizes based on contention.495lb # [ 0.064271] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.496client # [ 0.215504] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns497worker1 # [ 0.068579] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns498lb # [ 0.064273] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.499client # [ 0.217158] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)500worker1 # [ 0.068712] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)501lb # [ 0.064274] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.502worker1 # [ 0.072512] Console: colour VGA+ 80x25503lb # [ 0.068969] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16504worker1 # [ 0.072516] printk: legacy console [tty0] enabled505client # [ 0.218250] x86/cpu: User Mode Instruction Prevention (UMIP) activated506worker1 # [ 0.104253] printk: legacy console [ttyS0] enabled507lb # [ 0.069275] rcu: srcu_init: Setting srcu_struct sizes based on contention.508client # [ 0.219330] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127509worker1 # [ 0.206922] ACPI: Core revision 20250807510lb # [ 0.069284] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns511client # [ 0.220898] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0512lb # [ 0.069414] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)513worker1 # [ 0.207942] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns514lb # [ 0.073219] Console: colour VGA+ 80x25515client # [ 0.221938] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto516lb # [ 0.073223] printk: legacy console [tty0] enabled517worker1 # [ 0.209552] APIC: Switch to symmetric I/O mode setup518lb # [ 0.105014] printk: legacy console [ttyS0] enabled519worker1 # [ 0.210551] x2apic enabled520client # [ 0.223154] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl521lb # [ 0.207697] ACPI: Core revision 20250807522worker1 # [ 0.211354] APIC: Switched APIC routing to: physical x2apic523client # [ 0.224154] Transient Scheduler Attacks: Vulnerable: No microcode524client # [ 0.225880] Spectre V2 : Mitigation: Enhanced / Automatic IBRS525lb # [ 0.208627] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns526worker1 # [ 0.213248] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1527client # [ 0.226859] Speculative Return Stack Overflow: Mitigation: Safe RET528lb # [ 0.210199] APIC: Switch to symmetric I/O mode setup529lb # [ 0.211180] x2apic enabled530worker1 # [ 0.214256] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns531lb # [ 0.211969] APIC: Switched APIC routing to: physical x2apic532worker1 # [ 0.215930] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)533lb # [ 0.213859] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1534client # [ 0.227906] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization535worker1 # [ 0.217026] x86/cpu: User Mode Instruction Prevention (UMIP) activated536client # [ 0.229164] Spectre V2 : Enabling IBPB for BPF537worker1 # [ 0.218974] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127538lb # [ 0.214894] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns539client # [ 0.230702] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier540worker1 # [ 0.219884] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0541lb # [ 0.216566] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)542client # [ 0.231154] active return thunk: srso_alias_return_thunk543worker1 # [ 0.220694] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto544lb # [ 0.217653] x86/cpu: User Mode Instruction Prevention (UMIP) activated545client # [ 0.232032] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'546lb # [ 0.219445] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127547worker1 # [ 0.221927] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl548client # [ 0.233154] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'549lb # [ 0.220256] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0550worker1 # [ 0.222927] Transient Scheduler Attacks: Vulnerable: No microcode551client # [ 0.234154] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'552worker1 # [ 0.223850] Spectre V2 : Mitigation: Enhanced / Automatic IBRS553client # [ 0.235154] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'554lb # [ 0.221333] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto555worker1 # [ 0.224625] Speculative Return Stack Overflow: Mitigation: Safe RET556worker2 # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026557client # [ 0.236154] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'558lb # [ 0.222563] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl559client # [ 0.237154] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'560lb # [ 0.223563] Transient Scheduler Attacks: Vulnerable: No microcode561client # [ 0.238154] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'562lb # [ 0.225282] Spectre V2 : Mitigation: Enhanced / Automatic IBRS563client # [ 0.239156] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256564lb # [ 0.226260] Speculative Return Stack Overflow: Mitigation: Safe RET565worker1 # [ 0.225648] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization566client # [ 0.240154] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64567worker1 # [ 0.226935] Spectre V2 : Enabling IBPB for BPF568worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/24bj5klpiq603f5r4anb8fgirj501nj5-nixos-system-worker2-test/init regInfo=/nix/store/xn11lzr16q974jq38iwgxwjsfyi2rrj6-closure-info/registration console=ttyS0,115200n8 console=tty0569client # [ 0.241154] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512570worker2 # [ 0.000000] BIOS-provided physical RAM map:571worker1 # [ 0.228467] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier572client # [ 0.242154] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024573worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable574worker1 # [ 0.228928] active return thunk: srso_alias_return_thunk575client # [ 0.243153] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8576worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved577lb # [ 0.227302] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization578worker1 # [ 0.230803] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'579client # [ 0.244166] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.580lb # [ 0.228572] Spectre V2 : Enabling IBPB for BPF581worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved582worker1 # [ 0.231927] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'583worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable584lb # [ 0.230101] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier585worker1 # [ 0.232926] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'586lb # [ 0.230564] active return thunk: srso_alias_return_thunk587worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved588worker1 # [ 0.233926] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'589worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved590worker1 # [ 0.234926] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'591lb # [ 0.231460] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'592lb # [ 0.232563] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'593worker1 # [ 0.235927] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'594worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved595lb # [ 0.233563] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'596worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved597worker1 # [ 0.236926] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'598lb # [ 0.234563] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'599worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved600worker1 # [ 0.237927] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256601lb # [ 0.235563] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'602worker1 # [ 0.238926] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64603worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved604worker2 # [ 0.000000] NX (Execute Disable) protection: active605lb # [ 0.236563] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'606worker1 # [ 0.239926] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512607worker2 # [ 0.000000] APIC: Static calls initialized608worker1 # [ 0.240926] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024609worker2 # [ 0.000000] SMBIOS 2.8 present.610lb # [ 0.237564] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'611worker1 # [ 0.241927] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8612lb # [ 0.238563] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256613worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014614lb # [ 0.240302] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64615worker2 # [ 0.000000] DMI: Memory slots populated: 1/1616worker1 # [ 0.242927] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.617worker2 # [ 0.000000] Hypervisor detected: KVM618lb # [ 0.241292] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512619worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000620lb # [ 0.241563] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024621worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00622lb # [ 0.242563] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8623worker2 # [ 0.000001] kvm-clock: using sched offset of 568531747 cycles624lb # [ 0.243562] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.625worker2 # [ 0.000004] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns626worker2 # [ 0.000009] tsc: Detected 2400.012 MHz processor627worker2 # [ 0.001169] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000628worker2 # [ 0.001213] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs629worker2 # [ 0.001217] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT630worker2 # [ 0.003715] found SMP MP-table at [mem 0x000f5450-0x000f545f]631worker2 # [ 0.003738] Using GB pages for direct mapping632worker2 # [ 0.003870] RAMDISK: [mem 0x5e367000-0x5ffcffff]633worker2 # [ 0.003889] ACPI: Early table checksum verification disabled634worker2 # [ 0.003895] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )635worker2 # [ 0.003902] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)636worker2 # [ 0.003909] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)637worker2 # [ 0.003922] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)638worker2 # [ 0.003926] ACPI: FACS 0x000000005FFE0000 000040639worker2 # [ 0.003929] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)640worker2 # [ 0.003933] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)641worker2 # [ 0.003936] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)642worker2 # [ 0.003939] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)643worker2 # [ 0.003942] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]644client # [ 0.282861] Freeing SMP alternatives memory: 44K645worker2 # [ 0.003943] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]646client # [ 0.283159] pid_max: default: 32768 minimum: 301647worker2 # [ 0.003944] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]648client # [ 0.284268] LSM: initializing lsm=capability,landlock,yama,bpf,ima649client # [ 0.285267] landlock: Up and running.650worker2 # [ 0.003945] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]651client # [ 0.286154] Yama: becoming mindful.652client # [ 0.287308] LSM support for eBPF active653worker2 # [ 0.003946] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]654worker2 # [ 0.003946] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]655client # [ 0.288094] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)656worker2 # [ 0.003947] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]657client # [ 0.289201] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)658worker2 # [ 0.004310] No NUMA configuration found659worker2 # [ 0.004312] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]660worker2 # [ 0.004317] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]661client # [ 0.291681] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)662worker2 # [ 0.007388] Zone ranges:663worker2 # [ 0.007391] DMA [mem 0x0000000000001000-0x0000000000ffffff]664client # [ 0.292688] Performance Events: Fam17h+ core perfctr, AMD PMU driver.665worker1 # [ 0.279757] Freeing SMP alternatives memory: 44K666worker2 # [ 0.007394] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]667client # [ 0.293160] ... version: 2668worker1 # [ 0.280509] pid_max: default: 32768 minimum: 301669worker2 # [ 0.007395] Normal empty670client # [ 0.293878] ... bit width: 48671worker2 # [ 0.007396] Device empty672worker1 # [ 0.281025] LSM: initializing lsm=capability,landlock,yama,bpf,ima673client # [ 0.294157] ... generic counters: 6674worker2 # [ 0.007397] Movable zone start for each node675worker2 # [ 0.007398] Early memory node ranges676worker1 # [ 0.282047] landlock: Up and running.677client # [ 0.294854] ... generic bitmap: 000000000000003f678worker1 # [ 0.282927] Yama: becoming mindful.679worker2 # [ 0.007399] node 0: [mem 0x0000000000001000-0x000000000009efff]680client # [ 0.295219] ... fixed-purpose counters: 0681worker1 # [ 0.284066] LSM support for eBPF active682lb # [ 0.281556] Freeing SMP alternatives memory: 44K683client # [ 0.295948] ... fixed-purpose bitmap: 0000000000000000684worker2 # [ 0.007401] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]685lb # [ 0.282396] pid_max: default: 32768 minimum: 301686worker1 # [ 0.284850] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)687client # [ 0.296839] ... value mask: 0000ffffffffffff688worker2 # [ 0.007402] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]689lb # [ 0.283215] LSM: initializing lsm=capability,landlock,yama,bpf,ima690client # [ 0.297159] ... max period: 00007fffffffffff691worker2 # [ 0.007430] On node 0, zone DMA: 1 pages in unavailable ranges692worker1 # [ 0.285975] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)693lb # [ 0.283676] landlock: Up and running.694client # [ 0.298046] ... global_ctrl mask: 000000000000003f695lb # [ 0.285010] Yama: becoming mindful.696worker2 # [ 0.007800] On node 0, zone DMA: 97 pages in unavailable ranges697client # [ 0.298976] signal: max sigframe size: 3376698lb # [ 0.285791] LSM support for eBPF active699worker2 # [ 0.043856] On node 0, zone DMA32: 40 pages in unavailable ranges700client # [ 0.299756] rcu: Hierarchical SRCU implementation.701worker1 # [ 0.288323] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)702worker2 # [ 0.044564] ACPI: PM-Timer IO Port: 0x608703lb # [ 0.286676] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)704client # [ 0.300160] rcu: Max phase no-delay instances is 400.705worker2 # [ 0.044585] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])706worker1 # [ 0.289514] Performance Events: Fam17h+ core perfctr, AMD PMU driver.707lb # [ 0.287598] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)708worker2 # [ 0.044627] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23709worker1 # [ 0.289937] ... version: 2710worker1 # [ 0.290649] ... bit width: 48711worker2 # [ 0.044631] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)712worker1 # [ 0.290929] ... generic counters: 6713worker2 # [ 0.044633] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)714lb # [ 0.290472] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)715worker1 # [ 0.291633] ... generic bitmap: 000000000000003f716worker2 # [ 0.044635] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)717worker1 # [ 0.291929] ... fixed-purpose counters: 0718worker1 # [ 0.292637] ... fixed-purpose bitmap: 0000000000000000719worker2 # [ 0.044637] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)720lb # [ 0.291104] Performance Events: Fam17h+ core perfctr, AMD PMU driver.721client # [ 0.305886] smp: Bringing up secondary CPUs ...722lb # [ 0.291568] ... version: 2723worker1 # [ 0.292928] ... value mask: 0000ffffffffffff724client # [ 0.306174] smp: Brought up 1 node, 1 CPU725worker2 # [ 0.044638] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)726lb # [ 0.292254] ... bit width: 48727worker1 # [ 0.293800] ... max period: 00007fffffffffff728client # [ 0.306834] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)729worker2 # [ 0.044643] ACPI: Using ACPI (MADT) for SMP configuration information730lb # [ 0.292565] ... generic counters: 6731worker1 # [ 0.294581] ... global_ctrl mask: 000000000000003f732worker2 # [ 0.044645] ACPI: HPET id: 0x8086a201 base: 0xfed00000733lb # [ 0.293291] ... generic bitmap: 000000000000003f734worker2 # [ 0.044653] TSC deadline timer available735worker1 # [ 0.295039] signal: max sigframe size: 3376736lb # [ 0.293581] ... fixed-purpose counters: 0737worker2 # [ 0.044659] CPU topo: Max. logical packages: 1738client # [ 0.307474] Memory: 1454556K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)739worker1 # [ 0.295903] rcu: Hierarchical SRCU implementation.740lb # [ 0.294276] ... fixed-purpose bitmap: 0000000000000000741worker2 # [ 0.044660] CPU topo: Max. logical dies: 1742client # [ 0.308484] devtmpfs: initialized743worker1 # [ 0.296599] rcu: Max phase no-delay instances is 400.744lb # [ 0.294567] ... value mask: 0000ffffffffffff745worker2 # [ 0.044661] CPU topo: Max. dies per package: 1746client # [ 0.309296] x86/mm: Memory block size: 128MB747worker2 # [ 0.044666] CPU topo: Max. threads per core: 1748lb # [ 0.295460] ... max period: 00007fffffffffff749worker2 # [ 0.044667] CPU topo: Num. cores per package: 1750lb # [ 0.296306] ... global_ctrl mask: 000000000000003f751worker2 # [ 0.044667] CPU topo: Num. threads per package: 1752lb # [ 0.296696] signal: max sigframe size: 3376753client # [ 0.311142] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)754worker2 # [ 0.044668] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs755lb # [ 0.297594] rcu: Hierarchical SRCU implementation.756client # [ 0.312073] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).757worker2 # [ 0.044701] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()758lb # [ 0.298406] rcu: Max phase no-delay instances is 400.759client # [ 0.313293] pinctrl core: initialized pinctrl subsystem760worker1 # [ 0.301827] smp: Bringing up secondary CPUs ...761worker2 # [ 0.044758] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]762worker1 # [ 0.302525] smp: Brought up 1 node, 1 CPU763client # [ 0.314485] PM: RTC time: 08:03:45, date: 2026-09-20764worker2 # [ 0.044761] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]765worker1 # [ 0.302930] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)766worker2 # [ 0.044763] [mem 0x60000000-0xafffffff] available for PCI devices767worker2 # [ 0.044765] Booting paravirtualized kernel on KVM768worker1 # [ 0.304121] Memory: 1454592K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)769client # [ 0.317922] NET: Registered PF_NETLINK/PF_ROUTE protocol family770lb # [ 0.304044] smp: Bringing up secondary CPUs ...771worker2 # [ 0.044770] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns772worker1 # [ 0.305222] devtmpfs: initialized773lb # [ 0.304584] smp: Brought up 1 node, 1 CPU774worker1 # [ 0.306051] x86/mm: Memory block size: 128MB775worker2 # [ 0.049986] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1776client # [ 0.319411] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations777lb # [ 0.305252] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)778worker2 # [ 0.052959] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152779client # [ 0.320182] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations780worker2 # [ 0.053028] kvm-guest: PV spinlocks disabled, single CPU781worker1 # [ 0.307877] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)782lb # [ 0.305753] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)783client # [ 0.321430] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations784lb # [ 0.306847] devtmpfs: initialized785worker1 # [ 0.308825] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).786client # [ 0.322171] audit: initializing netlink subsys (disabled)787worker1 # [ 0.310026] pinctrl core: initialized pinctrl subsystem788lb # [ 0.307727] x86/mm: Memory block size: 128MB789client # [ 0.323159] audit: type=2000 audit(1789891426.273:1): state=initialized audit_enabled=0 res=1790worker1 # [ 0.311172] PM: RTC time: 08:03:45, date: 2026-09-20791client # [ 0.324342] thermal_sys: Registered thermal governor 'fair_share'792worker2 # [ 0.053031] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/24bj5klpiq603f5r4anb8fgirj501nj5-nixos-system-worker2-test/init regInfo=/nix/store/xn11lzr16q974jq38iwgxwjsfyi2rrj6-closure-info/registration console=ttyS0,115200n8 console=tty0793lb # [ 0.309546] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)794client # [ 0.324344] thermal_sys: Registered thermal governor 'bang_bang'795client # [ 0.325157] thermal_sys: Registered thermal governor 'step_wise'796lb # [ 0.310534] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).797worker2 # [ 0.053177] Unknown kernel command line parameters "regInfo=/nix/store/xn11lzr16q974jq38iwgxwjsfyi2rrj6-closure-info/registration", will be passed to user space.798client # [ 0.326079] thermal_sys: Registered thermal governor 'user_space'799worker1 # [ 0.314661] NET: Registered PF_NETLINK/PF_ROUTE protocol family800lb # [ 0.311690] pinctrl core: initialized pinctrl subsystem801worker2 # [ 0.053194] random: crng init done802client # [ 0.326850] thermal_sys: Registered thermal governor 'power_allocator'803lb # [ 0.312791] PM: RTC time: 08:03:45, date: 2026-09-20804worker2 # [ 0.053196] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes805client # [ 0.327175] cpuidle: using governor menu806worker1 # [ 0.315462] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations807worker2 # [ 0.056118] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)808worker1 # [ 0.315949] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations809client # [ 0.329402] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5810worker2 # [ 0.057826] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)811worker2 # [ 0.057878] Fallback order for Node 0: 0812worker1 # [ 0.317193] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations813client # [ 0.330418] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]814lb # [ 0.316349] NET: Registered PF_NETLINK/PF_ROUTE protocol family815worker1 # [ 0.317945] audit: initializing netlink subsys (disabled)816worker2 # [ 0.057883] Built 1 zonelists, mobility grouping on. Total pages: 393078817client # [ 0.331160] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry818worker2 # [ 0.057885] Policy zone: DMA32819worker1 # [ 0.319123] thermal_sys: Registered thermal governor 'fair_share'820lb # [ 0.317908] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations821client # [ 0.332171] PCI: Using configuration type 1 for base access822worker2 # [ 0.061260] mem auto-init: stack:all(zero), heap alloc:on, heap free:off823worker1 # [ 0.319125] thermal_sys: Registered thermal governor 'bang_bang'824lb # [ 0.318585] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations825worker2 # [ 0.065752] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1826client # [ 0.333329] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.827worker1 # [ 0.319930] thermal_sys: Registered thermal governor 'step_wise'828worker2 # [ 0.070228] allocated 3145728 bytes of page_ext829worker1 # [ 0.320862] thermal_sys: Registered thermal governor 'user_space'830lb # [ 0.319832] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations831worker2 # [ 0.082684] ftrace: allocating 48731 entries in 192 pages832worker2 # [ 0.082690] ftrace: allocated 192 pages with 2 groups833lb # [ 0.320581] audit: initializing netlink subsys (disabled)834worker1 # [ 0.321640] audit: type=2000 audit(1789891426.283:1): state=initialized audit_enabled=0 res=1835worker2 # [ 0.083969] Dynamic Preempt: lazy836lb # [ 0.321749] thermal_sys: Registered thermal governor 'fair_share'837worker1 # [ 0.322932] thermal_sys: Registered thermal governor 'power_allocator'838worker2 # [ 0.084183] rcu: Preemptible hierarchical RCU implementation.839worker1 # [ 0.322962] cpuidle: using governor menu840worker2 # [ 0.084185] rcu: RCU event tracing is enabled.841lb # [ 0.321752] thermal_sys: Registered thermal governor 'bang_bang'842worker2 # [ 0.084186] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.843lb # [ 0.322566] thermal_sys: Registered thermal governor 'step_wise'844worker1 # [ 0.325181] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5845worker2 # [ 0.084188] Trampoline variant of Tasks RCU enabled.846lb # [ 0.323524] thermal_sys: Registered thermal governor 'user_space'847worker2 # [ 0.084189] Rude variant of Tasks RCU enabled.848client # [ 0.340178] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages849worker1 # [ 0.326232] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]850worker2 # [ 0.084189] Tracing variant of Tasks RCU enabled.851lb # [ 0.324322] audit: type=2000 audit(1789891426.289:1): state=initialized audit_enabled=0 res=1852client # [ 0.341158] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page853worker1 # [ 0.326933] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry854worker2 # [ 0.084190] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.855lb # [ 0.325569] thermal_sys: Registered thermal governor 'power_allocator'856worker1 # [ 0.327947] PCI: Using configuration type 1 for base access857lb # [ 0.325586] cpuidle: using governor menu858worker2 # [ 0.084191] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1859worker1 # [ 0.329082] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.860worker2 # [ 0.084228] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.861lb # [ 0.327980] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5862worker2 # [ 0.084231] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.863lb # [ 0.328811] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]864client # [ 0.346158] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages865worker2 # [ 0.084233] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.866lb # [ 0.329569] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry867client # [ 0.347158] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page868worker2 # [ 0.090997] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16869lb # [ 0.330579] PCI: Using configuration type 1 for base access870worker2 # [ 0.091735] rcu: srcu_init: Setting srcu_struct sizes based on contention.871worker1 # [ 0.334200] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages872lb # [ 0.331760] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.873worker1 # [ 0.334932] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page874worker2 # [ 0.091746] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns875worker2 # [ 0.091890] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)876worker2 # [ 0.097044] Console: colour VGA+ 80x25877worker2 # [ 0.097051] printk: legacy console [tty0] enabled878worker2 # [ 0.138764] printk: legacy console [ttyS0] enabled879worker2 # [ 0.270843] ACPI: Core revision 20250807880worker1 # [ 0.339931] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages881worker1 # [ 0.340929] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page882lb # [ 0.338849] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages883client # [ 0.357571] ACPI: Added _OSI(Module Device)884worker2 # [ 0.272078] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns885lb # [ 0.339567] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page886client # [ 0.358158] ACPI: Added _OSI(Processor Device)887worker2 # [ 0.274216] APIC: Switch to symmetric I/O mode setup888client # [ 0.358918] ACPI: Added _OSI(Processor Aggregator Device)889worker2 # [ 0.275710] x2apic enabled890worker2 # [ 0.276751] APIC: Switched APIC routing to: physical x2apic891lb # [ 0.344567] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages892worker2 # [ 0.279233] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1893lb # [ 0.345567] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page894worker2 # [ 0.280523] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns895worker1 # [ 0.351297] ACPI: Added _OSI(Module Device)896worker1 # [ 0.351931] ACPI: Added _OSI(Processor Device)897worker2 # [ 0.282732] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)898worker1 # [ 0.352680] ACPI: Added _OSI(Processor Aggregator Device)899client # [ 0.367691] ACPI: 1 ACPI AML tables successfully acquired and loaded900worker2 # [ 0.284847] x86/cpu: User Mode Instruction Prevention (UMIP) activated901client # [ 0.369158] ACPI: Interpreter enabled902worker2 # [ 0.286831] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127903client # [ 0.369829] ACPI: PM: (supports S0 S3 S4 S5)904client # [ 0.370157] ACPI: Using IOAPIC for interrupt routing905worker2 # [ 0.287729] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0906lb # [ 0.354211] ACPI: Added _OSI(Module Device)907client # [ 0.371075] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug908lb # [ 0.354567] ACPI: Added _OSI(Processor Device)909worker2 # [ 0.288736] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto910lb # [ 0.355328] ACPI: Added _OSI(Processor Aggregator Device)911worker2 # [ 0.290730] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl912worker1 # [ 0.361433] ACPI: 1 ACPI AML tables successfully acquired and loaded913worker2 # [ 0.291730] Transient Scheduler Attacks: Vulnerable: No microcode914client # [ 0.376157] PCI: Using E820 reservations for host bridge windows915worker2 # [ 0.293729] Spectre V2 : Mitigation: Enhanced / Automatic IBRS916client # [ 0.377281] ACPI: Enabled 2 GPEs in block 00 to 3F917worker2 # [ 0.294729] Speculative Return Stack Overflow: Mitigation: Safe RET918worker1 # [ 0.365357] ACPI: Interpreter enabled919worker1 # [ 0.365946] ACPI: PM: (supports S0 S3 S4 S5)920worker1 # [ 0.366648] ACPI: Using IOAPIC for interrupt routing921lb # [ 0.364841] ACPI: 1 ACPI AML tables successfully acquired and loaded922worker2 # [ 0.295729] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization923worker1 # [ 0.368972] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug924worker2 # [ 0.297746] Spectre V2 : Enabling IBPB for BPF925worker1 # [ 0.369930] PCI: Using E820 reservations for host bridge windows926client # [ 0.385900] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])927worker2 # [ 0.298729] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier928lb # [ 0.369011] ACPI: Interpreter enabled929lb # [ 0.369582] ACPI: PM: (supports S0 S3 S4 S5)930worker2 # [ 0.299729] active return thunk: srso_alias_return_thunk931client # [ 0.386167] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]932lb # [ 0.370320] ACPI: Using IOAPIC for interrupt routing933worker1 # [ 0.373000] ACPI: Enabled 2 GPEs in block 00 to 3F934worker2 # [ 0.301480] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'935client # [ 0.387235] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]936lb # [ 0.370621] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug937worker2 # [ 0.302727] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'938client # [ 0.388279] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]939lb # [ 0.371566] PCI: Using E820 reservations for host bridge windows940worker2 # [ 0.303727] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'941client # [ 0.389537] PCI host bridge to bus 0000:00942worker2 # [ 0.304727] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'943client # [ 0.390162] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]944worker2 # [ 0.305726] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'945client # [ 0.391156] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]946lb # [ 0.374709] ACPI: Enabled 2 GPEs in block 00 to 3F947worker2 # [ 0.306726] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'948client # [ 0.392156] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]949worker2 # [ 0.307727] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'950client # [ 0.393156] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]951worker2 # [ 0.308728] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256952worker1 # [ 0.381241] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])953client # [ 0.394157] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]954worker2 # [ 0.309726] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64955worker1 # [ 0.381936] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]956client # [ 0.395157] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]957worker2 # [ 0.310726] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512958client # [ 0.396157] pci_bus 0000:00: root bus resource [bus 00-ff]959worker1 # [ 0.383010] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]960worker2 # [ 0.311727] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024961client # [ 0.397193] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint962worker1 # [ 0.384053] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]963worker2 # [ 0.312727] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8964worker1 # [ 0.385318] PCI host bridge to bus 0000:00965worker2 # [ 0.313726] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.966lb # [ 0.383453] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])967client # [ 0.398618] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint968worker1 # [ 0.385934] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]969worker1 # [ 0.386929] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]970lb # [ 0.384313] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]971worker1 # [ 0.387929] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]972lb # [ 0.386588] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]973worker1 # [ 0.388929] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]974lb # [ 0.387691] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]975worker1 # [ 0.389929] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]976lb # [ 0.388953] PCI host bridge to bus 0000:00977worker1 # [ 0.390929] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]978worker1 # [ 0.391930] pci_bus 0000:00: root bus resource [bus 00-ff]979lb # [ 0.389570] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]980lb # [ 0.390565] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]981client # [ 0.401263] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]982worker1 # [ 0.392931] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint983client # [ 0.402171] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]984lb # [ 0.391565] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]985worker1 # [ 0.394384] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint986client # [ 0.403140] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]987lb # [ 0.392565] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]988lb # [ 0.393566] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]989client # [ 0.404464] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]990lb # [ 0.394565] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]991lb # [ 0.395573] pci_bus 0000:00: root bus resource [bus 00-ff]992client # [ 0.406009] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint993lb # [ 0.396576] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint994worker1 # [ 0.397131] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]995lb # [ 0.398084] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint996client # [ 0.409196] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]997worker1 # [ 0.397942] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]998client # [ 0.410063] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]999worker1 # [ 0.398900] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1000client # [ 0.410920] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1001worker1 # [ 0.400024] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1002client # [ 0.412023] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1003client # [ 0.413788] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1004worker1 # [ 0.401703] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1005lb # [ 0.400628] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1006lb # [ 0.401577] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1007worker1 # [ 0.403946] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1008client # [ 0.416187] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1009lb # [ 0.402570] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1010worker1 # [ 0.404906] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1011client # [ 0.417105] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1012worker1 # [ 0.405678] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1013client # [ 0.417959] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1014lb # [ 0.403829] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1015worker1 # [ 0.405935] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1016client # [ 0.419660] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1017lb # [ 0.405371] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1018worker1 # [ 0.407536] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1019lb # [ 0.408595] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1020worker1 # [ 0.408937] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1021client # [ 0.422169] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1022lb # [ 0.409521] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1023worker1 # [ 0.409824] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1024client # [ 0.423151] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1025lb # [ 0.410366] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1026worker1 # [ 0.410696] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1027client # [ 0.424183] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1028worker2 # [ 0.352310] Freeing SMP alternatives memory: 44K1029lb # [ 0.411502] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1030worker2 # [ 0.352730] pid_max: default: 32768 minimum: 3011031worker1 # [ 0.412364] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1032client # [ 0.425797] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1033worker2 # [ 0.353843] LSM: initializing lsm=capability,landlock,yama,bpf,ima1034worker2 # [ 0.354862] landlock: Up and running.1035lb # [ 0.413152] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1036worker2 # [ 0.355460] Yama: becoming mindful.1037worker2 # [ 0.356362] LSM support for eBPF active1038client # [ 0.428172] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1039worker1 # [ 0.414949] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1040worker2 # [ 0.356871] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)1041lb # [ 0.416535] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1042worker1 # [ 0.415936] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1043client # [ 0.429141] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1044worker2 # [ 0.357765] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)1045lb # [ 0.417229] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1046worker1 # [ 0.416875] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1047client # [ 0.430180] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1048lb # [ 0.417672] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1049worker2 # [ 0.360885] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)1050worker1 # [ 0.418408] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1051client # [ 0.431790] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1052lb # [ 0.419155] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1053worker2 # [ 0.362292] Performance Events: Fam17h+ core perfctr, AMD PMU driver.1054worker2 # [ 0.362736] ... version: 21055worker1 # [ 0.419937] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1056worker2 # [ 0.363424] ... bit width: 481057worker1 # [ 0.420780] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1058client # [ 0.433873] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1059worker2 # [ 0.363728] ... generic counters: 61060client # [ 0.434164] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1061worker1 # [ 0.421647] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1062worker2 # [ 0.364421] ... generic bitmap: 000000000000003f1063lb # [ 0.421600] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1064worker2 # [ 0.364728] ... fixed-purpose counters: 01065client # [ 0.435084] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1066worker2 # [ 0.365399] ... fixed-purpose bitmap: 00000000000000001067lb # [ 0.423255] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1068worker1 # [ 0.422519] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1069worker2 # [ 0.365728] ... value mask: 0000ffffffffffff1070lb # [ 0.423588] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1071client # [ 0.436659] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1072worker2 # [ 0.366663] ... max period: 00007fffffffffff1073worker2 # [ 0.367430] ... global_ctrl mask: 000000000000003f1074lb # [ 0.425204] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1075worker1 # [ 0.423938] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1076worker2 # [ 0.367841] signal: max sigframe size: 33761077worker1 # [ 0.424782] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1078worker2 # [ 0.368664] rcu: Hierarchical SRCU implementation.1079worker2 # [ 0.369322] rcu: Max phase no-delay instances is 400.1080worker1 # [ 0.425649] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1081client # [ 0.440169] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1082lb # [ 0.426595] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1083client # [ 0.441027] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1084lb # [ 0.427477] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1085worker1 # [ 0.426546] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1086client # [ 0.441897] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1087lb # [ 0.428339] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1088client # [ 0.442856] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1089lb # [ 0.430834] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1090worker2 # [ 0.374680] smp: Bringing up secondary CPUs ...1091worker1 # [ 0.428948] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1092worker2 # [ 0.375308] smp: Brought up 1 node, 1 CPU1093worker1 # [ 0.429954] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1094client # [ 0.446032] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1095worker2 # [ 0.375731] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)1096client # [ 0.446784] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1097worker1 # [ 0.430905] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1098lb # [ 0.433393] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1099lb # [ 0.434225] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1100worker2 # [ 0.377013] Memory: 1454572K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)1101worker1 # [ 0.432387] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1102client # [ 0.447180] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1103lb # [ 0.434589] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1104worker2 # [ 0.378043] devtmpfs: initialized1105client # [ 0.448945] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1106worker2 # [ 0.378856] x86/mm: Memory block size: 128MB1107worker1 # [ 0.433951] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1108lb # [ 0.436187] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1109client # [ 0.450514] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1110worker1 # [ 0.434817] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1111worker2 # [ 0.380614] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)1112worker1 # [ 0.435693] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1113worker1 # [ 0.436778] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1114worker2 # [ 0.381659] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).1115client # [ 0.452205] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1116lb # [ 0.438566] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1117worker2 # [ 0.382852] pinctrl core: initialized pinctrl subsystem1118client # [ 0.453179] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1119lb # [ 0.440247] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1120worker2 # [ 0.384006] PM: RTC time: 08:03:45, date: 2026-09-201121worker1 # [ 0.438745] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1122lb # [ 0.440589] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1123client # [ 0.454764] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1124worker1 # [ 0.440943] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1125lb # [ 0.442189] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1126worker1 # [ 0.441952] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1127worker2 # [ 0.387455] NET: Registered PF_NETLINK/PF_ROUTE protocol family1128client # [ 0.457181] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1129worker2 # [ 0.388228] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations1130worker1 # [ 0.443507] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1131lb # [ 0.444346] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1132client # [ 0.458050] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1133client # [ 0.458863] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1134lb # [ 0.445180] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1135worker2 # [ 0.388752] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1136lb # [ 0.445587] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1137worker1 # [ 0.444944] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1138worker2 # [ 0.390023] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1139client # [ 0.459766] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1140worker1 # [ 0.445779] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1141worker2 # [ 0.390747] audit: initializing netlink subsys (disabled)1142lb # [ 0.446571] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1143worker1 # [ 0.446654] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1144worker2 # [ 0.392075] thermal_sys: Registered thermal governor 'fair_share'1145client # [ 0.461167] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1146lb # [ 0.448218] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1147worker2 # [ 0.392078] thermal_sys: Registered thermal governor 'bang_bang'1148client # [ 0.462060] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1149worker1 # [ 0.448361] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1150worker2 # [ 0.392730] thermal_sys: Registered thermal governor 'step_wise'1151worker2 # [ 0.393679] thermal_sys: Registered thermal governor 'user_space'1152lb # [ 0.449595] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1153client # [ 0.462880] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1154worker1 # [ 0.449937] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1155worker2 # [ 0.394486] audit: type=2000 audit(1789891426.398:1): state=initialized audit_enabled=0 res=11156lb # [ 0.450588] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1157worker1 # [ 0.450775] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1158client # [ 0.463764] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1159worker2 # [ 0.395734] thermal_sys: Registered thermal governor 'power_allocator'1160worker1 # [ 0.451655] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1161worker2 # [ 0.395754] cpuidle: using governor menu1162lb # [ 0.452127] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1163client # [ 0.465058] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1164worker1 # [ 0.452512] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1165worker2 # [ 0.397974] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51166client # [ 0.466068] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1167worker2 # [ 0.399023] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1168worker1 # [ 0.453529] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1169lb # [ 0.454261] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1170client # [ 0.467892] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1171worker2 # [ 0.399733] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1172lb # [ 0.455197] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1173worker1 # [ 0.454153] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1174worker2 # [ 0.400746] PCI: Using configuration type 1 for base access1175lb # [ 0.455588] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1176worker1 # [ 0.455606] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1177worker2 # [ 0.401892] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1178lb # [ 0.457176] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1179lb # [ 0.459279] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1180lb # [ 0.459573] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1181worker2 # [ 0.407020] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1182lb # [ 0.460587] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1183worker2 # [ 0.407731] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1184lb # [ 0.462705] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1185lb # [ 0.464461] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1186client # [ 0.468370] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1187worker2 # [ 0.412731] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1188lb # [ 0.465481] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1189worker1 # [ 0.456123] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1190worker2 # [ 0.413729] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1191client # [ 0.471185] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1192worker1 # [ 0.458552] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1193lb # [ 0.467314] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1194client # [ 0.472299] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1195worker1 # [ 0.459172] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1196client # [ 0.473827] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1197worker1 # [ 0.460498] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1198client # [ 0.474482] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1199worker1 # [ 0.461201] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1200client # [ 0.475455] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1201worker1 # [ 0.462220] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1202client # [ 0.476455] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1203worker1 # [ 0.463189] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1204worker2 # [ 0.424206] ACPI: Added _OSI(Module Device)1205client # [ 0.478918] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1206worker2 # [ 0.424731] ACPI: Added _OSI(Processor Device)1207worker1 # [ 0.464870] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1208client # [ 0.479164] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1209worker2 # [ 0.425505] ACPI: Added _OSI(Processor Aggregator Device)1210worker1 # [ 0.465547] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1211client # [ 0.480430] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1212worker1 # [ 0.466271] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1213lb # [ 0.467769] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1214client # [ 0.481916] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1215worker1 # [ 0.467958] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1216lb # [ 0.470414] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1217lb # [ 0.471542] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1218client # [ 0.485293] ACPI: PCI: Interrupt link LNKA configured for IRQ 101219worker1 # [ 0.471901] ACPI: PCI: Interrupt link LNKA configured for IRQ 101220lb # [ 0.473312] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1221client # [ 0.486593] ACPI: PCI: Interrupt link LNKB configured for IRQ 101222worker1 # [ 0.472806] ACPI: PCI: Interrupt link LNKB configured for IRQ 101223lb # [ 0.473809] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1224client # [ 0.487256] ACPI: PCI: Interrupt link LNKC configured for IRQ 111225worker1 # [ 0.473772] ACPI: PCI: Interrupt link LNKC configured for IRQ 111226client # [ 0.488255] ACPI: PCI: Interrupt link LNKD configured for IRQ 111227worker1 # [ 0.474756] ACPI: PCI: Interrupt link LNKD configured for IRQ 111228lb # [ 0.474860] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1229worker2 # [ 0.437182] ACPI: 1 ACPI AML tables successfully acquired and loaded1230client # [ 0.489257] ACPI: PCI: Interrupt link LNKE configured for IRQ 101231worker1 # [ 0.475732] ACPI: PCI: Interrupt link LNKE configured for IRQ 101232client # [ 0.490253] ACPI: PCI: Interrupt link LNKF configured for IRQ 101233lb # [ 0.475873] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1234worker1 # [ 0.476734] ACPI: PCI: Interrupt link LNKF configured for IRQ 101235client # [ 0.491258] ACPI: PCI: Interrupt link LNKG configured for IRQ 111236worker1 # [ 0.477957] ACPI: PCI: Interrupt link LNKG configured for IRQ 111237client # [ 0.492253] ACPI: PCI: Interrupt link LNKH configured for IRQ 111238worker1 # [ 0.479037] ACPI: PCI: Interrupt link LNKH configured for IRQ 111239lb # [ 0.478342] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1240client # [ 0.493195] ACPI: PCI: Interrupt link GSIA configured for IRQ 161241worker1 # [ 0.479967] ACPI: PCI: Interrupt link GSIA configured for IRQ 161242lb # [ 0.479248] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1243client # [ 0.494127] ACPI: PCI: Interrupt link GSIB configured for IRQ 171244client # [ 0.494966] ACPI: PCI: Interrupt link GSIC configured for IRQ 181245lb # [ 0.479917] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1246worker1 # [ 0.480936] ACPI: PCI: Interrupt link GSIB configured for IRQ 171247client # [ 0.496105] ACPI: PCI: Interrupt link GSID configured for IRQ 191248worker1 # [ 0.481953] ACPI: PCI: Interrupt link GSIC configured for IRQ 181249worker2 # [ 0.443345] ACPI: Interpreter enabled1250lb # [ 0.481363] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1251client # [ 0.496981] ACPI: PCI: Interrupt link GSIE configured for IRQ 201252worker2 # [ 0.443758] ACPI: PM: (supports S0 S3 S4 S5)1253worker1 # [ 0.482906] ACPI: PCI: Interrupt link GSID configured for IRQ 191254client # [ 0.497889] ACPI: PCI: Interrupt link GSIF configured for IRQ 211255worker2 # [ 0.444624] ACPI: Using IOAPIC for interrupt routing1256worker1 # [ 0.483690] ACPI: PCI: Interrupt link GSIE configured for IRQ 201257client # [ 0.498168] ACPI: PCI: Interrupt link GSIG configured for IRQ 221258worker1 # [ 0.483942] ACPI: PCI: Interrupt link GSIF configured for IRQ 211259client # [ 0.499134] ACPI: PCI: Interrupt link GSIH configured for IRQ 231260worker2 # [ 0.445520] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1261worker1 # [ 0.484898] ACPI: PCI: Interrupt link GSIG configured for IRQ 221262lb # [ 0.485354] ACPI: PCI: Interrupt link LNKA configured for IRQ 101263worker1 # [ 0.485942] ACPI: PCI: Interrupt link GSIH configured for IRQ 231264lb # [ 0.486399] ACPI: PCI: Interrupt link LNKB configured for IRQ 101265lb # [ 0.487668] ACPI: PCI: Interrupt link LNKC configured for IRQ 111266client # [ 0.502239] iommu: Default domain type: Translated1267worker1 # [ 0.488045] iommu: Default domain type: Translated1268lb # [ 0.488665] ACPI: PCI: Interrupt link LNKD configured for IRQ 111269client # [ 0.503166] iommu: DMA domain TLB invalidation policy: lazy mode1270worker1 # [ 0.488857] iommu: DMA domain TLB invalidation policy: lazy mode1271worker2 # [ 0.450731] PCI: Using E820 reservations for host bridge windows1272lb # [ 0.489667] ACPI: PCI: Interrupt link LNKE configured for IRQ 101273client # [ 0.504430] ACPI: bus type USB registered1274worker1 # [ 0.489902] ACPI: bus type USB registered1275worker2 # [ 0.451923] ACPI: Enabled 2 GPEs in block 00 to 3F1276client # [ 0.505149] usbcore: registered new interface driver usbfs1277worker1 # [ 0.490442] usbcore: registered new interface driver usbfs1278lb # [ 0.490664] ACPI: PCI: Interrupt link LNKF configured for IRQ 101279client # [ 0.506015] usbcore: registered new interface driver hub1280worker1 # [ 0.490945] usbcore: registered new interface driver hub1281lb # [ 0.491667] ACPI: PCI: Interrupt link LNKG configured for IRQ 111282client # [ 0.506805] usbcore: registered new device driver usb1283worker1 # [ 0.491792] usbcore: registered new device driver usb1284lb # [ 0.492665] ACPI: PCI: Interrupt link LNKH configured for IRQ 111285client # [ 0.508146] NetLabel: Initializing1286worker1 # [ 0.493475] NetLabel: Initializing1287lb # [ 0.493600] ACPI: PCI: Interrupt link GSIA configured for IRQ 161288client # [ 0.508583] NetLabel: domain hash size = 1281289worker1 # [ 0.493930] NetLabel: domain hash size = 1281290lb # [ 0.494587] ACPI: PCI: Interrupt link GSIB configured for IRQ 171291client # [ 0.509156] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1292worker1 # [ 0.494825] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1293client # [ 0.510182] NetLabel: unlabeled traffic allowed by default1294worker1 # [ 0.495756] NetLabel: unlabeled traffic allowed by default1295client # [ 0.511103] PCI: Using ACPI for IRQ routing1296worker1 # [ 0.496671] PCI: Using ACPI for IRQ routing1297worker2 # [ 0.460812] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1298lb # [ 0.495569] ACPI: PCI: Interrupt link GSIC configured for IRQ 181299worker2 # [ 0.461737] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1300lb # [ 0.496591] ACPI: PCI: Interrupt link GSID configured for IRQ 191301worker2 # [ 0.462830] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1302lb # [ 0.497579] ACPI: PCI: Interrupt link GSIE configured for IRQ 201303lb # [ 0.498578] ACPI: PCI: Interrupt link GSIF configured for IRQ 211304worker2 # [ 0.463872] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1305lb # [ 0.499638] ACPI: PCI: Interrupt link GSIG configured for IRQ 221306worker2 # [ 0.465507] PCI host bridge to bus 0000:001307lb # [ 0.500798] ACPI: PCI: Interrupt link GSIH configured for IRQ 231308worker2 # [ 0.466355] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1309worker2 # [ 0.466730] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1310lb # [ 0.502726] iommu: Default domain type: Translated1311lb # [ 0.503573] iommu: DMA domain TLB invalidation policy: lazy mode1312worker2 # [ 0.467730] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1313worker2 # [ 0.468730] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1314lb # [ 0.504823] ACPI: bus type USB registered1315worker2 # [ 0.469740] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1316lb # [ 0.506207] usbcore: registered new interface driver usbfs1317lb # [ 0.506584] usbcore: registered new interface driver hub1318worker2 # [ 0.470731] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1319lb # [ 0.507577] usbcore: registered new device driver usb1320worker2 # [ 0.471769] pci_bus 0000:00: root bus resource [bus 00-ff]1321lb # [ 0.509602] NetLabel: Initializing1322worker2 # [ 0.472857] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1323lb # [ 0.510253] NetLabel: domain hash size = 1281324lb # [ 0.510565] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1325worker2 # [ 0.474360] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1326lb # [ 0.511586] NetLabel: unlabeled traffic allowed by default1327lb # [ 0.512514] PCI: Using ACPI for IRQ routing1328worker2 # [ 0.477949] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1329worker2 # [ 0.478743] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1330worker2 # [ 0.479752] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1331worker2 # [ 0.481009] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1332worker2 # [ 0.482663] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1333worker2 # [ 0.485764] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1334client # [ 0.555326] pci 0000:00:01.0: vgaarb: setting as boot VGA device1335worker2 # [ 0.486743] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1336client # [ 0.556153] pci 0000:00:01.0: vgaarb: bridge control possible1337worker1 # [ 0.541675] pci 0000:00:01.0: vgaarb: setting as boot VGA device1338worker2 # [ 0.487760] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1339client # [ 0.556153] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1340worker1 # [ 0.541925] pci 0000:00:01.0: vgaarb: bridge control possible1341client # [ 0.556162] vgaarb: loaded1342worker2 # [ 0.488740] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1343client # [ 0.556890] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01344worker1 # [ 0.541925] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1345worker1 # [ 0.541940] vgaarb: loaded1346client # [ 0.557156] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1347worker2 # [ 0.490598] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1348worker1 # [ 0.542748] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01349worker1 # [ 0.543538] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1350worker2 # [ 0.493757] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1351client # [ 0.561267] clocksource: Switched to clocksource kvm-clock1352worker2 # [ 0.494739] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1353worker1 # [ 0.548112] clocksource: Switched to clocksource kvm-clock1354worker2 # [ 0.495761] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1355client # [ 0.564911] VFS: Disk quotas dquot_6.6.01356client # [ 0.565687] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1357worker2 # [ 0.497524] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1358client # [ 0.567130] pnp: PnP ACPI init1359worker1 # [ 0.552461] VFS: Disk quotas dquot_6.6.01360client # [ 0.567988] ACPI: IRQ 4 override to edge(!), high(!)1361worker1 # [ 0.553285] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1362client # [ 0.569006] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1363worker2 # [ 0.500757] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1364worker1 # [ 0.554821] pnp: PnP ACPI init1365client # [ 0.570502] pnp: PnP ACPI: found 6 devices1366worker2 # [ 0.501759] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1367worker1 # [ 0.555746] ACPI: IRQ 4 override to edge(!), high(!)1368worker1 # [ 0.556776] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1369worker2 # [ 0.502759] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1370worker1 # [ 0.558364] pnp: PnP ACPI: found 6 devices1371lb # [ 0.557441] pci 0000:00:01.0: vgaarb: setting as boot VGA device1372worker2 # [ 0.504483] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1373lb # [ 0.557562] pci 0000:00:01.0: vgaarb: bridge control possible1374lb # [ 0.557562] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1375lb # [ 0.557570] vgaarb: loaded1376lb # [ 0.558373] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01377worker2 # [ 0.507759] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1378client # [ 0.578294] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1379lb # [ 0.559163] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1380client # [ 0.579742] clocksource: Switched to clocksource acpi_pm1381worker2 # [ 0.508754] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1382client # [ 0.580693] NET: Registered PF_INET protocol family1383worker2 # [ 0.509755] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1384worker1 # [ 0.566232] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1385client # [ 0.581890] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1386worker1 # [ 0.567707] clocksource: Switched to clocksource acpi_pm1387lb # [ 0.562672] clocksource: Switched to clocksource kvm-clock1388worker1 # [ 0.568763] NET: Registered PF_INET protocol family1389worker2 # [ 0.511549] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1390worker1 # [ 0.569964] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1391lb # [ 0.566319] VFS: Disk quotas dquot_6.6.01392worker2 # [ 0.514756] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1393lb # [ 0.567057] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1394worker2 # [ 0.515741] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1395lb # [ 0.568465] pnp: PnP ACPI init1396worker2 # [ 0.516760] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1397lb # [ 0.569254] ACPI: IRQ 4 override to edge(!), high(!)1398lb # [ 0.570348] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1399worker2 # [ 0.518594] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1400lb # [ 0.571952] pnp: PnP ACPI: found 6 devices1401worker2 # [ 0.522764] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1402worker2 # [ 0.523702] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1403client # [ 0.598782] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1404worker2 # [ 0.524514] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1405client # [ 0.600394] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1406lb # [ 0.579705] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1407worker2 # [ 0.526339] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1408client # [ 0.601700] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1409lb # [ 0.581205] clocksource: Switched to clocksource acpi_pm1410lb # [ 0.582213] NET: Registered PF_INET protocol family1411client # [ 0.603013] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1412client # [ 0.604243] TCP: Hash tables configured (established 16384 bind 16384)1413worker1 # [ 0.588777] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1414worker2 # [ 0.529758] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1415lb # [ 0.583453] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1416client # [ 0.605412] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1417worker1 # [ 0.590459] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1418worker2 # [ 0.530641] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1419client # [ 0.606723] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1420worker1 # [ 0.591764] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1421worker2 # [ 0.531753] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1422client # [ 0.607896] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1423worker1 # [ 0.593084] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1424worker2 # [ 0.532736] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1425client # [ 0.609096] NET: Registered PF_UNIX/PF_LOCAL protocol family1426worker1 # [ 0.594385] TCP: Hash tables configured (established 16384 bind 16384)1427client # [ 0.610023] NET: Registered PF_XDP protocol family1428client # [ 0.610867] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1429worker1 # [ 0.595529] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1430worker2 # [ 0.534397] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1431client # [ 0.611834] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1432worker1 # [ 0.596765] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1433client # [ 0.612855] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1434worker1 # [ 0.597839] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1435worker2 # [ 0.537341] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1436client # [ 0.613943] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1437worker1 # [ 0.599104] NET: Registered PF_UNIX/PF_LOCAL protocol family1438worker2 # [ 0.537753] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1439client # [ 0.615005] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1440worker1 # [ 0.600099] NET: Registered PF_XDP protocol family1441worker1 # [ 0.600969] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1442client # [ 0.616060] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1443worker1 # [ 0.601961] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1444worker2 # [ 0.539440] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1445client # [ 0.617788] ACPI: \_SB_.GSIA: Enabled at IRQ 161446worker1 # [ 0.602991] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1447worker1 # [ 0.604167] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1448worker2 # [ 0.542683] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1449client # [ 0.620568] ACPI: \_SB_.GSIB: Enabled at IRQ 171450worker1 # [ 0.605325] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1451worker2 # [ 0.543370] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1452worker1 # [ 0.606418] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1453lb # [ 0.600473] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1454worker2 # [ 0.543751] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1455client # [ 0.622719] ACPI: \_SB_.GSIC: Enabled at IRQ 181456worker1 # [ 0.608409] ACPI: \_SB_.GSIA: Enabled at IRQ 161457lb # [ 0.602011] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1458worker2 # [ 0.545346] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1459client # [ 0.624695] ACPI: \_SB_.GSID: Enabled at IRQ 191460lb # [ 0.603500] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1461client # [ 0.626419] PCI: CLS 0 bytes, default 641462worker1 # [ 0.611195] ACPI: \_SB_.GSIB: Enabled at IRQ 171463lb # [ 0.604991] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1464lb # [ 0.606302] TCP: Hash tables configured (established 16384 bind 16384)1465client # [ 0.627306] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1466worker1 # [ 0.613299] ACPI: \_SB_.GSIC: Enabled at IRQ 181467worker2 # [ 0.547751] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1468lb # [ 0.607553] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1469worker2 # [ 0.548737] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1470client # [ 0.629003] Trying to unpack rootfs image as initramfs...1471lb # [ 0.608847] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1472worker1 # [ 0.615420] ACPI: \_SB_.GSID: Enabled at IRQ 191473worker2 # [ 0.549706] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1474lb # [ 0.609967] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1475worker1 # [ 0.617161] PCI: CLS 0 bytes, default 641476lb # [ 0.611316] NET: Registered PF_UNIX/PF_LOCAL protocol family1477worker2 # [ 0.551217] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1478lb # [ 0.612329] NET: Registered PF_XDP protocol family1479worker1 # [ 0.618131] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1480lb # [ 0.613188] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1481worker2 # [ 0.552643] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1482lb # [ 0.614220] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1483worker1 # [ 0.619950] Trying to unpack rootfs image as initramfs...1484worker2 # [ 0.553612] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1485lb # [ 0.615255] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1486worker2 # [ 0.555465] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1487lb # [ 0.616491] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1488lb # [ 0.617641] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1489lb # [ 0.618833] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1490lb # [ 0.620679] ACPI: \_SB_.GSIA: Enabled at IRQ 161491lb # [ 0.622970] ACPI: \_SB_.GSIB: Enabled at IRQ 171492lb # [ 0.625014] ACPI: \_SB_.GSIC: Enabled at IRQ 181493lb # [ 0.627063] ACPI: \_SB_.GSID: Enabled at IRQ 191494lb # [ 0.628814] PCI: CLS 0 bytes, default 641495worker2 # [ 0.555945] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1496lb # [ 0.629769] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1497lb # [ 0.631601] Trying to unpack rootfs image as initramfs...1498worker2 # [ 0.559495] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1499worker2 # [ 0.560047] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1500worker2 # [ 0.561737] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1501worker2 # [ 0.563018] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1502worker2 # [ 0.564032] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1503worker2 # [ 0.565009] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1504worker2 # [ 0.567731] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1505worker2 # [ 0.568747] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1506worker2 # [ 0.570131] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1507worker2 # [ 0.571759] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1508client # [ 0.675301] Initialise system trusted keyrings1509client # [ 0.678881] workingset: timestamp_bits=40 max_order=19 bucket_order=01510worker2 # [ 0.576975] ACPI: PCI: Interrupt link LNKA configured for IRQ 101511worker2 # [ 0.577855] ACPI: PCI: Interrupt link LNKB configured for IRQ 101512worker2 # [ 0.578837] ACPI: PCI: Interrupt link LNKC configured for IRQ 111513worker2 # [ 0.579835] ACPI: PCI: Interrupt link LNKD configured for IRQ 111514worker2 # [ 0.580843] ACPI: PCI: Interrupt link LNKE configured for IRQ 101515worker2 # [ 0.581833] ACPI: PCI: Interrupt link LNKF configured for IRQ 101516worker1 # [ 0.669849] Initialise system trusted keyrings1517worker2 # [ 0.582853] ACPI: PCI: Interrupt link LNKG configured for IRQ 111518worker2 # [ 0.584036] ACPI: PCI: Interrupt link LNKH configured for IRQ 111519worker1 # [ 0.673466] workingset: timestamp_bits=40 max_order=19 bucket_order=01520worker2 # [ 0.584844] ACPI: PCI: Interrupt link GSIA configured for IRQ 161521worker2 # [ 0.585763] ACPI: PCI: Interrupt link GSIB configured for IRQ 171522worker2 # [ 0.586755] ACPI: PCI: Interrupt link GSIC configured for IRQ 181523worker2 # [ 0.587763] ACPI: PCI: Interrupt link GSID configured for IRQ 191524worker2 # [ 0.588756] ACPI: PCI: Interrupt link GSIE configured for IRQ 201525worker2 # [ 0.589765] ACPI: PCI: Interrupt link GSIF configured for IRQ 211526worker2 # [ 0.590755] ACPI: PCI: Interrupt link GSIG configured for IRQ 221527worker2 # [ 0.591755] ACPI: PCI: Interrupt link GSIH configured for IRQ 231528client # [ 0.701281] Key type asymmetric registered1529worker2 # [ 0.594642] iommu: Default domain type: Translated1530client # [ 0.702078] Asymmetric key parser 'x509' registered1531lb # [ 0.681366] Initialise system trusted keyrings1532worker2 # [ 0.595547] iommu: DMA domain TLB invalidation policy: lazy mode1533client # [ 0.702997] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1534worker2 # [ 0.597184] ACPI: bus type USB registered1535lb # [ 0.684471] workingset: timestamp_bits=40 max_order=19 bucket_order=01536worker2 # [ 0.597833] usbcore: registered new interface driver usbfs1537worker2 # [ 0.598762] usbcore: registered new interface driver hub1538worker2 # [ 0.599767] usbcore: registered new device driver usb1539client # [ 0.708981] io scheduler mq-deadline registered1540client # [ 0.709792] io scheduler kyber registered1541worker2 # [ 0.602072] NetLabel: Initializing1542worker2 # [ 0.602733] NetLabel: domain hash size = 1281543worker2 # [ 0.603711] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1544client # [ 0.712493] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1545worker1 # [ 0.697971] Key type asymmetric registered1546worker2 # [ 0.604782] NetLabel: unlabeled traffic allowed by default1547worker1 # [ 0.698816] Asymmetric key parser 'x509' registered1548worker2 # [ 0.605753] PCI: Using ACPI for IRQ routing1549worker1 # [ 0.699815] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1550client # [ 0.715864] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1551worker1 # [ 0.704465] io scheduler mq-deadline registered1552worker1 # [ 0.705325] io scheduler kyber registered1553client # [ 0.721034] Linux agpgart interface v0.1031554client # [ 0.721896] ACPI: bus type drm_connector registered1555worker1 # [ 0.708710] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1556client # [ 0.725216] usbcore: registered new interface driver usbserial_generic1557worker1 # [ 0.710106] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1558client # [ 0.726365] usbserial: USB Serial support registered for generic1559lb # [ 0.709537] Key type asymmetric registered1560client # [ 0.729830] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1561client # [ 0.729830] Please enable it if your BIOS has the CPPC option.1562worker1 # [ 0.716196] Linux agpgart interface v0.1031563worker1 # [ 0.717040] ACPI: bus type drm_connector registered1564client # [ 0.732063] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1565lb # [ 0.712340] Asymmetric key parser 'x509' registered1566lb # [ 0.713240] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1567client # [ 0.736036] drop_monitor: Initializing network drop monitor service1568client # [ 0.737425] NET: Registered PF_INET6 protocol family1569lb # [ 0.716513] io scheduler mq-deadline registered1570worker1 # [ 0.722120] usbcore: registered new interface driver usbserial_generic1571lb # [ 0.717322] io scheduler kyber registered1572worker1 # [ 0.723341] usbserial: USB Serial support registered for generic1573client # [ 0.740385] Segment Routing with IPv61574lb # [ 0.719955] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1575worker1 # [ 0.726389] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1576worker1 # [ 0.726389] Please enable it if your BIOS has the CPPC option.1577client # [ 0.743857] In-situ OAM (IOAM) with IPv61578worker1 # [ 0.728709] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1579lb # [ 0.723312] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1580client # [ 0.745071] IPI shorthand broadcast: enabled1581worker1 # [ 0.733700] drop_monitor: Initializing network drop monitor service1582lb # [ 0.728391] Linux agpgart interface v0.1031583worker1 # [ 0.735038] NET: Registered PF_INET6 protocol family1584lb # [ 0.729257] ACPI: bus type drm_connector registered1585worker1 # [ 0.739088] Segment Routing with IPv61586client # [ 0.753745] sched_clock: Marking stable (610017441, 142801486)->(825468643, -72649716)1587lb # [ 0.733026] usbcore: registered new interface driver usbserial_generic1588client # [ 0.757075] registered taskstats version 11589worker1 # [ 0.742403] In-situ OAM (IOAM) with IPv61590lb # [ 0.736340] usbserial: USB Serial support registered for generic1591client # [ 0.758173] Loading compiled-in X.509 certificates1592worker1 # [ 0.743791] IPI shorthand broadcast: enabled1593lb # [ 0.737559] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1594lb # [ 0.737559] Please enable it if your BIOS has the CPPC option.1595lb # [ 0.746333] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1596worker1 # [ 0.752816] sched_clock: Marking stable (610018511, 142346673)->(826768266, -74403082)1597lb # [ 0.748232] drop_monitor: Initializing network drop monitor service1598lb # [ 0.751524] NET: Registered PF_INET6 protocol family1599worker2 # [ 0.664094] pci 0000:00:01.0: vgaarb: setting as boot VGA device1600worker1 # [ 0.760000] registered taskstats version 11601worker2 # [ 0.664725] pci 0000:00:01.0: vgaarb: bridge control possible1602lb # [ 0.755773] Segment Routing with IPv61603worker2 # [ 0.664725] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1604worker2 # [ 0.664745] vgaarb: loaded1605worker1 # [ 0.763502] Loading compiled-in X.509 certificates1606lb # [ 0.758351] In-situ OAM (IOAM) with IPv61607worker2 # [ 0.665765] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01608client # [ 0.779743] Demotion targets for Node 0: null1609client # [ 0.780756] Key type .fscrypt registered1610worker2 # [ 0.666731] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1611lb # [ 0.759702] IPI shorthand broadcast: enabled1612client # [ 0.782852] Key type fscrypt-provisioning registered1613client # [ 0.783836] ima: No TPM chip found, activating TPM-bypass!1614client # [ 0.784741] ima: Allocated hash algorithm: sha11615worker2 # [ 0.672923] clocksource: Switched to clocksource kvm-clock1616client # [ 0.788874] ima: No architecture policies found1617client # [ 0.790012] PM: Magic number: 10:880:681618client # [ 0.791682] RAS: Correctable Errors collector initialized.1619worker2 # [ 0.677858] VFS: Disk quotas dquot_6.6.01620worker2 # [ 0.679013] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1621lb # [ 0.772362] sched_clock: Marking stable (630023655, 142278024)->(849192494, -76890815)1622worker2 # [ 0.681259] pnp: PnP ACPI init1623worker2 # [ 0.682315] ACPI: IRQ 4 override to edge(!), high(!)1624worker2 # [ 0.683543] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1625lb # [ 0.777474] registered taskstats version 11626worker2 # [ 0.685462] pnp: PnP ACPI: found 6 devices1627lb # [ 0.778715] Loading compiled-in X.509 certificates1628client # [ 0.802794] clk: Disabling unused clocks1629worker1 # [ 0.788159] Demotion targets for Node 0: null1630worker1 # [ 0.789496] Key type .fscrypt registered1631client # [ 0.804837] PM: genpd: Disabling unused power domains1632worker1 # [ 0.792387] Key type fscrypt-provisioning registered1633worker1 # [ 0.793697] ima: No TPM chip found, activating TPM-bypass!1634worker2 # [ 0.694380] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1635worker2 # [ 0.696307] clocksource: Switched to clocksource acpi_pm1636worker2 # [ 0.697666] NET: Registered PF_INET protocol family1637worker1 # [ 0.798392] ima: Allocated hash algorithm: sha11638worker2 # [ 0.699280] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1639worker1 # [ 0.799588] ima: No architecture policies found1640worker1 # [ 0.803679] PM: Magic number: 10:880:681641worker1 # [ 0.806193] RAS: Correctable Errors collector initialized.1642lb # [ 0.801336] Demotion targets for Node 0: null1643lb # [ 0.804366] Key type .fscrypt registered1644lb # [ 0.805045] Key type fscrypt-provisioning registered1645lb # [ 0.806009] ima: No TPM chip found, activating TPM-bypass!1646lb # [ 0.808320] ima: Allocated hash algorithm: sha11647lb # [ 0.809104] ima: No architecture policies found1648worker1 # [ 0.817541] clk: Disabling unused clocks1649lb # [ 0.813313] PM: Magic number: 10:880:681650worker1 # [ 0.820399] PM: genpd: Disabling unused power domains1651lb # [ 0.814976] RAS: Correctable Errors collector initialized.1652worker2 # [ 0.722603] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1653worker2 # [ 0.724758] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1654worker2 # [ 0.726478] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1655worker2 # [ 0.728234] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1656lb # [ 0.823068] clk: Disabling unused clocks1657worker2 # [ 0.729849] TCP: Hash tables configured (established 16384 bind 16384)1658worker2 # [ 0.731378] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1659worker2 # [ 0.733037] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1660lb # [ 0.827319] PM: genpd: Disabling unused power domains1661worker2 # [ 0.734464] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1662worker2 # [ 0.736141] NET: Registered PF_UNIX/PF_LOCAL protocol family1663worker2 # [ 0.737420] NET: Registered PF_XDP protocol family1664worker2 # [ 0.738552] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1665worker2 # [ 0.739836] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1666worker2 # [ 0.741191] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1667worker2 # [ 0.742619] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1668worker2 # [ 0.744005] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1669worker2 # [ 0.745473] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1670worker2 # [ 0.747759] ACPI: \_SB_.GSIA: Enabled at IRQ 161671worker2 # [ 0.750658] ACPI: \_SB_.GSIB: Enabled at IRQ 171672worker2 # [ 0.753635] ACPI: \_SB_.GSIC: Enabled at IRQ 181673worker2 # [ 0.756444] ACPI: \_SB_.GSID: Enabled at IRQ 191674worker2 # [ 0.758785] PCI: CLS 0 bytes, default 641675worker2 # [ 0.760191] Trying to unpack rootfs image as initramfs...1676worker2 # [ 0.764235] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1677worker2 # [ 0.823735] Initialise system trusted keyrings1678worker2 # [ 0.826090] workingset: timestamp_bits=40 max_order=19 bucket_order=01679worker2 # [ 0.858800] Key type asymmetric registered1680worker2 # [ 0.859767] Asymmetric key parser 'x509' registered1681worker2 # [ 0.863105] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1682worker2 # [ 0.867093] io scheduler mq-deadline registered1683worker2 # [ 0.868433] io scheduler kyber registered1684worker2 # [ 0.872924] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1685worker2 # [ 0.874668] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1686worker2 # [ 0.881789] Linux agpgart interface v0.1031687client # [ 0.998023] Freeing initrd memory: 29096K1688worker2 # [ 0.885135] ACPI: bus type drm_connector registered1689client # [ 1.001289] Freeing unused decrypted memory: 2028K1690worker2 # [ 0.889241] usbcore: registered new interface driver usbserial_generic1691client # [ 1.003983] Freeing unused kernel image (initmem) memory: 3644K1692worker2 # [ 0.890435] usbserial: USB Serial support registered for generic1693client # [ 1.005086] Write protecting the kernel read-only data: 32768k1694client # [ 1.007071] Freeing unused kernel image (text/rodata gap) memory: 1208K1695worker2 # [ 0.893055] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1696worker2 # [ 0.893055] Please enable it if your BIOS has the CPPC option.1697client # [ 1.008674] Freeing unused kernel image (rodata/data gap) memory: 752K1698worker2 # [ 0.895364] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1699worker2 # [ 0.899250] drop_monitor: Initializing network drop monitor service1700worker2 # [ 0.900484] NET: Registered PF_INET6 protocol family1701worker2 # [ 0.904779] Segment Routing with IPv61702worker2 # [ 0.905556] In-situ OAM (IOAM) with IPv61703lb # [ 0.999820] Freeing initrd memory: 29092K1704worker2 # [ 0.909424] IPI shorthand broadcast: enabled1705lb # [ 1.002982] Freeing unused decrypted memory: 2028K1706lb # [ 1.005551] Freeing unused kernel image (initmem) memory: 3644K1707lb # [ 1.006619] Write protecting the kernel read-only data: 32768k1708lb # [ 1.008523] Freeing unused kernel image (text/rodata gap) memory: 1208K1709worker2 # [ 0.916280] sched_clock: Marking stable (730021165, 185980015)->(1004154109, -88152929)1710lb # [ 1.010011] Freeing unused kernel image (rodata/data gap) memory: 752K1711worker1 # [ 1.018931] Freeing initrd memory: 29092K1712worker2 # [ 0.921329] registered taskstats version 11713worker2 # [ 0.922521] Loading compiled-in X.509 certificates1714worker1 # [ 1.022292] Freeing unused decrypted memory: 2028K1715worker1 # [ 1.025007] Freeing unused kernel image (initmem) memory: 3644K1716worker1 # [ 1.026147] Write protecting the kernel read-only data: 32768k1717worker1 # [ 1.028056] Freeing unused kernel image (text/rodata gap) memory: 1208K1718worker1 # [ 1.029608] Freeing unused kernel image (rodata/data gap) memory: 752K1719worker2 # [ 0.946011] Demotion targets for Node 0: null1720client # [ 1.060331] x86/mm: Checked W+X mappings: passed, no W+X pages found.1721worker2 # [ 0.947029] Key type .fscrypt registered1722client # [ 1.061512] Run /init as init process1723worker2 # [ 0.947714] Key type fscrypt-provisioning registered1724worker2 # [ 0.948833] ima: No TPM chip found, activating TPM-bypass!1725worker2 # [ 0.952017] ima: Allocated hash algorithm: sha11726worker2 # [ 0.952845] ima: No architecture policies found1727worker2 # [ 0.956052] PM: Magic number: 10:880:681728worker2 # [ 0.957911] RAS: Correctable Errors collector initialized.1729client # [ 1.072517] systemd[1]: Inserted module 'autofs4'1730lb # [ 1.062095] x86/mm: Checked W+X mappings: passed, no W+X pages found.1731lb # [ 1.063181] Run /init as init process1732worker2 # [ 0.971966] clk: Disabling unused clocks1733worker2 # [ 0.972732] PM: genpd: Disabling unused power domains1734client # [ 1.089886] fuse: init (API version 7.45)1735lb # [ 1.073885] systemd[1]: Inserted module 'autofs4'1736client # [ 1.096065] ACPI: \_SB_.GSIG: Enabled at IRQ 221737worker1 # [ 1.081984] x86/mm: Checked W+X mappings: passed, no W+X pages found.1738worker1 # [ 1.083046] Run /init as init process1739client # [ 1.098787] ACPI: \_SB_.GSIH: Enabled at IRQ 231740client # [ 1.102859] ACPI: \_SB_.GSIE: Enabled at IRQ 201741client # [ 1.105376] ACPI: \_SB_.GSIF: Enabled at IRQ 211742worker1 # [ 1.093677] systemd[1]: Inserted module 'autofs4'1743lb # [ 1.090493] fuse: init (API version 7.45)1744lb # [ 1.096512] ACPI: \_SB_.GSIG: Enabled at IRQ 221745lb # [ 1.099139] ACPI: \_SB_.GSIH: Enabled at IRQ 231746lb # [ 1.102314] ACPI: \_SB_.GSIE: Enabled at IRQ 201747lb # [ 1.104623] ACPI: \_SB_.GSIF: Enabled at IRQ 211748worker1 # [ 1.110707] fuse: init (API version 7.45)1749worker1 # [ 1.117020] ACPI: \_SB_.GSIG: Enabled at IRQ 221750worker1 # [ 1.119487] ACPI: \_SB_.GSIH: Enabled at IRQ 231751worker1 # [ 1.122523] ACPI: \_SB_.GSIE: Enabled at IRQ 201752client # [ 1.137727] systemd[1]: Successfully made /usr/ read-only.1753worker1 # [ 1.124760] ACPI: \_SB_.GSIF: Enabled at IRQ 211754lb # [ 1.139797] systemd[1]: Successfully made /usr/ read-only.1755worker1 # [ 1.152319] systemd[1]: Successfully made /usr/ read-only.1756worker2 # [ 1.177619] Freeing initrd memory: 29092K1757worker2 # [ 1.180873] Freeing unused decrypted memory: 2028K1758worker2 # [ 1.183440] Freeing unused kernel image (initmem) memory: 3644K1759worker2 # [ 1.184498] Write protecting the kernel read-only data: 32768k1760worker2 # [ 1.186339] Freeing unused kernel image (text/rodata gap) memory: 1208K1761worker2 # [ 1.187823] Freeing unused kernel image (rodata/data gap) memory: 752K1762worker2 # [ 1.240546] x86/mm: Checked W+X mappings: passed, no W+X pages found.1763worker2 # [ 1.241715] Run /init as init process1764worker2 # [ 1.252897] systemd[1]: Inserted module 'autofs4'1765worker2 # [ 1.270558] fuse: init (API version 7.45)1766worker2 # [ 1.277088] ACPI: \_SB_.GSIG: Enabled at IRQ 221767worker2 # [ 1.279922] ACPI: \_SB_.GSIH: Enabled at IRQ 231768worker2 # [ 1.283790] ACPI: \_SB_.GSIE: Enabled at IRQ 201769worker2 # [ 1.286749] ACPI: \_SB_.GSIF: Enabled at IRQ 211770worker2 # [ 1.322600] systemd[1]: Successfully made /usr/ read-only.1771client # [ 1.473761] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772client # [ 1.478607] systemd[1]: Detected virtualization kvm.1773client # [ 1.479464] systemd[1]: Detected architecture x86-64.1774client # [ 1.480368] systemd[1]: Running in initrd.1775client # [ 1.481584] systemd[1]: Initializing machine ID from random generator.1776client # [ 1.482790] systemd[1]: Hostname set to <client>.1777lb # [ 1.475895] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778lb # [ 1.482195] systemd[1]: Detected virtualization kvm.1779lb # [ 1.483404] systemd[1]: Detected architecture x86-64.1780lb # [ 1.484543] systemd[1]: Running in initrd.1781lb # [ 1.485954] systemd[1]: Initializing machine ID from random generator.1782lb # [ 1.487533] systemd[1]: Hostname set to <lb>.1783worker1 # [ 1.488951] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784worker1 # [ 1.503113] systemd[1]: Detected virtualization kvm.1785worker1 # [ 1.505711] systemd[1]: Detected architecture x86-64.1786worker1 # [ 1.508163] systemd[1]: Running in initrd.1787worker1 # [ 1.511110] systemd[1]: Initializing machine ID from random generator.1788worker1 # [ 1.514709] systemd[1]: Hostname set to <worker1>.1789client # [ 1.702266] systemd[1]: bpf-restrict-fs: LSM BPF program attached1790lb # [ 1.695099] systemd[1]: bpf-restrict-fs: LSM BPF program attached1791worker1 # [ 1.728782] systemd[1]: bpf-restrict-fs: LSM BPF program attached1792lb # [ 1.735445] systemd[1]: Queued start job for default target Initrd Default Target.1793client # [ 1.758972] systemd[1]: Queued start job for default target Initrd Default Target.1794lb # [ 1.739704] systemd[1]: Created slice Slice /system/modprobe.1795lb # [ 1.740876] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1796client # [ 1.763265] systemd[1]: Created slice Slice /system/modprobe.1797lb # [ 1.742258] systemd[1]: Expecting device /dev/disk/by-label/nixos...1798lb # [ 1.743448] systemd[1]: Reached target Path Units.1799client # [ 1.764461] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1800lb # [ 1.744292] systemd[1]: Reached target Slice Units.1801lb # [ 1.745134] systemd[1]: Reached target Swaps.1802client # [ 1.765904] systemd[1]: Expecting device /dev/disk/by-label/nixos...1803lb # [ 1.745920] systemd[1]: Reached target Timer Units.1804client # [ 1.767029] systemd[1]: Reached target Path Units.1805client # [ 1.767922] systemd[1]: Reached target Slice Units.1806lb # [ 1.746895] systemd[1]: Listening on D-Bus System Message Bus Socket.1807client # [ 1.768775] systemd[1]: Reached target Swaps.1808lb # [ 1.748081] systemd[1]: Listening on Journal Socket (/dev/log).1809client # [ 1.769606] systemd[1]: Reached target Timer Units.1810lb # [ 1.749219] systemd[1]: Listening on Journal Sockets.1811client # [ 1.770623] systemd[1]: Listening on D-Bus System Message Bus Socket.1812lb # [ 1.750193] systemd[1]: Listening on udev Control Socket.1813client # [ 1.771893] systemd[1]: Listening on Journal Socket (/dev/log).1814lb # [ 1.751186] systemd[1]: Listening on udev Kernel Socket.1815client # [ 1.773074] systemd[1]: Listening on Journal Sockets.1816lb # [ 1.752105] systemd[1]: Reached target Socket Units.1817client # [ 1.774064] systemd[1]: Listening on udev Control Socket.1818client # [ 1.775093] systemd[1]: Listening on udev Kernel Socket.1819lb # [ 1.753934] systemd[1]: Starting Create List of Static Device Nodes...1820client # [ 1.776056] systemd[1]: Reached target Socket Units.1821worker2 # [ 1.658861] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1822client # [ 1.778056] systemd[1]: Starting Create List of Static Device Nodes...1823worker2 # [ 1.665055] systemd[1]: Detected virtualization kvm.1824lb # [ 1.758361] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1825worker2 # [ 1.666140] systemd[1]: Detected architecture x86-64.1826worker2 # [ 1.667219] systemd[1]: Running in initrd.1827worker2 # [ 1.668734] systemd[1]: Initializing machine ID from random generator.1828client # [ 1.783915] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1829worker2 # [ 1.670375] systemd[1]: Hostname set to <worker2>.1830lb # [ 1.766385] systemd[1]: Starting Load Kernel Module configfs...1831client # [ 1.788011] systemd[1]: Starting Load Kernel Module configfs...1832worker1 # [ 1.782655] systemd[1]: Queued start job for default target Initrd Default Target.1833lb # [ 1.780505] systemd[1]: Starting Journal Service...1834worker1 # [ 1.786737] systemd[1]: Created slice Slice /system/modprobe.1835worker1 # [ 1.787965] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1836worker1 # [ 1.789441] systemd[1]: Expecting device /dev/disk/by-label/nixos...1837worker1 # [ 1.790593] systemd[1]: Reached target Path Units.1838client # [ 1.805945] systemd[1]: Starting Journal Service...1839worker1 # [ 1.791508] systemd[1]: Reached target Slice Units.1840worker1 # [ 1.792429] systemd[1]: Reached target Swaps.1841worker1 # [ 1.793193] systemd[1]: Reached target Timer Units.1842worker1 # [ 1.794171] systemd[1]: Listening on D-Bus System Message Bus Socket.1843worker1 # [ 1.795426] systemd[1]: Listening on Journal Socket (/dev/log).1844worker1 # [ 1.796606] systemd[1]: Listening on Journal Sockets.1845worker1 # [ 1.797603] systemd[1]: Listening on udev Control Socket.1846worker1 # [ 1.798618] systemd[1]: Listening on udev Kernel Socket.1847worker1 # [ 1.799551] systemd[1]: Reached target Socket Units.1848lb # [ 1.794371] systemd[1]: Starting Load Kernel Modules...1849worker1 # [ 1.801554] systemd[1]: Starting Create List of Static Device Nodes...1850client # [ 1.820919] systemd[1]: Starting Load Kernel Modules...1851worker1 # [ 1.807449] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1852worker1 # [ 1.813589] systemd[1]: Starting Load Kernel Module configfs...1853lb # [ 1.799380] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1854lb # [ 1.817502] netfs: FS-Cache loaded1855client # [ 1.827993] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1856lb # [ 1.822375] 9pnet: Installing 9P2000 support1857worker1 # [ 1.829447] systemd[1]: Starting Journal Service...1858client # [ 1.846896] netfs: FS-Cache loaded1859client # [ 1.852563] 9pnet: Installing 9P2000 support1860worker1 # [ 1.839779] systemd[1]: Starting Load Kernel Modules...1861lb # [ 1.836723] systemd[1]: Starting Coldplug All udev Devices...1862client # [ 1.868026] systemd[1]: Starting Coldplug All udev Devices...1863worker1 # [ 1.850505] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1864lb # [ 1.860576] systemd[1]: Finished Create List of Static Device Nodes.1865worker1 # [ 1.867451] netfs: FS-Cache loaded1866client # [ 1.883926] systemd[1]: Finished Create List of Static Device Nodes.1867worker1 # [ 1.872806] 9pnet: Installing 9P2000 support1868client # [ 1.892351] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1869lb # [ 1.876622] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1870client # [ 1.899921] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1871client # [ 1.901466] systemd-journald[67]: Collecting audit messages is disabled.1872worker1 # [ 1.887725] systemd[1]: Starting Coldplug All udev Devices...1873lb # [ 1.885454] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1874client # [ 1.908018] systemd[1]: modprobe@configfs.service: Deactivated successfully.1875lb # [ 1.892917] systemd[1]: modprobe@configfs.service: Deactivated successfully.1876client # [ 1.915374] systemd[1]: Finished Load Kernel Module configfs.1877lb # [ 1.897824] systemd-journald[67]: Collecting audit messages is disabled.1878client # [ 1.920046] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1879worker1 # [ 1.906492] systemd[1]: Finished Create List of Static Device Nodes.1880lb # [ 1.900847] systemd[1]: Finished Load Kernel Module configfs.1881client # [ 1.922773] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1882lb # [ 1.906928] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1883worker1 # [ 1.914226] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1884client # [ 1.931133] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1885lb # [ 1.909702] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1886client # [ 1.936103] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1887worker1 # [ 1.922489] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1888lb # [ 1.919355] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1889lb # [ 1.921430] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1890worker1 # [ 1.927714] systemd-journald[67]: Collecting audit messages is disabled.1891worker1 # [ 1.929684] systemd[1]: modprobe@configfs.service: Deactivated successfully.1892worker1 # [ 1.936968] systemd[1]: Finished Load Kernel Module configfs.1893worker1 # [ 1.945159] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1894worker1 # [ 1.948206] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1895client # [ 1.965504] systemd[1]: Finished Load Kernel Modules.1896worker1 # [ 1.958621] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1897client # [ 1.975099] systemd[1]: Starting Apply Kernel Variables...1898lb # [ 1.956015] systemd[1]: Finished Load Kernel Modules.1899worker1 # [ 1.961701] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1900lb # [ 1.965795] systemd[1]: Starting Apply Kernel Variables...1901client # [ 1.995941] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1902client # [ 2.006919] systemd[1]: Starting Create Static Device Nodes in /dev...1903lb # [ 1.987471] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1904worker1 # [ 1.998460] systemd[1]: Finished Load Kernel Modules.1905lb # [ 2.002422] systemd[1]: Starting Create Static Device Nodes in /dev...1906worker1 # [ 2.009998] systemd[1]: Starting Apply Kernel Variables...1907client # [ 2.025511] systemd[1]: Finished Apply Kernel Variables.1908worker2 # [ 1.916581] systemd[1]: bpf-restrict-fs: LSM BPF program attached1909client # [ 2.036110] systemd[1]: Started Journal Service.1910client # [ 1.894346] systemd-modules-load[68]: Inserted module 'dm_mod'1911client # [ 1.897350] systemd-modules-load[68]: Inserted module 'virtio_balloon'1912client # [ 1.899571] systemd-modules-load[68]: Inserted module 'virtio_gpu'1913worker1 # [ 2.034672] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1914lb # [ 1.889454] systemd-modules-load[68]: Inserted module 'dm_mod'1915lb # [ 2.033135] systemd[1]: Finished Apply Kernel Variables.1916lb # [ 1.893287] systemd-modules-load[68]: Inserted module 'virtio_balloon'1917lb # [ 1.894459] systemd-modules-load[68]: Inserted module 'virtio_gpu'1918lb # [ 2.039635] systemd[1]: Started Journal Service.1919client # [ 1.919115] systemd[1]: Finished Create Static Device Nodes in /dev.1920client # [ 1.920326] systemd[1]: Reached target Preparation for Local File Systems.1921worker1 # [ 2.048185] systemd[1]: Starting Create Static Device Nodes in /dev...1922client # [ 1.922218] systemd[1]: Reached target Local File Systems.1923client # [ 1.926086] systemd[1]: Starting Create System Files and Directories...1924client # [ 1.930290] systemd[1]: Starting Rule-based Manager for Device Events and Files...1925worker2 # [ 1.966731] systemd[1]: Queued start job for default target Initrd Default Target.1926lb # [ 1.918191] systemd[1]: Finished Create Static Device Nodes in /dev.1927lb # [ 1.919247] systemd[1]: Reached target Preparation for Local File Systems.1928lb # [ 1.920295] systemd[1]: Reached target Local File Systems.1929worker1 # [ 2.069652] systemd[1]: Finished Apply Kernel Variables.1930worker2 # [ 1.971625] systemd[1]: Created slice Slice /system/modprobe.1931lb # [ 1.922693] systemd[1]: Starting Create System Files and Directories...1932worker1 # [ 1.926750] systemd-modules-load[68]: Inserted module 'dm_mod'1933worker2 # [ 1.972840] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1934worker1 # [ 1.929993] systemd-modules-load[68]: Inserted module 'virtio_balloon'1935worker2 # [ 1.974269] systemd[1]: Expecting device /dev/disk/by-label/nixos...1936worker2 # [ 1.975387] systemd[1]: Reached target Path Units.1937worker2 # [ 1.976278] systemd[1]: Reached target Slice Units.1938worker1 # [ 1.933243] systemd-modules-load[68]: Inserted module 'virtio_gpu'1939worker2 # [ 1.977181] systemd[1]: Reached target Swaps.1940lb # [ 1.928335] systemd[1]: Starting Rule-based Manager for Device Events and Files...1941worker2 # [ 1.977950] systemd[1]: Reached target Timer Units.1942worker2 # [ 1.979013] systemd[1]: Listening on D-Bus System Message Bus Socket.1943worker2 # [ 1.980250] systemd[1]: Listening on Journal Socket (/dev/log).1944worker1 # [ 2.079874] systemd[1]: Started Journal Service.1945worker2 # [ 1.981453] systemd[1]: Listening on Journal Sockets.1946worker2 # [ 1.982479] systemd[1]: Listening on udev Control Socket.1947worker2 # [ 1.983510] systemd[1]: Listening on udev Kernel Socket.1948worker2 # [ 1.984512] systemd[1]: Reached target Socket Units.1949worker2 # [ 1.986812] systemd[1]: Starting Create List of Static Device Nodes...1950client # [ 1.963490] systemd[1]: Finished Create System Files and Directories.1951worker2 # [ 1.992500] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1952worker2 # [ 1.999104] systemd[1]: Starting Load Kernel Module configfs...1953worker1 # [ 1.962270] systemd[1]: Finished Create Static Device Nodes in /dev.1954worker1 # [ 1.963439] systemd[1]: Reached target Preparation for Local File Systems.1955worker1 # [ 1.964489] systemd[1]: Reached target Local File Systems.1956worker1 # [ 1.969123] systemd[1]: Starting Create System Files and Directories...1957worker2 # [ 2.017170] systemd[1]: Starting Journal Service...1958client # [ 1.990458] systemd-udevd[83]: Using default interface naming scheme 'v261'.1959lb # [ 1.970854] systemd[1]: Finished Create System Files and Directories.1960worker1 # [ 1.978462] systemd[1]: Starting Rule-based Manager for Device Events and Files...1961worker2 # [ 2.028568] systemd[1]: Starting Load Kernel Modules...1962lb # [ 1.995457] systemd-udevd[83]: Using default interface naming scheme 'v261'.1963client # [ 2.016418] systemd[1]: Started Rule-based Manager for Device Events and Files.1964worker2 # [ 2.035100] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1965worker2 # [ 2.058112] netfs: FS-Cache loaded1966worker1 # [ 2.017269] systemd[1]: Finished Create System Files and Directories.1967worker2 # [ 2.065357] 9pnet: Installing 9P2000 support1968lb # [ 2.022324] systemd[1]: Started Rule-based Manager for Device Events and Files.1969worker2 # [ 2.074256] systemd[1]: Starting Coldplug All udev Devices...1970worker1 # [ 2.042461] systemd-udevd[83]: Using default interface naming scheme 'v261'.1971client # [ 2.066935] systemd[1]: Finished Coldplug All udev Devices.1972client # [ 2.068491] systemd[1]: Reached target System Initialization.1973client # [ 2.070134] systemd[1]: Reached target Basic System.1974worker2 # [ 2.100160] systemd[1]: Finished Create List of Static Device Nodes.1975lb # [ 2.058246] systemd[1]: Finished Coldplug All udev Devices.1976lb # [ 2.059715] systemd[1]: Reached target System Initialization.1977lb # [ 2.061160] systemd[1]: Reached target Basic System.1978worker2 # [ 2.110076] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1979worker1 # [ 2.068530] systemd[1]: Started Rule-based Manager for Device Events and Files.1980worker2 # [ 2.118636] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1981worker2 # [ 2.128751] systemd[1]: modprobe@configfs.service: Deactivated successfully.1982worker2 # [ 2.132425] systemd-journald[67]: Collecting audit messages is disabled.1983worker2 # [ 2.136589] systemd[1]: Finished Load Kernel Module configfs.1984worker2 # [ 2.142897] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1985worker2 # [ 2.146533] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1986worker1 # [ 2.106805] systemd[1]: Finished Coldplug All udev Devices.1987worker1 # [ 2.108079] systemd[1]: Reached target System Initialization.1988worker1 # [ 2.109268] systemd[1]: Reached target Basic System.1989worker2 # [ 2.153896] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1990worker2 # [ 2.157270] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1991worker2 # [ 2.188659] systemd[1]: Finished Load Kernel Modules.1992worker2 # [ 2.198241] systemd[1]: Starting Apply Kernel Variables...1993worker2 # [ 2.214098] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1994worker2 # [ 2.223606] systemd[1]: Starting Create Static Device Nodes in /dev...1995worker2 # [ 2.249174] systemd[1]: Finished Apply Kernel Variables.1996worker2 # [ 2.257932] systemd[1]: Started Journal Service.1997worker2 # [ 2.075251] systemd-modules-load[68]: Inserted module 'dm_mod'1998worker2 # [ 2.080133] systemd-modules-load[68]: Inserted module 'virtio_balloon'1999worker2 # [ 2.081448] systemd-modules-load[68]: Inserted module 'virtio_gpu'2000worker2 # [ 2.092180] systemd[1]: Finished Create Static Device Nodes in /dev.
loading 2,157 hidden lines…
4158worker2 # [ 20.834503] nix-grpc-daemon[854]: ts=2026-09-20T08:04:06Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4159worker1 # [ 21.053840] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514160worker1 # [ 21.057426] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4161worker2 # [ 21.035731] nix-grpc-daemon[854]: ts=2026-09-20T08:04:06Z level=info event=scheduler_connected addr=lb:500514162worker2 # [ 21.039493] nix-grpc-daemon[854]: ts=2026-09-20T08:04:06Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4163worker1 # [ 21.259218] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514164worker1 # [ 21.263397] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4165worker2 # [ 21.240693] nix-grpc-daemon[854]: ts=2026-09-20T08:04:06Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 21.244546] nix-grpc-daemon[854]: ts=2026-09-20T08:04:06Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4167worker1 # [ 21.464490] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514168worker1 # [ 21.467723] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4169worker2 # [ 21.445770] nix-grpc-daemon[854]: ts=2026-09-20T08:04:06Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 21.449275] nix-grpc-daemon[854]: ts=2026-09-20T08:04:06Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4171worker1 # [ 21.668579] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514172worker1 # [ 21.671988] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4173worker2 # [ 21.650460] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 21.653518] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4175worker1 # [ 21.872690] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514176worker1 # [ 21.875399] nix-grpc-daemon[782]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4177worker2 # [ 21.856178] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514178worker2 # [ 21.858771] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4179worker1 # [ 22.076055] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514180worker1 # [ 22.079131] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4181worker2 # [ 22.059435] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 22.062899] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4183worker1 # [ 22.280705] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514184worker1 # [ 22.283796] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4185worker2 # [ 22.264044] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514186worker2 # [ 22.267522] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4187worker1 # [ 22.484740] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514188worker1 # [ 22.488282] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4189worker2 # [ 22.468708] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_connected addr=lb:500514190worker2 # [ 22.472238] nix-grpc-daemon[854]: ts=2026-09-20T08:04:07Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4191worker1 # [ 22.689362] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514192worker1 # [ 22.692725] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4193worker2 # [ 22.673464] nix-grpc-daemon[854]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514194worker2 # [ 22.676861] nix-grpc-daemon[854]: ts=2026-09-20T08:04:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4195worker1 # [ 22.893625] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514196worker1 # [ 22.897440] nix-grpc-daemon[782]: ts=2026-09-20T08:04:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4197worker2 # [ 22.878559] nix-grpc-daemon[854]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514198worker2 # [ 22.882114] nix-grpc-daemon[854]: ts=2026-09-20T08:04:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4199worker1 # [ 23.098485] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_connected addr=lb:500514200worker1 # [ 23.101862] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4201worker2 # [ 23.083170] nix-grpc-daemon[854]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 23.086797] nix-grpc-daemon[854]: ts=2026-09-20T08:04:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4203worker1 # [ 23.303505] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_connected addr=lb:500514204worker1 # [ 23.306593] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4205worker2 # [ 23.287765] nix-grpc-daemon[854]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 23.292487] nix-grpc-daemon[854]: ts=2026-09-20T08:04:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4207worker1 # [ 23.507281] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_connected addr=lb:500514208worker1 # [ 23.510497] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4209worker2 # [ 23.492970] nix-grpc-daemon[854]: ts=2026-09-20T08:04:08Z level=info event=scheduler_connected addr=lb:500514210worker2 # [ 23.496347] nix-grpc-daemon[854]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4211worker1 # [ 23.711363] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_connected addr=lb:500514212worker1 # [ 23.714761] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4213worker2 # [ 23.697360] nix-grpc-daemon[854]: ts=2026-09-20T08:04:09Z level=info event=scheduler_connected addr=lb:500514214worker2 # [ 23.700827] nix-grpc-daemon[854]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4215worker1 # [ 23.915593] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_connected addr=lb:500514216worker1 # [ 23.918723] nix-grpc-daemon[782]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4217worker2 # [ 23.904076] nix-grpc-daemon[854]: ts=2026-09-20T08:04:09Z level=info event=scheduler_connected addr=lb:500514218worker2 # [ 23.906514] nix-grpc-daemon[854]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4219worker1 # [ 24.119702] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514220worker1 # [ 24.122985] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4221worker2 # [ 24.106051] nix-grpc-daemon[854]: ts=2026-09-20T08:04:09Z level=info event=scheduler_connected addr=lb:500514222worker2 # [ 24.109863] nix-grpc-daemon[854]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4223worker1 # [ 24.324140] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514224worker1 # [ 24.326905] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4225worker2 # [ 24.310897] nix-grpc-daemon[854]: ts=2026-09-20T08:04:09Z level=info event=scheduler_connected addr=lb:500514226worker2 # [ 24.314205] nix-grpc-daemon[854]: ts=2026-09-20T08:04:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4227worker1 # [ 24.527711] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514228worker1 # [ 24.530700] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4229worker2 # [ 24.515282] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 24.518898] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4231worker1 # [ 24.731747] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514232worker1 # [ 24.735075] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4233worker2 # [ 24.720458] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 24.727758] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4235worker1 # [ 24.935725] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514236worker1 # [ 24.939082] nix-grpc-daemon[782]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4237worker2 # [ 24.929752] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514238worker2 # [ 24.933227] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4239worker1 # [ 25.139881] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514240worker1 # [ 25.142782] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4241worker2 # [ 25.134350] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514242worker2 # [ 25.137514] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4243worker1 # [ 25.344209] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514244worker1 # [ 25.347358] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4245worker2 # [ 25.338836] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_connected addr=lb:500514246worker2 # [ 25.342175] nix-grpc-daemon[854]: ts=2026-09-20T08:04:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4247worker1 # [ 25.548178] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514248worker1 # [ 25.551173] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4249worker2 # [ 25.543610] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514250worker2 # [ 25.547625] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4251worker1 # [ 25.752149] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514252worker1 # [ 25.755522] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4253worker2 # [ 25.749226] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514254worker2 # [ 25.752489] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4255worker1 # [ 25.956481] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514256worker1 # [ 25.960162] nix-grpc-daemon[782]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4257worker2 # [ 25.954430] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514258worker2 # [ 25.958499] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4259worker1 # [ 26.161106] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514260worker1 # [ 26.164467] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4261worker2 # [ 26.159799] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514262worker2 # [ 26.163605] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4263worker1 # [ 26.366437] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514264worker1 # [ 26.369865] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4265worker2 # [ 26.364693] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_connected addr=lb:500514266worker2 # [ 26.367687] nix-grpc-daemon[854]: ts=2026-09-20T08:04:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4267worker1 # [ 26.570775] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514268worker1 # [ 26.574214] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4269worker2 # [ 26.568923] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514270worker2 # [ 26.574508] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4271worker1 # [ 26.775143] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514272worker1 # [ 26.778392] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4273worker2 # [ 26.774679] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514274worker2 # [ 26.777980] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4275worker1 # [ 26.979327] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514276worker1 # [ 26.982546] nix-grpc-daemon[782]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4277worker2 # [ 26.979836] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514278worker2 # [ 26.984359] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4279worker1 # [ 27.183484] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514280worker1 # [ 27.186450] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4281worker2 # [ 27.185782] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514282worker2 # [ 27.191320] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4283worker1 # [ 27.388358] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514284worker1 # [ 27.391703] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4285worker2 # [ 27.391984] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_connected addr=lb:500514286worker2 # [ 27.397192] nix-grpc-daemon[854]: ts=2026-09-20T08:04:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4287worker1 # [ 27.592763] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514288worker1 # [ 27.595820] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4289worker2 # [ 27.602492] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514290worker2 # [ 27.606338] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4291worker1 # [ 27.796655] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514292worker1 # [ 27.799714] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4293worker2 # [ 27.807270] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514294worker2 # [ 27.810599] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4295worker1 # [ 28.000682] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514296worker1 # [ 28.003664] nix-grpc-daemon[782]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4297worker2 # [ 28.013192] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514298worker2 # [ 28.015693] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4299worker1 # [ 28.204530] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514300worker1 # [ 28.207928] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4301worker2 # [ 28.216590] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514302worker2 # [ 28.220238] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4303worker1 # [ 28.410247] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514304worker1 # [ 28.412566] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4305worker2 # [ 28.421488] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_connected addr=lb:500514306worker2 # [ 28.424925] nix-grpc-daemon[854]: ts=2026-09-20T08:04:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4307worker1 # [ 28.613322] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514308worker1 # [ 28.617780] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4309worker2 # [ 28.625823] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514310worker2 # [ 28.629914] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4311worker1 # [ 28.818539] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514312worker1 # [ 28.821759] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4313worker2 # [ 28.831339] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514314worker2 # [ 28.836892] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4315worker1 # [ 29.022532] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514316worker1 # [ 29.025435] nix-grpc-daemon[782]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4317worker2 # [ 29.038207] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514318worker2 # [ 29.041114] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4319worker1 # [ 29.226163] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514320worker1 # [ 29.229053] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4321worker2 # [ 29.242213] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514322worker2 # [ 29.246183] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4323worker1 # [ 29.432733] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514324worker1 # [ 29.436073] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4325worker2 # [ 29.447058] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_connected addr=lb:500514326worker2 # [ 29.450140] nix-grpc-daemon[854]: ts=2026-09-20T08:04:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4327worker1 # [ 29.633698] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514328worker1 # [ 29.636372] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4329worker2 # [ 29.651153] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514330worker2 # [ 29.654855] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4331worker1 # [ 29.837152] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514332worker1 # [ 29.840987] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4333worker2 # [ 29.855615] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514334worker2 # [ 29.858644] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4335worker1 # [ 30.042098] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514336worker1 # [ 30.044648] nix-grpc-daemon[782]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4337worker2 # [ 30.060340] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514338worker2 # [ 30.064306] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4339worker1 # [ 30.245541] nix-grpc-daemon[782]: ts=2026-09-20T08:04:16Z level=info event=scheduler_connected addr=lb:500514340worker1 # [ 30.248747] nix-grpc-daemon[782]: ts=2026-09-20T08:04:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4341worker2 # [ 30.265334] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514342worker2 # [ 30.268782] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4343worker1 # [ 30.450432] nix-grpc-daemon[782]: ts=2026-09-20T08:04:16Z level=info event=scheduler_connected addr=lb:500514344worker1 # [ 30.453875] nix-grpc-daemon[782]: ts=2026-09-20T08:04:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4345worker2 # [ 30.469719] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_connected addr=lb:500514346worker2 # [ 30.472471] nix-grpc-daemon[854]: ts=2026-09-20T08:04:15Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4347worker1 # [ 30.654783] nix-grpc-daemon[782]: ts=2026-09-20T08:04:16Z level=info event=scheduler_connected addr=lb:500514348worker1 # [ 30.657874] nix-grpc-daemon[782]: ts=2026-09-20T08:04:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4349worker2 # [ 30.673194] nix-grpc-daemon[854]: ts=2026-09-20T08:04:16Z level=info event=scheduler_connected addr=lb:500514350worker2 # [ 30.675868] nix-grpc-daemon[854]: ts=2026-09-20T08:04:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4351worker1 # [ 30.858518] nix-grpc-daemon[782]: ts=2026-09-20T08:04:16Z level=info event=scheduler_connected addr=lb:500514352worker1 # [ 30.861114] nix-grpc-daemon[782]: ts=2026-09-20T08:04:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4353worker2 # [ 30.877214] nix-grpc-daemon[854]: ts=2026-09-20T08:04:16Z level=info event=scheduler_connected addr=lb:500514354worker2 # [ 30.882369] nix-grpc-daemon[854]: ts=2026-09-20T08:04:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4355worker1 # [ 31.061716] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_connected addr=lb:500514356worker1 # [ 31.064441] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4357worker2 # [ 31.084273] nix-grpc-daemon[854]: ts=2026-09-20T08:04:16Z level=info event=scheduler_connected addr=lb:500514358worker2 # [ 31.088832] nix-grpc-daemon[854]: ts=2026-09-20T08:04:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4359worker1 # [ 31.265435] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_connected addr=lb:500514360worker1 # [ 31.268350] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4361worker2 # [ 31.290106] nix-grpc-daemon[854]: ts=2026-09-20T08:04:16Z level=info event=scheduler_connected addr=lb:500514362worker2 # [ 31.293994] nix-grpc-daemon[854]: ts=2026-09-20T08:04:16Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4363worker1 # [ 31.469614] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_connected addr=lb:500514364worker1 # [ 31.472187] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4365worker2 # [ 31.494834] nix-grpc-daemon[854]: ts=2026-09-20T08:04:16Z level=info event=scheduler_connected addr=lb:500514366worker2 # [ 31.497797] nix-grpc-daemon[854]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4367worker1 # [ 31.673070] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_connected addr=lb:500514368worker1 # [ 31.675515] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4369worker2 # [ 31.698762] nix-grpc-daemon[854]: ts=2026-09-20T08:04:17Z level=info event=scheduler_connected addr=lb:500514370worker2 # [ 31.702146] nix-grpc-daemon[854]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4371worker1 # [ 31.876277] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_connected addr=lb:500514372worker1 # [ 31.878825] nix-grpc-daemon[782]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4373worker2 # [ 31.903228] nix-grpc-daemon[854]: ts=2026-09-20T08:04:17Z level=info event=scheduler_connected addr=lb:500514374worker2 # [ 31.906130] nix-grpc-daemon[854]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4375worker1 # [ 32.079446] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514376worker1 # [ 32.081570] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4377worker2 # [ 32.109746] nix-grpc-daemon[854]: ts=2026-09-20T08:04:17Z level=info event=scheduler_connected addr=lb:500514378worker2 # [ 32.112871] nix-grpc-daemon[854]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4379worker1 # [ 32.282397] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514380worker1 # [ 32.285344] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4381worker2 # [ 32.310773] nix-grpc-daemon[854]: ts=2026-09-20T08:04:17Z level=info event=scheduler_connected addr=lb:500514382worker2 # [ 32.315689] nix-grpc-daemon[854]: ts=2026-09-20T08:04:17Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4383worker1 # [ 32.487083] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514384worker1 # [ 32.488889] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4385worker2 # [ 32.516718] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514386worker2 # [ 32.519571] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4387worker1 # [ 32.689469] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514388worker1 # [ 32.691820] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4389worker2 # [ 32.720435] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514390worker2 # [ 32.723301] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4391worker1 # [ 32.892663] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514392worker1 # [ 32.895777] nix-grpc-daemon[782]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4393worker2 # [ 32.923872] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514394worker2 # [ 32.926587] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4395worker1 # [ 33.096690] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514396worker1 # [ 33.099827] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4397worker2 # [ 33.128279] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514398worker2 # [ 33.131259] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4399worker1 # [ 33.300673] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514400worker1 # [ 33.304235] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4401worker2 # [ 33.332414] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_connected addr=lb:500514402worker2 # [ 33.335323] nix-grpc-daemon[854]: ts=2026-09-20T08:04:18Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4403worker1 # [ 33.507902] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514404worker1 # [ 33.511138] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4405worker2 # [ 33.536246] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514406worker2 # [ 33.539253] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4407worker1 # [ 33.708206] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514408worker1 # [ 33.710848] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4409worker2 # [ 33.739972] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514410worker2 # [ 33.742680] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4411worker1 # [ 33.911416] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514412worker1 # [ 33.913730] nix-grpc-daemon[782]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4413worker2 # [ 33.943447] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514414worker2 # [ 33.946151] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4415worker1 # [ 34.114569] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514416worker1 # [ 34.117582] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4417worker2 # [ 34.147587] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514418worker2 # [ 34.150292] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4419worker1 # [ 34.318353] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514420worker1 # [ 34.320899] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4421worker2 # [ 34.351059] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_connected addr=lb:500514422worker2 # [ 34.354578] nix-grpc-daemon[854]: ts=2026-09-20T08:04:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4423worker1 # [ 34.522193] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514424worker1 # [ 34.524314] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4425worker2 # [ 34.555789] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514426worker2 # [ 34.560709] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4427worker1 # [ 34.724956] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514428worker1 # [ 34.727315] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4429worker2 # [ 34.761589] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514430worker2 # [ 34.767265] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4431worker1 # [ 34.928164] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514432worker1 # [ 34.930783] nix-grpc-daemon[782]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4433worker2 # [ 34.968640] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514434worker2 # [ 34.973966] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4435worker1 # [ 35.131622] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514436worker1 # [ 35.135443] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4437worker2 # [ 35.175876] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514438worker2 # [ 35.180970] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4439worker1 # [ 35.336185] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514440worker1 # [ 35.338679] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4441worker2 # [ 35.382049] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_connected addr=lb:500514442worker2 # [ 35.385597] nix-grpc-daemon[854]: ts=2026-09-20T08:04:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4443worker1 # [ 35.541253] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514444worker1 # [ 35.544959] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4445worker2 # [ 35.586352] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514446worker2 # [ 35.589401] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4447worker1 # [ 35.745626] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514448worker1 # [ 35.751186] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4449worker2 # [ 35.790293] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514450worker2 # [ 35.793517] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4451worker1 # [ 35.952224] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514452worker1 # [ 35.957843] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4453worker1 # [ 36.004951] nix-grpc-daemon[782]: ts=2026-09-20T08:04:21Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"4454worker2 # [ 35.921287] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"4455worker2 # [ 35.994571] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514456worker2 # [ 35.997630] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4457worker1 # [ 36.158392] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514458worker1 # [ 36.160713] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4459worker2 # [ 36.199546] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514460worker2 # [ 36.202926] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4461worker1 # [ 36.361439] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514462worker1 # [ 36.364206] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4463worker2 # [ 36.404183] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_connected addr=lb:500514464worker2 # [ 36.409889] nix-grpc-daemon[854]: ts=2026-09-20T08:04:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4465worker1 # [ 36.565435] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514466worker1 # [ 36.567572] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4467worker2 # [ 36.610886] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514468worker2 # [ 36.616307] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4469worker1 # [ 36.768478] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514470worker1 # [ 36.771233] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4471worker2 # [ 36.816721] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514472worker2 # [ 36.819153] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4473worker1 # [ 36.971893] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514474worker1 # [ 36.974084] nix-grpc-daemon[782]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4475worker2 # [ 37.020840] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514476worker2 # [ 37.026597] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4477worker1 # [ 37.174711] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514478worker1 # [ 37.176987] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4479worker2 # [ 37.231149] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514480worker2 # [ 37.234329] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4481worker1 # [ 37.378150] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514482worker1 # [ 37.383258] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4483worker2 # [ 37.431970] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_connected addr=lb:500514484worker2 # [ 37.434930] nix-grpc-daemon[854]: ts=2026-09-20T08:04:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4485worker1 # [ 37.585764] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514486worker1 # [ 37.587338] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4487worker2 # [ 37.635705] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514488worker2 # [ 37.638586] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4489worker1 # [ 37.786488] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514490worker1 # [ 37.789687] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4491worker2 # [ 37.839933] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514492worker2 # [ 37.844705] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4493worker1 # [ 37.990572] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514494worker1 # [ 37.993589] nix-grpc-daemon[782]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4495worker2 # [ 38.045746] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514496worker2 # [ 38.050676] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4497worker1 # [ 38.194826] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514498worker1 # [ 38.199785] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4499worker2 # [ 38.251905] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514500worker2 # [ 38.254309] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4501worker1 # [ 38.400551] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514502worker1 # [ 38.403470] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4503worker2 # [ 38.454868] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_connected addr=lb:500514504worker2 # [ 38.457465] nix-grpc-daemon[854]: ts=2026-09-20T08:04:23Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4505worker1 # [ 38.605163] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514506worker1 # [ 38.608250] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4507worker2 # [ 38.658147] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514508worker2 # [ 38.660654] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4509worker1 # [ 38.809080] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514510worker1 # [ 38.811273] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4511worker2 # [ 38.861578] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514512worker2 # [ 38.864720] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4513worker1 # [ 39.012272] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514514worker1 # [ 39.017954] nix-grpc-daemon[782]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4515worker2 # [ 39.065555] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514516worker2 # [ 39.068056] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4517worker1 # [ 39.219097] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_connected addr=lb:500514518worker1 # [ 39.224128] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4519worker2 # [ 39.273563] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514520worker2 # [ 39.276755] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4521worker1 # [ 39.425396] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_connected addr=lb:500514522worker1 # [ 39.430555] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4523worker2 # [ 39.473160] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_connected addr=lb:500514524worker2 # [ 39.478107] nix-grpc-daemon[854]: ts=2026-09-20T08:04:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4525worker1 # [ 39.632597] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_connected addr=lb:500514526worker1 # [ 39.637298] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4527worker2 # [ 39.679161] nix-grpc-daemon[854]: ts=2026-09-20T08:04:25Z level=info event=scheduler_connected addr=lb:500514528worker2 # [ 39.684093] nix-grpc-daemon[854]: ts=2026-09-20T08:04:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4529worker1 # [ 39.837982] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_connected addr=lb:500514530worker1 # [ 39.842755] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4531worker2 # [ 39.884494] nix-grpc-daemon[854]: ts=2026-09-20T08:04:25Z level=info event=scheduler_connected addr=lb:500514532worker2 # [ 39.886966] nix-grpc-daemon[854]: ts=2026-09-20T08:04:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4533worker1 # [ 40.044234] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_connected addr=lb:500514534worker1 # [ 40.049295] nix-grpc-daemon[782]: ts=2026-09-20T08:04:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4535worker2 # [ 40.087925] nix-grpc-daemon[854]: ts=2026-09-20T08:04:25Z level=info event=scheduler_connected addr=lb:500514536worker2 # [ 40.092739] nix-grpc-daemon[854]: ts=2026-09-20T08:04:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4537worker1 # [ 40.250433] nix-grpc-daemon[782]: ts=2026-09-20T08:04:26Z level=info event=scheduler_connected addr=lb:500514538worker1 # [ 40.255745] nix-grpc-daemon[782]: ts=2026-09-20T08:04:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4539worker2 # [ 40.293699] nix-grpc-daemon[854]: ts=2026-09-20T08:04:25Z level=info event=scheduler_connected addr=lb:500514540worker2 # [ 40.296167] nix-grpc-daemon[854]: ts=2026-09-20T08:04:25Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4541worker1 # [ 40.456500] nix-grpc-daemon[782]: ts=2026-09-20T08:04:26Z level=info event=scheduler_connected addr=lb:500514542worker1 # [ 40.458947] nix-grpc-daemon[782]: ts=2026-09-20T08:04:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4543worker2 # [ 40.496949] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_connected addr=lb:500514544worker2 # [ 40.499246] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4545worker1 # [ 40.662888] nix-grpc-daemon[782]: ts=2026-09-20T08:04:26Z level=info event=scheduler_connected addr=lb:500514546worker1 # [ 40.666315] nix-grpc-daemon[782]: ts=2026-09-20T08:04:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4547worker2 # [ 40.699945] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_connected addr=lb:500514548worker2 # [ 40.702583] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4549worker1 # [ 40.863607] nix-grpc-daemon[782]: ts=2026-09-20T08:04:26Z level=info event=scheduler_connected addr=lb:500514550worker1 # [ 40.868518] nix-grpc-daemon[782]: ts=2026-09-20T08:04:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4551worker2 # [ 40.903347] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_connected addr=lb:500514552worker2 # [ 40.905939] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4553worker1 # [ 41.069193] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514554worker1 # [ 41.071498] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4555worker2 # [ 41.106748] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_connected addr=lb:500514556worker2 # [ 41.109575] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4557worker1 # [ 41.272478] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514558worker1 # [ 41.275778] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4559worker2 # [ 41.310779] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_connected addr=lb:500514560worker2 # [ 41.314062] nix-grpc-daemon[854]: ts=2026-09-20T08:04:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4561worker1 # [ 41.477070] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514562worker1 # [ 41.482314] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4563worker2 # [ 41.514785] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514564worker2 # [ 41.517302] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4565worker1 # [ 41.683471] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514566worker1 # [ 41.686048] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4567worker2 # [ 41.717926] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514568worker2 # [ 41.720338] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4569worker1 # [ 41.886864] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514570worker1 # [ 41.891900] nix-grpc-daemon[782]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4571worker2 # [ 41.921053] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514572worker2 # [ 41.923560] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4573worker1 # [ 42.092677] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514574worker1 # [ 42.095818] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4575worker2 # [ 42.124800] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514576worker2 # [ 42.130215] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4577worker1 # [ 42.296894] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514578worker1 # [ 42.302166] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4579worker2 # [ 42.331554] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_connected addr=lb:500514580worker2 # [ 42.334290] nix-grpc-daemon[854]: ts=2026-09-20T08:04:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4581worker1 # [ 42.502695] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514582worker1 # [ 42.505451] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4583worker2 # [ 42.534953] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514584worker2 # [ 42.537323] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4585worker1 # [ 42.706591] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514586worker1 # [ 42.709255] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4587worker2 # [ 42.738450] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514588worker2 # [ 42.743793] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4589worker1 # [ 42.910246] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514590worker1 # [ 42.915217] nix-grpc-daemon[782]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4591worker2 # [ 42.944797] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514592worker2 # [ 42.948801] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4593worker1 # [ 43.115840] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514594worker1 # [ 43.118374] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4595worker2 # [ 43.149396] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514596worker2 # [ 43.153276] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4597worker1 # [ 43.319355] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514598worker1 # [ 43.323960] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4599worker2 # [ 43.354400] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_connected addr=lb:500514600worker2 # [ 43.356988] nix-grpc-daemon[854]: ts=2026-09-20T08:04:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4601worker1 # [ 43.525062] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514602worker1 # [ 43.530237] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4603worker2 # [ 43.558095] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514604worker2 # [ 43.561484] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4605worker1 # [ 43.732874] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514606worker1 # [ 43.736833] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4607worker2 # [ 43.762396] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514608worker2 # [ 43.765747] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4609worker1 # [ 43.937553] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514610worker1 # [ 43.942503] nix-grpc-daemon[782]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4611worker2 # [ 43.966659] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514612worker2 # [ 43.970259] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4613worker1 # [ 44.143034] nix-grpc-daemon[782]: ts=2026-09-20T08:04:30Z level=info event=scheduler_connected addr=lb:500514614worker1 # [ 44.145617] nix-grpc-daemon[782]: ts=2026-09-20T08:04:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4615worker2 # [ 44.171089] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514616worker2 # [ 44.174260] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4617worker1 # [ 44.346558] nix-grpc-daemon[782]: ts=2026-09-20T08:04:30Z level=info event=scheduler_connected addr=lb:500514618worker1 # [ 44.351371] nix-grpc-daemon[782]: ts=2026-09-20T08:04:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4619lb # [ 44.356564] rustfs-bucket-start[1175]: mb s3://farm4620lb # [ 44.361846] systemd[1]: Finished rustfs-bucket.service.4621lb # [ 44.367845] systemd[1]: Starting niks3 server...4622worker2 # [ 44.376675] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_connected addr=lb:500514623worker2 # [ 44.379076] nix-grpc-daemon[854]: ts=2026-09-20T08:04:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4624worker1 # [ 44.552213] nix-grpc-daemon[782]: ts=2026-09-20T08:04:30Z level=info event=scheduler_connected addr=lb:500514625worker1 # [ 44.554724] nix-grpc-daemon[782]: ts=2026-09-20T08:04:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4626lb # [ 44.602700] postgres[1194]: [1194] ERROR: relation "goose_db_version" does not exist at character 364627lb # [ 44.604356] postgres[1194]: [1194] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4628lb # [ 44.644517] niks3-server[1185]: 2026/09/20 08:04:30 OK 20241026095416_initial_model.sql (23.22ms)4629lb # [ 44.647955] niks3-server[1185]: 2026/09/20 08:04:30 OK 20251210153512_drop_unused_gin_index.sql (3.79ms)4630lb # [ 44.653404] niks3-server[1185]: 2026/09/20 08:04:30 OK 20251218171726_add_pins.sql (3.91ms)4631lb # [ 44.658481] niks3-server[1185]: 2026/09/20 08:04:30 OK 20260628120000_add_object_size_and_stats.sql (5.13ms)4632lb # [ 44.662869] niks3-server[1185]: 2026/09/20 08:04:30 OK 20260905000000_add_claims.sql (2.97ms)4633lb # [ 44.666919] niks3-server[1185]: 2026/09/20 08:04:30 OK 20260920000000_drop_claims.sql (3.97ms)4634lb # [ 44.668067] niks3-server[1185]: 2026/09/20 08:04:30 goose: successfully migrated database to version: 202609200000004635lb # [ 44.672948] niks3-server[1185]: 2026/09/20 08:04:30 OK 1_commit_pending_closure.sql (5.96ms)4636lb # [ 44.675724] niks3-server[1185]: 2026/09/20 08:04:30 OK 2_object_stats_trigger.sql (2.78ms)4637lb # [ 44.677104] niks3-server[1185]: 2026/09/20 08:04:30 goose: up to current file version: 24638lb # [ 44.682362] niks3-server[1185]: 2026/09/20 08:04:30 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4639worker2 # [ 44.579789] nix-grpc-daemon[854]: ts=2026-09-20T08:04:30Z level=info event=scheduler_connected addr=lb:500514640worker2 # [ 44.582799] nix-grpc-daemon[854]: ts=2026-09-20T08:04:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4641worker1 # [ 44.755906] nix-grpc-daemon[782]: ts=2026-09-20T08:04:30Z level=info event=scheduler_connected addr=lb:500514642worker1 # [ 44.758863] nix-grpc-daemon[782]: ts=2026-09-20T08:04:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4643lb # [ 44.756689] niks3-server[1185]: 2026/09/20 08:04:30 INFO Read proxy enabled — serving cache objects from S34644lb # [ 44.758443] niks3-server[1185]: 2026/09/20 08:04:30 INFO Using socket-activated listener address=0.0.0.0:57514645lb # [ 44.765124] systemd[1]: Started niks3 server.4646lb # [ 44.766235] systemd[1]: Reached target Multi-User System.4647lb # [ 44.767733] systemd[1]: Startup finished in 924ms (kernel) + 4.067s (initrd) + 39.775s (userspace) = 44.767s.4648lb # [ 44.769870] niks3-server[1185]: 2026/09/20 08:04:30 INFO Starting HTTP server address=0.0.0.0:57514649lb # [ 44.776262] niks3-server[1185]: 2026/09/20 08:04:30 INFO systemd watchdog enabled interval=15s4650lb # [ 44.779903] niks3-server[1185]: 2026/09/20 08:04:30 INFO lead: acquired remote=192.168.1.4:456284651worker2 # [ 44.650840] nix-grpc-daemon[854]: ts=2026-09-20T08:04:30Z level=info event=scheduler_take_over4652worker2 # [ 44.783637] nix-grpc-daemon[854]: ts=2026-09-20T08:04:30Z level=info event=scheduler_connected addr=lb:500514653worker2 # [ 44.787471] nix-grpc-daemon[854]: ts=2026-09-20T08:04:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04654worker1 # [ 44.959841] nix-grpc-daemon[782]: ts=2026-09-20T08:04:30Z level=info event=scheduler_connected addr=lb:500514655worker2 # [ 44.818637] nix-grpc-daemon[854]: ts=2026-09-20T08:04:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04656lb: (finished: waiting for unit niks3.service, in 34.67 seconds)4657lb: waiting for TCP port 5751 on localhost4658lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4659lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)4660lb: waiting for unit envoy.service4661lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4662lb: waiting for TCP port 50051 on localhost4663lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4664lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4665lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4666??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4667 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284668lb: must succeed: curl -sf localhost:9901/clusters4669lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4670lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.07 seconds)4671subtest: one node schedules and both builders hold a WorkerSession on it4672lb: must succeed: curl -sf localhost:9901/clusters4673lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4674worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4675worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.06 seconds)4676(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.11 seconds)4677subtest: balancer auth: client cert, bearer token, nothing, unknown CN4678worker2 # [ 46.041441] nix-grpc-daemon[854]: ts=2026-09-20T08:04:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338504679worker2 # [ 46.048428] systemd[1]: Started Nix Daemon.4680worker2 # [ 46.269117] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4681worker2 # [ 46.313213] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5)4682worker2 # [ 46.348919] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 2/5)4683worker2 # [ 46.472809] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 159 ms (attempt 3/5)4684worker2 # [ 46.638113] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 435 ms (attempt 4/5)4685worker2 # [ 47.084232] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4686worker1 # [ 47.265640] nix-grpc-daemon[782]: ts=2026-09-20T08:04:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582804687worker1 # [ 47.279909] systemd[1]: Started Nix Daemon.4688worker1 # [ 47.489319] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4689worker1 # [ 47.515926] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)4690worker1 # [ 47.584459] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 2/5)4691worker1 # [ 47.607748] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 223 ms (attempt 3/5)4692worker1 # [ 47.836560] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 454 ms (attempt 4/5)4693worker1 # [ 48.296554] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4694worker1 # [ 48.313999] nix-grpc-daemon[782]: ts=2026-09-20T08:04:34Z level=debug event=path_info_miss path=pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test error="error: path '/nix/store/pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test' is required, but there is no substituter that can build it"4695worker2 # [ 48.180570] nix-grpc-daemon[854]: ts=2026-09-20T08:04:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:338504696worker2 # [ 48.196259] nix-grpc-daemon[854]: ts=2026-09-20T08:04:33Z level=debug event=path_info_miss path=pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test error="error: path '/nix/store/pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test' is required, but there is no substituter that can build it"4697client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4698client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)4699client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4700lb # [ 48.495333] envoy[924]: conn peer=[::1]:50294 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24701lb # [ 48.497171] envoy[924]: conn peer=[2001:db8:1::1]:41162 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=22044702client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4703worker2 # [ 48.567887] nix-grpc-daemon[854]: ts=2026-09-20T08:04:34Z level=info event=oidc_jwks provider=mock keys=14704worker2 # [ 48.572985] nix-grpc-daemon[854]: ts=2026-09-20T08:04:34Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:338504705worker2 # [ 48.585872] nix-grpc-daemon[854]: ts=2026-09-20T08:04:34Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:338504706worker2 # [ 48.593982] nix-grpc-daemon[854]: ts=2026-09-20T08:04:34Z level=debug event=path_info_miss path=pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test error="error: path '/nix/store/pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test' is required, but there is no substituter that can build it"4707worker2 # [ 48.603186] nix-grpc-daemon[854]: ts=2026-09-20T08:04:34Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:338504708worker2 # [ 48.611284] nix-grpc-daemon[854]: ts=2026-09-20T08:04:34Z level=debug event=path_info_miss path=pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test error="error: path '/nix/store/pkvyx00qdzg2xbr96azisdpjk46xfshp-nixos-system-client-test' is required, but there is no substituter that can build it"4709client: must succeed: rm -r /var/lib/nix-grpc-store4710client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4711worker2 # [ 50.863457] nix-grpc-daemon[854]: ts=2026-09-20T08:04:36Z level=info event=denied method=QueryMissing cn=- role=none4712worker1 # [ 51.202946] nix-grpc-daemon[782]: ts=2026-09-20T08:04:37Z level=info event=denied method=QueryMissing cn=stranger role=none4713(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.32 seconds)4714subtest: DAG build is scheduled across workers and lands in the cache4715client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 >&24716worker2 # [ 51.298546] nix-grpc-daemon[854]: ts=2026-09-20T08:04:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338504717client # don't know how to build these paths:4718client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4719worker2 # [ 51.332093] nix-grpc-daemon[854]: ts=2026-09-20T08:04:36Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04720worker2 # [ 51.336984] nix-grpc-daemon[854]: ts=2026-09-20T08:04:36Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.4:50051 clients=14721worker2 # [ 51.345471] nix-grpc-daemon[854]: ts=2026-09-20T08:04:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338504722worker2 # [ 51.372601] nix-grpc-daemon[854]: ts=2026-09-20T08:04:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338504723client # copying 1 paths to 'lb:50051'...4724worker2 # [ 51.387196] nix-grpc-daemon[854]: ts=2026-09-20T08:04:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338504725client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4726worker2 # [ 51.413183] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4727worker2 # [ 51.437644] nix-grpc-daemon[854]: ts=2026-09-20T08:04:36Z level=info event=niks3_push_started pid=9344728worker2 # [ 51.885463] nix-daemon[923]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4729worker2 # [ 51.902690] nix-daemon[923]: reaped child process 949, status = succeeded4730worker2 # [ 52.065508] nix-daemon[923]: accepted connection from pid 952, user nix-grpc-daemon (trusted)4731worker2 # [ 52.080086] nix-daemon[923]: reaped child process 955, status = succeeded4732lb # [ 52.229465] niks3-server[1185]: 2026/09/20 08:04:38 INFO Received uploads request method=POST path=/api/pending_closures4733worker2 # [ 52.133684] nix-grpc-daemon[934]: time=2026-09-20T08:04:37.637Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4734worker2 # [ 52.137209] nix-grpc-daemon[934]: time=2026-09-20T08:04:37.641Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4735lb # [ 52.323308] niks3-server[1185]: 2026/09/20 08:04:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4736lb # [ 52.331426] niks3-server[1185]: 2026/09/20 08:04:38 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4737lb # [ 52.340473] niks3-server[1185]: 2026/09/20 08:04:38 INFO Signed narinfos id=1 count=14738worker2 # [ 52.205886] nix-grpc-daemon[934]: time=2026-09-20T08:04:37.710Z level=INFO msg="Uploading 1 narinfos"4739lb # [ 52.350400] niks3-server[1185]: 2026/09/20 08:04:38 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4740lb # [ 52.357086] niks3-server[1185]: 2026/09/20 08:04:38 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4741lb # [ 52.367714] niks3-server[1185]: 2026/09/20 08:04:38 INFO Completed upload id=14742worker2 # [ 52.231626] nix-grpc-daemon[934]: time=2026-09-20T08:04:37.736Z level=INFO msg="Upload complete. (508ms)"4743lb # [ 52.375666] niks3-server[1185]: 2026/09/20 08:04:38 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4744worker2 # [ 52.238195] nix-grpc-daemon[854]: ts=2026-09-20T08:04:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 paths=1 nar_bytes_in=5044745worker2 # [ 52.246113] nix-grpc-daemon[854]: ts=2026-09-20T08:04:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338504746worker2 # [ 52.256738] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4747worker2 # [ 52.265501] nix-daemon[923]: reaped child process 932, status = succeeded4748client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4749client # farm-a> worker2: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4750worker2 # [ 52.276598] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4751client # farm-a> LOG-farm-a-t14752worker2 # [ 53.494491] nix-daemon[923]: reaped child process 959, status = succeeded4753worker2 # [ 53.635769] nix-daemon[923]: accepted connection from pid 974, user nix-grpc-daemon (trusted)4754worker2 # [ 53.649441] nix-daemon[923]: reaped child process 977, status = succeeded4755lb # [ 53.798871] niks3-server[1185]: 2026/09/20 08:04:39 INFO Received uploads request method=POST path=/api/pending_closures4756worker2 # [ 53.686657] nix-grpc-daemon[934]: time=2026-09-20T08:04:39.191Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4757worker2 # [ 53.688287] nix-grpc-daemon[934]: time=2026-09-20T08:04:39.193Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4758lb # [ 53.850552] niks3-server[1185]: 2026/09/20 08:04:39 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4759lb # [ 53.855577] niks3-server[1185]: 2026/09/20 08:04:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4760lb # [ 53.858247] niks3-server[1185]: 2026/09/20 08:04:39 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4761lb # [ 53.860676] niks3-server[1185]: 2026/09/20 08:04:39 INFO Signed narinfos id=2 count=14762worker2 # [ 53.724483] nix-grpc-daemon[934]: time=2026-09-20T08:04:39.229Z level=INFO msg="Uploading 1 narinfos"4763lb # [ 53.865470] niks3-server[1185]: 2026/09/20 08:04:39 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4764lb # [ 53.869063] niks3-server[1185]: 2026/09/20 08:04:39 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4765lb # [ 53.876476] niks3-server[1185]: 2026/09/20 08:04:39 INFO Completed upload id=24766worker2 # [ 53.740171] nix-grpc-daemon[934]: time=2026-09-20T08:04:39.245Z level=INFO msg="Upload complete. (251ms)"4767lb # [ 53.879282] niks3-server[1185]: 2026/09/20 08:04:39 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4768worker2 # [ 53.744074] nix-grpc-daemon[854]: ts=2026-09-20T08:04:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14769worker2 # [ 53.754032] nix-daemon[923]: reaped child process 957, status = succeeded4770worker2 # [ 53.758358] nix-grpc-daemon[854]: ts=2026-09-20T08:04:39Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04771worker2 # [ 53.760424] nix-grpc-daemon[854]: ts=2026-09-20T08:04:39Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.4:50051 clients=14772worker2 # [ 53.802577] nix-grpc-daemon[854]: ts=2026-09-20T08:04:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338504773worker2 # [ 53.811601] nix-grpc-daemon[854]: ts=2026-09-20T08:04:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338504774client # copying 1 paths to 'lb:50051'...4775worker2 # [ 53.821552] nix-grpc-daemon[934]: time=2026-09-20T08:04:39.326Z level=INFO msg="All 1 paths already cached"4776worker2 # [ 53.825399] nix-grpc-daemon[854]: ts=2026-09-20T08:04:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338504777client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4778worker2 # [ 53.845137] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4779worker2 # [ 54.016421] nix-daemon[923]: accepted connection from pid 983, user nix-grpc-daemon (trusted)4780worker2 # [ 54.030250] nix-daemon[923]: reaped child process 986, status = succeeded4781worker2 # [ 54.189140] nix-daemon[923]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4782worker2 # [ 54.201817] nix-daemon[923]: reaped child process 991, status = succeeded4783lb # [ 54.352943] niks3-server[1185]: 2026/09/20 08:04:40 INFO Received uploads request method=POST path=/api/pending_closures4784worker2 # [ 54.220967] nix-grpc-daemon[934]: time=2026-09-20T08:04:39.726Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4785worker2 # [ 54.222527] nix-grpc-daemon[934]: time=2026-09-20T08:04:39.727Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4786lb # [ 54.371452] niks3-server[1185]: 2026/09/20 08:04:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4787lb # [ 54.374745] niks3-server[1185]: 2026/09/20 08:04:40 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4788lb # [ 54.377070] niks3-server[1185]: 2026/09/20 08:04:40 INFO Signed narinfos id=3 count=14789worker2 # [ 54.240501] nix-grpc-daemon[934]: time=2026-09-20T08:04:39.745Z level=INFO msg="Uploading 1 narinfos"4790lb # [ 54.381285] niks3-server[1185]: 2026/09/20 08:04:40 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4791lb # [ 54.384052] niks3-server[1185]: 2026/09/20 08:04:40 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4792lb # [ 54.390420] niks3-server[1185]: 2026/09/20 08:04:40 INFO Completed upload id=34793worker2 # [ 54.253814] nix-grpc-daemon[934]: time=2026-09-20T08:04:39.758Z level=INFO msg="Upload complete. (397ms)"4794lb # [ 54.391703] niks3-server[1185]: 2026/09/20 08:04:40 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4795worker2 # [ 54.255882] nix-grpc-daemon[854]: ts=2026-09-20T08:04:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 paths=1 nar_bytes_in=6244796worker2 # [ 54.259224] nix-grpc-daemon[854]: ts=2026-09-20T08:04:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338504797worker2 # [ 54.265792] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4798worker2 # [ 54.268211] nix-daemon[923]: reaped child process 981, status = succeeded4799worker2 # [ 54.273921] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4800client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4801client # farm-b> worker2: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4802client # farm-b> LOG-farm-b-t14803worker2 # [ 55.089456] nix-daemon[923]: reaped child process 995, status = succeeded4804worker2 # [ 55.204260] nix-daemon[923]: accepted connection from pid 1010, user nix-grpc-daemon (trusted)4805worker2 # [ 55.217174] nix-daemon[923]: reaped child process 1013, status = succeeded4806lb # [ 55.368971] niks3-server[1185]: 2026/09/20 08:04:41 INFO Received uploads request method=POST path=/api/pending_closures4807worker2 # [ 55.241916] nix-grpc-daemon[934]: time=2026-09-20T08:04:40.746Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4808worker2 # [ 55.243885] nix-grpc-daemon[934]: time=2026-09-20T08:04:40.748Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4809lb # [ 55.405760] niks3-server[1185]: 2026/09/20 08:04:41 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4810lb # [ 55.424124] niks3-server[1185]: 2026/09/20 08:04:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4811lb # [ 55.430640] niks3-server[1185]: 2026/09/20 08:04:41 INFO Signed narinfos id=4 count=14812worker2 # [ 55.298193] nix-grpc-daemon[934]: time=2026-09-20T08:04:40.803Z level=INFO msg="Uploading 1 narinfos"4813lb # [ 55.435490] niks3-server[1185]: 2026/09/20 08:04:41 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4814lb # [ 55.441509] niks3-server[1185]: 2026/09/20 08:04:41 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4815lb # [ 55.448061] niks3-server[1185]: 2026/09/20 08:04:41 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4816lb # [ 55.452277] niks3-server[1185]: 2026/09/20 08:04:41 INFO Completed upload id=44817lb # [ 55.454091] niks3-server[1185]: 2026/09/20 08:04:41 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4818worker2 # [ 55.315720] nix-grpc-daemon[934]: time=2026-09-20T08:04:40.820Z level=INFO msg="Upload complete. (235ms)"4819worker2 # [ 55.325320] nix-grpc-daemon[854]: ts=2026-09-20T08:04:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14820worker2 # [ 55.333772] nix-grpc-daemon[854]: ts=2026-09-20T08:04:40Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04821worker2 # [ 55.338733] nix-grpc-daemon[854]: ts=2026-09-20T08:04:40Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.4:50051 clients=14822worker2 # [ 55.343333] nix-grpc-daemon[854]: ts=2026-09-20T08:04:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338504823worker2 # [ 55.349394] nix-daemon[923]: reaped child process 993, status = succeeded4824worker2 # [ 55.354267] nix-grpc-daemon[854]: ts=2026-09-20T08:04:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338504825worker2 # [ 55.362429] nix-grpc-daemon[934]: time=2026-09-20T08:04:40.867Z level=INFO msg="All 2 paths already cached"4826client # copying 1 paths to 'lb:50051'...4827worker2 # [ 55.368492] nix-grpc-daemon[854]: ts=2026-09-20T08:04:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338504828client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4829worker2 # [ 55.387170] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4830worker2 # [ 55.514583] nix-daemon[923]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4831worker2 # [ 55.528118] nix-daemon[923]: reaped child process 1022, status = succeeded4832worker2 # [ 55.658974] nix-daemon[923]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4833worker2 # [ 55.672535] nix-daemon[923]: reaped child process 1027, status = succeeded4834lb # [ 55.819741] niks3-server[1185]: 2026/09/20 08:04:41 INFO Received uploads request method=POST path=/api/pending_closures4835worker2 # [ 55.691323] nix-grpc-daemon[934]: time=2026-09-20T08:04:41.196Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4836worker2 # [ 55.692904] nix-grpc-daemon[934]: time=2026-09-20T08:04:41.197Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4837lb # [ 55.851248] niks3-server[1185]: 2026/09/20 08:04:41 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4838lb # [ 55.862792] niks3-server[1185]: 2026/09/20 08:04:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4839worker2 # [ 55.733210] nix-grpc-daemon[934]: time=2026-09-20T08:04:41.238Z level=INFO msg="Uploading 1 narinfos"4840lb # [ 55.869925] niks3-server[1185]: 2026/09/20 08:04:41 INFO Signed narinfos id=5 count=14841lb # [ 55.878657] niks3-server[1185]: 2026/09/20 08:04:41 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4842lb # [ 55.885955] niks3-server[1185]: 2026/09/20 08:04:41 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4843lb # [ 55.890798] niks3-server[1185]: 2026/09/20 08:04:41 INFO Completed upload id=54844worker2 # [ 55.756435] nix-grpc-daemon[934]: time=2026-09-20T08:04:41.261Z level=INFO msg="Upload complete. (361ms)"4845lb # [ 55.895430] niks3-server[1185]: 2026/09/20 08:04:41 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4846worker2 # [ 55.760587] nix-grpc-daemon[854]: ts=2026-09-20T08:04:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 paths=1 nar_bytes_in=7524847worker2 # [ 55.766250] nix-grpc-daemon[854]: ts=2026-09-20T08:04:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338504848worker2 # [ 55.776882] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4849worker2 # [ 55.781142] nix-daemon[923]: reaped child process 1017, status = succeeded4850worker2 # [ 55.786761] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)4851client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4852client # farm-top> worker2: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4853worker2 # [ 56.400822] nix-daemon[923]: reaped child process 1031, status = succeeded4854client # farm-top> LOG-farm-top-t14855worker2 # [ 56.542695] nix-daemon[923]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4856worker2 # [ 56.556245] nix-daemon[923]: reaped child process 1049, status = succeeded4857lb # [ 56.703403] niks3-server[1185]: 2026/09/20 08:04:42 INFO Received uploads request method=POST path=/api/pending_closures4858worker2 # [ 56.579627] nix-grpc-daemon[934]: time=2026-09-20T08:04:42.084Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4859worker2 # [ 56.581141] nix-grpc-daemon[934]: time=2026-09-20T08:04:42.086Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4860lb # [ 56.750537] niks3-server[1185]: 2026/09/20 08:04:42 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4861lb # [ 56.758134] niks3-server[1185]: 2026/09/20 08:04:42 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4862lb # [ 56.765303] niks3-server[1185]: 2026/09/20 08:04:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4863lb # [ 56.769619] niks3-server[1185]: 2026/09/20 08:04:42 INFO Signed narinfos id=6 count=14864worker2 # [ 56.635860] nix-grpc-daemon[934]: time=2026-09-20T08:04:42.140Z level=INFO msg="Uploading 1 narinfos"4865lb # [ 56.779080] niks3-server[1185]: 2026/09/20 08:04:42 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4866lb # [ 56.781346] niks3-server[1185]: 2026/09/20 08:04:42 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4867lb # [ 56.785285] niks3-server[1185]: 2026/09/20 08:04:42 INFO Completed upload id=64868lb # [ 56.786426] niks3-server[1185]: 2026/09/20 08:04:42 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4869worker2 # [ 56.648927] nix-grpc-daemon[934]: time=2026-09-20T08:04:42.153Z level=INFO msg="Upload complete. (254ms)"4870worker2 # [ 56.656489] nix-grpc-daemon[854]: ts=2026-09-20T08:04:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14871worker2 # [ 56.667442] nix-daemon[923]: reaped child process 1029, status = succeeded4872client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 >&2, in 5.62 seconds)4873client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 outPath4874client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 outPath, in 0.18 seconds)4875client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4876client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4877client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4878client # copying 3 paths...4879client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4880client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4881client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4882client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.23 seconds)4883(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.13 seconds)4884subtest: repeat is answered Cached by the scheduler without building4885worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4886worker1 # finding garbage collector roots...4887worker1 # deleting specified paths...4888worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4889worker1 # deleting unused links...4890worker1 # note: hard linking is currently saving 0.0 KiB4891worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.22 seconds)4892worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4893worker2 # finding garbage collector roots...4894worker2 # deleting specified paths...4895worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4896worker2 # deleting unused links...4897worker2 # note: hard linking is currently saving 0.0 KiB4898worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4899client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 >&24900worker2 # [ 57.866567] nix-grpc-daemon[854]: ts=2026-09-20T08:04:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338504901client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4902client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4903worker2 # [ 57.889736] nix-grpc-daemon[854]: ts=2026-09-20T08:04:43Z level=debug event=want client=2 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=14904worker2 # [ 57.900755] nix-grpc-daemon[854]: ts=2026-09-20T08:04:43Z level=debug event=want client=2 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=14905worker2 # [ 57.911009] nix-grpc-daemon[854]: ts=2026-09-20T08:04:43Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14906client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 >&2, in 0.24 seconds)4907client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 outPath4908client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag t1 outPath, in 0.15 seconds)4909lb: must succeed: curl -sf localhost:9901/clusters4910lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4911lb: must succeed: curl -sf localhost:9901/clusters4912lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4913worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true4914worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true, in 0.07 seconds)4915(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.12 seconds)4916subtest: two clients wanting the same drv build it once4917client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4918lb # [ 58.496678] envoy[924]: conn peer=[2001:db8:1::1]:41176 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=363 ms=714919lb # [ 58.500782] envoy[924]: conn peer=[2001:db8:1::1]:41184 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=94920lb # [ 58.504644] envoy[924]: conn peer=[2001:db8:1::1]:41196 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=64921client # [ 58.526233] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4922client # Running as unit: dedup1.service; invocation ID: 903523192a0a4d3d805cc2d5c926681c4923lb # [ 58.508652] envoy[924]: conn peer=[2001:db8:1::1]:41210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9381 tx=3783 ms=54134924client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.04 seconds)4925lb # [ 58.512709] envoy[924]: conn peer=[2001:db8:1::1]:54822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1158 tx=481 ms=594926client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4927worker2 # [ 58.568709] nix-grpc-daemon[854]: ts=2026-09-20T08:04:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338504928client # [ 58.733380] nix[877]: don't know how to build these paths:4929client # [ 58.734393] nix[877]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4930worker2 # [ 58.584279] nix-grpc-daemon[854]: ts=2026-09-20T08:04:44Z level=debug event=want client=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04931worker2 # [ 58.589626] nix-grpc-daemon[854]: ts=2026-09-20T08:04:44Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14932worker1 # [ 58.778474] nix-grpc-daemon[782]: ts=2026-09-20T08:04:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582804933worker1 # [ 58.790378] nix-grpc-daemon[782]: ts=2026-09-20T08:04:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582804934client # [ 58.813279] nix[877]: copying 1 paths to 'lb:50051'...4935worker1 # [ 58.798837] nix-grpc-daemon[782]: ts=2026-09-20T08:04:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582804936client # [ 58.815223] nix[877]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4937worker1 # [ 58.810367] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4938worker1 # [ 58.823309] nix-grpc-daemon[782]: ts=2026-09-20T08:04:44Z level=info event=niks3_push_started pid=8684939worker1 # [ 59.233991] nix-daemon[846]: accepted connection from pid 879, user nix-grpc-daemon (trusted)4940worker1 # [ 59.250721] nix-daemon[846]: reaped child process 883, status = succeeded4941worker1 # [ 59.373340] nix-daemon[846]: accepted connection from pid 885, user nix-grpc-daemon (trusted)4942worker1 # [ 59.386560] nix-daemon[846]: reaped child process 888, status = succeeded4943lb # [ 59.394347] niks3-server[1185]: 2026/09/20 08:04:45 INFO Received uploads request method=POST path=/api/pending_closures4944worker1 # [ 59.410317] nix-grpc-daemon[868]: time=2026-09-20T08:04:45.357Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4945worker1 # [ 59.413467] nix-grpc-daemon[868]: time=2026-09-20T08:04:45.360Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4946lb # [ 59.448413] niks3-server[1185]: 2026/09/20 08:04:45 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4947lb # [ 59.454953] niks3-server[1185]: 2026/09/20 08:04:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4948lb # [ 59.458537] niks3-server[1185]: 2026/09/20 08:04:45 INFO Signed narinfos id=7 count=14949worker1 # [ 59.467512] nix-grpc-daemon[868]: time=2026-09-20T08:04:45.415Z level=INFO msg="Uploading 1 narinfos"4950lb # [ 59.472489] niks3-server[1185]: 2026/09/20 08:04:45 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4951lb # [ 59.474450] niks3-server[1185]: 2026/09/20 08:04:45 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4952lb # [ 59.483126] niks3-server[1185]: 2026/09/20 08:04:45 INFO Completed upload id=74953worker1 # [ 59.489151] nix-grpc-daemon[868]: time=2026-09-20T08:04:45.436Z level=INFO msg="Upload complete. (412ms)"4954worker1 # [ 59.491119] nix-grpc-daemon[782]: ts=2026-09-20T08:04:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=0 paths=1 nar_bytes_in=4324955lb # [ 59.488719] niks3-server[1185]: 2026/09/20 08:04:45 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4956worker1 # [ 59.498438] nix-grpc-daemon[782]: ts=2026-09-20T08:04:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582804957worker1 # [ 59.503078] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4958worker1 # [ 59.505735] nix-daemon[846]: reaped child process 866, status = succeeded4959worker1 # [ 59.511988] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4960client # [ 59.526282] nix[877]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4961client # [ 59.596261] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4962client # Running as unit: dedup2.service; invocation ID: 120bd3bd67094e45992851e150b82cf44963client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.08 seconds)4964worker2 # [ 59.670099] nix-grpc-daemon[854]: ts=2026-09-20T08:04:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:338504965client # [ 59.831845] nix[893]: don't know how to build these paths:4966client # [ 59.832889] nix[893]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4967worker2 # [ 59.681720] nix-grpc-daemon[854]: ts=2026-09-20T08:04:45Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04968worker1 # [ 59.830605] nix-grpc-daemon[782]: ts=2026-09-20T08:04:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582804969client # [ 59.850403] nix[893]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4970client: must succeed: sleep 24971client: (finished: must succeed: sleep 2, in 2.03 seconds)4972??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4973 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394974client: waiting for success: ! systemctl is-active dedup1 dedup24975??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4976 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394977worker1 # [ 63.129130] nix-daemon[846]: reaped child process 892, status = succeeded4978worker1 # [ 63.277108] nix-daemon[846]: accepted connection from pid 921, user nix-grpc-daemon (trusted)4979worker1 # [ 63.291449] nix-daemon[846]: reaped child process 924, status = succeeded4980lb # [ 63.298884] niks3-server[1185]: 2026/09/20 08:04:49 INFO Received uploads request method=POST path=/api/pending_closures4981worker1 # [ 63.309211] nix-grpc-daemon[868]: time=2026-09-20T08:04:49.256Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4982worker1 # [ 63.310670] nix-grpc-daemon[868]: time=2026-09-20T08:04:49.258Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4983lb # [ 63.325060] niks3-server[1185]: 2026/09/20 08:04:49 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4984lb # [ 63.328667] niks3-server[1185]: 2026/09/20 08:04:49 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4985lb # [ 63.332484] niks3-server[1185]: 2026/09/20 08:04:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4986lb # [ 63.334101] niks3-server[1185]: 2026/09/20 08:04:49 INFO Signed narinfos id=8 count=14987worker1 # [ 63.341220] nix-grpc-daemon[868]: time=2026-09-20T08:04:49.289Z level=INFO msg="Uploading 1 narinfos"4988lb # [ 63.337184] niks3-server[1185]: 2026/09/20 08:04:49 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4989lb # [ 63.342625] niks3-server[1185]: 2026/09/20 08:04:49 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4990lb # [ 63.346290] niks3-server[1185]: 2026/09/20 08:04:49 INFO Completed upload id=84991worker1 # [ 63.352184] nix-grpc-daemon[868]: time=2026-09-20T08:04:49.300Z level=INFO msg="Upload complete. (228ms)"4992lb # [ 63.347759] niks3-server[1185]: 2026/09/20 08:04:49 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4993worker1 # [ 63.354812] nix-grpc-daemon[782]: ts=2026-09-20T08:04:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=4 outputs=14994worker1 # [ 63.358148] nix-grpc-daemon[782]: ts=2026-09-20T08:04:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=4 outputs=14995worker1 # [ 63.364403] nix-daemon[846]: reaped child process 890, status = succeeded4996client # [ 63.407974] systemd[1]: dedup2.service: Deactivated successfully.4997client # [ 63.411167] systemd[1]: dedup2.service: Consumed 122ms CPU time over 3.815s wall clock time, 33.1M memory peak, 7.4K incoming IP traffic, 6.4K outgoing IP traffic.4998client # [ 63.416215] systemd[1]: dedup1.service: Deactivated successfully.4999client # [ 63.417738] systemd[1]: dedup1.service: Consumed 107ms CPU time over 4.891s wall clock time, 37.3M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.5000client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.07 seconds)5001client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5002client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5003worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5004worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)5005worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5006worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)5007worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true5008worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.05 seconds)5009worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true5010worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.05 seconds)5011(finished: subtest: two clients wanting the same drv build it once, in 5.80 seconds)5012subtest: build hook: nix-daemon with builders = grpc://lb5013client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag hook 2>/tmp/hook.log5014client # [ 64.436733] systemd[1]: Started Nix Daemon.5015client # [ 64.586140] nix-daemon[935]: accepted connection from pid 932, user root (trusted)5016worker2 # [ 64.595662] nix-grpc-daemon[854]: ts=2026-09-20T08:04:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:338505017worker1 # [ 64.744349] nix-grpc-daemon[782]: ts=2026-09-20T08:04:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582805018worker2 # [ 64.609046] nix-grpc-daemon[854]: ts=2026-09-20T08:04:50Z level=debug event=want client=5 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05019worker2 # [ 64.611779] nix-grpc-daemon[854]: ts=2026-09-20T08:04:50Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15020worker2 # [ 64.614731] nix-grpc-daemon[854]: ts=2026-09-20T08:04:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338505021worker2 # [ 64.618308] nix-grpc-daemon[854]: ts=2026-09-20T08:04:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338505022worker2 # [ 64.623478] nix-grpc-daemon[854]: ts=2026-09-20T08:04:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338505023worker2 # [ 64.627470] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5024worker2 # [ 64.793574] nix-daemon[923]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5025worker2 # [ 64.809412] nix-daemon[923]: reaped child process 1109, status = succeeded5026worker2 # [ 64.966234] nix-daemon[923]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5027worker2 # [ 64.979860] nix-daemon[923]: reaped child process 1114, status = succeeded5028lb # [ 65.127721] niks3-server[1185]: 2026/09/20 08:04:51 INFO Received uploads request method=POST path=/api/pending_closures5029worker2 # [ 64.999961] nix-grpc-daemon[934]: time=2026-09-20T08:04:50.504Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5030worker2 # [ 65.001685] nix-grpc-daemon[934]: time=2026-09-20T08:04:50.506Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5031lb # [ 65.157826] niks3-server[1185]: 2026/09/20 08:04:51 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5032lb # [ 65.168415] niks3-server[1185]: 2026/09/20 08:04:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign5033lb # [ 65.173829] niks3-server[1185]: 2026/09/20 08:04:51 INFO Signed narinfos id=9 count=15034worker2 # [ 65.040245] nix-grpc-daemon[934]: time=2026-09-20T08:04:50.545Z level=INFO msg="Uploading 1 narinfos"5035lb # [ 65.183610] niks3-server[1185]: 2026/09/20 08:04:51 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5036lb # [ 65.191826] niks3-server[1185]: 2026/09/20 08:04:51 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete5037lb # [ 65.198348] niks3-server[1185]: 2026/09/20 08:04:51 INFO Completed upload id=95038worker2 # [ 65.061912] nix-grpc-daemon[934]: time=2026-09-20T08:04:50.566Z level=INFO msg="Upload complete. (427ms)"5039lb # [ 65.200379] niks3-server[1185]: 2026/09/20 08:04:51 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5040worker2 # [ 65.065560] nix-grpc-daemon[854]: ts=2026-09-20T08:04:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 paths=1 nar_bytes_in=5125041worker2 # [ 65.071343] nix-grpc-daemon[854]: ts=2026-09-20T08:04:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338505042worker2 # [ 65.079589] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5043worker2 # [ 65.085737] nix-daemon[923]: reaped child process 1104, status = succeeded5044worker2 # [ 65.089833] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5045worker2 # [ 66.314664] nix-daemon[923]: reaped child process 1118, status = succeeded5046worker2 # [ 66.463041] nix-daemon[923]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5047worker2 # [ 66.474219] nix-daemon[923]: reaped child process 1136, status = succeeded5048lb # [ 66.625652] niks3-server[1185]: 2026/09/20 08:04:52 INFO Received uploads request method=POST path=/api/pending_closures5049worker2 # [ 66.496571] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.001Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5050worker2 # [ 66.498412] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.003Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5051lb # [ 66.664218] niks3-server[1185]: 2026/09/20 08:04:52 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5052lb # [ 66.678534] niks3-server[1185]: 2026/09/20 08:04:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign5053lb # [ 66.683522] niks3-server[1185]: 2026/09/20 08:04:52 INFO Signed narinfos id=10 count=15054worker2 # [ 66.550317] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.055Z level=INFO msg="Uploading 1 narinfos"5055lb # [ 66.690985] niks3-server[1185]: 2026/09/20 08:04:52 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5056lb # [ 66.695492] niks3-server[1185]: 2026/09/20 08:04:52 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5057lb # [ 66.705487] niks3-server[1185]: 2026/09/20 08:04:52 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete5058lb # [ 66.711554] niks3-server[1185]: 2026/09/20 08:04:52 INFO Completed upload id=105059worker2 # [ 66.575256] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.080Z level=INFO msg="Upload complete. (278ms)"5060lb # [ 66.713964] niks3-server[1185]: 2026/09/20 08:04:52 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5061worker1 # [ 66.726576] nix-grpc-daemon[782]: ts=2026-09-20T08:04:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582805062worker2 # [ 66.582623] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=5 outputs=15063worker1 # [ 66.734502] nix-grpc-daemon[782]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5064worker2 # [ 66.594841] nix-daemon[923]: reaped child process 1116, status = succeeded5065worker1 # [ 66.761365] nix-grpc-daemon[782]: ts=2026-09-20T08:04:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:582805066worker1 # [ 66.766981] nix-grpc-daemon[782]: ts=2026-09-20T08:04:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=0 paths=1 nar_bytes_out=1285067worker2 # [ 66.901648] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:338505068worker2 # [ 66.907835] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338505069worker2 # [ 66.913234] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.417Z level=INFO msg="All 1 paths already cached"5070worker2 # [ 66.923195] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=debug event=want client=6 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05071worker2 # [ 66.928585] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15072worker2 # [ 66.933991] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338505073worker2 # [ 66.945648] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338505074worker2 # [ 66.957713] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.462Z level=INFO msg="All 1 paths already cached"5075worker2 # [ 66.964509] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338505076worker2 # [ 66.972800] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5077worker2 # [ 67.128191] nix-daemon[923]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)5078worker2 # [ 67.146278] nix-daemon[923]: reaped child process 1145, status = succeeded5079worker2 # [ 67.290031] nix-daemon[923]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5080worker2 # [ 67.303801] nix-daemon[923]: reaped child process 1150, status = succeeded5081lb # [ 67.451731] niks3-server[1185]: 2026/09/20 08:04:53 INFO Received uploads request method=POST path=/api/pending_closures5082worker2 # [ 67.323772] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.828Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5083worker2 # [ 67.325775] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.828Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5084lb # [ 67.482285] niks3-server[1185]: 2026/09/20 08:04:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign5085worker2 # [ 67.353234] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.858Z level=INFO msg="Uploading 1 narinfos"5086lb # [ 67.491136] niks3-server[1185]: 2026/09/20 08:04:53 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5087lb # [ 67.496175] niks3-server[1185]: 2026/09/20 08:04:53 INFO Signed narinfos id=11 count=15088lb # [ 67.507272] niks3-server[1185]: 2026/09/20 08:04:53 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5089lb # [ 67.513221] niks3-server[1185]: 2026/09/20 08:04:53 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete5090worker2 # [ 67.383678] nix-grpc-daemon[934]: time=2026-09-20T08:04:52.888Z level=INFO msg="Upload complete. (398ms)"5091lb # [ 67.520234] niks3-server[1185]: 2026/09/20 08:04:53 INFO Completed upload id=115092worker2 # [ 67.386442] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 paths=1 nar_bytes_in=6245093lb # [ 67.523712] niks3-server[1185]: 2026/09/20 08:04:53 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5094worker2 # [ 67.391214] nix-grpc-daemon[854]: ts=2026-09-20T08:04:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338505095worker2 # [ 67.400371] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5096worker2 # [ 67.401960] nix-daemon[923]: reaped child process 1140, status = succeeded5097worker2 # [ 67.408632] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5098lb # [ 68.496371] envoy[924]: conn peer=[2001:db8:1::1]:54844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1085 ms=35595099lb # [ 68.500206] envoy[924]: conn peer=[2001:db8:1::1]:54834 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1329 ms=46615100lb # [ 68.503755] envoy[924]: conn peer=[2001:db8:1::1]:37750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=333 ms=555101lb # [ 68.508484] envoy[924]: conn peer=[2001:db8:1::1]:37734 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3423 tx=1750 ms=20815102worker2 # [ 69.025512] nix-daemon[923]: reaped child process 1154, status = succeeded5103worker2 # [ 69.190304] nix-daemon[923]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5104worker2 # [ 69.203225] nix-daemon[923]: reaped child process 1172, status = succeeded5105lb # [ 69.357193] niks3-server[1185]: 2026/09/20 08:04:55 INFO Received uploads request method=POST path=/api/pending_closures5106worker2 # [ 69.226089] nix-grpc-daemon[934]: time=2026-09-20T08:04:54.730Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5107worker2 # [ 69.227582] nix-grpc-daemon[934]: time=2026-09-20T08:04:54.732Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5108lb # [ 69.383342] niks3-server[1185]: 2026/09/20 08:04:55 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5109lb # [ 69.387362] niks3-server[1185]: 2026/09/20 08:04:55 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5110lb # [ 69.391100] niks3-server[1185]: 2026/09/20 08:04:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5111lb # [ 69.392927] niks3-server[1185]: 2026/09/20 08:04:55 INFO Signed narinfos id=12 count=15112worker2 # [ 69.257941] nix-grpc-daemon[934]: time=2026-09-20T08:04:54.762Z level=INFO msg="Uploading 1 narinfos"5113lb # [ 69.395563] niks3-server[1185]: 2026/09/20 08:04:55 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5114lb # [ 69.401668] niks3-server[1185]: 2026/09/20 08:04:55 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5115lb # [ 69.404824] niks3-server[1185]: 2026/09/20 08:04:55 INFO Completed upload id=125116worker2 # [ 69.268204] nix-grpc-daemon[934]: time=2026-09-20T08:04:54.773Z level=INFO msg="Upload complete. (250ms)"5117lb # [ 69.406474] niks3-server[1185]: 2026/09/20 08:04:55 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5118worker2 # [ 69.270515] nix-grpc-daemon[854]: ts=2026-09-20T08:04:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=6 outputs=15119worker2 # [ 69.277566] nix-daemon[923]: reaped child process 1152, status = succeeded5120worker2 # [ 69.279093] nix-grpc-daemon[854]: ts=2026-09-20T08:04:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:338505121worker2 # [ 69.293315] nix-grpc-daemon[854]: ts=2026-09-20T08:04:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:338505122worker2 # [ 69.295968] nix-grpc-daemon[854]: ts=2026-09-20T08:04:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 paths=1 nar_bytes_out=1765123worker1 # [ 69.678886] nix-grpc-daemon[782]: ts=2026-09-20T08:04:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:582805124worker2 # [ 69.539784] nix-grpc-daemon[854]: ts=2026-09-20T08:04:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338505125worker2 # [ 69.555443] nix-grpc-daemon[934]: time=2026-09-20T08:04:55.060Z level=INFO msg="All 2 paths already cached"5126worker2 # [ 69.569523] nix-grpc-daemon[854]: ts=2026-09-20T08:04:55Z level=debug event=want client=7 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05127worker2 # [ 69.574844] nix-grpc-daemon[854]: ts=2026-09-20T08:04:55Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15128worker2 # [ 69.580544] nix-grpc-daemon[854]: ts=2026-09-20T08:04:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338505129worker2 # [ 69.592700] nix-grpc-daemon[854]: ts=2026-09-20T08:04:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338505130worker2 # [ 69.946883] nix-grpc-daemon[934]: time=2026-09-20T08:04:55.451Z level=INFO msg="All 2 paths already cached"5131worker2 # [ 69.955357] nix-grpc-daemon[854]: ts=2026-09-20T08:04:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338505132worker2 # [ 69.965160] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5133worker2 # [ 70.138810] nix-daemon[923]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5134worker2 # [ 70.152229] nix-daemon[923]: reaped child process 1181, status = succeeded5135worker2 # [ 70.302269] nix-daemon[923]: accepted connection from pid 1183, user nix-grpc-daemon (trusted)5136worker2 # [ 70.315507] nix-daemon[923]: reaped child process 1186, status = succeeded5137lb # [ 70.468555] niks3-server[1185]: 2026/09/20 08:04:56 INFO Received uploads request method=POST path=/api/pending_closures5138worker2 # [ 70.341862] nix-grpc-daemon[934]: time=2026-09-20T08:04:55.846Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5139worker2 # [ 70.343335] nix-grpc-daemon[934]: time=2026-09-20T08:04:55.846Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5140lb # [ 70.496798] niks3-server[1185]: 2026/09/20 08:04:56 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5141lb # [ 70.510703] niks3-server[1185]: 2026/09/20 08:04:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5142lb # [ 70.515920] niks3-server[1185]: 2026/09/20 08:04:56 INFO Signed narinfos id=13 count=15143worker2 # [ 70.382365] nix-grpc-daemon[934]: time=2026-09-20T08:04:55.887Z level=INFO msg="Uploading 1 narinfos"5144lb # [ 70.523798] niks3-server[1185]: 2026/09/20 08:04:56 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5145lb # [ 70.533228] niks3-server[1185]: 2026/09/20 08:04:56 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5146lb # [ 70.539892] niks3-server[1185]: 2026/09/20 08:04:56 INFO Completed upload id=135147worker2 # [ 70.403465] nix-grpc-daemon[934]: time=2026-09-20T08:04:55.908Z level=INFO msg="Upload complete. (426ms)"5148lb # [ 70.542040] niks3-server[1185]: 2026/09/20 08:04:56 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5149worker2 # [ 70.408326] nix-grpc-daemon[854]: ts=2026-09-20T08:04:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 paths=1 nar_bytes_in=7525150worker2 # [ 70.414326] nix-grpc-daemon[854]: ts=2026-09-20T08:04:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338505151worker2 # [ 70.422533] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5152worker2 # [ 70.431144] nix-daemon[923]: reaped child process 1176, status = succeeded5153worker2 # [ 70.434237] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5154worker2 # [ 71.460236] nix-daemon[923]: reaped child process 1190, status = succeeded5155worker2 # [ 71.610335] nix-daemon[923]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5156worker2 # [ 71.623901] nix-daemon[923]: reaped child process 1208, status = succeeded5157lb # [ 71.774465] niks3-server[1185]: 2026/09/20 08:04:57 INFO Received uploads request method=POST path=/api/pending_closures5158worker2 # [ 71.643096] nix-grpc-daemon[934]: time=2026-09-20T08:04:57.147Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5159worker2 # [ 71.645240] nix-grpc-daemon[934]: time=2026-09-20T08:04:57.149Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5160lb # [ 71.800912] niks3-server[1185]: 2026/09/20 08:04:57 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5161lb # [ 71.804972] niks3-server[1185]: 2026/09/20 08:04:57 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5162lb # [ 71.809082] niks3-server[1185]: 2026/09/20 08:04:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5163lb # [ 71.811281] niks3-server[1185]: 2026/09/20 08:04:57 INFO Signed narinfos id=14 count=15164worker2 # [ 71.676179] nix-grpc-daemon[934]: time=2026-09-20T08:04:57.181Z level=INFO msg="Uploading 1 narinfos"5165lb # [ 71.816596] niks3-server[1185]: 2026/09/20 08:04:57 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5166lb # [ 71.821595] niks3-server[1185]: 2026/09/20 08:04:57 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5167lb # [ 71.825625] niks3-server[1185]: 2026/09/20 08:04:57 INFO Completed upload id=145168worker2 # [ 71.688983] nix-grpc-daemon[934]: time=2026-09-20T08:04:57.194Z level=INFO msg="Upload complete. (247ms)"5169lb # [ 71.828919] niks3-server[1185]: 2026/09/20 08:04:57 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5170worker2 # [ 71.692445] nix-grpc-daemon[854]: ts=2026-09-20T08:04:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=7 outputs=15171worker2 # [ 71.698596] nix-daemon[923]: reaped child process 1188, status = succeeded5172worker1 # [ 71.882335] nix-grpc-daemon[782]: ts=2026-09-20T08:04:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582805173worker1 # [ 71.899465] nix-grpc-daemon[782]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5174worker1 # [ 71.912967] nix-grpc-daemon[782]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5175worker2 # [ 71.798172] nix-grpc-daemon[854]: ts=2026-09-20T08:04:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:338505176worker2 # [ 71.802261] nix-grpc-daemon[854]: ts=2026-09-20T08:04:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 paths=1 nar_bytes_out=2325177client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.74 seconds)5178client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5179client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5180client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5181client # [ 72.077278] nix-daemon[935]: reaped child process 937, status = succeeded5182client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)5183client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5184client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5185(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.83 seconds)5186subtest: an input only one worker has still reaches the builder5187client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5188client # this derivation will be built:5189client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5190client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5191client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.60 seconds)5192client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5193client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.16 seconds)5194worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5195worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)5196worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5197worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5198client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25199client # [ 73.302310] nix-daemon[935]: accepted connection from pid 1018, user root (trusted)5200client # this derivation will be built:5201client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5202worker1 # [ 73.448385] nix-grpc-daemon[782]: ts=2026-09-20T08:04:59Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:582805203worker2 # [ 73.307749] nix-grpc-daemon[854]: ts=2026-09-20T08:04:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338505204client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5205client # copying 1 paths...5206client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5207worker1 # [ 73.458546] nix-grpc-daemon[782]: ts=2026-09-20T08:04:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582805208worker1 # [ 73.470749] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5209worker1 # [ 73.640054] nix-daemon[846]: accepted connection from pid 947, user nix-grpc-daemon (trusted)5210worker1 # [ 73.654837] nix-daemon[846]: reaped child process 950, status = succeeded5211lb # [ 73.662568] niks3-server[1185]: 2026/09/20 08:04:59 INFO Received uploads request method=POST path=/api/pending_closures5212worker1 # [ 73.673726] nix-grpc-daemon[868]: time=2026-09-20T08:04:59.621Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5213worker1 # [ 73.675246] nix-grpc-daemon[868]: time=2026-09-20T08:04:59.621Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5214lb # [ 73.677865] niks3-server[1185]: 2026/09/20 08:04:59 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5215lb # [ 73.683060] niks3-server[1185]: 2026/09/20 08:04:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5216lb # [ 73.684969] niks3-server[1185]: 2026/09/20 08:04:59 INFO Signed narinfos id=15 count=15217worker1 # [ 73.692561] nix-grpc-daemon[868]: time=2026-09-20T08:04:59.640Z level=INFO msg="Uploading 1 narinfos"5218lb # [ 73.689501] niks3-server[1185]: 2026/09/20 08:04:59 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5219lb # [ 73.693262] niks3-server[1185]: 2026/09/20 08:04:59 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5220lb # [ 73.696620] niks3-server[1185]: 2026/09/20 08:04:59 INFO Completed upload id=155221worker1 # [ 73.702671] nix-grpc-daemon[868]: time=2026-09-20T08:04:59.650Z level=INFO msg="Upload complete. (221ms)"5222lb # [ 73.698861] niks3-server[1185]: 2026/09/20 08:04:59 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5223worker1 # [ 73.705061] nix-grpc-daemon[782]: ts=2026-09-20T08:04:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=0 paths=1 nar_bytes_in=1205224worker1 # [ 73.711830] nix-daemon[846]: reaped child process 945, status = succeeded5225worker2 # [ 73.576273] nix-grpc-daemon[854]: ts=2026-09-20T08:04:59Z level=debug event=want client=8 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05226worker2 # [ 73.580687] nix-grpc-daemon[854]: ts=2026-09-20T08:04:59Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15227worker1 # [ 73.762151] nix-grpc-daemon[782]: ts=2026-09-20T08:04:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582805228worker1 # [ 73.772425] nix-grpc-daemon[782]: ts=2026-09-20T08:04:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582805229client # copying 2 paths to 'lb:50051'...5230client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5231worker1 # [ 73.785104] nix-grpc-daemon[782]: ts=2026-09-20T08:04:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582805232client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5233worker1 # [ 73.798193] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5234worker1 # [ 73.977904] nix-daemon[846]: accepted connection from pid 955, user nix-grpc-daemon (trusted)5235worker1 # [ 73.994577] nix-daemon[846]: reaped child process 958, status = succeeded5236worker1 # [ 74.168883] nix-daemon[846]: accepted connection from pid 960, user nix-grpc-daemon (trusted)5237worker1 # [ 74.184226] nix-daemon[846]: reaped child process 963, status = succeeded5238lb # [ 74.191801] niks3-server[1185]: 2026/09/20 08:05:00 INFO Received uploads request method=POST path=/api/pending_closures5239lb # [ 74.200840] niks3-server[1185]: 2026/09/20 08:05:00 INFO Received uploads request method=POST path=/api/pending_closures5240worker1 # [ 74.215075] nix-grpc-daemon[868]: time=2026-09-20T08:05:00.162Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5241worker1 # [ 74.216731] nix-grpc-daemon[868]: time=2026-09-20T08:05:00.162Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5242worker1 # [ 74.218663] nix-grpc-daemon[868]: time=2026-09-20T08:05:00.162Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5243lb # [ 74.238638] niks3-server[1185]: 2026/09/20 08:05:00 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5244lb # [ 74.250351] niks3-server[1185]: 2026/09/20 08:05:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5245lb # [ 74.256214] niks3-server[1185]: 2026/09/20 08:05:00 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5246lb # [ 74.259624] niks3-server[1185]: 2026/09/20 08:05:00 INFO Signed narinfos id=16 count=15247lb # [ 74.261300] niks3-server[1185]: 2026/09/20 08:05:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5248lb # [ 74.263562] niks3-server[1185]: 2026/09/20 08:05:00 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5249lb # [ 74.265569] niks3-server[1185]: 2026/09/20 08:05:00 INFO Signed narinfos id=17 count=25250lb # [ 74.267247] niks3-server[1185]: 2026/09/20 08:05:00 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5251worker1 # [ 74.273609] nix-grpc-daemon[868]: time=2026-09-20T08:05:00.220Z level=INFO msg="Uploading 3 narinfos"5252lb # [ 74.284680] niks3-server[1185]: 2026/09/20 08:05:00 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5253lb # [ 74.303220] niks3-server[1185]: 2026/09/20 08:05:00 INFO Completed upload id=165254lb # [ 74.304719] niks3-server[1185]: 2026/09/20 08:05:00 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5255lb # [ 74.314075] niks3-server[1185]: 2026/09/20 08:05:00 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5256lb # [ 74.320627] niks3-server[1185]: 2026/09/20 08:05:00 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5257lb # [ 74.326797] niks3-server[1185]: 2026/09/20 08:05:00 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5258lb # [ 74.334532] niks3-server[1185]: 2026/09/20 08:05:00 INFO Completed upload id=175259worker1 # [ 74.340770] nix-grpc-daemon[868]: time=2026-09-20T08:05:00.288Z level=INFO msg="Upload complete. (528ms)"5260worker1 # [ 74.346204] nix-grpc-daemon[782]: ts=2026-09-20T08:05:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=0 paths=2 nar_bytes_in=9765261worker1 # [ 74.352484] nix-grpc-daemon[782]: ts=2026-09-20T08:05:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582805262worker1 # [ 74.362564] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5263worker1 # [ 74.377790] nix-daemon[846]: reaped child process 952, status = succeeded5264client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5265worker1 # [ 74.383471] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5266client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5267worker1 # [ 75.415785] nix-daemon[846]: reaped child process 967, status = succeeded5268worker1 # [ 75.563993] nix-daemon[846]: accepted connection from pid 982, user nix-grpc-daemon (trusted)5269worker1 # [ 75.576231] nix-daemon[846]: reaped child process 985, status = succeeded5270lb # [ 75.588826] niks3-server[1185]: 2026/09/20 08:05:01 INFO Received uploads request method=POST path=/api/pending_closures5271worker1 # [ 75.602152] nix-grpc-daemon[868]: time=2026-09-20T08:05:01.549Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5272lb # [ 75.621793] niks3-server[1185]: 2026/09/20 08:05:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5273worker1 # [ 75.632257] nix-grpc-daemon[868]: time=2026-09-20T08:05:01.579Z level=INFO msg="Uploading 1 narinfos"5274lb # [ 75.627182] niks3-server[1185]: 2026/09/20 08:05:01 INFO Signed narinfos id=18 count=15275lb # [ 75.638633] niks3-server[1185]: 2026/09/20 08:05:01 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5276lb # [ 75.647931] niks3-server[1185]: 2026/09/20 08:05:01 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5277lb # [ 75.656307] niks3-server[1185]: 2026/09/20 08:05:01 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5278lb # [ 75.692647] niks3-server[1185]: 2026/09/20 08:05:01 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5279lb # [ 75.708268] niks3-server[1185]: 2026/09/20 08:05:01 INFO Completed upload id=185280worker1 # [ 75.714602] nix-grpc-daemon[868]: time=2026-09-20T08:05:01.662Z level=INFO msg="Upload complete. (320ms)"5281worker1 # [ 75.721498] nix-grpc-daemon[782]: ts=2026-09-20T08:05:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=8 outputs=15282worker1 # [ 75.735284] nix-daemon[846]: reaped child process 965, status = succeeded5283client # copying 1 paths...5284worker1 # [ 75.766179] nix-grpc-daemon[782]: ts=2026-09-20T08:05:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582805285client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5286worker2 # [ 75.645798] nix-grpc-daemon[854]: ts=2026-09-20T08:05:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:338505287worker2 # [ 75.654327] nix-grpc-daemon[854]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5288worker2 # [ 75.664944] nix-grpc-daemon[854]: ts=2026-09-20T08:05:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=0 paths=1 nar_bytes_out=1205289client # [ 75.919179] nix-daemon[935]: reaped child process 1021, status = succeeded5290client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.82 seconds)5291client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25292client # [ 76.074936] nix-daemon[935]: accepted connection from pid 1040, user root (trusted)5293client # this derivation will be built:5294client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5295worker1 # [ 76.216083] nix-grpc-daemon[782]: ts=2026-09-20T08:05:02Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:582805296client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5297worker2 # [ 76.076582] nix-grpc-daemon[854]: ts=2026-09-20T08:05:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338505298worker2 # [ 76.105122] nix-grpc-daemon[854]: ts=2026-09-20T08:05:01Z level=debug event=want client=9 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05299worker2 # [ 76.107397] nix-grpc-daemon[854]: ts=2026-09-20T08:05:01Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15300worker2 # [ 76.110099] nix-grpc-daemon[854]: ts=2026-09-20T08:05:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338505301worker2 # [ 76.117410] nix-grpc-daemon[854]: ts=2026-09-20T08:05:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338505302client # copying 1 paths to 'lb:50051'...5303client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5304worker2 # [ 76.135138] nix-grpc-daemon[854]: ts=2026-09-20T08:05:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:338505305worker2 # [ 76.143909] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5306worker2 # [ 76.160340] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)5307worker2 # [ 76.224898] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5)5308worker2 # [ 76.232293] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 254 ms (attempt 3/5)5309worker2 # [ 76.490332] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 674 ms (attempt 4/5)5310worker2 # [ 77.165910] nix-grpc-daemon[854]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5311worker2 # [ 77.172461] nix-grpc-daemon[854]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5312worker2 # [ 77.374179] nix-daemon[923]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5313worker2 # [ 77.387131] nix-daemon[923]: reaped child process 1227, status = succeeded5314worker2 # [ 77.538213] nix-daemon[923]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5315worker2 # [ 77.553495] nix-daemon[923]: reaped child process 1232, status = succeeded5316lb # [ 77.706389] niks3-server[1185]: 2026/09/20 08:05:03 INFO Received uploads request method=POST path=/api/pending_closures5317worker2 # [ 77.579783] nix-grpc-daemon[934]: time=2026-09-20T08:05:03.084Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5318worker2 # [ 77.581391] nix-grpc-daemon[934]: time=2026-09-20T08:05:03.086Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5319lb # [ 77.734396] niks3-server[1185]: 2026/09/20 08:05:03 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5320lb # [ 77.738769] niks3-server[1185]: 2026/09/20 08:05:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5321lb # [ 77.740923] niks3-server[1185]: 2026/09/20 08:05:03 INFO Signed narinfos id=19 count=15322worker2 # [ 77.605883] nix-grpc-daemon[934]: time=2026-09-20T08:05:03.110Z level=INFO msg="Uploading 1 narinfos"5323lb # [ 77.743487] niks3-server[1185]: 2026/09/20 08:05:03 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5324lb # [ 77.749277] niks3-server[1185]: 2026/09/20 08:05:03 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5325lb # [ 77.753170] niks3-server[1185]: 2026/09/20 08:05:03 INFO Completed upload id=195326worker2 # [ 77.617770] nix-grpc-daemon[934]: time=2026-09-20T08:05:03.122Z level=INFO msg="Upload complete. (422ms)"5327lb # [ 77.755894] niks3-server[1185]: 2026/09/20 08:05:03 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5328worker2 # [ 77.620267] nix-grpc-daemon[854]: ts=2026-09-20T08:05:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=1 paths=1 nar_bytes_in=5605329worker2 # [ 77.625145] nix-grpc-daemon[854]: ts=2026-09-20T08:05:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:338505330worker2 # [ 77.632530] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5331worker2 # [ 77.634748] nix-daemon[923]: reaped child process 1218, status = succeeded5332worker2 # [ 77.638842] nix-grpc-daemon[854]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5333worker2 # [ 77.646830] nix-daemon[923]: accepted connection from pid 854, user nix-grpc-daemon (trusted)5334client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5335client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5336lb # [ 78.497252] envoy[924]: conn peer=[2001:db8:1::1]:37772 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=545337lb # [ 78.500892] envoy[924]: conn peer=[2001:db8:1::1]:37758 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3867 tx=1851 ms=24505338lb # [ 78.504520] envoy[924]: conn peer=[2001:db8:1::1]:37790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=165339lb # [ 78.507938] envoy[924]: conn peer=[2001:db8:1::1]:37784 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4302 tx=2225 ms=22805340lb # [ 78.511485] envoy[924]: conn peer=[2001:db8:1::1]:33436 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4190 tx=1750 ms=24135341lb # [ 78.514926] envoy[924]: conn peer=[2001:db8:1::1]:33442 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=715342worker2 # [ 79.061476] nix-daemon[923]: reaped child process 1237, status = succeeded5343worker2 # [ 79.197911] nix-daemon[923]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5344worker2 # [ 79.209776] nix-daemon[923]: reaped child process 1255, status = succeeded5345lb # [ 79.362796] niks3-server[1185]: 2026/09/20 08:05:05 INFO Received uploads request method=POST path=/api/pending_closures5346worker2 # [ 79.234329] nix-grpc-daemon[934]: time=2026-09-20T08:05:04.739Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5347lb # [ 79.391334] niks3-server[1185]: 2026/09/20 08:05:05 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5348lb # [ 79.399450] niks3-server[1185]: 2026/09/20 08:05:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5349lb # [ 79.403634] niks3-server[1185]: 2026/09/20 08:05:05 INFO Signed narinfos id=20 count=15350worker2 # [ 79.270159] nix-grpc-daemon[934]: time=2026-09-20T08:05:04.775Z level=INFO msg="Uploading 1 narinfos"5351lb # [ 79.413161] niks3-server[1185]: 2026/09/20 08:05:05 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5352lb # [ 79.423772] niks3-server[1185]: 2026/09/20 08:05:05 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5353worker2 # [ 79.293954] nix-grpc-daemon[934]: time=2026-09-20T08:05:04.799Z level=INFO msg="Upload complete. (238ms)"5354lb # [ 79.430984] niks3-server[1185]: 2026/09/20 08:05:05 INFO Completed upload id=205355lb # [ 79.434105] niks3-server[1185]: 2026/09/20 08:05:05 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5356worker2 # [ 79.298503] nix-grpc-daemon[854]: ts=2026-09-20T08:05:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33850 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=9 outputs=15357worker2 # [ 79.309451] nix-daemon[923]: reaped child process 1234, status = succeeded5358client # copying 1 paths...5359worker1 # [ 79.486765] nix-grpc-daemon[782]: ts=2026-09-20T08:05:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582805360worker1 # [ 79.496823] nix-grpc-daemon[782]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5361client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5362worker1 # [ 79.527498] nix-grpc-daemon[782]: ts=2026-09-20T08:05:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:582805363worker1 # [ 79.532285] nix-grpc-daemon[782]: ts=2026-09-20T08:05:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=0 paths=1 nar_bytes_out=1205364client # [ 79.654440] nix-daemon[935]: reaped child process 1043, status = succeeded5365client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.73 seconds)5366client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25367client # [ 79.800288] nix-daemon[935]: accepted connection from pid 1062, user root (trusted)5368client # this derivation will be built:5369client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5370worker1 # [ 79.935540] nix-grpc-daemon[782]: ts=2026-09-20T08:05:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:582805371worker2 # [ 79.796248] nix-grpc-daemon[854]: ts=2026-09-20T08:05:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:338505372client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5373worker2 # [ 79.802309] nix-grpc-daemon[934]: time=2026-09-20T08:05:05.307Z level=INFO msg="All 1 paths already cached"5374worker2 # [ 79.812601] nix-grpc-daemon[854]: ts=2026-09-20T08:05:05Z level=debug event=want client=10 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05375worker2 # [ 79.814708] nix-grpc-daemon[854]: ts=2026-09-20T08:05:05Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15376worker1 # [ 79.959171] nix-grpc-daemon[782]: ts=2026-09-20T08:05:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582805377worker1 # [ 79.967125] nix-grpc-daemon[782]: ts=2026-09-20T08:05:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582805378worker1 # [ 79.976142] nix-grpc-daemon[868]: time=2026-09-20T08:05:05.923Z level=INFO msg="All 1 paths already cached"5379client # copying 1 paths to 'lb:50051'...5380client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5381worker1 # [ 79.981230] nix-grpc-daemon[782]: ts=2026-09-20T08:05:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582805382worker1 # [ 79.986458] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5383worker1 # [ 80.141296] nix-daemon[846]: accepted connection from pid 990, user nix-grpc-daemon (trusted)5384worker1 # [ 80.154330] nix-daemon[846]: reaped child process 993, status = succeeded5385worker1 # [ 80.314385] nix-daemon[846]: accepted connection from pid 995, user nix-grpc-daemon (trusted)5386worker1 # [ 80.327150] nix-daemon[846]: reaped child process 998, status = succeeded5387lb # [ 80.335828] niks3-server[1185]: 2026/09/20 08:05:06 INFO Received uploads request method=POST path=/api/pending_closures5388worker1 # [ 80.351121] nix-grpc-daemon[868]: time=2026-09-20T08:05:06.298Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5389worker1 # [ 80.352488] nix-grpc-daemon[868]: time=2026-09-20T08:05:06.298Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5390lb # [ 80.366968] niks3-server[1185]: 2026/09/20 08:05:06 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5391lb # [ 80.381355] niks3-server[1185]: 2026/09/20 08:05:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5392lb # [ 80.387095] niks3-server[1185]: 2026/09/20 08:05:06 INFO Signed narinfos id=21 count=15393worker1 # [ 80.396488] nix-grpc-daemon[868]: time=2026-09-20T08:05:06.344Z level=INFO msg="Uploading 1 narinfos"5394lb # [ 80.398529] niks3-server[1185]: 2026/09/20 08:05:06 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5395lb # [ 80.404811] niks3-server[1185]: 2026/09/20 08:05:06 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5396lb # [ 80.410284] niks3-server[1185]: 2026/09/20 08:05:06 INFO Completed upload id=215397worker1 # [ 80.416403] nix-grpc-daemon[868]: time=2026-09-20T08:05:06.364Z level=INFO msg="Upload complete. (424ms)"5398lb # [ 80.411884] niks3-server[1185]: 2026/09/20 08:05:06 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5399worker1 # [ 80.420277] nix-grpc-daemon[782]: ts=2026-09-20T08:05:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=0 paths=1 nar_bytes_in=5605400worker1 # [ 80.426549] nix-grpc-daemon[782]: ts=2026-09-20T08:05:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582805401worker1 # [ 80.435567] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5402worker1 # [ 80.444042] nix-daemon[846]: reaped child process 988, status = succeeded5403worker1 # [ 80.448258] nix-daemon[846]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5404client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5405client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5406worker1 # [ 81.276395] nix-daemon[846]: reaped child process 1002, status = succeeded5407worker1 # [ 81.415972] nix-daemon[846]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)5408worker1 # [ 81.429196] nix-daemon[846]: reaped child process 1020, status = succeeded5409lb # [ 81.436962] niks3-server[1185]: 2026/09/20 08:05:07 INFO Received uploads request method=POST path=/api/pending_closures5410worker1 # [ 81.448161] nix-grpc-daemon[868]: time=2026-09-20T08:05:07.395Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5411lb # [ 81.466188] niks3-server[1185]: 2026/09/20 08:05:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5412worker1 # [ 81.477241] nix-grpc-daemon[868]: time=2026-09-20T08:05:07.425Z level=INFO msg="Uploading 1 narinfos"5413lb # [ 81.471655] niks3-server[1185]: 2026/09/20 08:05:07 INFO Signed narinfos id=22 count=15414lb # [ 81.477538] niks3-server[1185]: 2026/09/20 08:05:07 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5415lb # [ 81.485333] niks3-server[1185]: 2026/09/20 08:05:07 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5416lb # [ 81.493055] niks3-server[1185]: 2026/09/20 08:05:07 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5417lb # [ 81.496187] niks3-server[1185]: 2026/09/20 08:05:07 INFO Completed upload id=225418worker1 # [ 81.503340] nix-grpc-daemon[868]: time=2026-09-20T08:05:07.451Z level=INFO msg="Upload complete. (246ms)"5419lb # [ 81.497643] niks3-server[1185]: 2026/09/20 08:05:07 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5420client # copying 1 paths...5421worker1 # [ 81.506785] nix-grpc-daemon[782]: ts=2026-09-20T08:05:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=10 outputs=15422worker2 # [ 81.369074] nix-grpc-daemon[854]: ts=2026-09-20T08:05:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:338505423worker1 # [ 81.517587] nix-daemon[846]: reaped child process 1000, status = succeeded5424worker2 # [ 81.376638] nix-grpc-daemon[854]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5425client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5426worker1 # [ 81.540822] nix-grpc-daemon[782]: ts=2026-09-20T08:05:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:582805427worker1 # [ 81.544249] nix-grpc-daemon[782]: ts=2026-09-20T08:05:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58280 duration_s=0 paths=1 nar_bytes_out=1205428client # [ 81.659526] nix-daemon[935]: reaped child process 1065, status = succeeded5429client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.01 seconds)5430(finished: subtest: an input only one worker has still reaches the builder, in 9.55 seconds)5431subtest: upload whose reference the worker lacks is completed from the cache5432client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5433client # this derivation will be built:5434client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5435client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5436client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.34 seconds)5437client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5438client # this derivation will be built:5439client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5440client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5441client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.34 seconds)5442client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5443client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5444worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5445worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5446worker2 # [ 82.626446] systemd[1]: Stopping Nix worker-protocol over gRPC...5447worker2 # [ 82.706684] nix-grpc-daemon[854]: ts=2026-09-20T08:05:08Z level=info event=signal_exit5448worker2 # [ 83.184881] nix-grpc-daemon[854]: ts=2026-09-20T08:05:08Z level=info event=scheduler_yield5449worker1 # [ 83.330591] nix-grpc-daemon[782]: ts=2026-09-20T08:05:09Z level=info event=scheduler_restarting addr=lb:500515450lb # [ 83.324982] niks3-server[1185]: 2026/09/20 08:05:09 INFO lead: released remote=192.168.1.4:456285451worker2 # [ 83.189947] nix-grpc-daemon[854]: ts=2026-09-20T08:05:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:483365452worker2 # [ 83.194116] nix-grpc-daemon[854]: ts=2026-09-20T08:05:08Z level=info event=worker_gone addr=192.168.1.3:500515453worker2 # [ 83.198459] nix-grpc-daemon[854]: ts=2026-09-20T08:05:08Z level=info event=scheduler_restarting addr=lb:500515454worker2 # [ 83.201541] nix-grpc-daemon[854]: ts=2026-09-20T08:05:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:483365455worker2 # [ 83.204760] nix-grpc-daemon[854]: ts=2026-09-20T08:05:08Z level=info event=worker_gone addr=192.168.1.4:500515456worker1 # [ 83.530824] nix-grpc-daemon[782]: ts=2026-09-20T08:05:09Z level=info event=scheduler_connected addr=lb:500515457worker2 # [ 83.394824] nix-grpc-daemon[854]: ts=2026-09-20T08:05:08Z level=info event=scheduler_connected addr=lb:500515458worker2 # [ 83.401594] nix-grpc-daemon[854]: ts=2026-09-20T08:05:08Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5459worker1 # [ 83.545112] nix-grpc-daemon[782]: ts=2026-09-20T08:05:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5460worker1 # [ 83.745672] nix-grpc-daemon[782]: ts=2026-09-20T08:05:09Z level=info event=scheduler_connected addr=lb:500515461worker2 # [ 83.602452] nix-grpc-daemon[854]: ts=2026-09-20T08:05:09Z level=info event=scheduler_connected addr=lb:500515462worker1 # [ 83.749137] nix-grpc-daemon[782]: ts=2026-09-20T08:05:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5463worker2 # [ 83.607639] nix-grpc-daemon[854]: ts=2026-09-20T08:05:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5464worker1 # [ 83.949454] nix-grpc-daemon[782]: ts=2026-09-20T08:05:09Z level=info event=scheduler_connected addr=lb:500515465worker1 # [ 83.953200] nix-grpc-daemon[782]: ts=2026-09-20T08:05:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5466worker2 # [ 83.810329] nix-grpc-daemon[854]: ts=2026-09-20T08:05:09Z level=info event=scheduler_connected addr=lb:500515467worker2 # [ 83.814195] nix-grpc-daemon[854]: ts=2026-09-20T08:05:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5468worker1 # [ 84.153904] nix-grpc-daemon[782]: ts=2026-09-20T08:05:10Z level=info event=scheduler_connected addr=lb:500515469worker1 # [ 84.156381] nix-grpc-daemon[782]: ts=2026-09-20T08:05:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5470worker2 # [ 84.014673] nix-grpc-daemon[854]: ts=2026-09-20T08:05:09Z level=info event=scheduler_connected addr=lb:500515471worker2 # [ 84.018598] nix-grpc-daemon[854]: ts=2026-09-20T08:05:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5472worker2 # [ 84.189209] nix-grpc-daemon[854]: ts=2026-09-20T08:05:09Z level=info event=scheduler_restarting5473worker2 # [ 84.208956] nix-daemon[923]: reaped child process 925, status = succeeded5474worker1 # [ 84.357492] nix-grpc-daemon[782]: ts=2026-09-20T08:05:10Z level=info event=scheduler_connected addr=lb:500515475worker2 # [ 84.216189] nix-grpc-daemon[854]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5476worker2 # [ 84.217558] nix-grpc-daemon[854]: E0000 00:00:1789891509.720797 897 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 11, started true, shutdown true5477worker2 # [ 84.225976] nix-grpc-daemon[854]: ts=2026-09-20T08:05:09Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5478lb # [ 84.776946] niks3-server[1185]: 2026/09/20 08:05:10 INFO lead: acquired remote=192.168.1.3:363965479worker1 # [ 84.782857] nix-grpc-daemon[782]: ts=2026-09-20T08:05:10Z level=info event=scheduler_take_over5480worker2 # [ 84.836900] nix-grpc-daemon[854]: E0000 00:00:1789891510.340363 897 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5481worker2 # [ 84.842128] nix-grpc-daemon[854]: E0000 00:00:1789891510.341857 897 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5482worker1 # [ 86.312567] nix-grpc-daemon[782]: ts=2026-09-20T08:05:12Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5483worker2 # [ 86.179778] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5484worker2 # [ 86.182676] systemd[1]: Stopped Nix worker-protocol over gRPC.5485worker2 # [ 86.183885] systemd[1]: nix-grpc-daemon.service: Consumed 2.658s CPU time over 1min 7.830s wall clock time, 57.7M memory peak, 98.2K incoming IP traffic, 198K outgoing IP traffic.5486worker2 # [ 86.191257] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5487worker2 # [ 86.193099] systemd[1]: Closed Nix worker-protocol over gRPC.5488worker2 # [ 86.194425] systemd[1]: nix-grpc-daemon.socket: Consumed 384us CPU time over 1min 16.862s wall clock time, 256K memory peak, 151.3K incoming IP traffic, 112.6K outgoing IP traffic.5489lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5490lb: must succeed: curl -sf localhost:9901/clusters5491lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5492lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5493client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25494worker1 # [ 86.512824] nix-grpc-daemon[782]: ts=2026-09-20T08:05:12Z level=info event=scheduler_connected addr=lb:500515495worker1 # [ 86.517319] nix-grpc-daemon[782]: ts=2026-09-20T08:05:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05496worker1 # [ 86.619033] nix-grpc-daemon[782]: ts=2026-09-20T08:05:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582805497worker1 # [ 86.623450] nix-grpc-daemon[782]: ts=2026-09-20T08:05:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:582805498client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5499client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.23 seconds)5500worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5501worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5502worker2 # [ 86.584286] systemd[1]: Listening on Nix worker-protocol over gRPC.5503worker1 # [ 86.762986] systemd[1]: Stopping Nix worker-protocol over gRPC...5504worker1 # [ 86.792547] nix-grpc-daemon[782]: ts=2026-09-20T08:05:12Z level=info event=signal_exit5505worker1 # [ 87.274552] nix-grpc-daemon[782]: ts=2026-09-20T08:05:13Z level=info event=scheduler_yield5506worker1 # [ 87.278713] nix-grpc-daemon[782]: ts=2026-09-20T08:05:13Z level=info event=scheduler_restarting addr=lb:500515507lb # [ 87.271327] niks3-server[1185]: 2026/09/20 08:05:13 INFO lead: released remote=192.168.1.3:363965508worker1 # [ 87.283566] nix-grpc-daemon[782]: ts=2026-09-20T08:05:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:580085509worker1 # [ 87.286871] nix-grpc-daemon[782]: ts=2026-09-20T08:05:13Z level=info event=worker_gone addr=192.168.1.3:500515510worker2 # [ 87.171875] systemd[1]: Starting Nix worker-protocol over gRPC...5511worker1 # [ 87.480386] nix-grpc-daemon[782]: ts=2026-09-20T08:05:13Z level=info event=scheduler_connected addr=lb:500515512worker2 # [ 87.340614] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5513worker2 # [ 87.344578] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15514worker2 # [ 87.350795] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5515worker1 # [ 87.491889] nix-grpc-daemon[782]: ts=2026-09-20T08:05:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5516worker2 # [ 87.355238] systemd[1]: Started Nix worker-protocol over gRPC.5517worker2 # [ 87.363687] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:12Z level=info event=scheduler_take_over5518lb # [ 87.500290] niks3-server[1185]: 2026/09/20 08:05:13 INFO lead: acquired remote=192.168.1.4:472145519worker2 # [ 87.373669] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:12Z level=info event=scheduler_connected addr=lb:500515520worker2 # [ 87.376541] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05521worker1 # [ 87.692949] nix-grpc-daemon[782]: ts=2026-09-20T08:05:13Z level=info event=scheduler_connected addr=lb:500515522worker1 # [ 87.698415] nix-grpc-daemon[782]: ts=2026-09-20T08:05:13Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5523worker1 # [ 87.899139] nix-grpc-daemon[782]: ts=2026-09-20T08:05:13Z level=info event=scheduler_connected addr=lb:500515524worker2 # [ 87.758086] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05525worker1 # [ 88.278796] nix-grpc-daemon[782]: ts=2026-09-20T08:05:14Z level=info event=scheduler_restarting5526worker2 # [ 88.145206] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:367065527worker2 # [ 88.146797] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:13Z level=info event=worker_gone addr=192.168.1.3:500515528worker1 # [ 88.299302] nix-grpc-daemon[782]: ts=2026-09-20T08:05:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5529worker1 # [ 88.321637] nix-daemon[846]: reaped child process 848, status = succeeded5530lb # [ 88.498484] envoy[924]: conn peer=[2001:db8:1::1]:33446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3711 tx=1915 ms=33695531lb # [ 88.502381] envoy[924]: conn peer=[2001:db8:1::1]:33460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=585532lb # [ 88.506237] envoy[924]: conn peer=[2001:db8:1::1]:33466 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3699 tx=1891 ms=16575533lb # [ 88.510490] envoy[924]: conn peer=[2001:db8:1::1]:33470 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=515534lb # [ 88.514592] envoy[924]: conn peer=[2001:db8:1::4]:35498 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=27330 tx=5152 ms=655055535lb # [ 88.518725] envoy[924]: conn peer=[2001:db8:1::1]:35628 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=165536lb # [ 88.522311] envoy[924]: conn peer=[2001:db8:1::3]:44506 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=27119 tx=4917 ms=685795537worker1 # [ 88.933489] nix-grpc-daemon[782]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5538worker1 # [ 88.936901] nix-grpc-daemon[782]: E0000 00:00:1789891514.880057 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 11, started true, shutdown true5539worker1 # [ 88.941411] nix-grpc-daemon[782]: E0000 00:00:1789891514.880560 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5540worker1 # [ 90.276462] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5541worker1 # [ 90.277948] systemd[1]: Stopped Nix worker-protocol over gRPC.5542worker1 # [ 90.279942] systemd[1]: nix-grpc-daemon.service: Consumed 1.666s CPU time over 1min 11.793s wall clock time, 47.4M memory peak, 71K incoming IP traffic, 132.2K outgoing IP traffic.5543worker1 # [ 90.283724] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5544worker1 # [ 90.285579] systemd[1]: Closed Nix worker-protocol over gRPC.5545worker1 # [ 90.287261] systemd[1]: nix-grpc-daemon.socket: Consumed 333us CPU time over 1min 21.402s wall clock time, 256K memory peak, 128.7K incoming IP traffic, 104.1K outgoing IP traffic.5546lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5547lb: must succeed: curl -sf localhost:9901/clusters5548lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5549lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.06 seconds)5550client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25551worker2 # [ 90.393967] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:367185552worker2 # [ 90.405776] nix-daemon[923]: accepted connection from pid 1279, user nix-grpc-daemon (trusted)5553worker2 # [ 90.422458] nix-grpc-daemon[1279]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5)5554worker2 # [ 90.521360] nix-grpc-daemon[1279]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 144 ms (attempt 2/5)5555worker2 # [ 90.670978] nix-grpc-daemon[1279]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 274 ms (attempt 3/5)5556worker2 # [ 90.947995] nix-grpc-daemon[1279]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 181 ms (attempt 4/5)5557worker2 # [ 91.129654] nix-grpc-daemon[1279]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5558client # copying 2 paths...5559client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5560worker2 # [ 91.150159] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:367185561client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5562worker2 # [ 91.160779] nix-daemon[923]: accepted connection from pid 1279, user nix-grpc-daemon (trusted)5563worker2 # [ 91.183272] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:16Z level=info event=niks3_push_started pid=13355564worker2 # [ 91.476171] nix-daemon[923]: accepted connection from pid 1348, user nix-grpc-daemon (trusted)5565worker2 # [ 91.489308] nix-daemon[923]: reaped child process 1351, status = succeeded5566lb # [ 91.641575] niks3-server[1185]: 2026/09/20 08:05:17 INFO Received uploads request method=POST path=/api/pending_closures5567lb # [ 91.651281] niks3-server[1185]: 2026/09/20 08:05:17 INFO Received uploads request method=POST path=/api/pending_closures5568worker2 # [ 91.526333] nix-grpc-daemon[1335]: time=2026-09-20T08:05:17.030Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5569worker2 # [ 91.527708] nix-grpc-daemon[1335]: time=2026-09-20T08:05:17.030Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5570worker2 # [ 91.531656] nix-grpc-daemon[1335]: time=2026-09-20T08:05:17.030Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5571lb # [ 91.693214] niks3-server[1185]: 2026/09/20 08:05:17 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5572lb # [ 91.710370] niks3-server[1185]: 2026/09/20 08:05:17 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5573lb # [ 91.718963] niks3-server[1185]: 2026/09/20 08:05:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5574lb # [ 91.720602] niks3-server[1185]: 2026/09/20 08:05:17 INFO Signed narinfos id=24 count=25575lb # [ 91.722684] niks3-server[1185]: 2026/09/20 08:05:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5576lb # [ 91.733471] niks3-server[1185]: 2026/09/20 08:05:17 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5577lb # [ 91.739439] niks3-server[1185]: 2026/09/20 08:05:17 INFO Signed narinfos id=23 count=15578worker2 # [ 91.603776] nix-grpc-daemon[1335]: time=2026-09-20T08:05:17.108Z level=INFO msg="Uploading 3 narinfos"5579lb # [ 91.743611] niks3-server[1185]: 2026/09/20 08:05:17 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5580lb # [ 91.759471] niks3-server[1185]: 2026/09/20 08:05:17 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5581lb # [ 91.764276] niks3-server[1185]: 2026/09/20 08:05:17 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5582lb # [ 91.770097] niks3-server[1185]: 2026/09/20 08:05:17 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5583lb # [ 91.773168] niks3-server[1185]: 2026/09/20 08:05:17 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5584lb # [ 91.780313] niks3-server[1185]: 2026/09/20 08:05:17 INFO Completed upload id=245585lb # [ 91.781828] niks3-server[1185]: 2026/09/20 08:05:17 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5586lb # [ 91.785199] niks3-server[1185]: 2026/09/20 08:05:17 INFO Completed upload id=235587worker2 # [ 91.648983] nix-grpc-daemon[1335]: time=2026-09-20T08:05:17.153Z level=INFO msg="Upload complete. (318ms)"5588worker2 # [ 91.654411] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36718 duration_s=0 paths=2 nar_bytes_in=3125589worker2 # [ 91.667217] nix-daemon[923]: reaped child process 1333, status = succeeded5590client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.48 seconds)5591worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5592worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5593worker1 # [ 91.920690] systemd[1]: Listening on Nix worker-protocol over gRPC.5594worker1 # [ 91.928288] systemd[1]: Starting Nix worker-protocol over gRPC...5595worker1 # [ 92.071750] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5596worker1 # [ 92.074802] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15597worker1 # [ 92.079810] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5598worker1 # [ 92.083355] systemd[1]: Started Nix worker-protocol over gRPC.5599lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5600lb: must succeed: curl -sf localhost:9901/clusters5601worker1 # [ 92.099482] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:18Z level=info event=scheduler_connected addr=lb:500515602worker2 # [ 91.960173] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05603lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5604lb: must succeed: curl -sf localhost:9901/clusters5605lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5606lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)5607(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.54 seconds)5608subtest: scheduler down: the other node takes the lock and keeps it5609lb: must succeed: curl -sf localhost:9901/clusters5610lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5611lb: must succeed: curl -sf localhost:9901/clusters5612lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5613lb: must succeed: curl -sf localhost:9901/clusters5614lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5615lb: must succeed: curl -sf localhost:9901/clusters5616lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5617worker2 # [ 93.223878] systemd[1]: Stopping Nix worker-protocol over gRPC...5618worker2 # [ 93.355826] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:18Z level=info event=signal_exit5619worker2 # [ 93.566891] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_yield5620lb # [ 93.706512] niks3-server[1185]: 2026/09/20 08:05:19 INFO lead: released remote=192.168.1.4:472145621worker1 # [ 93.715443] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:19Z level=info event=scheduler_restarting addr=lb:500515622worker2 # [ 93.573089] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:367065623worker2 # [ 93.574621] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=worker_gone addr=192.168.1.3:500515624worker2 # [ 93.577548] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_restarting addr=lb:500515625worker2 # [ 93.578973] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:367065626worker2 # [ 93.580504] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=worker_gone addr=192.168.1.4:500515627worker1 # [ 93.918296] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:19Z level=info event=scheduler_connected addr=lb:500515628worker2 # [ 93.777857] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_connected addr=lb:500515629worker2 # [ 93.786279] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5630worker1 # [ 93.927686] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5631worker1 # [ 94.128283] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:20Z level=info event=scheduler_connected addr=lb:500515632worker2 # [ 93.987174] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_connected addr=lb:500515633worker1 # [ 94.131262] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5634worker2 # [ 93.990245] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5635worker1 # [ 94.331836] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:20Z level=info event=scheduler_connected addr=lb:500515636worker2 # [ 94.191365] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_connected addr=lb:500515637worker1 # [ 94.334288] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5638worker2 # [ 94.193989] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5639worker1 # [ 94.535626] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:20Z level=info event=scheduler_connected addr=lb:500515640worker2 # [ 94.396237] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_connected addr=lb:500515641worker2 # [ 94.397587] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5642worker1 # [ 94.540442] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5643worker2 # [ 94.569606] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:20Z level=info event=scheduler_restarting5644worker2 # [ 94.583321] nix-daemon[923]: reaped child process 1328, status = succeeded5645worker2 # [ 94.600896] nix-grpc-daemon[1279]: ts=2026-09-20T08:05:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5646worker1 # [ 94.743791] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:20Z level=info event=scheduler_connected addr=lb:500515647worker1 # [ 94.746893] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5648worker1 # [ 94.944457] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:20Z level=info event=scheduler_connected addr=lb:500515649worker2 # [ 94.803564] nix-grpc-daemon[1279]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5650worker2 # [ 94.805179] nix-grpc-daemon[1279]: E0000 00:00:1789891520.308360 1322 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 11, started true, shutdown true5651worker1 # [ 95.504496] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5652worker2 # [ 95.381701] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5653worker2 # [ 95.384903] systemd[1]: Stopped Nix worker-protocol over gRPC.5654worker2 # [ 95.388866] systemd[1]: nix-grpc-daemon.service: Consumed 314ms CPU time over 8.208s wall clock time, 30.4M memory peak, 17.4K incoming IP traffic, 22.2K outgoing IP traffic.5655worker2 # [ 95.400808] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5656worker2 # [ 95.404362] systemd[1]: Closed Nix worker-protocol over gRPC.5657worker2 # [ 95.407309] systemd[1]: nix-grpc-daemon.socket: Consumed 412us CPU time over 8.825s wall clock time, 256K memory peak, 26.3K incoming IP traffic, 20.2K outgoing IP traffic.5658worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5659worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.04 seconds)5660worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5661worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5662worker1 # [ 95.705052] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:21Z level=info event=scheduler_connected addr=lb:500515663worker1 # [ 95.708025] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5664worker1 # [ 95.909248] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:21Z level=info event=scheduler_connected addr=lb:500515665worker1 # [ 95.910949] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:21Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5666worker1 # [ 96.111659] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_connected addr=lb:500515667worker1 # [ 96.114332] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5668worker1 # [ 96.314877] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_connected addr=lb:500515669worker1 # [ 96.316835] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5670worker1 # [ 96.517521] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_connected addr=lb:500515671worker1 # [ 96.520111] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5672worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5673worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.05 seconds)5674worker1 # [ 96.720574] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_connected addr=lb:500515675worker1 # [ 96.723595] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5676worker1 # [ 96.925900] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_connected addr=lb:500515677worker1 # [ 96.930843] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5678lb # [ 97.084500] niks3-server[1185]: 2026/09/20 08:05:22 INFO lead: acquired remote=192.168.1.3:436385679worker1 # [ 97.093476] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:23Z level=info event=scheduler_take_over5680worker1 # [ 97.131314] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:23Z level=info event=scheduler_connected addr=lb:500515681worker1 # [ 97.133714] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:23Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5682worker1 # [ 97.334365] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:23Z level=info event=scheduler_connected addr=lb:500515683worker1 # [ 97.336377] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:23Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05684worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5685worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5686client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag on-standby >&25687worker1 # [ 97.921157] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447065688worker1 # [ 97.933493] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5689client # don't know how to build these paths:5690client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5691worker1 # [ 97.956380] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:23Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05692worker1 # [ 97.962306] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:23Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15693worker1 # [ 98.005077] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447065694worker1 # [ 98.010855] nix-grpc-daemon[1043]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)5695worker1 # [ 98.013168] nix-grpc-daemon[1043]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 2/5)5696worker1 # [ 98.068786] nix-grpc-daemon[1043]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 3/5)5697worker1 # [ 98.148627] nix-grpc-daemon[1043]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 516 ms (attempt 4/5)5698lb # [ 98.499884] envoy[924]: conn peer=[2001:db8:1::1]:35636 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=305 ms=12755699lb # [ 98.505361] envoy[924]: conn peer=[2001:db8:1::4]:50614 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1323 tx=339 ms=72305700worker1 # [ 98.670993] nix-grpc-daemon[1043]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5701worker1 # [ 98.688530] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447065702client # copying 1 paths to 'lb:50051'...5703client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5704worker1 # [ 98.704150] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447065705worker1 # [ 98.714170] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5706worker1 # [ 98.733752] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:24Z level=info event=niks3_push_started pid=11225707worker1 # [ 99.060878] nix-daemon[846]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)5708worker1 # [ 99.074986] nix-daemon[846]: reaped child process 1138, status = succeeded5709worker1 # [ 99.206961] nix-daemon[846]: accepted connection from pid 1140, user nix-grpc-daemon (trusted)5710worker1 # [ 99.220153] nix-daemon[846]: reaped child process 1143, status = succeeded5711lb # [ 99.223518] niks3-server[1185]: 2026/09/20 08:05:25 INFO Received uploads request method=POST path=/api/pending_closures5712worker1 # [ 99.236350] nix-grpc-daemon[1122]: time=2026-09-20T08:05:25.182Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5713worker1 # [ 99.238387] nix-grpc-daemon[1122]: time=2026-09-20T08:05:25.183Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5714lb # [ 99.245824] niks3-server[1185]: 2026/09/20 08:05:25 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5715lb # [ 99.248917] niks3-server[1185]: 2026/09/20 08:05:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5716lb # [ 99.251418] niks3-server[1185]: 2026/09/20 08:05:25 INFO Signed narinfos id=25 count=15717worker1 # [ 99.258953] nix-grpc-daemon[1122]: time=2026-09-20T08:05:25.206Z level=INFO msg="Uploading 1 narinfos"5718lb # [ 99.255851] niks3-server[1185]: 2026/09/20 08:05:25 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5719lb # [ 99.260729] niks3-server[1185]: 2026/09/20 08:05:25 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5720lb # [ 99.264542] niks3-server[1185]: 2026/09/20 08:05:25 INFO Completed upload id=255721worker1 # [ 99.270409] nix-grpc-daemon[1122]: time=2026-09-20T08:05:25.218Z level=INFO msg="Upload complete. (356ms)"5722lb # [ 99.266433] niks3-server[1185]: 2026/09/20 08:05:25 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5723worker1 # [ 99.273401] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44706 duration_s=0 paths=1 nar_bytes_in=5205724worker1 # [ 99.277308] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447065725worker1 # [ 99.284256] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5726worker1 # [ 99.286762] nix-daemon[846]: reaped child process 1120, status = succeeded5727worker1 # [ 99.293861] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5728client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5729client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5730client # farm-a> LOG-farm-a-on-standby5731worker1 # [ 100.309891] nix-daemon[846]: reaped child process 1147, status = succeeded5732worker1 # [ 100.439512] nix-daemon[846]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5733worker1 # [ 100.453201] nix-daemon[846]: reaped child process 1165, status = succeeded5734lb # [ 100.461245] niks3-server[1185]: 2026/09/20 08:05:26 INFO Received uploads request method=POST path=/api/pending_closures5735worker1 # [ 100.475955] nix-grpc-daemon[1122]: time=2026-09-20T08:05:26.423Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5736worker1 # [ 100.477470] nix-grpc-daemon[1122]: time=2026-09-20T08:05:26.425Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5737lb # [ 100.493476] niks3-server[1185]: 2026/09/20 08:05:26 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5738lb # [ 100.512069] niks3-server[1185]: 2026/09/20 08:05:26 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5739lb # [ 100.522425] niks3-server[1185]: 2026/09/20 08:05:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5740lb # [ 100.526766] niks3-server[1185]: 2026/09/20 08:05:26 INFO Signed narinfos id=26 count=15741worker1 # [ 100.536283] nix-grpc-daemon[1122]: time=2026-09-20T08:05:26.483Z level=INFO msg="Uploading 1 narinfos"5742lb # [ 100.531602] niks3-server[1185]: 2026/09/20 08:05:26 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5743lb # [ 100.541069] niks3-server[1185]: 2026/09/20 08:05:26 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5744lb # [ 100.548056] niks3-server[1185]: 2026/09/20 08:05:26 INFO Completed upload id=265745worker1 # [ 100.554278] nix-grpc-daemon[1122]: time=2026-09-20T08:05:26.502Z level=INFO msg="Upload complete. (253ms)"5746lb # [ 100.550204] niks3-server[1185]: 2026/09/20 08:05:26 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5747worker1 # [ 100.562551] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44706 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15748worker1 # [ 100.575217] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:26Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05749worker1 # [ 100.577440] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:26Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15750worker1 # [ 100.581541] nix-daemon[846]: reaped child process 1145, status = succeeded5751worker1 # [ 100.620112] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447065752worker1 # [ 100.631837] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447065753worker1 # [ 100.641238] nix-grpc-daemon[1122]: time=2026-09-20T08:05:26.589Z level=INFO msg="All 1 paths already cached"5754client # copying 1 paths to 'lb:50051'...5755client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5756worker1 # [ 100.646108] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447065757worker1 # [ 100.651075] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5758worker1 # [ 100.784160] nix-daemon[846]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5759worker1 # [ 100.798502] nix-daemon[846]: reaped child process 1174, status = succeeded5760worker1 # [ 100.943271] nix-daemon[846]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5761worker1 # [ 100.957692] nix-daemon[846]: reaped child process 1179, status = succeeded5762lb # [ 100.959736] niks3-server[1185]: 2026/09/20 08:05:26 INFO Received uploads request method=POST path=/api/pending_closures5763worker1 # [ 100.970370] nix-grpc-daemon[1122]: time=2026-09-20T08:05:26.917Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5764worker1 # [ 100.971789] nix-grpc-daemon[1122]: time=2026-09-20T08:05:26.917Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5765lb # [ 100.981710] niks3-server[1185]: 2026/09/20 08:05:26 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5766lb # [ 100.991612] niks3-server[1185]: 2026/09/20 08:05:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5767lb # [ 100.996379] niks3-server[1185]: 2026/09/20 08:05:26 INFO Signed narinfos id=27 count=15768worker1 # [ 101.005579] nix-grpc-daemon[1122]: time=2026-09-20T08:05:26.953Z level=INFO msg="Uploading 1 narinfos"5769lb # [ 101.009552] niks3-server[1185]: 2026/09/20 08:05:26 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5770lb # [ 101.014163] niks3-server[1185]: 2026/09/20 08:05:26 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5771worker1 # [ 101.025746] nix-grpc-daemon[1122]: time=2026-09-20T08:05:26.973Z level=INFO msg="Upload complete. (369ms)"5772lb # [ 101.019840] niks3-server[1185]: 2026/09/20 08:05:26 INFO Completed upload id=275773worker1 # [ 101.029075] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44706 duration_s=0 paths=1 nar_bytes_in=6405774lb # [ 101.025081] niks3-server[1185]: 2026/09/20 08:05:26 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5775worker1 # [ 101.033198] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447065776worker1 # [ 101.038819] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5777worker1 # [ 101.041147] nix-daemon[846]: reaped child process 1169, status = succeeded5778client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5779worker1 # [ 101.046520] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5780client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5781client # farm-b> LOG-farm-b-on-standby5782worker1 # [ 102.676364] nix-daemon[846]: reaped child process 1183, status = succeeded5783worker1 # [ 102.803660] nix-daemon[846]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)5784worker1 # [ 102.815975] nix-daemon[846]: reaped child process 1201, status = succeeded5785lb # [ 102.824831] niks3-server[1185]: 2026/09/20 08:05:28 INFO Received uploads request method=POST path=/api/pending_closures5786worker1 # [ 102.838860] nix-grpc-daemon[1122]: time=2026-09-20T08:05:28.786Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5787worker1 # [ 102.840393] nix-grpc-daemon[1122]: time=2026-09-20T08:05:28.788Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5788lb # [ 102.848479] niks3-server[1185]: 2026/09/20 08:05:28 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5789lb # [ 102.852135] niks3-server[1185]: 2026/09/20 08:05:28 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5790lb # [ 102.856057] niks3-server[1185]: 2026/09/20 08:05:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5791lb # [ 102.859573] niks3-server[1185]: 2026/09/20 08:05:28 INFO Signed narinfos id=28 count=15792worker1 # [ 102.867204] nix-grpc-daemon[1122]: time=2026-09-20T08:05:28.814Z level=INFO msg="Uploading 1 narinfos"5793lb # [ 102.863542] niks3-server[1185]: 2026/09/20 08:05:28 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5794lb # [ 102.868592] niks3-server[1185]: 2026/09/20 08:05:28 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5795lb # [ 102.872664] niks3-server[1185]: 2026/09/20 08:05:28 INFO Completed upload id=285796worker1 # [ 102.878693] nix-grpc-daemon[1122]: time=2026-09-20T08:05:28.826Z level=INFO msg="Upload complete. (224ms)"5797lb # [ 102.874175] niks3-server[1185]: 2026/09/20 08:05:28 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5798worker1 # [ 102.881677] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44706 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15799worker1 # [ 102.888421] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:28Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05800worker1 # [ 102.890473] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:28Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15801worker1 # [ 102.895840] nix-daemon[846]: reaped child process 1181, status = succeeded5802worker1 # [ 102.932383] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447065803worker1 # [ 102.939096] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447065804worker1 # [ 102.945322] nix-grpc-daemon[1122]: time=2026-09-20T08:05:28.893Z level=INFO msg="All 2 paths already cached"5805client # copying 1 paths to 'lb:50051'...5806client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5807worker1 # [ 102.949651] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447065808worker1 # [ 102.955086] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5809worker1 # [ 103.079182] nix-daemon[846]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)5810worker1 # [ 103.094527] nix-daemon[846]: reaped child process 1210, status = succeeded5811worker1 # [ 103.228355] nix-daemon[846]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)5812worker1 # [ 103.242832] nix-daemon[846]: reaped child process 1215, status = succeeded5813lb # [ 103.246427] niks3-server[1185]: 2026/09/20 08:05:29 INFO Received uploads request method=POST path=/api/pending_closures5814worker1 # [ 103.256076] nix-grpc-daemon[1122]: time=2026-09-20T08:05:29.203Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5815worker1 # [ 103.257566] nix-grpc-daemon[1122]: time=2026-09-20T08:05:29.205Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5816lb # [ 103.262105] niks3-server[1185]: 2026/09/20 08:05:29 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5817lb # [ 103.265701] niks3-server[1185]: 2026/09/20 08:05:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5818lb # [ 103.267376] niks3-server[1185]: 2026/09/20 08:05:29 INFO Signed narinfos id=29 count=15819worker1 # [ 103.274547] nix-grpc-daemon[1122]: time=2026-09-20T08:05:29.222Z level=INFO msg="Uploading 1 narinfos"5820lb # [ 103.271455] niks3-server[1185]: 2026/09/20 08:05:29 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5821lb # [ 103.277096] niks3-server[1185]: 2026/09/20 08:05:29 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5822lb # [ 103.279959] niks3-server[1185]: 2026/09/20 08:05:29 INFO Completed upload id=295823worker1 # [ 103.285906] nix-grpc-daemon[1122]: time=2026-09-20T08:05:29.233Z level=INFO msg="Upload complete. (325ms)"5824worker1 # [ 103.287869] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44706 duration_s=0 paths=1 nar_bytes_in=7685825lb # [ 103.282927] niks3-server[1185]: 2026/09/20 08:05:29 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5826worker1 # [ 103.291584] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447065827worker1 # [ 103.298300] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5828worker1 # [ 103.303153] nix-daemon[846]: reaped child process 1205, status = succeeded5829worker1 # [ 103.309102] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5830client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5831client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5832client # farm-top> LOG-farm-top-on-standby5833worker1 # [ 104.527083] nix-daemon[846]: reaped child process 1219, status = succeeded5834worker1 # [ 104.671250] nix-daemon[846]: accepted connection from pid 1234, user nix-grpc-daemon (trusted)5835worker1 # [ 104.687146] nix-daemon[846]: reaped child process 1237, status = succeeded5836lb # [ 104.689975] niks3-server[1185]: 2026/09/20 08:05:30 INFO Received uploads request method=POST path=/api/pending_closures5837worker1 # [ 104.702711] nix-grpc-daemon[1122]: time=2026-09-20T08:05:30.650Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5838worker1 # [ 104.704315] nix-grpc-daemon[1122]: time=2026-09-20T08:05:30.652Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5839lb # [ 104.719164] niks3-server[1185]: 2026/09/20 08:05:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5840lb # [ 104.723041] niks3-server[1185]: 2026/09/20 08:05:30 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5841lb # [ 104.725820] niks3-server[1185]: 2026/09/20 08:05:30 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5842lb # [ 104.733744] niks3-server[1185]: 2026/09/20 08:05:30 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5843lb # [ 104.737955] niks3-server[1185]: 2026/09/20 08:05:30 INFO Signed narinfos id=30 count=15844worker1 # [ 104.744350] nix-grpc-daemon[1122]: time=2026-09-20T08:05:30.691Z level=INFO msg="Uploading 1 narinfos"5845lb # [ 104.744592] niks3-server[1185]: 2026/09/20 08:05:30 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5846lb # [ 104.753073] niks3-server[1185]: 2026/09/20 08:05:30 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5847lb # [ 104.757086] niks3-server[1185]: 2026/09/20 08:05:30 INFO Completed upload id=305848worker1 # [ 104.763091] nix-grpc-daemon[1122]: time=2026-09-20T08:05:30.710Z level=INFO msg="Upload complete. (245ms)"5849worker1 # [ 104.766554] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44706 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15850worker1 # [ 104.774420] nix-daemon[846]: reaped child process 1217, status = succeeded5851client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag on-standby >&2, in 7.06 seconds)5852client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag on-standby outPath5853client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag on-standby outPath, in 0.16 seconds)5854worker2 # [ 104.866689] systemd[1]: Listening on Nix worker-protocol over gRPC.5855worker2 # [ 104.871937] systemd[1]: Starting Nix worker-protocol over gRPC...5856worker2 # [ 105.041516] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5857worker2 # [ 105.043781] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15858worker2 # [ 105.050875] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5859worker2 # [ 105.052551] systemd[1]: Started Nix worker-protocol over gRPC.5860lb: must succeed: curl -sf localhost:9901/clusters5861worker2 # [ 105.066772] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:30Z level=info event=scheduler_connected addr=lb:500515862worker1 # [ 105.210875] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05863lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5864worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5865worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.06 seconds)5866lb: must succeed: curl -sf localhost:9901/clusters5867lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5868lb: must succeed: curl -sf localhost:9901/clusters5869lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5870worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5871worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5872(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.28 seconds)5873subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5874client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5875client # [ 105.537825] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5876client # Running as unit: rs.service; invocation ID: 7831c46651ab4d3d94aef19d724fd0df5877client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5878worker1 # [ 105.746601] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447065879client # [ 105.769499] nix[1165]: don't know how to build these paths:5880client # [ 105.770916] nix[1165]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5881worker1 # [ 105.770090] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:31Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05882worker1 # [ 105.775382] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:31Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15883worker1 # [ 105.780309] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447065884worker1 # [ 105.794210] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447065885client # [ 105.820284] nix[1165]: copying 1 paths to 'lb:50051'...5886client # [ 105.822274] nix[1165]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5887worker1 # [ 105.807163] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447065888worker1 # [ 105.819205] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5889worker1 # [ 105.972889] nix-daemon[846]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5890worker1 # [ 105.990572] nix-daemon[846]: reaped child process 1255, status = succeeded5891worker1 # [ 106.138100] nix-daemon[846]: accepted connection from pid 1257, user nix-grpc-daemon (trusted)5892worker1 # [ 106.151667] nix-daemon[846]: reaped child process 1260, status = succeeded5893lb # [ 106.160536] niks3-server[1185]: 2026/09/20 08:05:32 INFO Received uploads request method=POST path=/api/pending_closures5894worker1 # [ 106.177471] nix-grpc-daemon[1122]: time=2026-09-20T08:05:32.125Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5895worker1 # [ 106.179440] nix-grpc-daemon[1122]: time=2026-09-20T08:05:32.126Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5896lb # [ 106.192781] niks3-server[1185]: 2026/09/20 08:05:32 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5897lb # [ 106.205722] niks3-server[1185]: 2026/09/20 08:05:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5898worker1 # [ 106.217096] nix-grpc-daemon[1122]: time=2026-09-20T08:05:32.164Z level=INFO msg="Uploading 1 narinfos"5899lb # [ 106.210996] niks3-server[1185]: 2026/09/20 08:05:32 INFO Signed narinfos id=31 count=15900lb # [ 106.224677] niks3-server[1185]: 2026/09/20 08:05:32 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5901lb # [ 106.240849] niks3-server[1185]: 2026/09/20 08:05:32 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5902worker1 # [ 106.251539] nix-grpc-daemon[1122]: time=2026-09-20T08:05:32.199Z level=INFO msg="Upload complete. (421ms)"5903lb # [ 106.245539] niks3-server[1185]: 2026/09/20 08:05:32 INFO Completed upload id=315904lb # [ 106.249775] niks3-server[1185]: 2026/09/20 08:05:32 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5905worker1 # [ 106.255126] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44706 duration_s=0 paths=1 nar_bytes_in=4165906worker1 # [ 106.261528] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447065907worker1 # [ 106.272120] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5908worker1 # [ 106.283187] nix-daemon[846]: reaped child process 1250, status = succeeded5909worker1 # [ 106.288310] nix-daemon[846]: accepted connection from pid 1043, user nix-grpc-daemon (trusted)5910client # [ 106.302897] nix[1165]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5911lb: must succeed: curl -sf localhost:9901/clusters5912lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5913lb: must succeed: curl -sf localhost:9901/clusters5914lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5915lb: must succeed: curl -sf localhost:9901/clusters5916lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5917lb: must succeed: curl -sf localhost:9901/clusters5918lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5919worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5920worker2 # [ 108.195280] systemd-journald[287]: Received client request to rotate journal, rotating.5921worker2 # [ 108.317633] systemd-journald[287]: Vacuuming done, freed 0B of archived journals from /var/log/journal/f005de3fe2be45e88412cb3c5518cc13.5922worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.16 seconds)5923worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5924worker1 # [ 108.312551] systemd[1]: Stopping Nix worker-protocol over gRPC...5925worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5926??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5927 File "/nix/store/3gycnalwgpix76jjky7ljgh88gwgdb0l-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735928client: must succeed: sleep 5.05929lb # [ 108.500796] envoy[924]: conn peer=[2001:db8:1::1]:44808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9451 tx=3777 ms=68625930worker1 # [ 109.087314] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:35Z level=info event=draining5931client: (finished: must succeed: sleep 5.0, in 5.04 seconds)5932worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5933worker1 # [ 113.714883] nix-daemon[846]: reaped child process 1264, status = succeeded5934worker1 # [ 113.853073] nix-daemon[846]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5935worker1 # [ 113.865473] nix-daemon[846]: reaped child process 1307, status = succeeded5936lb # [ 113.869231] niks3-server[1185]: 2026/09/20 08:05:39 INFO Received uploads request method=POST path=/api/pending_closures5937worker1 # [ 113.880574] nix-grpc-daemon[1122]: time=2026-09-20T08:05:39.828Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5938worker1 # [ 113.882208] nix-grpc-daemon[1122]: time=2026-09-20T08:05:39.830Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5939lb # [ 113.896302] niks3-server[1185]: 2026/09/20 08:05:39 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5940lb # [ 113.906298] niks3-server[1185]: 2026/09/20 08:05:39 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5941lb # [ 113.911242] niks3-server[1185]: 2026/09/20 08:05:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5942lb # [ 113.914178] niks3-server[1185]: 2026/09/20 08:05:39 INFO Signed narinfos id=32 count=15943worker1 # [ 113.920210] nix-grpc-daemon[1122]: time=2026-09-20T08:05:39.868Z level=INFO msg="Uploading 1 narinfos"5944lb # [ 113.918933] niks3-server[1185]: 2026/09/20 08:05:39 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5945lb # [ 113.923959] niks3-server[1185]: 2026/09/20 08:05:39 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5946lb # [ 113.927406] niks3-server[1185]: 2026/09/20 08:05:39 INFO Completed upload id=325947worker1 # [ 113.933367] nix-grpc-daemon[1122]: time=2026-09-20T08:05:39.881Z level=INFO msg="Upload complete. (230ms)"5948lb # [ 113.930719] niks3-server[1185]: 2026/09/20 08:05:39 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5949worker1 # [ 113.937174] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44706 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15950worker1 # [ 113.944599] nix-daemon[846]: reaped child process 1262, status = succeeded5951client # [ 113.983802] systemd[1]: rs.service: Deactivated successfully.5952client # [ 113.985622] systemd[1]: rs.service: Consumed 132ms CPU time over 8.445s wall clock time, 37.1M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.5953worker1 # [ 114.090772] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=signal_exit5954lb # [ 114.305244] niks3-server[1185]: 2026/09/20 08:05:40 INFO lead: released remote=192.168.1.3:436385955worker1 # [ 114.310177] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=scheduler_yield5956worker2 # [ 114.171768] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:39Z level=info event=scheduler_restarting addr=lb:500515957worker1 # [ 114.315585] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:446905958worker1 # [ 114.320332] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=worker_gone addr=192.168.1.4:500515959worker1 # [ 114.323441] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=scheduler_restarting addr=lb:500515960worker1 # [ 114.326479] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:446905961worker1 # [ 114.329776] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=worker_gone addr=192.168.1.3:500515962worker2 # [ 114.374033] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:39Z level=info event=scheduler_connected addr=lb:500515963worker1 # [ 114.519788] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=scheduler_connected addr=lb:500515964worker2 # [ 114.380808] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5965worker1 # [ 114.523415] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5966worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.09 seconds)5967client: waiting for success: ! systemctl is-active rs5968client: (finished: waiting for success: ! systemctl is-active rs, in 0.05 seconds)5969client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5970client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5971client: must fail: journalctl -u rs | grep -q 'reconnecting'5972worker1 # [ 114.724609] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=scheduler_connected addr=lb:500515973worker2 # [ 114.582425] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:40Z level=info event=scheduler_connected addr=lb:500515974worker2 # [ 114.588615] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5975worker1 # [ 114.730753] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5976client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5977lb: must succeed: curl -sf localhost:9901/clusters5978lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5979worker2 # [ 114.788816] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:40Z level=info event=scheduler_connected addr=lb:500515980worker1 # [ 114.931797] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=scheduler_connected addr=lb:500515981worker2 # [ 114.794608] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5982worker1 # [ 114.937971] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5983worker1 # [ 115.136891] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:41Z level=info event=scheduler_connected addr=lb:500515984worker2 # [ 114.995358] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:40Z level=info event=scheduler_connected addr=lb:500515985worker1 # [ 115.139785] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5986worker2 # [ 114.998070] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5987worker2 # [ 115.060555] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:40Z level=info event=scheduler_take_over5988lb # [ 115.197476] niks3-server[1185]: 2026/09/20 08:05:41 INFO lead: acquired remote=192.168.1.4:593725989worker1 # [ 115.313825] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:41Z level=info event=scheduler_restarting5990worker1 # [ 115.340392] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:41Z level=info event=scheduler_connected addr=lb:500515991worker1 # [ 115.342203] nix-grpc-daemon[1043]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5992worker2 # [ 115.200094] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:40Z level=info event=scheduler_connected addr=lb:500515993worker1 # [ 115.343523] nix-grpc-daemon[1043]: E0000 00:00:1789891541.289978 1086 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5994worker2 # [ 115.201608] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05995worker1 # [ 115.364210] nix-grpc-daemon[1043]: ts=2026-09-20T08:05:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5996worker1 # [ 115.459613] nix-daemon[846]: reaped child process 1113, status = succeeded5997lb: must succeed: curl -sf localhost:9901/clusters5998lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5999worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6000worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.06 seconds)6001worker1 # [ 116.310964] nix-grpc-daemon[1043]: E0000 00:00:1789891542.258286 1086 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 12, started true, shutdown true6002worker1 # [ 116.313100] nix-grpc-daemon[1043]: E0000 00:00:1789891542.258453 1086 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6003lb: must succeed: curl -sf localhost:9901/clusters6004lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6005worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6006worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)6007worker1 # [ 117.364583] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6008worker1 # [ 117.367448] systemd[1]: Stopped Nix worker-protocol over gRPC.6009worker1 # [ 117.370420] systemd[1]: nix-grpc-daemon.service: Consumed 1.310s CPU time over 25.435s wall clock time, 33.3M memory peak, 47.4K incoming IP traffic, 90.8K outgoing IP traffic.6010worker1 # [ 117.387823] systemd[1]: Starting Nix worker-protocol over gRPC...6011worker1 # [ 117.545899] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6012worker1 # [ 117.548902] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16013worker1 # [ 117.555054] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6014worker1 # [ 117.559108] systemd[1]: Started Nix worker-protocol over gRPC.6015worker1 # [ 117.567353] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:43Z level=info event=scheduler_connected addr=lb:500516016worker2 # [ 117.427324] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06017lb: must succeed: curl -sf localhost:9901/clusters6018lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6019worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6020worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)6021(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.65 seconds)6022subtest: scheduler restart mid-build: build finishes, no double build6023client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid6024client # [ 118.186397] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.6025client # Running as unit: mid.service; invocation ID: 19d7779bc94944ad956134a70c1ded1f6026client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)6027worker2 # [ 118.232115] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:352966028worker2 # [ 118.237831] nix-daemon[923]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)6029client # [ 118.404552] nix[1201]: don't know how to build these paths:6030client # [ 118.405642] nix[1201]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6031worker2 # [ 118.255079] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:43Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06032worker2 # [ 118.257146] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:43Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16033worker2 # [ 118.303829] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966034worker2 # [ 118.310836] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)6035worker2 # [ 118.363577] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 2/5)6036lb # [ 118.501044] envoy[924]: conn peer=[2001:db8:1::1]:33148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2934 tx=1325 ms=82076037lb # [ 118.502757] envoy[924]: conn peer=[2001:db8:1::3]:59552 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5380 tx=1319 ms=232626038worker2 # [ 118.415221] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 3/5)6039worker2 # [ 118.462675] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 263 ms (attempt 4/5)6040worker2 # [ 118.726053] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6041worker2 # [ 118.741158] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352966042client # [ 118.910508] nix[1201]: copying 1 paths to 'lb:50051'...6043client # [ 118.914153] nix[1201]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6044worker2 # [ 118.757905] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352966045worker2 # [ 118.768176] nix-daemon[923]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)6046worker2 # [ 118.793523] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:44Z level=info event=niks3_push_started pid=14956047worker2 # [ 119.132623] nix-daemon[923]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)6048worker2 # [ 119.146598] nix-daemon[923]: reaped child process 1511, status = succeeded6049worker2 # [ 119.317430] nix-daemon[923]: accepted connection from pid 1513, user nix-grpc-daemon (trusted)6050worker2 # [ 119.331785] nix-daemon[923]: reaped child process 1520, status = succeeded6051lb # [ 119.483883] niks3-server[1185]: 2026/09/20 08:05:45 INFO Received uploads request method=POST path=/api/pending_closures6052worker2 # [ 119.353412] nix-grpc-daemon[1495]: time=2026-09-20T08:05:44.857Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6053worker2 # [ 119.355638] nix-grpc-daemon[1495]: time=2026-09-20T08:05:44.857Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6054lb # [ 119.509183] niks3-server[1185]: 2026/09/20 08:05:45 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6055worker2 # [ 119.376194] nix-grpc-daemon[1495]: time=2026-09-20T08:05:44.881Z level=INFO msg="Uploading 1 narinfos"6056lb # [ 119.512957] niks3-server[1185]: 2026/09/20 08:05:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign6057lb # [ 119.514728] niks3-server[1185]: 2026/09/20 08:05:45 INFO Signed narinfos id=33 count=16058lb # [ 119.523361] niks3-server[1185]: 2026/09/20 08:05:45 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete6059lb # [ 119.528492] niks3-server[1185]: 2026/09/20 08:05:45 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6060lb # [ 119.531368] niks3-server[1185]: 2026/09/20 08:05:45 INFO Completed upload id=336061worker2 # [ 119.394905] nix-grpc-daemon[1495]: time=2026-09-20T08:05:44.899Z level=INFO msg="Upload complete. (415ms)"6062worker2 # [ 119.398086] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=0 paths=1 nar_bytes_in=4246063worker2 # [ 119.400811] nix-grpc-daemon[1372]: ts=2026-09-20T08:05:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966064worker2 # [ 119.405190] nix-grpc-daemon[1495]: time=2026-09-20T08:05:44.910Z level=WARN msg="Failed to register uploaded object" key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo error="server returned 404: object is not pending upload\n"6065worker2 # [ 119.412095] nix-daemon[923]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)6066worker2 # [ 119.414632] nix-daemon[923]: reaped child process 1493, status = succeeded6067worker2 # [ 119.420787] nix-daemon[923]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)6068client # [ 119.578263] nix[1201]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6069lb: must succeed: curl -sf localhost:9901/clusters6070lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6071lb: must succeed: curl -sf localhost:9901/clusters6072lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6073worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6074worker2 # [ 120.623394] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1372 (nix-grpc-daemon) on client request.6075worker2 # [ 120.632532] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6076lb # [ 120.772476] niks3-server[1185]: 2026/09/20 08:05:46 INFO lead: released remote=192.168.1.4:593726077client # [ 120.792912] nix[1201]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6078worker1 # [ 120.777276] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:46Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6079client # [ 120.797354] nix[1201]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6080worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)6081client: must succeed: sleep 5.06082worker2 # [ 120.656404] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6083worker2 # [ 120.659952] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6084worker2 # [ 120.661990] systemd[1]: nix-grpc-daemon.service: Consumed 463ms CPU time over 15.783s wall clock time, 30.7M memory peak, 14.9K incoming IP traffic, 22.6K outgoing IP traffic.6085worker2 # [ 120.673515] nix-daemon[923]: reaped child process 1486, status = succeeded6086worker2 # [ 120.674968] nix-daemon[923]: reaped child process 1522, status = succeeded6087worker2 # [ 120.688428] nix-daemon[1524]: unexpected Nix daemon error: error: interrupted by the user6088worker2 # [ 120.693935] nix-daemon[923]: reaped child process 1524, status = failed with exit code 16089worker1 # [ 120.977675] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:46Z level=info event=scheduler_connected addr=lb:500516090worker2 # [ 120.838045] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6091worker2 # [ 120.843062] systemd[1]: Starting Nix worker-protocol over gRPC...6092worker2 # [ 120.986915] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6093worker2 # [ 120.990419] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16094worker2 # [ 120.999333] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6095worker2 # [ 121.002453] systemd[1]: Started Nix worker-protocol over gRPC.6096worker1 # [ 121.152969] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6097lb # [ 121.149194] niks3-server[1185]: 2026/09/20 08:05:47 INFO lead: acquired remote=192.168.1.4:376226098worker2 # [ 121.015853] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=info event=scheduler_take_over6099worker2 # [ 121.020496] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=info event=scheduler_connected addr=lb:500516100worker2 # [ 121.022268] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06101worker2 # [ 121.143205] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06102worker2 # [ 121.145265] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16103worker2 # [ 121.197928] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:353266104worker2 # [ 121.203083] nix-daemon[923]: accepted connection from pid 1556, user nix-grpc-daemon (trusted)6105worker2 # [ 121.209540] nix-daemon[923]: accepted connection from pid 1556, user nix-grpc-daemon (trusted)6106worker2 # [ 121.212513] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06107worker1 # [ 121.353971] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:47Z level=info event=scheduler_connected addr=lb:500516108client # [ 121.376213] nix[1201]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6109worker2 # [ 121.219360] nix-daemon[923]: accepted connection from pid 1556, user nix-grpc-daemon (trusted)6110client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6111client: waiting for success: ! systemctl is-active mid6112worker2 # [ 126.036983] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:51Z level=info event=niks3_push_started pid=16356113worker2 # [ 126.069555] nix-daemon[923]: reaped child process 1611, status = succeeded6114worker2 # [ 126.391959] nix-daemon[923]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6115worker2 # [ 126.405754] nix-daemon[923]: reaped child process 1651, status = succeeded6116lb # [ 126.554357] niks3-server[1185]: 2026/09/20 08:05:52 INFO Received uploads request method=POST path=/api/pending_closures6117worker2 # [ 126.425894] nix-grpc-daemon[1635]: time=2026-09-20T08:05:51.930Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6118worker2 # [ 126.431121] nix-grpc-daemon[1635]: time=2026-09-20T08:05:51.932Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6119lb # [ 126.599901] niks3-server[1185]: 2026/09/20 08:05:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign6120lb # [ 126.610653] niks3-server[1185]: 2026/09/20 08:05:52 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6121worker2 # [ 126.481871] nix-grpc-daemon[1635]: time=2026-09-20T08:05:51.986Z level=INFO msg="Uploading 1 narinfos"6122lb # [ 126.617901] niks3-server[1185]: 2026/09/20 08:05:52 INFO Signed narinfos id=34 count=16123lb # [ 126.635708] niks3-server[1185]: 2026/09/20 08:05:52 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6124lb # [ 126.637403] niks3-server[1185]: 2026/09/20 08:05:52 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete6125lb # [ 126.641859] niks3-server[1185]: 2026/09/20 08:05:52 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6126lb # [ 126.645276] niks3-server[1185]: 2026/09/20 08:05:52 INFO Completed upload id=346127lb # [ 126.647301] niks3-server[1185]: 2026/09/20 08:05:52 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6128worker2 # [ 126.510473] nix-grpc-daemon[1635]: time=2026-09-20T08:05:52.015Z level=INFO msg="Upload complete. (277ms)"6129worker2 # [ 126.517745] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35326 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16130worker2 # [ 126.530841] nix-daemon[923]: reaped child process 1609, status = succeeded6131client # [ 126.708513] systemd[1]: mid.service: Deactivated successfully.6132client # [ 126.710496] systemd[1]: mid.service: Consumed 128ms CPU time over 8.524s wall clock time, 39.3M memory peak, 9.6K incoming IP traffic, 10.8K outgoing IP traffic.6133client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6134client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6135client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6136(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.01 seconds)6137subtest: interrupting the client stops the build on the worker6138client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6139client # [ 127.169567] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6140client # Running as unit: intr.service; invocation ID: ac8df6407bc2455f91bda893026511f86141client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6142client # [ 127.362918] nix[1237]: don't know how to build these paths:6143client # [ 127.363852] nix[1237]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6144worker2 # [ 127.204450] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:353266145worker2 # [ 127.217166] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:52Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06146worker2 # [ 127.221435] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:52Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16147worker1 # [ 127.416384] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886148worker1 # [ 127.421735] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6149worker1 # [ 127.429964] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)6150worker1 # [ 127.513576] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 2/5)6151worker1 # [ 127.592198] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 365 ms (attempt 3/5)6152worker1 # [ 127.957597] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 256 ms (attempt 4/5)6153worker1 # [ 128.217704] nix-grpc-daemon[1314]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6154worker1 # [ 128.233243] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886155client # [ 128.257135] nix[1237]: copying 1 paths to 'lb:50051'...6156client # [ 128.258611] nix[1237]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6157worker1 # [ 128.243669] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886158worker1 # [ 128.251112] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6159worker1 # [ 128.266474] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:54Z level=info event=niks3_push_started pid=13986160lb # [ 128.501898] envoy[924]: conn peer=[2001:db8:1::4]:48362 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1560 tx=496 ms=155706161lb # [ 128.504086] envoy[924]: conn peer=[2001:db8:1::1]:46262 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3865 tx=1653 ms=83026162worker1 # [ 128.607425] nix-daemon[846]: accepted connection from pid 1415, user nix-grpc-daemon (trusted)6163worker1 # [ 128.622728] nix-daemon[846]: reaped child process 1418, status = succeeded6164worker1 # [ 128.789660] nix-daemon[846]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)6165worker1 # [ 128.804358] nix-daemon[846]: reaped child process 1423, status = succeeded6166lb # [ 128.813722] niks3-server[1185]: 2026/09/20 08:05:54 INFO Received uploads request method=POST path=/api/pending_closures6167worker1 # [ 128.831771] nix-grpc-daemon[1398]: time=2026-09-20T08:05:54.779Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6168worker1 # [ 128.834843] nix-grpc-daemon[1398]: time=2026-09-20T08:05:54.781Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6169lb # [ 128.853628] niks3-server[1185]: 2026/09/20 08:05:54 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6170lb # [ 128.864955] niks3-server[1185]: 2026/09/20 08:05:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign6171lb # [ 128.870268] niks3-server[1185]: 2026/09/20 08:05:54 INFO Signed narinfos id=35 count=16172worker1 # [ 128.880047] nix-grpc-daemon[1398]: time=2026-09-20T08:05:54.827Z level=INFO msg="Uploading 1 narinfos"6173lb # [ 128.881199] niks3-server[1185]: 2026/09/20 08:05:54 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6174lb # [ 128.892407] niks3-server[1185]: 2026/09/20 08:05:54 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete6175lb # [ 128.898306] niks3-server[1185]: 2026/09/20 08:05:54 INFO Completed upload id=356176worker1 # [ 128.906262] nix-grpc-daemon[1398]: time=2026-09-20T08:05:54.854Z level=INFO msg="Upload complete. (451ms)"6177lb # [ 128.901914] niks3-server[1185]: 2026/09/20 08:05:54 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6178worker1 # [ 128.910356] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=4246179worker1 # [ 128.913290] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886180worker1 # [ 128.917875] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6181worker1 # [ 128.925695] nix-daemon[846]: reaped child process 1396, status = succeeded6182worker1 # [ 128.929062] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6183client # [ 128.944630] nix[1237]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6184client: must succeed: systemctl kill -s INT intr6185client # [ 130.722609] systemd[1]: intr.service: Sent signal SIGINT to main process 1237 (nix) on client request.6186worker1 # [ 130.716121] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:56Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6187client # [ 130.732369] nix[1237]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6188client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6189client # [ 130.740503] nix[1237]: error: interrupted by the user6190client # [ 130.769363] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6191client # [ 130.771090] systemd[1]: intr.service: Failed with result 'exit-code'.6192worker1 # [ 130.757451] nix-daemon[846]: reaped child process 1425, status = succeeded6193client # [ 130.773991] systemd[1]: intr.service: Consumed 118ms CPU time over 3.603s wall clock time, 39.1M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.6194worker1 # [ 130.762302] nix-daemon[1427]: unexpected Nix daemon error: error: interrupted by the user6195worker1 # [ 130.773237] nix-daemon[846]: reaped child process 1427, status = failed with exit code 16196(finished: subtest: interrupting the client stops the build on the worker, in 3.70 seconds)6197subtest: a deploy mid-build drains: the build finishes, the new generation takes over6198client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6199client # [ 130.876102] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6200client # Running as unit: sw.service; invocation ID: e1c51db655894bc0a4c3b521255970e06201client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6202worker1 # [ 131.043045] nix-grpc-daemon[1314]: ts=2026-09-20T08:05:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:410886203client # [ 131.062418] nix[1258]: don't know how to build these paths:6204client # [ 131.063437] nix[1258]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6205worker2 # [ 130.916976] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:56Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06206worker2 # [ 130.919717] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:56Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16207worker2 # [ 130.960609] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:353266208worker2 # [ 130.971472] nix-grpc-daemon[1556]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)6209worker2 # [ 131.009563] nix-grpc-daemon[1556]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 2/5)6210worker2 # [ 131.015392] nix-grpc-daemon[1556]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 384 ms (attempt 3/5)6211worker2 # [ 131.403096] nix-grpc-daemon[1556]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 459 ms (attempt 4/5)6212worker2 # [ 131.867652] nix-grpc-daemon[1556]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6213worker2 # [ 131.891238] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:353266214client # [ 132.060808] nix[1258]: copying 1 paths to 'lb:50051'...6215client # [ 132.066835] nix[1258]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6216worker2 # [ 131.909709] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:353266217worker2 # [ 131.920307] nix-daemon[923]: accepted connection from pid 1556, user nix-grpc-daemon (trusted)6218worker2 # [ 132.095167] nix-daemon[923]: accepted connection from pid 1690, user nix-grpc-daemon (trusted)6219worker2 # [ 132.110230] nix-daemon[923]: reaped child process 1693, status = succeeded6220worker2 # [ 132.268673] nix-daemon[923]: accepted connection from pid 1695, user nix-grpc-daemon (trusted)6221worker2 # [ 132.284348] nix-daemon[923]: reaped child process 1698, status = succeeded6222lb # [ 132.434164] niks3-server[1185]: 2026/09/20 08:05:58 INFO Received uploads request method=POST path=/api/pending_closures6223worker2 # [ 132.305309] nix-grpc-daemon[1635]: time=2026-09-20T08:05:57.809Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6224worker2 # [ 132.306984] nix-grpc-daemon[1635]: time=2026-09-20T08:05:57.809Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6225lb # [ 132.454257] niks3-server[1185]: 2026/09/20 08:05:58 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6226lb # [ 132.463100] niks3-server[1185]: 2026/09/20 08:05:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6227lb # [ 132.467816] niks3-server[1185]: 2026/09/20 08:05:58 INFO Signed narinfos id=36 count=16228worker2 # [ 132.331298] nix-grpc-daemon[1635]: time=2026-09-20T08:05:57.836Z level=INFO msg="Uploading 1 narinfos"6229lb # [ 132.472479] niks3-server[1185]: 2026/09/20 08:05:58 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6230lb # [ 132.482334] niks3-server[1185]: 2026/09/20 08:05:58 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6231lb # [ 132.487428] niks3-server[1185]: 2026/09/20 08:05:58 INFO Completed upload id=366232worker2 # [ 132.350985] nix-grpc-daemon[1635]: time=2026-09-20T08:05:57.856Z level=INFO msg="Upload complete. (419ms)"6233worker2 # [ 132.355203] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35326 duration_s=0 paths=1 nar_bytes_in=4166234lb # [ 132.493239] niks3-server[1185]: 2026/09/20 08:05:58 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6235worker2 # [ 132.359946] nix-grpc-daemon[1556]: ts=2026-09-20T08:05:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:353266236worker2 # [ 132.367427] nix-daemon[923]: accepted connection from pid 1556, user nix-grpc-daemon (trusted)6237worker2 # [ 132.370448] nix-daemon[923]: reaped child process 1688, status = succeeded6238worker2 # [ 132.377125] nix-daemon[923]: accepted connection from pid 1556, user nix-grpc-daemon (trusted)6239client # [ 132.534494] nix[1258]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6240worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26241worker2 # Checking switch inhibitors... done6242worker2 # [ 134.881436] nixos[1733]: switching to system configuration /nix/store/5b0jnp4s12lkrp1p488h19rpqmjis2zq-nixos-system-worker2-test6243worker2 # activating the configuration...6244worker2 # [ 134.885370] systemd[1]: Stopped target Remote File Systems.6245worker2 # [ 134.887157] systemd[1]: Stopped target Local File Systems.6246worker2 # setting up /etc...6247worker2 # [ 135.523578] systemd[1]: Reload requested from client PID 1733 ('.switch-to-conf') (unit backdoor.service)...6248worker2 # [ 135.525289] systemd[1]: Reloading...6249worker2 # [ 136.236459] systemd[1]: Reloading finished in 708 ms.6250worker2 # restarting sysinit-reactivation.target6251worker2 # [ 136.293813] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6252worker2 # [ 136.431485] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6253worker2 # [ 136.434350] systemd[1]: Reached target Reactivate sysinit units.6254worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6255worker2 # [ 136.440433] systemd[1]: Reloading D-Bus System Message Bus...6256worker2 # [ 136.441559] dbus-broker-launch[581]: Caught SIGHUP, trigger reload.6257worker2 # [ 136.446728] systemd[1]: Reloading Nix worker-protocol over gRPC...6258worker2 # [ 136.470757] dbus-broker-launch[581]: Looking up NSS user entry for 'systemd-timesync'...6259worker2 # [ 136.478177] dbus-broker-launch[581]: NSS returned no entry for 'systemd-timesync'6260worker2 # [ 136.479262] dbus-broker-launch[581]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6261worker2 # [ 136.497371] systemd[1]: Reloaded Nix worker-protocol over gRPC.6262worker2 # [ 136.512512] systemd[1]: Reloaded D-Bus System Message Bus.6263worker2 # [ 136.519555] systemd[1]: Reached target Local File Systems.6264worker2 # [ 136.527185] systemd[1]: Reached target Remote File Systems.6265worker2 # [ 136.532310] systemd[1]: Starting linger-users.service...6266worker2 # [ 136.539122] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6267worker2 # [ 136.545467] systemd[1]: Starting Create SUID/SGID Wrappers...6268worker2 # [ 136.547176] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6269worker2 # [ 136.548510] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676270worker2 # [ 136.552589] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6271worker2 # [ 136.556402] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6272worker2 # [ 136.559576] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6273worker2 # [ 136.561967] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6274worker2 # [ 136.564248] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6275worker2 # [ 136.607710] systemd[1]: linger-users.service: Deactivated successfully.6276worker2 # [ 136.611420] systemd[1]: Finished linger-users.service.6277worker2 # [ 136.972271] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6278worker2 # [ 136.973393] systemd[1]: Finished Create SUID/SGID Wrappers.6279worker2 # [ 137.008788] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:02Z level=info event=draining6280worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6281worker2 # [ 137.230358] nixos[1733]: finished switching to system configuration /nix/store/5b0jnp4s12lkrp1p488h19rpqmjis2zq-nixos-system-worker2-test6282worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.97 seconds)6283worker2: must succeed: pgrep -f 'read -t [9]0 x'6284worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6285client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag during-drain >&26286worker2 # [ 137.474898] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:353266287client # don't know how to build these paths:6288client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6289worker2 # [ 137.490302] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:02Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06290worker2 # [ 137.492780] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:02Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16291worker1 # [ 137.678257] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886292worker1 # [ 137.689300] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886293client # copying 1 paths to 'lb:50051'...6294worker1 # [ 137.701366] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886295client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6296worker1 # [ 137.720987] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6297worker1 # [ 137.897982] nix-daemon[846]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6298worker1 # [ 137.914625] nix-daemon[846]: reaped child process 1480, status = succeeded6299worker1 # [ 138.098687] nix-daemon[846]: accepted connection from pid 1482, user nix-grpc-daemon (trusted)6300worker1 # [ 138.115274] nix-daemon[846]: reaped child process 1485, status = succeeded6301lb # [ 138.119259] niks3-server[1185]: 2026/09/20 08:06:04 INFO Received uploads request method=POST path=/api/pending_closures6302worker1 # [ 138.130860] nix-grpc-daemon[1398]: time=2026-09-20T08:06:04.078Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6303worker1 # [ 138.132386] nix-grpc-daemon[1398]: time=2026-09-20T08:06:04.078Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6304lb # [ 138.134831] niks3-server[1185]: 2026/09/20 08:06:04 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6305lb # [ 138.138537] niks3-server[1185]: 2026/09/20 08:06:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6306lb # [ 138.141620] niks3-server[1185]: 2026/09/20 08:06:04 INFO Signed narinfos id=37 count=16307worker1 # [ 138.148505] nix-grpc-daemon[1398]: time=2026-09-20T08:06:04.095Z level=INFO msg="Uploading 1 narinfos"6308lb # [ 138.143917] niks3-server[1185]: 2026/09/20 08:06:04 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6309lb # [ 138.149172] niks3-server[1185]: 2026/09/20 08:06:04 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6310lb # [ 138.153395] niks3-server[1185]: 2026/09/20 08:06:04 INFO Completed upload id=376311worker1 # [ 138.159533] nix-grpc-daemon[1398]: time=2026-09-20T08:06:04.107Z level=INFO msg="Upload complete. (430ms)"6312lb # [ 138.155272] niks3-server[1185]: 2026/09/20 08:06:04 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6313worker1 # [ 138.162564] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=5286314worker1 # [ 138.167101] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886315worker1 # [ 138.174432] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6316worker1 # [ 138.183312] nix-daemon[846]: reaped child process 1475, status = succeeded6317client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6318worker1 # [ 138.190285] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6319client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6320lb # [ 138.502629] envoy[924]: conn peer=[2001:db8:1::1]:38156 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2937 tx=730 ms=33946321client # farm-a> LOG-farm-a-during-drain6322worker1 # [ 139.414122] nix-daemon[846]: reaped child process 1489, status = succeeded6323worker1 # [ 139.538093] nix-daemon[846]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)6324worker1 # [ 139.552625] nix-daemon[846]: reaped child process 1507, status = succeeded6325lb # [ 139.556850] niks3-server[1185]: 2026/09/20 08:06:05 INFO Received uploads request method=POST path=/api/pending_closures6326worker1 # [ 139.588379] nix-grpc-daemon[1398]: time=2026-09-20T08:06:05.534Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6327worker1 # [ 139.589831] nix-grpc-daemon[1398]: time=2026-09-20T08:06:05.534Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6328lb # [ 139.602604] niks3-server[1185]: 2026/09/20 08:06:05 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6329lb # [ 139.605167] niks3-server[1185]: 2026/09/20 08:06:05 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6330lb # [ 139.608345] niks3-server[1185]: 2026/09/20 08:06:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6331lb # [ 139.610156] niks3-server[1185]: 2026/09/20 08:06:05 INFO Signed narinfos id=38 count=16332worker1 # [ 139.617837] nix-grpc-daemon[1398]: time=2026-09-20T08:06:05.565Z level=INFO msg="Uploading 1 narinfos"6333lb # [ 139.614687] niks3-server[1185]: 2026/09/20 08:06:05 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6334lb # [ 139.619247] niks3-server[1185]: 2026/09/20 08:06:05 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6335lb # [ 139.623811] niks3-server[1185]: 2026/09/20 08:06:05 INFO Completed upload id=386336worker1 # [ 139.629689] nix-grpc-daemon[1398]: time=2026-09-20T08:06:05.577Z level=INFO msg="Upload complete. (229ms)"6337lb # [ 139.625079] niks3-server[1185]: 2026/09/20 08:06:05 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6338worker1 # [ 139.633200] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16339worker1 # [ 139.640681] nix-daemon[846]: reaped child process 1487, status = succeeded6340worker2 # [ 139.496303] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:05Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06341worker2 # [ 139.499328] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:05Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16342worker1 # [ 139.689669] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886343worker1 # [ 139.703388] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886344worker1 # [ 139.719767] nix-grpc-daemon[1398]: time=2026-09-20T08:06:05.667Z level=INFO msg="All 1 paths already cached"6345client # copying 1 paths to 'lb:50051'...6346client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6347worker1 # [ 139.727668] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886348worker1 # [ 139.738166] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6349worker1 # [ 139.892248] nix-daemon[846]: accepted connection from pid 1511, user nix-grpc-daemon (trusted)6350worker1 # [ 139.907132] nix-daemon[846]: reaped child process 1514, status = succeeded6351worker1 # [ 140.064549] nix-daemon[846]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6352worker1 # [ 140.079806] nix-daemon[846]: reaped child process 1519, status = succeeded6353lb # [ 140.081921] niks3-server[1185]: 2026/09/20 08:06:05 INFO Received uploads request method=POST path=/api/pending_closures6354worker1 # [ 140.092345] nix-grpc-daemon[1398]: time=2026-09-20T08:06:06.040Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6355worker1 # [ 140.094166] nix-grpc-daemon[1398]: time=2026-09-20T08:06:06.042Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6356lb # [ 140.101730] niks3-server[1185]: 2026/09/20 08:06:06 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6357lb # [ 140.106206] niks3-server[1185]: 2026/09/20 08:06:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6358lb # [ 140.109028] niks3-server[1185]: 2026/09/20 08:06:06 INFO Signed narinfos id=39 count=16359worker1 # [ 140.116459] nix-grpc-daemon[1398]: time=2026-09-20T08:06:06.064Z level=INFO msg="Uploading 1 narinfos"6360lb # [ 140.114121] niks3-server[1185]: 2026/09/20 08:06:06 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6361lb # [ 140.120749] niks3-server[1185]: 2026/09/20 08:06:06 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6362lb # [ 140.123700] niks3-server[1185]: 2026/09/20 08:06:06 INFO Completed upload id=396363worker1 # [ 140.131200] nix-grpc-daemon[1398]: time=2026-09-20T08:06:06.079Z level=INFO msg="Upload complete. (381ms)"6364lb # [ 140.126739] niks3-server[1185]: 2026/09/20 08:06:06 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6365worker1 # [ 140.134719] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=6406366worker1 # [ 140.139120] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886367worker1 # [ 140.145940] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6368worker1 # [ 140.155432] nix-daemon[846]: reaped child process 1509, status = succeeded6369client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6370client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6371worker1 # [ 140.162442] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6372client # farm-b> LOG-farm-b-during-drain6373worker1 # [ 140.782121] nix-daemon[846]: reaped child process 1523, status = succeeded6374worker1 # [ 140.942598] nix-daemon[846]: accepted connection from pid 1538, user nix-grpc-daemon (trusted)6375worker1 # [ 140.963978] nix-daemon[846]: reaped child process 1541, status = succeeded6376lb # [ 140.969117] niks3-server[1185]: 2026/09/20 08:06:06 INFO Received uploads request method=POST path=/api/pending_closures6377worker1 # [ 140.980669] nix-grpc-daemon[1398]: time=2026-09-20T08:06:06.928Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6378worker1 # [ 140.982398] nix-grpc-daemon[1398]: time=2026-09-20T08:06:06.930Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6379lb # [ 140.993785] niks3-server[1185]: 2026/09/20 08:06:06 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6380lb # [ 141.008409] niks3-server[1185]: 2026/09/20 08:06:06 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6381lb # [ 141.013287] niks3-server[1185]: 2026/09/20 08:06:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6382lb # [ 141.017445] niks3-server[1185]: 2026/09/20 08:06:06 INFO Signed narinfos id=40 count=16383worker1 # [ 141.023662] nix-grpc-daemon[1398]: time=2026-09-20T08:06:06.971Z level=INFO msg="Uploading 1 narinfos"6384lb # [ 141.019552] niks3-server[1185]: 2026/09/20 08:06:06 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6385lb # [ 141.027789] niks3-server[1185]: 2026/09/20 08:06:06 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6386lb # [ 141.032775] niks3-server[1185]: 2026/09/20 08:06:06 INFO Completed upload id=406387worker1 # [ 141.038942] nix-grpc-daemon[1398]: time=2026-09-20T08:06:06.986Z level=INFO msg="Upload complete. (265ms)"6388lb # [ 141.035327] niks3-server[1185]: 2026/09/20 08:06:06 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6389worker1 # [ 141.043430] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16390worker2 # [ 140.906318] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:06Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06391worker2 # [ 140.909272] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:06Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16392worker1 # [ 141.053735] nix-daemon[846]: reaped child process 1521, status = succeeded6393worker1 # [ 141.092894] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886394worker1 # [ 141.101659] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886395worker1 # [ 141.110060] nix-grpc-daemon[1398]: time=2026-09-20T08:06:07.057Z level=INFO msg="All 2 paths already cached"6396client # copying 1 paths to 'lb:50051'...6397worker1 # [ 141.115312] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886398client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6399worker1 # [ 141.124608] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6400worker1 # [ 141.269324] nix-daemon[846]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6401worker1 # [ 141.286298] nix-daemon[846]: reaped child process 1548, status = succeeded6402worker1 # [ 141.433257] nix-daemon[846]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6403worker1 # [ 141.447533] nix-daemon[846]: reaped child process 1553, status = succeeded6404lb # [ 141.457326] niks3-server[1185]: 2026/09/20 08:06:07 INFO Received uploads request method=POST path=/api/pending_closures6405worker1 # [ 141.467679] nix-grpc-daemon[1398]: time=2026-09-20T08:06:07.415Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6406worker1 # [ 141.469690] nix-grpc-daemon[1398]: time=2026-09-20T08:06:07.415Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6407lb # [ 141.472285] niks3-server[1185]: 2026/09/20 08:06:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6408lb # [ 141.474282] niks3-server[1185]: 2026/09/20 08:06:07 INFO Signed narinfos id=41 count=16409worker1 # [ 141.481866] nix-grpc-daemon[1398]: time=2026-09-20T08:06:07.429Z level=INFO msg="Uploading 1 narinfos"6410lb # [ 141.481710] niks3-server[1185]: 2026/09/20 08:06:07 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6411lb # [ 141.504999] niks3-server[1185]: 2026/09/20 08:06:07 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6412lb # [ 141.510443] niks3-server[1185]: 2026/09/20 08:06:07 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6413lb # [ 141.513709] niks3-server[1185]: 2026/09/20 08:06:07 INFO Completed upload id=416414worker1 # [ 141.519814] nix-grpc-daemon[1398]: time=2026-09-20T08:06:07.467Z level=INFO msg="Upload complete. (385ms)"6415lb # [ 141.516063] niks3-server[1185]: 2026/09/20 08:06:07 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6416worker1 # [ 141.522883] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=7686417worker1 # [ 141.527316] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886418worker1 # [ 141.533795] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6419worker1 # [ 141.540796] nix-daemon[846]: reaped child process 1543, status = succeeded6420client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6421client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6422worker1 # [ 141.550083] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6423client # farm-top> LOG-farm-top-during-drain6424worker1 # [ 142.570705] nix-daemon[846]: reaped child process 1557, status = succeeded6425worker1 # [ 142.705392] nix-daemon[846]: accepted connection from pid 1572, user nix-grpc-daemon (trusted)6426worker1 # [ 142.721486] nix-daemon[846]: reaped child process 1575, status = succeeded6427lb # [ 142.728945] niks3-server[1185]: 2026/09/20 08:06:08 INFO Received uploads request method=POST path=/api/pending_closures6428worker1 # [ 142.740358] nix-grpc-daemon[1398]: time=2026-09-20T08:06:08.687Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6429worker1 # [ 142.742790] nix-grpc-daemon[1398]: time=2026-09-20T08:06:08.687Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6430lb # [ 142.745990] niks3-server[1185]: 2026/09/20 08:06:08 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6431lb # [ 142.748590] niks3-server[1185]: 2026/09/20 08:06:08 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6432lb # [ 142.753848] niks3-server[1185]: 2026/09/20 08:06:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6433lb # [ 142.756504] niks3-server[1185]: 2026/09/20 08:06:08 INFO Signed narinfos id=42 count=16434worker1 # [ 142.762748] nix-grpc-daemon[1398]: time=2026-09-20T08:06:08.710Z level=INFO msg="Uploading 1 narinfos"6435lb # [ 142.758317] niks3-server[1185]: 2026/09/20 08:06:08 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6436lb # [ 142.762869] niks3-server[1185]: 2026/09/20 08:06:08 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6437lb # [ 142.766422] niks3-server[1185]: 2026/09/20 08:06:08 INFO Completed upload id=426438worker1 # [ 142.773565] nix-grpc-daemon[1398]: time=2026-09-20T08:06:08.721Z level=INFO msg="Upload complete. (215ms)"6439lb # [ 142.768875] niks3-server[1185]: 2026/09/20 08:06:08 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6440worker1 # [ 142.776177] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16441worker1 # [ 142.783660] nix-daemon[846]: reaped child process 1555, status = succeeded6442client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag during-drain >&2, in 5.39 seconds)6443client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag during-drain outPath6444client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6445worker2: must succeed: pkill -f 'read -t [9]0 x'6446worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6447client: waiting for success: ! systemctl is-active sw6448worker2 # [ 143.013117] nix-daemon[923]: reaped child process 1702, status = succeeded6449worker2 # [ 143.165447] nix-daemon[923]: accepted connection from pid 1880, user nix-grpc-daemon (trusted)6450worker2 # [ 143.181189] nix-daemon[923]: reaped child process 1883, status = succeeded6451lb # [ 143.326224] niks3-server[1185]: 2026/09/20 08:06:09 INFO Received uploads request method=POST path=/api/pending_closures6452worker2 # [ 143.194853] nix-grpc-daemon[1635]: time=2026-09-20T08:06:08.699Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6453worker2 # [ 143.197436] nix-grpc-daemon[1635]: time=2026-09-20T08:06:08.701Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6454lb # [ 143.348104] niks3-server[1185]: 2026/09/20 08:06:09 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6455lb # [ 143.352517] niks3-server[1185]: 2026/09/20 08:06:09 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6456lb # [ 143.355861] niks3-server[1185]: 2026/09/20 08:06:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6457lb # [ 143.358291] niks3-server[1185]: 2026/09/20 08:06:09 INFO Signed narinfos id=43 count=16458worker2 # [ 143.222352] nix-grpc-daemon[1635]: time=2026-09-20T08:06:08.726Z level=INFO msg="Uploading 1 narinfos"6459lb # [ 143.360751] niks3-server[1185]: 2026/09/20 08:06:09 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6460lb # [ 143.366888] niks3-server[1185]: 2026/09/20 08:06:09 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6461lb # [ 143.369852] niks3-server[1185]: 2026/09/20 08:06:09 INFO Completed upload id=436462worker2 # [ 143.233114] nix-grpc-daemon[1635]: time=2026-09-20T08:06:08.738Z level=INFO msg="Upload complete. (234ms)"6463lb # [ 143.373704] niks3-server[1185]: 2026/09/20 08:06:09 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6464worker2 # [ 143.238215] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35326 duration_s=10 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16465worker2 # [ 143.244881] nix-daemon[923]: reaped child process 1700, status = succeeded6466client # [ 143.425369] systemd[1]: sw.service: Deactivated successfully.6467client # [ 143.427482] systemd[1]: sw.service: Consumed 133ms CPU time over 12.549s wall clock time, 39.1M memory peak, 8.6K incoming IP traffic, 9K outgoing IP traffic.6468client: (finished: waiting for success: ! systemctl is-active sw, in 1.07 seconds)6469client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6470worker2 # [ 144.011511] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=signal_exit6471worker2 # [ 144.030812] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=scheduler_yield6472client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6473lb # [ 144.169627] niks3-server[1185]: 2026/09/20 08:06:10 INFO lead: released remote=192.168.1.4:376226474worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6475worker1 # [ 144.176131] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:10Z level=info event=scheduler_restarting addr=lb:500516476worker2 # [ 144.036025] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:353126477worker2 # [ 144.038160] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=worker_gone addr=192.168.1.3:500516478worker2 # [ 144.040188] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=scheduler_restarting addr=lb:500516479worker2 # [ 144.041562] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:353126480worker2 # [ 144.042959] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=worker_gone addr=192.168.1.4:500516481worker1 # [ 144.376464] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:10Z level=info event=scheduler_connected addr=lb:500516482worker1 # [ 144.379160] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6483worker2 # [ 144.239230] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=scheduler_connected addr=lb:500516484worker2 # [ 144.240910] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6485worker1 # [ 144.579805] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:10Z level=info event=scheduler_connected addr=lb:500516486worker2 # [ 144.439908] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=scheduler_connected addr=lb:500516487worker1 # [ 144.582139] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6488worker2 # [ 144.442056] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6489worker1 # [ 144.782830] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:10Z level=info event=scheduler_connected addr=lb:500516490worker2 # [ 144.642699] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:10Z level=info event=scheduler_connected addr=lb:500516491worker1 # [ 144.789644] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6492worker2 # [ 144.648445] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6493worker1 # [ 144.990425] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:10Z level=info event=scheduler_connected addr=lb:500516494worker2 # [ 144.849239] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:10Z level=info event=scheduler_connected addr=lb:500516495worker1 # [ 144.993956] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6496worker2 # [ 144.852961] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6497worker2 # [ 145.032216] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:10Z level=info event=scheduler_restarting6498worker2 # [ 145.051830] nix-daemon[923]: reaped child process 1607, status = succeeded6499worker1 # [ 145.195218] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:11Z level=info event=scheduler_connected addr=lb:500516500worker1 # [ 145.199376] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6501worker2 # [ 145.059939] nix-grpc-daemon[1556]: ts=2026-09-20T08:06:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6502worker1 # [ 145.400578] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:11Z level=info event=scheduler_connected addr=lb:500516503worker2 # [ 145.260712] nix-grpc-daemon[1556]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6504worker2 # [ 145.264679] nix-grpc-daemon[1556]: E0000 00:00:1789891570.764873 1599 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6505worker2 # [ 145.504262] nix-grpc-daemon[1556]: E0000 00:00:1789891571.009222 1599 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6506worker2 # [ 145.506744] nix-grpc-daemon[1556]: E0000 00:00:1789891571.009349 1599 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 8, started true, shutdown true6507worker1 # [ 147.167438] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:13Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"6508worker2 # [ 147.044707] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6509worker2 # [ 147.048715] systemd[1]: nix-grpc-daemon.service: Consumed 722ms CPU time over 26.388s wall clock time, 31M memory peak, 24.1K incoming IP traffic, 32.9K outgoing IP traffic.6510worker2 # [ 147.065966] systemd[1]: Starting Nix worker-protocol over gRPC...6511worker2 # [ 147.195109] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6512worker2 # [ 147.199162] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16513worker2 # [ 147.205788] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6514worker2 # [ 147.210870] systemd[1]: Started Nix worker-protocol over gRPC.6515worker2 # [ 147.215168] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:12Z level=info event=scheduler_take_over6516lb # [ 147.352367] niks3-server[1185]: 2026/09/20 08:06:13 INFO lead: acquired remote=192.168.1.4:423766517worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.19 seconds)6518lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6519lb: must succeed: curl -sf localhost:9901/clusters6520worker2 # [ 147.222686] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:12Z level=info event=scheduler_connected addr=lb:500516521worker1 # [ 147.367411] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:13Z level=info event=scheduler_connected addr=lb:500516522worker2 # [ 147.227261] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6523worker2 # [ 147.233174] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06524lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6525worker2 # [ 147.428084] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:12Z level=info event=scheduler_connected addr=lb:500516526worker2 # [ 147.430285] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:12Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6527worker2 # [ 147.630954] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:13Z level=info event=scheduler_connected addr=lb:500516528worker2 # [ 147.633288] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06529lb: must succeed: curl -sf localhost:9901/clusters6530lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6531lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)6532lb: must succeed: curl -sf localhost:9901/clusters6533lb # [ 148.501606] envoy[924]: conn peer=[2001:db8:1::1]:37942 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9416 tx=3786 ms=51756534lb # [ 148.504157] envoy[924]: conn peer=[2001:db8:1::1]:38160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2936 tx=1377 ms=123516535lb # [ 148.505673] envoy[924]: conn peer=[2001:db8:1::4]:55134 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1721 tx=529 ms=240446536lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6537worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6538worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)6539(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.73 seconds)6540subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6541client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16542worker2 # [ 148.636823] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537166543worker2 # [ 148.648520] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6544worker2 # [ 148.669429] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:14Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06545worker2 # [ 148.674240] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:14Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16546worker2 # [ 148.678657] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:14Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06547worker2 # [ 148.682449] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:14Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06548worker1 # [ 148.862096] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886549worker1 # [ 148.871812] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886550worker1 # [ 148.879385] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886551worker1 # [ 148.883544] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6552worker1 # [ 149.047564] nix-daemon[846]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6553worker1 # [ 149.061696] nix-daemon[846]: reaped child process 1582, status = succeeded6554worker1 # [ 149.211676] nix-daemon[846]: accepted connection from pid 1584, user nix-grpc-daemon (trusted)6555worker1 # [ 149.225716] nix-daemon[846]: reaped child process 1587, status = succeeded6556lb # [ 149.233338] niks3-server[1185]: 2026/09/20 08:06:15 INFO Received uploads request method=POST path=/api/pending_closures6557worker1 # [ 149.245901] nix-grpc-daemon[1398]: time=2026-09-20T08:06:15.193Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6558worker1 # [ 149.247314] nix-grpc-daemon[1398]: time=2026-09-20T08:06:15.193Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6559lb # [ 149.252183] niks3-server[1185]: 2026/09/20 08:06:15 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6560lb # [ 149.255727] niks3-server[1185]: 2026/09/20 08:06:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6561lb # [ 149.259217] niks3-server[1185]: 2026/09/20 08:06:15 INFO Signed narinfos id=44 count=16562worker1 # [ 149.265356] nix-grpc-daemon[1398]: time=2026-09-20T08:06:15.213Z level=INFO msg="Uploading 1 narinfos"6563lb # [ 149.263166] niks3-server[1185]: 2026/09/20 08:06:15 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6564lb # [ 149.268063] niks3-server[1185]: 2026/09/20 08:06:15 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6565lb # [ 149.272181] niks3-server[1185]: 2026/09/20 08:06:15 INFO Completed upload id=446566worker1 # [ 149.278172] nix-grpc-daemon[1398]: time=2026-09-20T08:06:15.226Z level=INFO msg="Upload complete. (387ms)"6567worker1 # [ 149.280199] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=5046568lb # [ 149.275123] niks3-server[1185]: 2026/09/20 08:06:15 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6569worker1 # [ 149.283955] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886570worker1 # [ 149.291062] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6571worker1 # [ 149.294640] nix-daemon[846]: reaped child process 1577, status = succeeded6572worker1 # [ 149.299789] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6573worker1 # [ 150.116135] nix-daemon[846]: reaped child process 1591, status = succeeded6574worker1 # [ 150.246864] nix-daemon[846]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6575worker1 # [ 150.260369] nix-daemon[846]: reaped child process 1609, status = succeeded6576lb # [ 150.265078] niks3-server[1185]: 2026/09/20 08:06:16 INFO Received uploads request method=POST path=/api/pending_closures6577worker1 # [ 150.276638] nix-grpc-daemon[1398]: time=2026-09-20T08:06:16.224Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6578worker1 # [ 150.278174] nix-grpc-daemon[1398]: time=2026-09-20T08:06:16.226Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6579lb # [ 150.291801] niks3-server[1185]: 2026/09/20 08:06:16 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6580lb # [ 150.300356] niks3-server[1185]: 2026/09/20 08:06:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6581lb # [ 150.302956] niks3-server[1185]: 2026/09/20 08:06:16 INFO Signed narinfos id=45 count=16582worker1 # [ 150.310904] nix-grpc-daemon[1398]: time=2026-09-20T08:06:16.258Z level=INFO msg="Uploading 1 narinfos"6583lb # [ 150.308379] niks3-server[1185]: 2026/09/20 08:06:16 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6584lb # [ 150.311708] niks3-server[1185]: 2026/09/20 08:06:16 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6585lb # [ 150.319585] niks3-server[1185]: 2026/09/20 08:06:16 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6586lb # [ 150.322933] niks3-server[1185]: 2026/09/20 08:06:16 INFO Completed upload id=456587worker1 # [ 150.328946] nix-grpc-daemon[1398]: time=2026-09-20T08:06:16.276Z level=INFO msg="Upload complete. (222ms)"6588lb # [ 150.324785] niks3-server[1185]: 2026/09/20 08:06:16 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6589worker1 # [ 150.333342] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16590worker1 # [ 150.340853] nix-daemon[846]: reaped child process 1589, status = succeeded6591worker2 # [ 150.198675] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:15Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16592worker2 # [ 150.203395] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:15Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06593worker1 # [ 150.382867] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886594worker1 # [ 150.396509] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886595worker1 # [ 150.407705] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886596worker1 # [ 150.420151] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6597worker1 # [ 150.584178] nix-daemon[846]: accepted connection from pid 1613, user nix-grpc-daemon (trusted)6598worker1 # [ 150.597774] nix-daemon[846]: reaped child process 1616, status = succeeded6599worker1 # [ 150.760520] nix-daemon[846]: accepted connection from pid 1618, user nix-grpc-daemon (trusted)6600worker1 # [ 150.775124] nix-daemon[846]: reaped child process 1621, status = succeeded6601lb # [ 150.785740] niks3-server[1185]: 2026/09/20 08:06:16 INFO Received uploads request method=POST path=/api/pending_closures6602worker1 # [ 150.799641] nix-grpc-daemon[1398]: time=2026-09-20T08:06:16.747Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6603worker1 # [ 150.801266] nix-grpc-daemon[1398]: time=2026-09-20T08:06:16.749Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6604lb # [ 150.815299] niks3-server[1185]: 2026/09/20 08:06:16 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6605lb # [ 150.829479] niks3-server[1185]: 2026/09/20 08:06:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6606lb # [ 150.834269] niks3-server[1185]: 2026/09/20 08:06:16 INFO Signed narinfos id=46 count=16607worker1 # [ 150.843943] nix-grpc-daemon[1398]: time=2026-09-20T08:06:16.791Z level=INFO msg="Uploading 1 narinfos"6608lb # [ 150.839908] niks3-server[1185]: 2026/09/20 08:06:16 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6609lb # [ 150.851210] niks3-server[1185]: 2026/09/20 08:06:16 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6610lb # [ 150.857312] niks3-server[1185]: 2026/09/20 08:06:16 INFO Completed upload id=466611worker1 # [ 150.863471] nix-grpc-daemon[1398]: time=2026-09-20T08:06:16.811Z level=INFO msg="Upload complete. (431ms)"6612lb # [ 150.860256] niks3-server[1185]: 2026/09/20 08:06:16 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6613worker1 # [ 150.868234] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=5046614worker1 # [ 150.873610] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886615worker1 # [ 150.883414] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6616worker1 # [ 150.891486] nix-daemon[846]: reaped child process 1611, status = succeeded6617worker1 # [ 150.897072] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6618worker1 # [ 151.713722] nix-daemon[846]: reaped child process 1625, status = succeeded6619worker1 # [ 151.822377] nix-daemon[846]: accepted connection from pid 1640, user nix-grpc-daemon (trusted)6620worker1 # [ 151.836485] nix-daemon[846]: reaped child process 1643, status = succeeded6621lb # [ 151.839079] niks3-server[1185]: 2026/09/20 08:06:17 INFO Received uploads request method=POST path=/api/pending_closures6622worker1 # [ 151.849456] nix-grpc-daemon[1398]: time=2026-09-20T08:06:17.797Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6623worker1 # [ 151.850943] nix-grpc-daemon[1398]: time=2026-09-20T08:06:17.798Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6624lb # [ 151.864352] niks3-server[1185]: 2026/09/20 08:06:17 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6625lb # [ 151.867962] niks3-server[1185]: 2026/09/20 08:06:17 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6626lb # [ 151.871635] niks3-server[1185]: 2026/09/20 08:06:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6627lb # [ 151.873433] niks3-server[1185]: 2026/09/20 08:06:17 INFO Signed narinfos id=47 count=16628worker1 # [ 151.880604] nix-grpc-daemon[1398]: time=2026-09-20T08:06:17.828Z level=INFO msg="Uploading 1 narinfos"6629lb # [ 151.877196] niks3-server[1185]: 2026/09/20 08:06:17 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6630lb # [ 151.882192] niks3-server[1185]: 2026/09/20 08:06:17 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6631lb # [ 151.884573] niks3-server[1185]: 2026/09/20 08:06:17 INFO Completed upload id=476632worker1 # [ 151.891958] nix-grpc-daemon[1398]: time=2026-09-20T08:06:17.839Z level=INFO msg="Upload complete. (188ms)"6633lb # [ 151.887254] niks3-server[1185]: 2026/09/20 08:06:17 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6634worker1 # [ 151.894441] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16635worker1 # [ 151.902139] nix-daemon[846]: reaped child process 1623, status = succeeded6636worker2 # [ 151.754856] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:17Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16637worker1 # [ 151.904091] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886638worker2 # [ 151.764554] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:17Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06639worker1 # [ 151.910444] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886640worker1 # [ 151.916180] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886641worker1 # [ 151.919974] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6642worker1 # [ 152.043133] nix-daemon[846]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)6643worker1 # [ 152.058089] nix-daemon[846]: reaped child process 1650, status = succeeded6644worker1 # [ 152.175088] nix-daemon[846]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6645worker1 # [ 152.189480] nix-daemon[846]: reaped child process 1655, status = succeeded6646lb # [ 152.191226] niks3-server[1185]: 2026/09/20 08:06:18 INFO Received uploads request method=POST path=/api/pending_closures6647worker1 # [ 152.200755] nix-grpc-daemon[1398]: time=2026-09-20T08:06:18.148Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6648worker1 # [ 152.202582] nix-grpc-daemon[1398]: time=2026-09-20T08:06:18.150Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6649lb # [ 152.206210] niks3-server[1185]: 2026/09/20 08:06:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6650lb # [ 152.210186] niks3-server[1185]: 2026/09/20 08:06:18 INFO Signed narinfos id=48 count=16651worker1 # [ 152.216063] nix-grpc-daemon[1398]: time=2026-09-20T08:06:18.163Z level=INFO msg="Uploading 1 narinfos"6652lb # [ 152.211224] niks3-server[1185]: 2026/09/20 08:06:18 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6653lb # [ 152.221464] niks3-server[1185]: 2026/09/20 08:06:18 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6654worker1 # [ 152.230795] nix-grpc-daemon[1398]: time=2026-09-20T08:06:18.178Z level=INFO msg="Upload complete. (304ms)"6655lb # [ 152.225452] niks3-server[1185]: 2026/09/20 08:06:18 INFO Completed upload id=486656worker1 # [ 152.232777] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=5046657lb # [ 152.227753] niks3-server[1185]: 2026/09/20 08:06:18 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6658worker1 # [ 152.236567] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886659lb # [ 152.232056] niks3-server[1185]: 2026/09/20 08:06:18 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6660worker1 # [ 152.243642] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6661worker1 # [ 152.247196] nix-daemon[846]: reaped child process 1645, status = succeeded6662worker1 # [ 152.251148] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6663worker1 # [ 152.877506] nix-daemon[846]: reaped child process 1659, status = succeeded6664worker1 # [ 153.011109] nix-daemon[846]: accepted connection from pid 1674, user nix-grpc-daemon (trusted)6665worker1 # [ 153.023165] nix-daemon[846]: reaped child process 1677, status = succeeded6666lb # [ 153.026921] niks3-server[1185]: 2026/09/20 08:06:18 INFO Received uploads request method=POST path=/api/pending_closures6667worker1 # [ 153.039114] nix-grpc-daemon[1398]: time=2026-09-20T08:06:18.986Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6668worker1 # [ 153.042278] nix-grpc-daemon[1398]: time=2026-09-20T08:06:18.988Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6669lb # [ 153.049245] niks3-server[1185]: 2026/09/20 08:06:18 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6670lb # [ 153.053520] niks3-server[1185]: 2026/09/20 08:06:18 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6671lb # [ 153.059252] niks3-server[1185]: 2026/09/20 08:06:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6672lb # [ 153.061907] niks3-server[1185]: 2026/09/20 08:06:18 INFO Signed narinfos id=49 count=16673worker1 # [ 153.069199] nix-grpc-daemon[1398]: time=2026-09-20T08:06:19.016Z level=INFO msg="Uploading 1 narinfos"6674lb # [ 153.065152] niks3-server[1185]: 2026/09/20 08:06:18 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6675lb # [ 153.070053] niks3-server[1185]: 2026/09/20 08:06:18 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6676lb # [ 153.073326] niks3-server[1185]: 2026/09/20 08:06:18 INFO Completed upload id=496677worker1 # [ 153.080452] nix-grpc-daemon[1398]: time=2026-09-20T08:06:19.028Z level=INFO msg="Upload complete. (222ms)"6678lb # [ 153.074650] niks3-server[1185]: 2026/09/20 08:06:18 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6679worker1 # [ 153.084064] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16680worker1 # [ 153.088033] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886681worker2 # [ 152.945449] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:18Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16682worker1 # [ 153.093827] nix-daemon[846]: reaped child process 1657, status = succeeded6683worker1 # [ 153.095783] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886684worker2 # [ 152.954109] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:18Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06685worker1 # [ 153.101423] nix-grpc-daemon[1398]: time=2026-09-20T08:06:19.049Z level=INFO msg="All 1 paths already cached"6686worker1 # [ 153.105223] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886687worker1 # [ 153.109178] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6688worker1 # [ 153.259134] nix-daemon[846]: accepted connection from pid 1681, user nix-grpc-daemon (trusted)6689worker1 # [ 153.272217] nix-daemon[846]: reaped child process 1684, status = succeeded6690worker1 # [ 153.393253] nix-daemon[846]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6691worker1 # [ 153.407434] nix-daemon[846]: reaped child process 1689, status = succeeded6692lb # [ 153.410763] niks3-server[1185]: 2026/09/20 08:06:19 INFO Received uploads request method=POST path=/api/pending_closures6693worker1 # [ 153.426895] nix-grpc-daemon[1398]: time=2026-09-20T08:06:19.374Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6694worker1 # [ 153.428452] nix-grpc-daemon[1398]: time=2026-09-20T08:06:19.376Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6695lb # [ 153.444189] niks3-server[1185]: 2026/09/20 08:06:19 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6696lb # [ 153.454821] niks3-server[1185]: 2026/09/20 08:06:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6697worker1 # [ 153.469279] nix-grpc-daemon[1398]: time=2026-09-20T08:06:19.416Z level=INFO msg="Uploading 1 narinfos"6698lb # [ 153.463123] niks3-server[1185]: 2026/09/20 08:06:19 INFO Signed narinfos id=50 count=16699lb # [ 153.467715] niks3-server[1185]: 2026/09/20 08:06:19 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6700lb # [ 153.478948] niks3-server[1185]: 2026/09/20 08:06:19 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6701lb # [ 153.484646] niks3-server[1185]: 2026/09/20 08:06:19 INFO Completed upload id=506702worker1 # [ 153.490061] nix-grpc-daemon[1398]: time=2026-09-20T08:06:19.437Z level=INFO msg="Upload complete. (374ms)"6703worker1 # [ 153.492801] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=6246704lb # [ 153.487377] niks3-server[1185]: 2026/09/20 08:06:19 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6705worker1 # [ 153.497489] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886706worker1 # [ 153.503470] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6707worker1 # [ 153.505906] nix-daemon[846]: reaped child process 1679, status = succeeded6708worker1 # [ 153.511155] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6709worker1 # [ 154.739422] nix-daemon[846]: reaped child process 1693, status = succeeded6710worker1 # [ 154.905690] nix-daemon[846]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6711worker1 # [ 154.920775] nix-daemon[846]: reaped child process 1711, status = succeeded6712lb # [ 154.929580] niks3-server[1185]: 2026/09/20 08:06:20 INFO Received uploads request method=POST path=/api/pending_closures6713worker1 # [ 154.944940] nix-grpc-daemon[1398]: time=2026-09-20T08:06:20.892Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6714worker1 # [ 154.949583] nix-grpc-daemon[1398]: time=2026-09-20T08:06:20.894Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6715lb # [ 154.973455] niks3-server[1185]: 2026/09/20 08:06:20 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6716lb # [ 154.982302] niks3-server[1185]: 2026/09/20 08:06:20 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6717lb # [ 154.992110] niks3-server[1185]: 2026/09/20 08:06:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6718lb # [ 154.998879] niks3-server[1185]: 2026/09/20 08:06:20 INFO Signed narinfos id=51 count=16719worker1 # [ 155.007517] nix-grpc-daemon[1398]: time=2026-09-20T08:06:20.955Z level=INFO msg="Uploading 1 narinfos"6720lb # [ 155.005270] niks3-server[1185]: 2026/09/20 08:06:20 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6721lb # [ 155.013859] niks3-server[1185]: 2026/09/20 08:06:20 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6722lb # [ 155.022257] niks3-server[1185]: 2026/09/20 08:06:20 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6723lb # [ 155.025092] niks3-server[1185]: 2026/09/20 08:06:20 INFO Completed upload id=516724worker1 # [ 155.031114] nix-grpc-daemon[1398]: time=2026-09-20T08:06:20.978Z level=INFO msg="Upload complete. (312ms)"6725worker1 # [ 155.034375] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16726worker2 # [ 154.892747] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:20Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16727worker1 # [ 155.043068] nix-daemon[846]: reaped child process 1691, status = succeeded6728worker1 # [ 155.077469] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886729worker2 # [ 154.940522] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:20Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06730worker1 # [ 155.087308] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886731worker1 # [ 155.094043] nix-grpc-daemon[1398]: time=2026-09-20T08:06:21.041Z level=INFO msg="All 1 paths already cached"6732worker1 # [ 155.098942] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886733worker1 # [ 155.105088] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6734worker1 # [ 155.251876] nix-daemon[846]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6735worker1 # [ 155.268348] nix-daemon[846]: reaped child process 1718, status = succeeded6736worker1 # [ 155.409523] nix-daemon[846]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6737worker1 # [ 155.423519] nix-daemon[846]: reaped child process 1723, status = succeeded6738lb # [ 155.430368] niks3-server[1185]: 2026/09/20 08:06:21 INFO Received uploads request method=POST path=/api/pending_closures6739worker1 # [ 155.444824] nix-grpc-daemon[1398]: time=2026-09-20T08:06:21.392Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6740worker1 # [ 155.446665] nix-grpc-daemon[1398]: time=2026-09-20T08:06:21.394Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6741lb # [ 155.456648] niks3-server[1185]: 2026/09/20 08:06:21 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6742lb # [ 155.469078] niks3-server[1185]: 2026/09/20 08:06:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6743lb # [ 155.474826] niks3-server[1185]: 2026/09/20 08:06:21 INFO Signed narinfos id=52 count=16744worker1 # [ 155.483896] nix-grpc-daemon[1398]: time=2026-09-20T08:06:21.431Z level=INFO msg="Uploading 1 narinfos"6745lb # [ 155.479734] niks3-server[1185]: 2026/09/20 08:06:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6746lb # [ 155.490938] niks3-server[1185]: 2026/09/20 08:06:21 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6747worker1 # [ 155.505468] nix-grpc-daemon[1398]: time=2026-09-20T08:06:21.453Z level=INFO msg="Upload complete. (394ms)"6748lb # [ 155.499676] niks3-server[1185]: 2026/09/20 08:06:21 INFO Completed upload id=526749worker1 # [ 155.507848] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=6246750lb # [ 155.502171] niks3-server[1185]: 2026/09/20 08:06:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6751worker1 # [ 155.515532] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886752worker1 # [ 155.520552] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6753worker1 # [ 155.523198] nix-daemon[846]: reaped child process 1713, status = succeeded6754worker1 # [ 155.528572] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6755worker1 # [ 156.743115] nix-daemon[846]: reaped child process 1727, status = succeeded6756worker1 # [ 156.867455] nix-daemon[846]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6757worker1 # [ 156.881608] nix-daemon[846]: reaped child process 1745, status = succeeded6758lb # [ 156.889488] niks3-server[1185]: 2026/09/20 08:06:22 INFO Received uploads request method=POST path=/api/pending_closures6759worker1 # [ 156.900543] nix-grpc-daemon[1398]: time=2026-09-20T08:06:22.848Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6760worker1 # [ 156.903431] nix-grpc-daemon[1398]: time=2026-09-20T08:06:22.849Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6761lb # [ 156.908852] niks3-server[1185]: 2026/09/20 08:06:22 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6762lb # [ 156.913068] niks3-server[1185]: 2026/09/20 08:06:22 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6763lb # [ 156.916456] niks3-server[1185]: 2026/09/20 08:06:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6764lb # [ 156.919298] niks3-server[1185]: 2026/09/20 08:06:22 INFO Signed narinfos id=53 count=16765worker1 # [ 156.925402] nix-grpc-daemon[1398]: time=2026-09-20T08:06:22.873Z level=INFO msg="Uploading 1 narinfos"6766lb # [ 156.922500] niks3-server[1185]: 2026/09/20 08:06:22 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6767lb # [ 156.927082] niks3-server[1185]: 2026/09/20 08:06:22 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6768lb # [ 156.931094] niks3-server[1185]: 2026/09/20 08:06:22 INFO Completed upload id=536769worker1 # [ 156.936840] nix-grpc-daemon[1398]: time=2026-09-20T08:06:22.884Z level=INFO msg="Upload complete. (201ms)"6770lb # [ 156.932617] niks3-server[1185]: 2026/09/20 08:06:22 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6771worker1 # [ 156.939452] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16772worker1 # [ 156.946750] nix-daemon[846]: reaped child process 1725, status = succeeded6773worker2 # [ 156.799823] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:22Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16774worker2 # [ 156.809119] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:22Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06775worker1 # [ 156.993260] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886776worker1 # [ 157.004987] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886777worker1 # [ 157.017764] nix-grpc-daemon[1398]: time=2026-09-20T08:06:22.965Z level=INFO msg="All 1 paths already cached"6778worker1 # [ 157.024415] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886779worker1 # [ 157.032343] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6780worker1 # [ 157.183312] nix-daemon[846]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6781worker1 # [ 157.197881] nix-daemon[846]: reaped child process 1752, status = succeeded6782worker1 # [ 157.349711] nix-daemon[846]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)6783worker1 # [ 157.363776] nix-daemon[846]: reaped child process 1757, status = succeeded6784lb # [ 157.370909] niks3-server[1185]: 2026/09/20 08:06:23 INFO Received uploads request method=POST path=/api/pending_closures6785worker1 # [ 157.381545] nix-grpc-daemon[1398]: time=2026-09-20T08:06:23.329Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6786worker1 # [ 157.384035] nix-grpc-daemon[1398]: time=2026-09-20T08:06:23.331Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6787lb # [ 157.388789] niks3-server[1185]: 2026/09/20 08:06:23 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6788lb # [ 157.391453] niks3-server[1185]: 2026/09/20 08:06:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6789lb # [ 157.393650] niks3-server[1185]: 2026/09/20 08:06:23 INFO Signed narinfos id=54 count=16790worker1 # [ 157.401326] nix-grpc-daemon[1398]: time=2026-09-20T08:06:23.348Z level=INFO msg="Uploading 1 narinfos"6791lb # [ 157.399732] niks3-server[1185]: 2026/09/20 08:06:23 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6792lb # [ 157.403445] niks3-server[1185]: 2026/09/20 08:06:23 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6793lb # [ 157.406599] niks3-server[1185]: 2026/09/20 08:06:23 INFO Completed upload id=546794worker1 # [ 157.412616] nix-grpc-daemon[1398]: time=2026-09-20T08:06:23.360Z level=INFO msg="Upload complete. (368ms)"6795lb # [ 157.409306] niks3-server[1185]: 2026/09/20 08:06:23 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6796worker1 # [ 157.415205] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=6246797worker1 # [ 157.419734] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886798worker1 # [ 157.429355] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6799worker1 # [ 157.432482] nix-daemon[846]: reaped child process 1747, status = succeeded6800worker1 # [ 157.438350] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6801worker1 # [ 158.656822] nix-daemon[846]: reaped child process 1761, status = succeeded6802worker1 # [ 158.786773] nix-daemon[846]: accepted connection from pid 1776, user nix-grpc-daemon (trusted)6803worker1 # [ 158.799152] nix-daemon[846]: reaped child process 1779, status = succeeded6804lb # [ 158.808409] niks3-server[1185]: 2026/09/20 08:06:24 INFO Received uploads request method=POST path=/api/pending_closures6805worker1 # [ 158.822188] nix-grpc-daemon[1398]: time=2026-09-20T08:06:24.769Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6806worker1 # [ 158.823713] nix-grpc-daemon[1398]: time=2026-09-20T08:06:24.771Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6807lb # [ 158.854064] niks3-server[1185]: 2026/09/20 08:06:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6808lb # [ 158.862085] niks3-server[1185]: 2026/09/20 08:06:24 INFO Signed narinfos id=55 count=16809worker1 # [ 158.870602] nix-grpc-daemon[1398]: time=2026-09-20T08:06:24.818Z level=INFO msg="Uploading 1 narinfos"6810lb # [ 158.869213] niks3-server[1185]: 2026/09/20 08:06:24 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6811lb # [ 158.873108] niks3-server[1185]: 2026/09/20 08:06:24 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6812lb # [ 158.880102] niks3-server[1185]: 2026/09/20 08:06:24 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6813lb # [ 158.886167] niks3-server[1185]: 2026/09/20 08:06:24 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6814lb # [ 158.889933] niks3-server[1185]: 2026/09/20 08:06:24 INFO Completed upload id=556815lb # [ 158.891168] niks3-server[1185]: 2026/09/20 08:06:24 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6816worker1 # [ 158.896068] nix-grpc-daemon[1398]: time=2026-09-20T08:06:24.843Z level=INFO msg="Upload complete. (247ms)"6817worker1 # [ 158.901596] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16818worker2 # [ 158.759833] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:24Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16819worker1 # [ 158.916167] nix-daemon[846]: reaped child process 1759, status = succeeded6820worker1 # [ 158.945777] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886821worker2 # [ 158.807076] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:24Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06822worker1 # [ 158.958121] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886823worker1 # [ 158.970227] nix-grpc-daemon[1398]: time=2026-09-20T08:06:24.918Z level=INFO msg="All 2 paths already cached"6824worker1 # [ 158.977903] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886825worker1 # [ 158.986229] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6826worker1 # [ 159.129606] nix-daemon[846]: accepted connection from pid 1783, user nix-grpc-daemon (trusted)6827worker1 # [ 159.142832] nix-daemon[846]: reaped child process 1786, status = succeeded6828worker1 # [ 159.262583] nix-daemon[846]: accepted connection from pid 1788, user nix-grpc-daemon (trusted)6829worker1 # [ 159.275806] nix-daemon[846]: reaped child process 1791, status = succeeded6830lb # [ 159.278465] niks3-server[1185]: 2026/09/20 08:06:25 INFO Received uploads request method=POST path=/api/pending_closures6831worker1 # [ 159.288336] nix-grpc-daemon[1398]: time=2026-09-20T08:06:25.235Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6832worker1 # [ 159.289721] nix-grpc-daemon[1398]: time=2026-09-20T08:06:25.236Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6833lb # [ 159.290931] niks3-server[1185]: 2026/09/20 08:06:25 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6834lb # [ 159.295542] niks3-server[1185]: 2026/09/20 08:06:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6835lb # [ 159.297597] niks3-server[1185]: 2026/09/20 08:06:25 INFO Signed narinfos id=56 count=16836worker1 # [ 159.303695] nix-grpc-daemon[1398]: time=2026-09-20T08:06:25.251Z level=INFO msg="Uploading 1 narinfos"6837lb # [ 159.302354] niks3-server[1185]: 2026/09/20 08:06:25 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6838lb # [ 159.310984] niks3-server[1185]: 2026/09/20 08:06:25 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6839lb # [ 159.313415] niks3-server[1185]: 2026/09/20 08:06:25 INFO Completed upload id=566840worker1 # [ 159.319351] nix-grpc-daemon[1398]: time=2026-09-20T08:06:25.267Z level=INFO msg="Upload complete. (320ms)"6841lb # [ 159.315339] niks3-server[1185]: 2026/09/20 08:06:25 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6842worker1 # [ 159.321341] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=7526843worker1 # [ 159.325199] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886844worker1 # [ 159.330872] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6845worker1 # [ 159.333193] nix-daemon[846]: reaped child process 1781, status = succeeded6846worker1 # [ 159.338219] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6847worker1 # [ 160.554106] nix-daemon[846]: reaped child process 1795, status = succeeded6848worker1 # [ 160.669840] nix-daemon[846]: accepted connection from pid 1810, user nix-grpc-daemon (trusted)6849worker1 # [ 160.682624] nix-daemon[846]: reaped child process 1813, status = succeeded6850lb # [ 160.688711] niks3-server[1185]: 2026/09/20 08:06:26 INFO Received uploads request method=POST path=/api/pending_closures6851worker1 # [ 160.704160] nix-grpc-daemon[1398]: time=2026-09-20T08:06:26.651Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6852worker1 # [ 160.706532] nix-grpc-daemon[1398]: time=2026-09-20T08:06:26.653Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6853lb # [ 160.724863] niks3-server[1185]: 2026/09/20 08:06:26 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6854lb # [ 160.732559] niks3-server[1185]: 2026/09/20 08:06:26 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6855lb # [ 160.746418] niks3-server[1185]: 2026/09/20 08:06:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6856worker1 # [ 160.756036] nix-grpc-daemon[1398]: time=2026-09-20T08:06:26.703Z level=INFO msg="Uploading 1 narinfos"6857lb # [ 160.751105] niks3-server[1185]: 2026/09/20 08:06:26 INFO Signed narinfos id=57 count=16858lb # [ 160.755487] niks3-server[1185]: 2026/09/20 08:06:26 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6859lb # [ 160.765317] niks3-server[1185]: 2026/09/20 08:06:26 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6860lb # [ 160.769761] niks3-server[1185]: 2026/09/20 08:06:26 INFO Completed upload id=576861worker1 # [ 160.775868] nix-grpc-daemon[1398]: time=2026-09-20T08:06:26.723Z level=INFO msg="Upload complete. (230ms)"6862lb # [ 160.771539] niks3-server[1185]: 2026/09/20 08:06:26 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6863worker2 # [ 160.639372] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:26Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16864worker1 # [ 160.780929] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16865worker1 # [ 160.795194] nix-daemon[846]: reaped child process 1793, status = succeeded6866worker1 # [ 160.825660] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886867worker1 # [ 160.836890] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886868worker1 # [ 160.846735] nix-grpc-daemon[1398]: time=2026-09-20T08:06:26.794Z level=INFO msg="All 2 paths already cached"6869worker1 # [ 160.854278] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886870worker1 # [ 160.864167] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6871worker1 # [ 161.022030] nix-daemon[846]: accepted connection from pid 1817, user nix-grpc-daemon (trusted)6872worker1 # [ 161.035475] nix-daemon[846]: reaped child process 1820, status = succeeded6873worker1 # [ 161.196801] nix-daemon[846]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)6874worker1 # [ 161.209428] nix-daemon[846]: reaped child process 1825, status = succeeded6875lb # [ 161.218890] niks3-server[1185]: 2026/09/20 08:06:27 INFO Received uploads request method=POST path=/api/pending_closures6876worker1 # [ 161.231401] nix-grpc-daemon[1398]: time=2026-09-20T08:06:27.178Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6877worker1 # [ 161.232767] nix-grpc-daemon[1398]: time=2026-09-20T08:06:27.179Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6878lb # [ 161.239876] niks3-server[1185]: 2026/09/20 08:06:27 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6879lb # [ 161.246940] niks3-server[1185]: 2026/09/20 08:06:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6880lb # [ 161.250367] niks3-server[1185]: 2026/09/20 08:06:27 INFO Signed narinfos id=58 count=16881worker1 # [ 161.258839] nix-grpc-daemon[1398]: time=2026-09-20T08:06:27.206Z level=INFO msg="Uploading 1 narinfos"6882lb # [ 161.259936] niks3-server[1185]: 2026/09/20 08:06:27 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6883lb # [ 161.267689] niks3-server[1185]: 2026/09/20 08:06:27 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6884lb # [ 161.273461] niks3-server[1185]: 2026/09/20 08:06:27 INFO Completed upload id=586885worker1 # [ 161.279477] nix-grpc-daemon[1398]: time=2026-09-20T08:06:27.227Z level=INFO msg="Upload complete. (404ms)"6886lb # [ 161.274926] niks3-server[1185]: 2026/09/20 08:06:27 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6887worker1 # [ 161.281601] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=7526888worker1 # [ 161.285325] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886889worker1 # [ 161.291873] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6890worker1 # [ 161.294208] nix-daemon[846]: reaped child process 1815, status = succeeded6891worker1 # [ 161.300043] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6892worker1 # [ 162.114776] nix-daemon[846]: reaped child process 1829, status = succeeded6893worker1 # [ 162.253305] nix-daemon[846]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6894worker1 # [ 162.266837] nix-daemon[846]: reaped child process 1847, status = succeeded6895lb # [ 162.276373] niks3-server[1185]: 2026/09/20 08:06:28 INFO Received uploads request method=POST path=/api/pending_closures6896worker1 # [ 162.289330] nix-grpc-daemon[1398]: time=2026-09-20T08:06:28.236Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6897worker1 # [ 162.290858] nix-grpc-daemon[1398]: time=2026-09-20T08:06:28.238Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6898lb # [ 162.300561] niks3-server[1185]: 2026/09/20 08:06:28 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6899lb # [ 162.306113] niks3-server[1185]: 2026/09/20 08:06:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6900lb # [ 162.308350] niks3-server[1185]: 2026/09/20 08:06:28 INFO Signed narinfos id=59 count=16901worker1 # [ 162.316216] nix-grpc-daemon[1398]: time=2026-09-20T08:06:28.264Z level=INFO msg="Uploading 1 narinfos"6902lb # [ 162.313981] niks3-server[1185]: 2026/09/20 08:06:28 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6903lb # [ 162.318082] niks3-server[1185]: 2026/09/20 08:06:28 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6904lb # [ 162.322186] niks3-server[1185]: 2026/09/20 08:06:28 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6905lb # [ 162.326283] niks3-server[1185]: 2026/09/20 08:06:28 INFO Completed upload id=596906worker1 # [ 162.333719] nix-grpc-daemon[1398]: time=2026-09-20T08:06:28.281Z level=INFO msg="Upload complete. (226ms)"6907lb # [ 162.329192] niks3-server[1185]: 2026/09/20 08:06:28 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6908worker1 # [ 162.336155] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16909worker2 # [ 162.197248] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:27Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16910worker1 # [ 162.343755] nix-daemon[846]: reaped child process 1827, status = succeeded6911worker1 # [ 162.381807] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886912worker1 # [ 162.392390] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:410886913worker1 # [ 162.404636] nix-grpc-daemon[1398]: time=2026-09-20T08:06:28.352Z level=INFO msg="All 2 paths already cached"6914worker1 # [ 162.411607] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:410886915worker1 # [ 162.419308] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6916worker1 # [ 162.581798] nix-daemon[846]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6917worker1 # [ 162.596046] nix-daemon[846]: reaped child process 1854, status = succeeded6918worker1 # [ 162.744127] nix-daemon[846]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)6919worker1 # [ 162.758164] nix-daemon[846]: reaped child process 1859, status = succeeded6920lb # [ 162.767120] niks3-server[1185]: 2026/09/20 08:06:28 INFO Received uploads request method=POST path=/api/pending_closures6921worker1 # [ 162.781324] nix-grpc-daemon[1398]: time=2026-09-20T08:06:28.728Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6922worker1 # [ 162.782705] nix-grpc-daemon[1398]: time=2026-09-20T08:06:28.728Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6923lb # [ 162.794877] niks3-server[1185]: 2026/09/20 08:06:28 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6924lb # [ 162.803351] niks3-server[1185]: 2026/09/20 08:06:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6925worker1 # [ 162.814891] nix-grpc-daemon[1398]: time=2026-09-20T08:06:28.762Z level=INFO msg="Uploading 1 narinfos"6926lb # [ 162.809643] niks3-server[1185]: 2026/09/20 08:06:28 INFO Signed narinfos id=60 count=16927lb # [ 162.825215] niks3-server[1185]: 2026/09/20 08:06:28 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6928lb # [ 162.917885] niks3-server[1185]: 2026/09/20 08:06:28 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6929lb # [ 162.926613] niks3-server[1185]: 2026/09/20 08:06:28 INFO Completed upload id=606930worker1 # [ 162.932853] nix-grpc-daemon[1398]: time=2026-09-20T08:06:28.880Z level=INFO msg="Upload complete. (501ms)"6931lb # [ 162.933063] niks3-server[1185]: 2026/09/20 08:06:28 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6932worker1 # [ 162.938906] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=0 paths=1 nar_bytes_in=7526933worker1 # [ 162.945518] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:410886934worker1 # [ 162.960252] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6935worker1 # [ 162.969967] nix-daemon[846]: reaped child process 1849, status = succeeded6936worker1 # [ 162.981096] nix-daemon[846]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)6937worker1 # [ 164.400252] nix-daemon[846]: reaped child process 1863, status = succeeded6938worker1 # [ 164.533739] nix-daemon[846]: accepted connection from pid 1878, user nix-grpc-daemon (trusted)6939worker1 # [ 164.545344] nix-daemon[846]: reaped child process 1881, status = succeeded6940lb # [ 164.555748] niks3-server[1185]: 2026/09/20 08:06:30 INFO Received uploads request method=POST path=/api/pending_closures6941worker1 # [ 164.572736] nix-grpc-daemon[1398]: time=2026-09-20T08:06:30.520Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6942worker1 # [ 164.575774] nix-grpc-daemon[1398]: time=2026-09-20T08:06:30.523Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6943lb # [ 164.593105] niks3-server[1185]: 2026/09/20 08:06:30 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6944lb # [ 164.605450] niks3-server[1185]: 2026/09/20 08:06:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6945lb # [ 164.609220] niks3-server[1185]: 2026/09/20 08:06:30 INFO Signed narinfos id=61 count=16946worker1 # [ 164.617325] nix-grpc-daemon[1398]: time=2026-09-20T08:06:30.565Z level=INFO msg="Uploading 1 narinfos"6947lb # [ 164.615985] niks3-server[1185]: 2026/09/20 08:06:30 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6948lb # [ 164.620673] niks3-server[1185]: 2026/09/20 08:06:30 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6949lb # [ 164.623164] niks3-server[1185]: 2026/09/20 08:06:30 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6950lb # [ 164.626386] niks3-server[1185]: 2026/09/20 08:06:30 INFO Completed upload id=616951worker1 # [ 164.633598] nix-grpc-daemon[1398]: time=2026-09-20T08:06:30.581Z level=INFO msg="Upload complete. (240ms)"6952lb # [ 164.630272] niks3-server[1185]: 2026/09/20 08:06:30 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6953worker1 # [ 164.637745] nix-grpc-daemon[1314]: ts=2026-09-20T08:06:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41088 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16954worker1 # [ 164.648778] nix-daemon[846]: reaped child process 1861, status = succeeded6955client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.13 seconds)6956worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1'6957worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1', in 0.06 seconds)6958client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16959worker2 # [ 164.775093] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537166960worker2 # [ 164.781700] nix-grpc-daemon[1897]: ts=2026-09-20T08:06:30Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06961lb # [ 168.502927] envoy[924]: conn peer=[2001:db8:1::1]:55598 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=26791 tx=10623 ms=158816962client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 120.21 seconds)6963(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 136.40 seconds)6964subtest: low disk drains a worker and builds go to the other6965worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6966worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.99 seconds)6967worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6968worker1 # [ 286.615714] nix-grpc-daemon[1314]: ts=2026-09-20T08:08:32Z level=info event=unhealthy reason=min_free6969worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6970client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag drain >&26971worker2 # [ 287.082475] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537166972client # don't know how to build these paths:6973client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6974worker2 # [ 287.092820] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:32Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06975worker2 # [ 287.094740] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:32Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16976worker2 # [ 287.135645] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537166977worker2 # [ 287.141925] nix-grpc-daemon[1897]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5)6978worker2 # [ 287.188533] nix-grpc-daemon[1897]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 2/5)6979worker2 # [ 287.248097] nix-grpc-daemon[1897]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 3/5)6980worker2 # [ 287.251905] nix-grpc-daemon[1897]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 295 ms (attempt 4/5)6981worker2 # [ 287.548134] nix-grpc-daemon[1897]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6982worker2 # [ 287.565197] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537166983client # copying 1 paths to 'lb:50051'...6984client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6985worker2 # [ 287.574814] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537166986worker2 # [ 287.583876] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6987worker2 # [ 287.604358] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:33Z level=info event=niks3_push_started pid=19766988worker2 # [ 287.901325] nix-daemon[923]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6989worker2 # [ 287.912511] nix-daemon[923]: reaped child process 1992, status = succeeded6990worker2 # [ 288.071323] nix-daemon[923]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6991worker2 # [ 288.083522] nix-daemon[923]: reaped child process 1997, status = succeeded6992lb # [ 288.239701] niks3-server[1185]: 2026/09/20 08:08:34 INFO Received uploads request method=POST path=/api/pending_closures6993worker2 # [ 288.111094] nix-grpc-daemon[1976]: time=2026-09-20T08:08:33.615Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6994worker2 # [ 288.114338] nix-grpc-daemon[1976]: time=2026-09-20T08:08:33.617Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6995lb # [ 288.279802] niks3-server[1185]: 2026/09/20 08:08:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6996lb # [ 288.288310] niks3-server[1185]: 2026/09/20 08:08:34 INFO Signed narinfos id=62 count=16997worker2 # [ 288.154847] nix-grpc-daemon[1976]: time=2026-09-20T08:08:33.659Z level=INFO msg="Uploading 1 narinfos"6998lb # [ 288.306347] niks3-server[1185]: 2026/09/20 08:08:34 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6999lb # [ 288.385458] niks3-server[1185]: 2026/09/20 08:08:34 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7000lb # [ 288.436164] niks3-server[1185]: 2026/09/20 08:08:34 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7001lb # [ 288.452887] niks3-server[1185]: 2026/09/20 08:08:34 INFO Completed upload id=627002worker2 # [ 288.316314] nix-grpc-daemon[1976]: time=2026-09-20T08:08:33.821Z level=INFO msg="Upload complete. (547ms)"7003worker2 # [ 288.319313] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53716 duration_s=0 paths=1 nar_bytes_in=5127004worker2 # [ 288.322398] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537167005lb # [ 288.462421] niks3-server[1185]: 2026/09/20 08:08:34 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7006worker2 # [ 288.329797] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7007worker2 # [ 288.332185] nix-daemon[923]: reaped child process 1974, status = succeeded7008client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7009client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7010worker2 # [ 288.337325] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7011lb # [ 288.514069] envoy[924]: conn peer=[2001:db8:1::1]:53770 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=725 tx=455 ms=1200227012client # farm-a> LOG-farm-a-drain7013worker2 # [ 289.151848] nix-daemon[923]: reaped child process 2001, status = succeeded7014worker2 # [ 289.296269] nix-daemon[923]: accepted connection from pid 2016, user nix-grpc-daemon (trusted)7015worker2 # [ 289.309559] nix-daemon[923]: reaped child process 2019, status = succeeded7016lb # [ 289.462188] niks3-server[1185]: 2026/09/20 08:08:35 INFO Received uploads request method=POST path=/api/pending_closures7017worker2 # [ 289.335827] nix-grpc-daemon[1976]: time=2026-09-20T08:08:34.840Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7018worker2 # [ 289.337562] nix-grpc-daemon[1976]: time=2026-09-20T08:08:34.842Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7019lb # [ 289.498461] niks3-server[1185]: 2026/09/20 08:08:35 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7020lb # [ 289.512691] niks3-server[1185]: 2026/09/20 08:08:35 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7021lb # [ 289.523348] niks3-server[1185]: 2026/09/20 08:08:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7022lb # [ 289.527986] niks3-server[1185]: 2026/09/20 08:08:35 INFO Signed narinfos id=63 count=17023worker2 # [ 289.395441] nix-grpc-daemon[1976]: time=2026-09-20T08:08:34.899Z level=INFO msg="Uploading 1 narinfos"7024lb # [ 289.534635] niks3-server[1185]: 2026/09/20 08:08:35 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7025lb # [ 289.542291] niks3-server[1185]: 2026/09/20 08:08:35 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7026lb # [ 289.546086] niks3-server[1185]: 2026/09/20 08:08:35 INFO Completed upload id=637027lb # [ 289.548463] niks3-server[1185]: 2026/09/20 08:08:35 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7028worker2 # [ 289.410986] nix-grpc-daemon[1976]: time=2026-09-20T08:08:34.915Z level=INFO msg="Upload complete. (267ms)"7029worker2 # [ 289.421623] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53716 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17030worker2 # [ 289.428935] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:34Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07031worker2 # [ 289.432429] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:34Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17032worker2 # [ 289.440292] nix-daemon[923]: reaped child process 1999, status = succeeded7033worker2 # [ 289.472108] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537167034worker2 # [ 289.485069] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537167035worker2 # [ 289.498280] nix-grpc-daemon[1976]: time=2026-09-20T08:08:35.003Z level=INFO msg="All 1 paths already cached"7036client # copying 1 paths to 'lb:50051'...7037client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7038worker2 # [ 289.506162] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537167039worker2 # [ 289.516185] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7040worker2 # [ 289.659320] nix-daemon[923]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)7041worker2 # [ 289.673638] nix-daemon[923]: reaped child process 2028, status = succeeded7042worker2 # [ 289.803488] nix-daemon[923]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)7043worker2 # [ 289.817071] nix-daemon[923]: reaped child process 2033, status = succeeded7044lb # [ 289.961820] niks3-server[1185]: 2026/09/20 08:08:35 INFO Received uploads request method=POST path=/api/pending_closures7045worker2 # [ 289.834620] nix-grpc-daemon[1976]: time=2026-09-20T08:08:35.339Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7046worker2 # [ 289.835980] nix-grpc-daemon[1976]: time=2026-09-20T08:08:35.339Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7047lb # [ 289.991655] niks3-server[1185]: 2026/09/20 08:08:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7048lb # [ 289.996904] niks3-server[1185]: 2026/09/20 08:08:35 INFO Signed narinfos id=64 count=17049worker2 # [ 289.863366] nix-grpc-daemon[1976]: time=2026-09-20T08:08:35.368Z level=INFO msg="Uploading 1 narinfos"7050lb # [ 290.003919] niks3-server[1185]: 2026/09/20 08:08:35 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7051lb # [ 290.014438] niks3-server[1185]: 2026/09/20 08:08:35 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7052lb # [ 290.019559] niks3-server[1185]: 2026/09/20 08:08:35 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7053worker2 # [ 289.889246] nix-grpc-daemon[1976]: time=2026-09-20T08:08:35.394Z level=INFO msg="Upload complete. (362ms)"7054lb # [ 290.025918] niks3-server[1185]: 2026/09/20 08:08:35 INFO Completed upload id=647055worker2 # [ 289.892092] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53716 duration_s=0 paths=1 nar_bytes_in=6247056worker2 # [ 289.894588] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537167057lb # [ 290.031598] niks3-server[1185]: 2026/09/20 08:08:35 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7058worker2 # [ 289.900239] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7059worker2 # [ 289.904647] nix-daemon[923]: reaped child process 2023, status = succeeded7060client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7061worker2 # [ 289.911372] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7062client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7063client # farm-b> LOG-farm-b-drain7064worker2 # [ 290.730318] nix-daemon[923]: reaped child process 2037, status = succeeded7065worker2 # [ 290.869453] nix-daemon[923]: accepted connection from pid 2052, user nix-grpc-daemon (trusted)7066worker2 # [ 290.880898] nix-daemon[923]: reaped child process 2055, status = succeeded7067lb # [ 291.032852] niks3-server[1185]: 2026/09/20 08:08:36 INFO Received uploads request method=POST path=/api/pending_closures7068worker2 # [ 290.904797] nix-grpc-daemon[1976]: time=2026-09-20T08:08:36.409Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7069worker2 # [ 290.906748] nix-grpc-daemon[1976]: time=2026-09-20T08:08:36.411Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7070lb # [ 291.070928] niks3-server[1185]: 2026/09/20 08:08:36 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7071lb # [ 291.075772] niks3-server[1185]: 2026/09/20 08:08:36 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7072lb # [ 291.090836] niks3-server[1185]: 2026/09/20 08:08:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7073worker2 # [ 290.960504] nix-grpc-daemon[1976]: time=2026-09-20T08:08:36.465Z level=INFO msg="Uploading 1 narinfos"7074lb # [ 291.097434] niks3-server[1185]: 2026/09/20 08:08:37 INFO Signed narinfos id=65 count=17075lb # [ 291.103029] niks3-server[1185]: 2026/09/20 08:08:37 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7076lb # [ 291.108165] niks3-server[1185]: 2026/09/20 08:08:37 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7077lb # [ 291.111433] niks3-server[1185]: 2026/09/20 08:08:37 INFO Completed upload id=657078worker2 # [ 290.975151] nix-grpc-daemon[1976]: time=2026-09-20T08:08:36.480Z level=INFO msg="Upload complete. (256ms)"7079lb # [ 291.114206] niks3-server[1185]: 2026/09/20 08:08:37 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7080worker2 # [ 290.984785] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53716 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17081worker2 # [ 290.992284] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:36Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07082worker2 # [ 290.995514] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:36Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17083worker2 # [ 291.003701] nix-daemon[923]: reaped child process 2035, status = succeeded7084worker2 # [ 291.034943] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537167085worker2 # [ 291.042522] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537167086worker2 # [ 291.050247] nix-grpc-daemon[1976]: time=2026-09-20T08:08:36.555Z level=INFO msg="All 2 paths already cached"7087client # copying 1 paths to 'lb:50051'...7088worker2 # [ 291.055626] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537167089client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7090worker2 # [ 291.062611] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7091worker2 # [ 291.208136] nix-daemon[923]: accepted connection from pid 2061, user nix-grpc-daemon (trusted)7092worker2 # [ 291.221498] nix-daemon[923]: reaped child process 2064, status = succeeded7093worker2 # [ 291.372093] nix-daemon[923]: accepted connection from pid 2066, user nix-grpc-daemon (trusted)7094worker2 # [ 291.385808] nix-daemon[923]: reaped child process 2069, status = succeeded7095lb # [ 291.535604] niks3-server[1185]: 2026/09/20 08:08:37 INFO Received uploads request method=POST path=/api/pending_closures7096worker2 # [ 291.406990] nix-grpc-daemon[1976]: time=2026-09-20T08:08:36.911Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7097worker2 # [ 291.408507] nix-grpc-daemon[1976]: time=2026-09-20T08:08:36.911Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7098lb # [ 291.559503] niks3-server[1185]: 2026/09/20 08:08:37 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7099lb # [ 291.571738] niks3-server[1185]: 2026/09/20 08:08:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7100lb # [ 291.577247] niks3-server[1185]: 2026/09/20 08:08:37 INFO Signed narinfos id=66 count=17101worker2 # [ 291.443893] nix-grpc-daemon[1976]: time=2026-09-20T08:08:36.948Z level=INFO msg="Uploading 1 narinfos"7102lb # [ 291.583709] niks3-server[1185]: 2026/09/20 08:08:37 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7103lb # [ 291.596394] niks3-server[1185]: 2026/09/20 08:08:37 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7104lb # [ 291.601311] niks3-server[1185]: 2026/09/20 08:08:37 INFO Completed upload id=667105worker2 # [ 291.467342] nix-grpc-daemon[1976]: time=2026-09-20T08:08:36.972Z level=INFO msg="Upload complete. (397ms)"7106lb # [ 291.605044] niks3-server[1185]: 2026/09/20 08:08:37 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7107worker2 # [ 291.471085] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53716 duration_s=0 paths=1 nar_bytes_in=7527108worker2 # [ 291.474972] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537167109worker2 # [ 291.481973] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7110worker2 # [ 291.484349] nix-daemon[923]: reaped child process 2059, status = succeeded7111worker2 # [ 291.489853] nix-daemon[923]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)7112client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7113client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7114client # farm-top> LOG-farm-top-drain7115worker2 # [ 292.318332] nix-daemon[923]: reaped child process 2073, status = succeeded7116worker2 # [ 292.456159] nix-daemon[923]: accepted connection from pid 2088, user nix-grpc-daemon (trusted)7117worker2 # [ 292.470896] nix-daemon[923]: reaped child process 2091, status = succeeded7118lb # [ 292.623134] niks3-server[1185]: 2026/09/20 08:08:38 INFO Received uploads request method=POST path=/api/pending_closures7119worker2 # [ 292.494921] nix-grpc-daemon[1976]: time=2026-09-20T08:08:37.999Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7120worker2 # [ 292.498070] nix-grpc-daemon[1976]: time=2026-09-20T08:08:38.001Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7121lb # [ 292.661768] niks3-server[1185]: 2026/09/20 08:08:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7122worker2 # [ 292.536119] nix-grpc-daemon[1976]: time=2026-09-20T08:08:38.040Z level=INFO msg="Uploading 1 narinfos"7123lb # [ 292.672713] niks3-server[1185]: 2026/09/20 08:08:38 INFO Signed narinfos id=67 count=17124lb # [ 292.678393] niks3-server[1185]: 2026/09/20 08:08:38 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7125lb # [ 292.683747] niks3-server[1185]: 2026/09/20 08:08:38 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7126lb # [ 292.688100] niks3-server[1185]: 2026/09/20 08:08:38 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7127lb # [ 292.698358] niks3-server[1185]: 2026/09/20 08:08:38 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7128lb # [ 292.703724] niks3-server[1185]: 2026/09/20 08:08:38 INFO Completed upload id=677129worker2 # [ 292.567082] nix-grpc-daemon[1976]: time=2026-09-20T08:08:38.071Z level=INFO msg="Upload complete. (269ms)"7130lb # [ 292.706572] niks3-server[1185]: 2026/09/20 08:08:38 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7131worker2 # [ 292.575238] nix-grpc-daemon[1897]: ts=2026-09-20T08:08:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53716 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17132worker2 # [ 292.587302] nix-daemon[923]: reaped child process 2071, status = succeeded7133client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag drain >&2, in 5.71 seconds)7134client: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag drain outPath7135client: (finished: must succeed: nix eval --raw -f /nix/store/ds1pbvns9yfiwin6gyw68m5d6l6a7x29-job.nix --argstr tag drain outPath, in 0.17 seconds)7136worker1: must succeed: rm /nix/.rw-store/fill7137worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.10 seconds)7138worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7139worker1 # [ 293.620229] nix-grpc-daemon[1314]: ts=2026-09-20T08:08:39Z level=info event=healthy reason=min_free7140worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)7141(finished: subtest: low disk drains a worker and builds go to the other, in 9.24 seconds)7142(finished: run the VM test script, in 295.18 seconds)7143test script finished in 295.47s7144cleanup7145kill QemuMachine (pid 49)7146client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7147kill QemuMachine (pid 50)7148lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7149kill QemuMachine (pid 51)7150worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7151kill QemuMachine (pid 52)7152worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7153(finished: cleanup, in 1.01 seconds)7154additionally exposed symbols:7155 client, lb, worker1, worker2,7156 vlan1,7157 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh