vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #473
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb # Formatting '/build/vm-state-lb/tmp.QMpq2NqxsS', fmt=raw size=107374182414worker1 # Disk image does not exist, creating the virtualisation disk image...15lb # mke2fs 1.47.4 (6-Mar-2025)16lb: QEMU running (pid 50)17worker1: QEMU running (pid 52)18worker1 # Formatting '/build/vm-state-worker1/tmp.iExclQbith', fmt=raw size=107374182419lb # Discarding device blocks: 0/262144 done20worker1 # mke2fs 1.47.4 (6-Mar-2025)21lb # Creating filesystem with 262144 4k blocks and 65536 inodes22client # Disk image does not exist, creating the virtualisation disk image...23client: QEMU running (pid 49)24worker1 # Discarding device blocks: 0/262144 done25lb # Filesystem UUID: 0ac7a1b0-a04b-40ed-8584-2d317d80ca4c26worker2 # Disk image does not exist, creating the virtualisation disk image...27client # Formatting '/build/vm-state-client/tmp.SsFdvtFJWB', fmt=raw size=107374182428lb # Superblock backups stored on blocks:29worker2 # Formatting '/build/vm-state-worker2/tmp.cFoSgWEVSz', fmt=raw size=107374182430client # mke2fs 1.47.4 (6-Mar-2025)31lb # 32768, 98304, 163840, 22937632worker2 # mke2fs 1.47.4 (6-Mar-2025)33client # Discarding device blocks: 0/262144 done34lb # 35worker2 # Discarding device blocks: 0/262144 done36client # Creating filesystem with 262144 4k blocks and 65536 inodes37worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes38worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes39client # Filesystem UUID: 73f3ba7c-b2be-464f-91fe-92efdb1f8b5e40worker1 # Filesystem UUID: a9e8d808-75f3-4cad-b064-c6a59208fdb941worker2 # Filesystem UUID: f1a55f63-4450-494c-88ba-d3b96e4dad0242client # Superblock backups stored on blocks:43worker1 # Superblock backups stored on blocks:44worker2 # Superblock backups stored on blocks:45client # 32768, 98304, 163840, 22937646worker1 # 32768, 98304, 163840, 22937647worker2 # 32768, 98304, 163840, 22937648client # 49worker1 # 50worker2 # 51client # Allocating group tables: 0/8 done52worker1 # Allocating group tables: 0/8 done53worker2 # Allocating group tables: 0/8 done54client # Writing inode tables: 0/8 done55worker1 # Writing inode tables: 0/8 done56worker2 # Writing inode tables: 0/8 done57client # Creating journal (8192 blocks): done58worker1 # Creating journal (8192 blocks): done59worker2 # Creating journal (8192 blocks): done60client # Writing superblocks and filesystem accounting information: 0/8 done61worker1 # Writing superblocks and filesystem accounting information: 0/8 done62worker2 # Writing superblocks and filesystem accounting information: 0/8 done63client # 64worker1 # 65worker2 # 66client # Virtualisation disk image created.67worker1 # Virtualisation disk image created.68worker2 # Virtualisation disk image created.69worker2: QEMU running (pid 51)70lb # Allocating group tables: 0/8 done71lb # Writing inode tables: 0/8 done72lb # Creating journal (8192 blocks): done73lb # Writing superblocks and filesystem accounting information: 0/8 done74lb # 75lb # Virtualisation disk image created.76(finished: start all VMs, in 0.27 seconds)77client: waiting for unit network-addresses-eth1.service78client: waiting for the VM to finish booting79lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker1 # 84worker1 # 85worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086lb # 87lb # 88lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90worker1 # 91worker1 # 92worker1 # 93worker1 # 94worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0095lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...96lb # 97lb # 98lb # 99lb # 100worker2 # 101worker2 # 102lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00103worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00104lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105lb # 106lb # 107worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...108worker1 # 109worker1 # 110client # 111client # 112worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...113worker2 # 114worker2 # 115worker2 # 116worker2 # 117client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00118worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00119client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...120client # 121client # 122client # 123client # 124worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...125worker2 # 126worker2 # 127client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129client # 130client # 131worker1 # Booting from ROM...132lb # Booting from ROM...133worker2 # Booting from ROM...134client # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026137worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/sv46bqjv56hpwfdlfbiqqk75c9wn49ss-nixos-system-worker1-test/init regInfo=/nix/store/9svqxhrgyf4k3m7ih0lqx23pv9hl752c-closure-info/registration console=ttyS0,115200n8 console=tty0138worker1 # [ 0.000000] BIOS-provided physical RAM map:139worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable140worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved141worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved142lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fj20bcn47ir2axl7b7sxwa0fvig2ds2j-nixos-system-lb-test/init regInfo=/nix/store/l5iprc9wdxb7nl0q1bx2grj7psaadl34-closure-info/registration console=ttyS0,115200n8 console=tty0143worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable144lb # [ 0.000000] BIOS-provided physical RAM map:145lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable146worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved147lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved148worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved149lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved150worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved151lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable152worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved153worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026154lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved155worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved156lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved157worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved158worker1 # [ 0.000000] NX (Execute Disable) protection: active159lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved160worker1 # [ 0.000000] APIC: Static calls initialized161client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026162worker1 # [ 0.000000] SMBIOS 2.8 present.163lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved164lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved165worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4p0rsf301ngzk6ydvvpd55qzgfl3sk2k-nixos-system-worker2-test/init regInfo=/nix/store/wsi5ixxy1rl93hi0l57rxfvizfzxgnaz-closure-info/registration console=ttyS0,115200n8 console=tty0166worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014167worker2 # [ 0.000000] BIOS-provided physical RAM map:168worker1 # [ 0.000000] DMI: Memory slots populated: 1/1169lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved170worker1 # [ 0.000000] Hypervisor detected: KVM171lb # [ 0.000000] NX (Execute Disable) protection: active172worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable173lb # [ 0.000000] APIC: Static calls initialized174worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000175lb # [ 0.000000] SMBIOS 2.8 present.176worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved177worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00178client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test/init regInfo=/nix/store/9balbfgs3w3ldwc5x160r8h4fh1s60j0-closure-info/registration console=ttyS0,115200n8 console=tty0179worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved180worker1 # [ 0.000001] kvm-clock: using sched offset of 504506977 cycles181client # [ 0.000000] BIOS-provided physical RAM map:182lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014183client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable184worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable185lb # [ 0.000000] DMI: Memory slots populated: 1/1186worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns187lb # [ 0.000000] Hypervisor detected: KVM188worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved189client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved190worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor191lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000192client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved194worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000195lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00196client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable197worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved198lb # [ 0.000000] kvm-clock: using sched offset of 500238529 cycles199worker1 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs200client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved201worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved202worker1 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT203lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns204client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved205worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved206worker1 # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f]207lb # [ 0.000005] tsc: Detected 2400.012 MHz processor208worker1 # [ 0.002741] Using GB pages for direct mapping209worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved210lb # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000211client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved212worker1 # [ 0.002883] RAMDISK: [mem 0x5e367000-0x5ffcffff]213worker2 # [ 0.000000] NX (Execute Disable) protection: active214client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved215lb # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs216worker1 # [ 0.002892] ACPI: Early table checksum verification disabled217worker2 # [ 0.000000] APIC: Static calls initialized218worker2 # [ 0.000000] SMBIOS 2.8 present.219worker1 # [ 0.002895] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )220client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved221lb # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT222client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved223lb # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f]224worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014225worker1 # [ 0.002899] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)226lb # [ 0.002740] Using GB pages for direct mapping227client # [ 0.000000] NX (Execute Disable) protection: active228worker2 # [ 0.000000] DMI: Memory slots populated: 1/1229client # [ 0.000000] APIC: Static calls initialized230worker2 # [ 0.000000] Hypervisor detected: KVM231worker1 # [ 0.002903] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)232client # [ 0.000000] SMBIOS 2.8 present.233lb # [ 0.002892] RAMDISK: [mem 0x5e367000-0x5ffcffff]234worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000235lb # [ 0.002901] ACPI: Early table checksum verification disabled236worker1 # [ 0.002910] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)237lb # [ 0.002904] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )238worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00239client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014240worker1 # [ 0.002912] ACPI: FACS 0x000000005FFE0000 000040241worker2 # [ 0.000001] kvm-clock: using sched offset of 502849965 cycles242client # [ 0.000000] DMI: Memory slots populated: 1/1243lb # [ 0.002908] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)244client # [ 0.000000] Hypervisor detected: KVM245worker1 # [ 0.002913] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)246client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000247worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns248lb # [ 0.002912] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)249worker1 # [ 0.002915] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)250client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00251worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor252lb # [ 0.002919] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)253client # [ 0.000001] kvm-clock: using sched offset of 507326574 cycles254worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000255worker1 # [ 0.002917] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)256lb # [ 0.002921] ACPI: FACS 0x000000005FFE0000 000040257worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs258client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns259worker1 # [ 0.002918] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)260lb # [ 0.002922] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)261client # [ 0.000005] tsc: Detected 2400.012 MHz processor262worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT263worker1 # [ 0.002919] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]264lb # [ 0.002924] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)265client # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000266worker2 # [ 0.002735] found SMP MP-table at [mem 0x000f5450-0x000f545f]267worker1 # [ 0.002920] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]268worker2 # [ 0.002747] Using GB pages for direct mapping269client # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs270lb # [ 0.002925] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)271worker2 # [ 0.002832] RAMDISK: [mem 0x5e367000-0x5ffcffff]272worker1 # [ 0.002921] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]273worker2 # [ 0.002841] ACPI: Early table checksum verification disabled274client # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT275lb # [ 0.002927] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)276worker1 # [ 0.002921] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]277client # [ 0.002731] found SMP MP-table at [mem 0x000f5450-0x000f545f]278worker2 # [ 0.002844] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )279lb # [ 0.002928] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]280worker1 # [ 0.002922] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]281client # [ 0.002743] Using GB pages for direct mapping282worker2 # [ 0.002848] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)283client # [ 0.002831] RAMDISK: [mem 0x5e366000-0x5ffcffff]284lb # [ 0.002929] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]285worker1 # [ 0.002922] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]286client # [ 0.002840] ACPI: Early table checksum verification disabled287worker2 # [ 0.002852] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)288lb # [ 0.002930] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]289worker1 # [ 0.002923] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]290client # [ 0.002843] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )291worker1 # [ 0.003153] No NUMA configuration found292lb # [ 0.002930] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]293worker2 # [ 0.002859] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)294worker1 # [ 0.003153] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]295client # [ 0.002847] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)296worker2 # [ 0.002861] ACPI: FACS 0x000000005FFE0000 000040297lb # [ 0.002931] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]298worker1 # [ 0.003156] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]299worker1 # [ 0.005552] Zone ranges:300client # [ 0.002851] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)301lb # [ 0.002931] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]302worker2 # [ 0.002862] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)303worker1 # [ 0.005553] DMA [mem 0x0000000000001000-0x0000000000ffffff]304lb # [ 0.002932] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]305client # [ 0.002859] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)306worker2 # [ 0.002864] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)307lb # [ 0.003163] No NUMA configuration found308worker1 # [ 0.005555] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]309client # [ 0.002861] ACPI: FACS 0x000000005FFE0000 000040310worker1 # [ 0.005556] Normal empty311worker1 # [ 0.005557] Device empty312lb # [ 0.003164] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]313worker2 # [ 0.002865] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)314worker1 # [ 0.005558] Movable zone start for each node315client # [ 0.002862] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)316lb # [ 0.003166] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]317worker1 # [ 0.005558] Early memory node ranges318lb # [ 0.005560] Zone ranges:319worker2 # [ 0.002867] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)320client # [ 0.002864] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)321worker1 # [ 0.005559] node 0: [mem 0x0000000000001000-0x000000000009efff]322lb # [ 0.005561] DMA [mem 0x0000000000001000-0x0000000000ffffff]323worker2 # [ 0.002868] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]324worker1 # [ 0.005560] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]325lb # [ 0.005563] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]326client # [ 0.002865] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)327worker2 # [ 0.002869] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]328lb # [ 0.005564] Normal empty329worker1 # [ 0.005561] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]330lb # [ 0.005565] Device empty331worker2 # [ 0.002870] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]332client # [ 0.002867] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)333lb # [ 0.005565] Movable zone start for each node334worker1 # [ 0.005582] On node 0, zone DMA: 1 pages in unavailable ranges335lb # [ 0.005566] Early memory node ranges336worker2 # [ 0.002870] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]337client # [ 0.002868] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]338worker1 # [ 0.005860] On node 0, zone DMA: 97 pages in unavailable ranges339lb # [ 0.005567] node 0: [mem 0x0000000000001000-0x000000000009efff]340client # [ 0.002869] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]341worker2 # [ 0.002871] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]342worker1 # [ 0.034198] On node 0, zone DMA32: 40 pages in unavailable ranges343lb # [ 0.005568] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]344worker1 # [ 0.034661] ACPI: PM-Timer IO Port: 0x608345worker2 # [ 0.002872] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]346client # [ 0.002870] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]347worker1 # [ 0.034672] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])348lb # [ 0.005569] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]349client # [ 0.002870] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]350worker2 # [ 0.002872] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]351lb # [ 0.005591] On node 0, zone DMA: 1 pages in unavailable ranges352worker1 # [ 0.034700] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23353worker2 # [ 0.003102] No NUMA configuration found354client # [ 0.002871] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]355lb # [ 0.005863] On node 0, zone DMA: 97 pages in unavailable ranges356worker1 # [ 0.034702] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)357worker2 # [ 0.003103] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]358lb # [ 0.034212] On node 0, zone DMA32: 40 pages in unavailable ranges359client # [ 0.002871] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]360worker1 # [ 0.034704] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)361worker2 # [ 0.003106] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]362lb # [ 0.034669] ACPI: PM-Timer IO Port: 0x608363worker2 # [ 0.005566] Zone ranges:364client # [ 0.002872] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]365worker1 # [ 0.034705] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)366lb # [ 0.034681] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])367worker2 # [ 0.005567] DMA [mem 0x0000000000001000-0x0000000000ffffff]368client # [ 0.003106] No NUMA configuration found369worker1 # [ 0.034706] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)370lb # [ 0.034709] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23371worker2 # [ 0.005569] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]372client # [ 0.003107] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]373worker2 # [ 0.005570] Normal empty374worker1 # [ 0.034707] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)375worker2 # [ 0.005571] Device empty376client # [ 0.003109] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]377lb # [ 0.034712] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)378client # [ 0.005561] Zone ranges:379worker2 # [ 0.005571] Movable zone start for each node380worker1 # [ 0.034709] ACPI: Using ACPI (MADT) for SMP configuration information381lb # [ 0.034714] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)382worker2 # [ 0.005572] Early memory node ranges383client # [ 0.005562] DMA [mem 0x0000000000001000-0x0000000000ffffff]384worker1 # [ 0.034710] ACPI: HPET id: 0x8086a201 base: 0xfed00000385lb # [ 0.034715] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)386worker2 # [ 0.005572] node 0: [mem 0x0000000000001000-0x000000000009efff]387client # [ 0.005563] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]388worker1 # [ 0.034713] TSC deadline timer available389worker1 # [ 0.034718] CPU topo: Max. logical packages: 1390client # [ 0.005564] Normal empty391lb # [ 0.034716] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)392worker2 # [ 0.005573] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]393client # [ 0.005565] Device empty394worker1 # [ 0.034719] CPU topo: Max. logical dies: 1395client # [ 0.005565] Movable zone start for each node396lb # [ 0.034717] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)397worker2 # [ 0.005574] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]398worker1 # [ 0.034719] CPU topo: Max. dies per package: 1399client # [ 0.005566] Early memory node ranges400lb # [ 0.034719] ACPI: Using ACPI (MADT) for SMP configuration information401worker2 # [ 0.005595] On node 0, zone DMA: 1 pages in unavailable ranges402worker1 # [ 0.034722] CPU topo: Max. threads per core: 1403client # [ 0.005567] node 0: [mem 0x0000000000001000-0x000000000009efff]404worker1 # [ 0.034723] CPU topo: Num. cores per package: 1405lb # [ 0.034720] ACPI: HPET id: 0x8086a201 base: 0xfed00000406worker2 # [ 0.005873] On node 0, zone DMA: 97 pages in unavailable ranges407lb # [ 0.034724] TSC deadline timer available408client # [ 0.005568] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]409worker1 # [ 0.034723] CPU topo: Num. threads per package: 1410worker2 # [ 0.034514] On node 0, zone DMA32: 40 pages in unavailable ranges411lb # [ 0.034728] CPU topo: Max. logical packages: 1412worker1 # [ 0.034724] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs413client # [ 0.005569] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]414worker2 # [ 0.034976] ACPI: PM-Timer IO Port: 0x608415lb # [ 0.034729] CPU topo: Max. logical dies: 1416client # [ 0.005590] On node 0, zone DMA: 1 pages in unavailable ranges417worker1 # [ 0.034742] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()418lb # [ 0.034729] CPU topo: Max. dies per package: 1419worker2 # [ 0.034987] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])420lb # [ 0.034733] CPU topo: Max. threads per core: 1421client # [ 0.005868] On node 0, zone DMA: 97 pages in unavailable ranges422worker2 # [ 0.035016] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23423worker1 # [ 0.034775] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]424lb # [ 0.034733] CPU topo: Num. cores per package: 1425client # [ 0.034139] On node 0, zone DMA32: 40 pages in unavailable ranges426worker2 # [ 0.035018] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)427lb # [ 0.034734] CPU topo: Num. threads per package: 1428client # [ 0.034600] ACPI: PM-Timer IO Port: 0x608429worker1 # [ 0.034777] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]430worker2 # [ 0.035020] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)431lb # [ 0.034734] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs432client # [ 0.034611] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])433worker1 # [ 0.034778] [mem 0x60000000-0xafffffff] available for PCI devices434worker1 # [ 0.034780] Booting paravirtualized kernel on KVM435worker2 # [ 0.035021] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)436lb # [ 0.034753] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()437client # [ 0.034641] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23438worker2 # [ 0.035022] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)439lb # [ 0.034787] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]440client # [ 0.034644] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)441worker1 # [ 0.034783] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns442worker2 # [ 0.035023] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)443client # [ 0.034646] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)444lb # [ 0.034788] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]445worker1 # [ 0.039231] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1446worker2 # [ 0.035025] ACPI: Using ACPI (MADT) for SMP configuration information447client # [ 0.034647] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)448lb # [ 0.034790] [mem 0x60000000-0xafffffff] available for PCI devices449worker1 # [ 0.041876] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152450worker2 # [ 0.035026] ACPI: HPET id: 0x8086a201 base: 0xfed00000451lb # [ 0.034792] Booting paravirtualized kernel on KVM452client # [ 0.034648] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)453worker2 # [ 0.035031] TSC deadline timer available454worker1 # [ 0.041922] kvm-guest: PV spinlocks disabled, single CPU455worker2 # [ 0.035035] CPU topo: Max. logical packages: 1456client # [ 0.034648] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)457lb # [ 0.034794] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns458worker2 # [ 0.035036] CPU topo: Max. logical dies: 1459client # [ 0.034651] ACPI: Using ACPI (MADT) for SMP configuration information460worker2 # [ 0.035036] CPU topo: Max. dies per package: 1461lb # [ 0.039274] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1462client # [ 0.034651] ACPI: HPET id: 0x8086a201 base: 0xfed00000463worker2 # [ 0.035041] CPU topo: Max. threads per core: 1464client # [ 0.034655] TSC deadline timer available465lb # [ 0.041846] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152466worker2 # [ 0.035042] CPU topo: Num. cores per package: 1467client # [ 0.034659] CPU topo: Max. logical packages: 1468worker2 # [ 0.035042] CPU topo: Num. threads per package: 1469lb # [ 0.041892] kvm-guest: PV spinlocks disabled, single CPU470worker1 # [ 0.041924] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/sv46bqjv56hpwfdlfbiqqk75c9wn49ss-nixos-system-worker1-test/init regInfo=/nix/store/9svqxhrgyf4k3m7ih0lqx23pv9hl752c-closure-info/registration console=ttyS0,115200n8 console=tty0471client # [ 0.034660] CPU topo: Max. logical dies: 1472worker2 # [ 0.035043] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs473client # [ 0.034660] CPU topo: Max. dies per package: 1474client # [ 0.034664] CPU topo: Max. threads per core: 1475worker2 # [ 0.035061] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()476client # [ 0.034664] CPU topo: Num. cores per package: 1477worker1 # [ 0.042018] Unknown kernel command line parameters "regInfo=/nix/store/9svqxhrgyf4k3m7ih0lqx23pv9hl752c-closure-info/registration", will be passed to user space.478worker1 # [ 0.042031] random: crng init done479worker2 # [ 0.035092] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]480client # [ 0.034665] CPU topo: Num. threads per package: 1481worker1 # [ 0.042032] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes482worker2 # [ 0.035094] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]483client # [ 0.034665] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs484lb # [ 0.041894] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fj20bcn47ir2axl7b7sxwa0fvig2ds2j-nixos-system-lb-test/init regInfo=/nix/store/l5iprc9wdxb7nl0q1bx2grj7psaadl34-closure-info/registration console=ttyS0,115200n8 console=tty0485client # [ 0.034684] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()486worker2 # [ 0.035096] [mem 0x60000000-0xafffffff] available for PCI devices487worker1 # [ 0.044289] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)488worker2 # [ 0.035097] Booting paravirtualized kernel on KVM489client # [ 0.034716] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]490worker1 # [ 0.045417] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)491lb # [ 0.041994] Unknown kernel command line parameters "regInfo=/nix/store/l5iprc9wdxb7nl0q1bx2grj7psaadl34-closure-info/registration", will be passed to user space.492lb # [ 0.042009] random: crng init done493worker1 # [ 0.045456] Fallback order for Node 0: 0494client # [ 0.034717] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]495worker2 # [ 0.035101] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns496worker1 # [ 0.045458] Built 1 zonelists, mobility grouping on. Total pages: 393078497lb # [ 0.042010] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes498client # [ 0.034719] [mem 0x60000000-0xafffffff] available for PCI devices499worker2 # [ 0.039580] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1500worker1 # [ 0.045459] Policy zone: DMA32501client # [ 0.034720] Booting paravirtualized kernel on KVM502worker2 # [ 0.041878] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152503lb # [ 0.044242] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)504worker1 # [ 0.048079] mem auto-init: stack:all(zero), heap alloc:on, heap free:off505worker2 # [ 0.041925] kvm-guest: PV spinlocks disabled, single CPU506lb # [ 0.045356] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)507client # [ 0.034723] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns508worker1 # [ 0.051381] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1509lb # [ 0.045388] Fallback order for Node 0: 0510worker1 # [ 0.055176] allocated 3145728 bytes of page_ext511client # [ 0.039175] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1512worker1 # [ 0.065474] ftrace: allocating 48731 entries in 192 pages513lb # [ 0.045391] Built 1 zonelists, mobility grouping on. Total pages: 393078514client # [ 0.041481] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152515lb # [ 0.045392] Policy zone: DMA32516worker1 # [ 0.065476] ftrace: allocated 192 pages with 2 groups517worker1 # [ 0.066336] Dynamic Preempt: lazy518client # [ 0.041528] kvm-guest: PV spinlocks disabled, single CPU519lb # [ 0.048007] mem auto-init: stack:all(zero), heap alloc:on, heap free:off520worker1 # [ 0.066511] rcu: Preemptible hierarchical RCU implementation.521lb # [ 0.051377] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1522worker2 # [ 0.041927] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4p0rsf301ngzk6ydvvpd55qzgfl3sk2k-nixos-system-worker2-test/init regInfo=/nix/store/wsi5ixxy1rl93hi0l57rxfvizfzxgnaz-closure-info/registration console=ttyS0,115200n8 console=tty0523worker1 # [ 0.066512] rcu: RCU event tracing is enabled.524lb # [ 0.055215] allocated 3145728 bytes of page_ext525worker1 # [ 0.066513] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.526lb # [ 0.065554] ftrace: allocating 48731 entries in 192 pages527worker1 # [ 0.066514] Trampoline variant of Tasks RCU enabled.528lb # [ 0.065556] ftrace: allocated 192 pages with 2 groups529worker2 # [ 0.042022] Unknown kernel command line parameters "regInfo=/nix/store/wsi5ixxy1rl93hi0l57rxfvizfzxgnaz-closure-info/registration", will be passed to user space.530lb # [ 0.066402] Dynamic Preempt: lazy531worker1 # [ 0.066515] Rude variant of Tasks RCU enabled.532worker2 # [ 0.042035] random: crng init done533worker1 # [ 0.066515] Tracing variant of Tasks RCU enabled.534lb # [ 0.066578] rcu: Preemptible hierarchical RCU implementation.535client # [ 0.041530] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test/init regInfo=/nix/store/9balbfgs3w3ldwc5x160r8h4fh1s60j0-closure-info/registration console=ttyS0,115200n8 console=tty0536worker2 # [ 0.042035] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes537lb # [ 0.066579] rcu: RCU event tracing is enabled.538worker1 # [ 0.066516] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.539lb # [ 0.066579] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.540worker2 # [ 0.044298] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)541worker1 # [ 0.066517] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1542client # [ 0.041624] Unknown kernel command line parameters "regInfo=/nix/store/9balbfgs3w3ldwc5x160r8h4fh1s60j0-closure-info/registration", will be passed to user space.543lb # [ 0.066581] Trampoline variant of Tasks RCU enabled.544client # [ 0.041637] random: crng init done545worker2 # [ 0.045705] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)546lb # [ 0.066582] Rude variant of Tasks RCU enabled.547worker1 # [ 0.066538] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.548worker2 # [ 0.045738] Fallback order for Node 0: 0549client # [ 0.041638] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes550lb # [ 0.066582] Tracing variant of Tasks RCU enabled.551worker2 # [ 0.045740] Built 1 zonelists, mobility grouping on. Total pages: 393078552worker1 # [ 0.066540] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.553worker2 # [ 0.045742] Policy zone: DMA32554client # [ 0.044120] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)555lb # [ 0.066582] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.556worker2 # [ 0.048391] mem auto-init: stack:all(zero), heap alloc:on, heap free:off557lb # [ 0.066583] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1558client # [ 0.045214] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)559worker1 # [ 0.066541] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.560client # [ 0.045246] Fallback order for Node 0: 0561worker2 # [ 0.051717] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1562worker1 # [ 0.070953] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16563lb # [ 0.066605] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.564worker2 # [ 0.055222] allocated 3145728 bytes of page_ext565client # [ 0.045249] Built 1 zonelists, mobility grouping on. Total pages: 393078566worker1 # [ 0.071235] rcu: srcu_init: Setting srcu_struct sizes based on contention.567client # [ 0.045250] Policy zone: DMA32568worker2 # [ 0.065578] ftrace: allocating 48731 entries in 192 pages569lb # [ 0.066607] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.570client # [ 0.047868] mem auto-init: stack:all(zero), heap alloc:on, heap free:off571worker2 # [ 0.065580] ftrace: allocated 192 pages with 2 groups572worker1 # [ 0.071244] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns573worker2 # [ 0.066446] Dynamic Preempt: lazy574lb # [ 0.066608] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.575client # [ 0.051024] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1576worker2 # [ 0.066624] rcu: Preemptible hierarchical RCU implementation.577worker1 # [ 0.071352] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)578lb # [ 0.071018] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16579client # [ 0.054792] allocated 3145728 bytes of page_ext580worker2 # [ 0.066624] rcu: RCU event tracing is enabled.581worker1 # [ 0.074902] Console: colour VGA+ 80x25582client # [ 0.064946] ftrace: allocating 48731 entries in 192 pages583lb # [ 0.071301] rcu: srcu_init: Setting srcu_struct sizes based on contention.584worker1 # [ 0.074905] printk: legacy console [tty0] enabled585worker2 # [ 0.066625] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.586client # [ 0.064948] ftrace: allocated 192 pages with 2 groups587worker1 # [ 0.104347] printk: legacy console [ttyS0] enabled588client # [ 0.065853] Dynamic Preempt: lazy589worker2 # [ 0.066627] Trampoline variant of Tasks RCU enabled.590lb # [ 0.071309] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns591worker2 # [ 0.066628] Rude variant of Tasks RCU enabled.592worker1 # [ 0.200169] ACPI: Core revision 20250807593client # [ 0.066027] rcu: Preemptible hierarchical RCU implementation.594worker2 # [ 0.066628] Tracing variant of Tasks RCU enabled.595client # [ 0.066028] rcu: RCU event tracing is enabled.596lb # [ 0.071418] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)597lb # [ 0.074975] Console: colour VGA+ 80x25598client # [ 0.066029] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.599worker2 # [ 0.066629] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.600worker1 # [ 0.201037] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns601lb # [ 0.074977] printk: legacy console [tty0] enabled602client # [ 0.066030] Trampoline variant of Tasks RCU enabled.603worker2 # [ 0.066630] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1604lb # [ 0.104461] printk: legacy console [ttyS0] enabled605client # [ 0.066031] Rude variant of Tasks RCU enabled.606worker1 # [ 0.202524] APIC: Switch to symmetric I/O mode setup607client # [ 0.066031] Tracing variant of Tasks RCU enabled.608worker1 # [ 0.203460] x2apic enabled609worker2 # [ 0.066644] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.610lb # [ 0.200081] ACPI: Core revision 20250807611client # [ 0.066032] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.612worker2 # [ 0.066646] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.613worker1 # [ 0.204188] APIC: Switched APIC routing to: physical x2apic614client # [ 0.066032] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1615lb # [ 0.200999] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns616worker2 # [ 0.066647] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.617lb # [ 0.202470] APIC: Switch to symmetric I/O mode setup618client # [ 0.066054] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.619worker2 # [ 0.071064] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16620worker1 # [ 0.205967] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1621lb # [ 0.203437] x2apic enabled622client # [ 0.066056] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623worker2 # [ 0.071347] rcu: srcu_init: Setting srcu_struct sizes based on contention.624lb # [ 0.204164] APIC: Switched APIC routing to: physical x2apic625worker1 # [ 0.206917] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns626client # [ 0.066057] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.627worker2 # [ 0.071355] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns628client # [ 0.070469] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16629worker1 # [ 0.208509] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)630worker2 # [ 0.071463] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)631lb # [ 0.205951] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1632client # [ 0.070757] rcu: srcu_init: Setting srcu_struct sizes based on contention.633worker2 # [ 0.075332] Console: colour VGA+ 80x25634worker2 # [ 0.075335] printk: legacy console [tty0] enabled635worker1 # [ 0.209595] x86/cpu: User Mode Instruction Prevention (UMIP) activated636client # [ 0.070765] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns637worker2 # [ 0.104822] printk: legacy console [ttyS0] enabled638lb # [ 0.206899] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns639worker1 # [ 0.210634] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127640worker2 # [ 0.200494] ACPI: Core revision 20250807641client # [ 0.070868] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)642client # [ 0.074433] Console: colour VGA+ 80x25643worker1 # [ 0.211507] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0644lb # [ 0.208522] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)645client # [ 0.074437] printk: legacy console [tty0] enabled646worker2 # [ 0.201393] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns647client # [ 0.103939] printk: legacy console [ttyS0] enabled648lb # [ 0.209608] x86/cpu: User Mode Instruction Prevention (UMIP) activated649worker1 # [ 0.212511] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto650worker2 # [ 0.202885] APIC: Switch to symmetric I/O mode setup651client # [ 0.199822] ACPI: Core revision 20250807652lb # [ 0.210642] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127653worker2 # [ 0.203825] x2apic enabled654worker1 # [ 0.213507] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl655lb # [ 0.211520] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0656client # [ 0.200653] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns657worker2 # [ 0.204569] APIC: Switched APIC routing to: physical x2apic658worker1 # [ 0.215507] Transient Scheduler Attacks: Vulnerable: No microcode659client # [ 0.202169] APIC: Switch to symmetric I/O mode setup660worker1 # [ 0.216506] Spectre V2 : Mitigation: Enhanced / Automatic IBRS661lb # [ 0.212523] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto662client # [ 0.203168] x2apic enabled663worker2 # [ 0.206353] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1664worker1 # [ 0.217506] Speculative Return Stack Overflow: Mitigation: Safe RET665client # [ 0.203924] APIC: Switched APIC routing to: physical x2apic666lb # [ 0.213520] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl667worker2 # [ 0.207325] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns668lb # [ 0.214519] Transient Scheduler Attacks: Vulnerable: No microcode669client # [ 0.205788] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1670lb # [ 0.215519] Spectre V2 : Mitigation: Enhanced / Automatic IBRS671worker2 # [ 0.209014] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)672lb # [ 0.216519] Speculative Return Stack Overflow: Mitigation: Safe RET673worker2 # [ 0.210101] x86/cpu: User Mode Instruction Prevention (UMIP) activated674worker1 # [ 0.218506] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization675client # [ 0.206782] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns676worker1 # [ 0.220515] Spectre V2 : Enabling IBPB for BPF677worker2 # [ 0.211151] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127678client # [ 0.208458] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)679worker2 # [ 0.212012] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0680worker1 # [ 0.221242] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier681client # [ 0.209544] x86/cpu: User Mode Instruction Prevention (UMIP) activated682worker1 # [ 0.222507] active return thunk: srso_alias_return_thunk683lb # [ 0.217519] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization684client # [ 0.210593] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127685worker2 # [ 0.213015] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto686lb # [ 0.219527] Spectre V2 : Enabling IBPB for BPF687worker1 # [ 0.223527] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'688client # [ 0.212128] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0689worker2 # [ 0.215012] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl690worker1 # [ 0.224506] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'691lb # [ 0.220520] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier692worker2 # [ 0.216012] Transient Scheduler Attacks: Vulnerable: No microcode693worker1 # [ 0.225506] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'694client # [ 0.213208] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto695lb # [ 0.221520] active return thunk: srso_alias_return_thunk696worker2 # [ 0.217011] Spectre V2 : Mitigation: Enhanced / Automatic IBRS697worker1 # [ 0.226506] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'698client # [ 0.214455] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl699lb # [ 0.222540] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'700worker2 # [ 0.218011] Speculative Return Stack Overflow: Mitigation: Safe RET701worker1 # [ 0.227507] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'702client # [ 0.215456] Transient Scheduler Attacks: Vulnerable: No microcode703lb # [ 0.223519] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'704worker1 # [ 0.228507] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'705client # [ 0.217164] Spectre V2 : Mitigation: Enhanced / Automatic IBRS706lb # [ 0.225276] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'707worker1 # [ 0.229506] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'708client # [ 0.218141] Speculative Return Stack Overflow: Mitigation: Safe RET709lb # [ 0.226257] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'710worker1 # [ 0.230507] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256711lb # [ 0.226519] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'712worker2 # [ 0.219011] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization713worker1 # [ 0.231506] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64714worker2 # [ 0.221020] Spectre V2 : Enabling IBPB for BPF715lb # [ 0.228254] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'716worker1 # [ 0.232506] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512717worker2 # [ 0.222012] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier718lb # [ 0.228519] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'719worker1 # [ 0.233506] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024720client # [ 0.218455] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization721worker2 # [ 0.223012] active return thunk: srso_alias_return_thunk722lb # [ 0.230519] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256723client # [ 0.220463] Spectre V2 : Enabling IBPB for BPF724worker1 # [ 0.234507] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8725lb # [ 0.231424] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64726worker2 # [ 0.224034] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'727client # [ 0.221456] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier728worker1 # [ 0.235506] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.729lb # [ 0.232240] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512730worker2 # [ 0.225011] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'731client # [ 0.222456] active return thunk: srso_alias_return_thunk732lb # [ 0.233216] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024733worker2 # [ 0.226744] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'734lb # [ 0.234265] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8735client # [ 0.224111] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'736worker2 # [ 0.227772] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'737client # [ 0.225362] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'738worker2 # [ 0.228791] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'739lb # [ 0.235245] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.740client # [ 0.226235] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'741worker2 # [ 0.229759] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'742client # [ 0.227455] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'743worker2 # [ 0.230864] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'744client # [ 0.228455] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'745worker2 # [ 0.232011] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256746client # [ 0.229445] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'747worker2 # [ 0.233011] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64748client # [ 0.230295] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'749worker2 # [ 0.234011] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512750client # [ 0.231455] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256751worker2 # [ 0.235011] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024752client # [ 0.232455] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64753worker2 # [ 0.236011] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8754client # [ 0.233455] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512755worker2 # [ 0.237011] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.756client # [ 0.234455] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757client # [ 0.235455] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758client # [ 0.236455] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker1 # [ 0.270574] Freeing SMP alternatives memory: 44K760worker1 # [ 0.271375] pid_max: default: 32768 minimum: 301761worker1 # [ 0.272156] LSM: initializing lsm=capability,landlock,yama,bpf,ima762worker1 # [ 0.272604] landlock: Up and running.763lb # [ 0.269541] Freeing SMP alternatives memory: 44K764worker1 # [ 0.273949] Yama: becoming mindful.765lb # [ 0.270322] pid_max: default: 32768 minimum: 301766worker1 # [ 0.274721] LSM support for eBPF active767lb # [ 0.270622] LSM: initializing lsm=capability,landlock,yama,bpf,ima768worker1 # [ 0.275637] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)769lb # [ 0.271633] landlock: Up and running.770lb # [ 0.272520] Yama: becoming mindful.771worker1 # [ 0.276547] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)772lb # [ 0.273732] LSM support for eBPF active773lb # [ 0.274657] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)774worker1 # [ 0.279209] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)775lb # [ 0.275560] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)776worker1 # [ 0.280070] Performance Events: Fam17h+ core perfctr, AMD PMU driver.777worker1 # [ 0.280511] ... version: 2778worker1 # [ 0.281232] ... bit width: 48779lb # [ 0.278332] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)780worker1 # [ 0.281509] ... generic counters: 6781worker2 # [ 0.272722] Freeing SMP alternatives memory: 44K782worker2 # [ 0.273015] pid_max: default: 32768 minimum: 301783worker1 # [ 0.282198] ... generic bitmap: 000000000000003f784lb # [ 0.279087] Performance Events: Fam17h+ core perfctr, AMD PMU driver.785worker1 # [ 0.282521] ... fixed-purpose counters: 0786lb # [ 0.279524] ... version: 2787worker2 # [ 0.274113] LSM: initializing lsm=capability,landlock,yama,bpf,ima788client # [ 0.270749] Freeing SMP alternatives memory: 44K789worker1 # [ 0.283216] ... fixed-purpose bitmap: 0000000000000000790lb # [ 0.280223] ... bit width: 48791worker2 # [ 0.275119] landlock: Up and running.792client # [ 0.271458] pid_max: default: 32768 minimum: 301793worker2 # [ 0.276012] Yama: becoming mindful.794lb # [ 0.280521] ... generic counters: 6795worker1 # [ 0.283509] ... value mask: 0000ffffffffffff796client # [ 0.272289] LSM: initializing lsm=capability,landlock,yama,bpf,ima797worker2 # [ 0.276797] LSM support for eBPF active798lb # [ 0.281234] ... generic bitmap: 000000000000003f799worker1 # [ 0.284375] ... max period: 00007fffffffffff800client # [ 0.273498] landlock: Up and running.801lb # [ 0.281532] ... fixed-purpose counters: 0802worker1 # [ 0.285174] ... global_ctrl mask: 000000000000003f803client # [ 0.274109] Yama: becoming mindful.804worker2 # [ 0.277586] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)805lb # [ 0.282212] ... fixed-purpose bitmap: 0000000000000000806worker1 # [ 0.285637] signal: max sigframe size: 3376807client # [ 0.274669] LSM support for eBPF active808lb # [ 0.282521] ... value mask: 0000ffffffffffff809worker2 # [ 0.278918] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)810worker1 # [ 0.286403] rcu: Hierarchical SRCU implementation.811lb # [ 0.283404] ... max period: 00007fffffffffff812client # [ 0.275582] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)813worker1 # [ 0.287054] rcu: Max phase no-delay instances is 400.814lb # [ 0.284230] ... global_ctrl mask: 000000000000003f815client # [ 0.276504] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)816lb # [ 0.284636] signal: max sigframe size: 3376817worker2 # [ 0.281306] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)818lb # [ 0.285444] rcu: Hierarchical SRCU implementation.819lb # [ 0.286142] rcu: Max phase no-delay instances is 400.820worker2 # [ 0.282597] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821client # [ 0.279337] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)822worker2 # [ 0.283021] ... version: 2823worker2 # [ 0.283750] ... bit width: 48824worker1 # [ 0.292106] smp: Bringing up secondary CPUs ...825client # [ 0.280015] Performance Events: Fam17h+ core perfctr, AMD PMU driver.826worker2 # [ 0.284013] ... generic counters: 6827worker1 # [ 0.292522] smp: Brought up 1 node, 1 CPU828client # [ 0.280460] ... version: 2829worker2 # [ 0.284710] ... generic bitmap: 000000000000003f830client # [ 0.281176] ... bit width: 48831worker1 # [ 0.293179] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)832worker2 # [ 0.285013] ... fixed-purpose counters: 0833client # [ 0.281458] ... generic counters: 6834worker2 # [ 0.285742] ... fixed-purpose bitmap: 0000000000000000835client # [ 0.282201] ... generic bitmap: 000000000000003f836lb # [ 0.291101] smp: Bringing up secondary CPUs ...837worker2 # [ 0.286013] ... value mask: 0000ffffffffffff838client # [ 0.282469] ... fixed-purpose counters: 0839worker1 # [ 0.293674] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)840lb # [ 0.291536] smp: Brought up 1 node, 1 CPU841worker2 # [ 0.286894] ... max period: 00007fffffffffff842client # [ 0.283130] ... fixed-purpose bitmap: 0000000000000000843worker1 # [ 0.294823] devtmpfs: initialized844lb # [ 0.292177] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)845worker2 # [ 0.287699] ... global_ctrl mask: 000000000000003f846client # [ 0.283457] ... value mask: 0000ffffffffffff847worker1 # [ 0.295618] x86/mm: Memory block size: 128MB848worker2 # [ 0.288112] signal: max sigframe size: 3376849client # [ 0.284360] ... max period: 00007fffffffffff850worker2 # [ 0.288921] rcu: Hierarchical SRCU implementation.851lb # [ 0.292722] Memory: 1454552K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)852client # [ 0.285176] ... global_ctrl mask: 000000000000003f853worker1 # [ 0.297342] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)854worker2 # [ 0.289587] rcu: Max phase no-delay instances is 400.855client # [ 0.285573] signal: max sigframe size: 3376856lb # [ 0.293830] devtmpfs: initialized857client # [ 0.286414] rcu: Hierarchical SRCU implementation.858lb # [ 0.294654] x86/mm: Memory block size: 128MB859worker1 # [ 0.298405] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).860client # [ 0.287053] rcu: Max phase no-delay instances is 400.861worker1 # [ 0.299596] pinctrl core: initialized pinctrl subsystem862worker1 # [ 0.300736] PM: RTC time: 12:53:25, date: 2026-09-21863lb # [ 0.296418] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)864lb # [ 0.297485] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).865worker2 # [ 0.294579] smp: Bringing up secondary CPUs ...866lb # [ 0.298629] pinctrl core: initialized pinctrl subsystem867worker2 # [ 0.295027] smp: Brought up 1 node, 1 CPU868lb # [ 0.299702] PM: RTC time: 12:53:25, date: 2026-09-21869worker2 # [ 0.295721] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)870client # [ 0.291968] smp: Bringing up secondary CPUs ...871worker1 # [ 0.304705] NET: Registered PF_NETLINK/PF_ROUTE protocol family872client # [ 0.292472] smp: Brought up 1 node, 1 CPU873client # [ 0.293143] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)874worker1 # [ 0.306055] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations875worker2 # [ 0.296181] Memory: 1454548K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)876worker2 # [ 0.297289] devtmpfs: initialized877worker1 # [ 0.306534] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations878worker2 # [ 0.298042] x86/mm: Memory block size: 128MB879lb # [ 0.303713] NET: Registered PF_NETLINK/PF_ROUTE protocol family880client # [ 0.293623] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)881worker1 # [ 0.307802] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations882client # [ 0.294759] devtmpfs: initialized883lb # [ 0.305045] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations884worker1 # [ 0.308519] audit: initializing netlink subsys (disabled)885client # [ 0.295602] x86/mm: Memory block size: 128MB886worker2 # [ 0.299808] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)887lb # [ 0.305546] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations888worker1 # [ 0.309623] thermal_sys: Registered thermal governor 'fair_share'889worker2 # [ 0.300924] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).890worker1 # [ 0.309625] thermal_sys: Registered thermal governor 'bang_bang'891client # [ 0.297377] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)892lb # [ 0.306809] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations893worker2 # [ 0.302096] pinctrl core: initialized pinctrl subsystem894worker1 # [ 0.310510] thermal_sys: Registered thermal governor 'step_wise'895lb # [ 0.307533] audit: initializing netlink subsys (disabled)896worker2 # [ 0.303199] PM: RTC time: 12:53:25, date: 2026-09-21897client # [ 0.298351] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).898worker1 # [ 0.311413] thermal_sys: Registered thermal governor 'user_space'899client # [ 0.299533] pinctrl core: initialized pinctrl subsystem900lb # [ 0.308637] thermal_sys: Registered thermal governor 'fair_share'901worker1 # [ 0.312194] audit: type=2000 audit(1789995205.709:1): state=initialized audit_enabled=0 res=1902lb # [ 0.308639] thermal_sys: Registered thermal governor 'bang_bang'903client # [ 0.300746] PM: RTC time: 12:53:25, date: 2026-09-21904worker1 # [ 0.313512] thermal_sys: Registered thermal governor 'power_allocator'905lb # [ 0.309522] thermal_sys: Registered thermal governor 'step_wise'906worker1 # [ 0.313540] cpuidle: using governor menu907lb # [ 0.310495] thermal_sys: Registered thermal governor 'user_space'908worker2 # [ 0.306960] NET: Registered PF_NETLINK/PF_ROUTE protocol family909worker1 # [ 0.315732] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5910lb # [ 0.311288] audit: type=2000 audit(1789995205.710:1): state=initialized audit_enabled=0 res=1911worker2 # [ 0.308296] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations912client # [ 0.304363] NET: Registered PF_NETLINK/PF_ROUTE protocol family913lb # [ 0.312525] thermal_sys: Registered thermal governor 'power_allocator'914worker1 # [ 0.316741] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]915worker2 # [ 0.309037] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations916lb # [ 0.312551] cpuidle: using governor menu917worker1 # [ 0.317512] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry918client # [ 0.305671] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations919worker1 # [ 0.318521] PCI: Using configuration type 1 for base access920worker2 # [ 0.310312] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations921lb # [ 0.314531] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5922client # [ 0.306482] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations923worker2 # [ 0.311024] audit: initializing netlink subsys (disabled)924worker1 # [ 0.319624] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.925lb # [ 0.315750] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]926client # [ 0.307758] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations927worker2 # [ 0.312257] thermal_sys: Registered thermal governor 'fair_share'928lb # [ 0.316525] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry929client # [ 0.308468] audit: initializing netlink subsys (disabled)930worker2 # [ 0.312260] thermal_sys: Registered thermal governor 'bang_bang'931lb # [ 0.317543] PCI: Using configuration type 1 for base access932worker2 # [ 0.313015] thermal_sys: Registered thermal governor 'step_wise'933client # [ 0.309630] thermal_sys: Registered thermal governor 'fair_share'934worker2 # [ 0.313944] thermal_sys: Registered thermal governor 'user_space'935client # [ 0.309632] thermal_sys: Registered thermal governor 'bang_bang'936lb # [ 0.318653] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.937client # [ 0.310459] thermal_sys: Registered thermal governor 'step_wise'938worker2 # [ 0.314759] audit: type=2000 audit(1789995205.722:1): state=initialized audit_enabled=0 res=1939client # [ 0.311418] thermal_sys: Registered thermal governor 'user_space'940worker2 # [ 0.316016] thermal_sys: Registered thermal governor 'power_allocator'941worker2 # [ 0.316031] cpuidle: using governor menu942client # [ 0.312215] audit: type=2000 audit(1789995205.722:1): state=initialized audit_enabled=0 res=1943worker1 # [ 0.326745] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages944client # [ 0.313461] thermal_sys: Registered thermal governor 'power_allocator'945worker1 # [ 0.327510] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page946worker2 # [ 0.318264] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5947client # [ 0.313475] cpuidle: using governor menu948worker2 # [ 0.319254] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]949client # [ 0.315722] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5950lb # [ 0.325781] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages951worker2 # [ 0.320017] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry952worker2 # [ 0.321026] PCI: Using configuration type 1 for base access953client # [ 0.316752] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]954lb # [ 0.326523] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page955client # [ 0.317461] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry956worker1 # [ 0.332510] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages957worker2 # [ 0.322152] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.958client # [ 0.318470] PCI: Using configuration type 1 for base access959worker1 # [ 0.333510] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page960client # [ 0.319558] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.961lb # [ 0.331523] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages962lb # [ 0.332523] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page963worker2 # [ 0.327283] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages964worker2 # [ 0.328023] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page965worker1 # [ 0.342185] ACPI: Added _OSI(Module Device)966client # [ 0.326755] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages967worker1 # [ 0.342511] ACPI: Added _OSI(Processor Device)968client # [ 0.327459] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page969worker1 # [ 0.343258] ACPI: Added _OSI(Processor Aggregator Device)970worker2 # [ 0.333015] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages971worker2 # [ 0.334013] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page972lb # [ 0.341157] ACPI: Added _OSI(Module Device)973lb # [ 0.341524] ACPI: Added _OSI(Processor Device)974lb # [ 0.342267] ACPI: Added _OSI(Processor Aggregator Device)975client # [ 0.332459] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages976client # [ 0.333458] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page977worker1 # [ 0.350070] ACPI: 1 ACPI AML tables successfully acquired and loaded978worker1 # [ 0.353553] ACPI: Interpreter enabled979lb # [ 0.349061] ACPI: 1 ACPI AML tables successfully acquired and loaded980worker1 # [ 0.354225] ACPI: PM: (supports S0 S3 S4 S5)981worker2 # [ 0.344364] ACPI: Added _OSI(Module Device)982worker1 # [ 0.354510] ACPI: Using IOAPIC for interrupt routing983worker2 # [ 0.345015] ACPI: Added _OSI(Processor Device)984worker2 # [ 0.345769] ACPI: Added _OSI(Processor Aggregator Device)985worker1 # [ 0.355390] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug986client # [ 0.342466] ACPI: Added _OSI(Module Device)987lb # [ 0.352458] ACPI: Interpreter enabled988client # [ 0.343174] ACPI: Added _OSI(Processor Device)989lb # [ 0.353003] ACPI: PM: (supports S0 S3 S4 S5)990client # [ 0.343458] ACPI: Added _OSI(Processor Aggregator Device)991lb # [ 0.353522] ACPI: Using IOAPIC for interrupt routing992lb # [ 0.354394] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug993worker1 # [ 0.360510] PCI: Using E820 reservations for host bridge windows994worker1 # [ 0.361617] ACPI: Enabled 2 GPEs in block 00 to 3F995worker2 # [ 0.354262] ACPI: 1 ACPI AML tables successfully acquired and loaded996lb # [ 0.359523] PCI: Using E820 reservations for host bridge windows997lb # [ 0.360633] ACPI: Enabled 2 GPEs in block 00 to 3F998client # [ 0.352910] ACPI: 1 ACPI AML tables successfully acquired and loaded999worker2 # [ 0.358421] ACPI: Interpreter enabled1000worker2 # [ 0.359030] ACPI: PM: (supports S0 S3 S4 S5)1001worker2 # [ 0.359739] ACPI: Using IOAPIC for interrupt routing1002worker1 # [ 0.370301] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1003client # [ 0.356708] ACPI: Interpreter enabled1004worker2 # [ 0.360098] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1005client # [ 0.357355] ACPI: PM: (supports S0 S3 S4 S5)1006worker1 # [ 0.371852] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1007client # [ 0.357966] ACPI: Using IOAPIC for interrupt routing1008worker2 # [ 0.361015] PCI: Using E820 reservations for host bridge windows1009worker1 # [ 0.372603] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1010client # [ 0.358513] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1011lb # [ 0.369324] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1012worker1 # [ 0.373634] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1013client # [ 0.359459] PCI: Using E820 reservations for host bridge windows1014worker2 # [ 0.364126] ACPI: Enabled 2 GPEs in block 00 to 3F1015worker1 # [ 0.374862] PCI host bridge to bus 0000:001016lb # [ 0.370884] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1017worker1 # [ 0.375520] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1018lb # [ 0.371620] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1019client # [ 0.362566] ACPI: Enabled 2 GPEs in block 00 to 3F1020worker1 # [ 0.376509] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1021lb # [ 0.372648] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1022worker1 # [ 0.377509] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1023lb # [ 0.373904] PCI host bridge to bus 0000:001024worker1 # [ 0.378509] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1025lb # [ 0.374526] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1026worker1 # [ 0.379509] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1027lb # [ 0.375522] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1028worker1 # [ 0.380509] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1029lb # [ 0.376521] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1030worker1 # [ 0.381510] pci_bus 0000:00: root bus resource [bus 00-ff]1031worker2 # [ 0.372816] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1032lb # [ 0.377523] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1033worker1 # [ 0.382511] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1034lb # [ 0.378522] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1035worker2 # [ 0.373743] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1036client # [ 0.371244] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1037lb # [ 0.379522] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1038worker1 # [ 0.383935] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1039lb # [ 0.380522] pci_bus 0000:00: root bus resource [bus 00-ff]1040worker2 # [ 0.375938] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1041client # [ 0.372180] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1042worker2 # [ 0.377022] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1043lb # [ 0.381475] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1044client # [ 0.373757] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1045worker2 # [ 0.378372] PCI host bridge to bus 0000:001046lb # [ 0.382968] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1047client # [ 0.374590] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1048worker2 # [ 0.379018] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1049client # [ 0.375839] PCI host bridge to bus 0000:001050worker2 # [ 0.380014] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1051client # [ 0.376463] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1052worker2 # [ 0.381014] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1053client # [ 0.377458] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1054worker2 # [ 0.382014] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1055client # [ 0.378458] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1056worker2 # [ 0.383014] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1057worker1 # [ 0.386590] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1058client # [ 0.379458] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1059worker1 # [ 0.387591] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1060worker2 # [ 0.384014] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1061client # [ 0.380457] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1062worker2 # [ 0.385014] pci_bus 0000:00: root bus resource [bus 00-ff]1063worker1 # [ 0.388574] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1064client # [ 0.381457] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1065worker2 # [ 0.386069] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1066client # [ 0.382458] pci_bus 0000:00: root bus resource [bus 00-ff]1067worker1 # [ 0.389836] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1068lb # [ 0.385585] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1069lb # [ 0.386533] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1070client # [ 0.383448] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1071worker2 # [ 0.387460] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1072worker1 # [ 0.391241] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1073lb # [ 0.387491] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1074client # [ 0.384912] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1075lb # [ 0.388822] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1076worker1 # [ 0.393541] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1077worker1 # [ 0.394426] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1078lb # [ 0.390440] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1079worker1 # [ 0.395264] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1080worker1 # [ 0.395515] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1081lb # [ 0.392542] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1082lb # [ 0.393427] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1083worker1 # [ 0.397152] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1084worker2 # [ 0.390176] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1085lb # [ 0.394287] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1086worker2 # [ 0.391026] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1087lb # [ 0.394528] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1088worker1 # [ 0.398533] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1089client # [ 0.387536] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1090worker2 # [ 0.392036] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1091worker1 # [ 0.399397] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1092client # [ 0.388597] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1093lb # [ 0.396076] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1094worker1 # [ 0.400252] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1095worker2 # [ 0.393346] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1096client # [ 0.389525] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1097worker1 # [ 0.401054] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1098client # [ 0.390803] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1099worker2 # [ 0.394787] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1100lb # [ 0.397553] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1101lb # [ 0.398438] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1102client # [ 0.392225] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1103lb # [ 0.399335] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1104worker1 # [ 0.403516] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1105worker2 # [ 0.397048] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1106worker1 # [ 0.404402] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1107worker2 # [ 0.397910] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1108lb # [ 0.401012] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1109worker1 # [ 0.405240] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1110client # [ 0.395490] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1111worker2 # [ 0.398793] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1112client # [ 0.396349] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1113worker2 # [ 0.400013] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1114worker1 # [ 0.406092] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1115client # [ 0.397234] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1116lb # [ 0.403538] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1117worker2 # [ 0.401570] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1118client # [ 0.398464] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1119lb # [ 0.404368] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1120lb # [ 0.405283] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1121worker1 # [ 0.408510] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1122client # [ 0.399994] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1123worker2 # [ 0.403046] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1124worker1 # [ 0.409375] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1125worker2 # [ 0.403895] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1126lb # [ 0.406090] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1127worker1 # [ 0.410251] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1128worker2 # [ 0.404799] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1129client # [ 0.401490] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1130client # [ 0.402335] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1131worker1 # [ 0.411076] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1132lb # [ 0.408424] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1133worker2 # [ 0.406442] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1134client # [ 0.403213] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1135lb # [ 0.409122] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1136worker1 # [ 0.412538] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1137client # [ 0.404052] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1138lb # [ 0.409544] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1139worker1 # [ 0.413405] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1140worker2 # [ 0.409060] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1141worker1 # [ 0.414308] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1142lb # [ 0.411103] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1143worker2 # [ 0.409931] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1144worker2 # [ 0.410757] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1145client # [ 0.406490] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1146worker1 # [ 0.415958] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1147lb # [ 0.412530] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1148client # [ 0.407343] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1149worker2 # [ 0.411586] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1150lb # [ 0.413346] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1151client # [ 0.408218] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1152lb # [ 0.414236] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1153worker1 # [ 0.418518] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1154client # [ 0.409032] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1155worker2 # [ 0.413044] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1156worker1 # [ 0.419376] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1157lb # [ 0.415131] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1158worker2 # [ 0.413860] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1159worker1 # [ 0.420207] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1160client # [ 0.410487] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1161worker2 # [ 0.414749] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1162client # [ 0.411323] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1163worker1 # [ 0.421117] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1164worker2 # [ 0.415585] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1165lb # [ 0.417813] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1166client # [ 0.412187] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1167lb # [ 0.419162] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1168worker1 # [ 0.423132] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1169worker2 # [ 0.417021] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1170lb # [ 0.419543] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1171client # [ 0.413004] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1172worker1 # [ 0.423516] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1173worker2 # [ 0.417874] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1174worker1 # [ 0.424440] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1175worker2 # [ 0.418776] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1176lb # [ 0.421069] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1177client # [ 0.414465] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1178worker1 # [ 0.425340] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1179client # [ 0.415296] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1180worker2 # [ 0.419599] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1181lb # [ 0.422539] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1182client # [ 0.416221] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1183worker1 # [ 0.426819] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1184lb # [ 0.423392] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1185lb # [ 0.424274] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1186client # [ 0.417050] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1187worker1 # [ 0.428251] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1188lb # [ 0.424527] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1189worker1 # [ 0.428532] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1190worker2 # [ 0.422022] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1191worker2 # [ 0.423020] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1192worker1 # [ 0.430126] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1193lb # [ 0.426680] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1194worker2 # [ 0.423975] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1195client # [ 0.419695] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1196client # [ 0.420464] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1197lb # [ 0.428535] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1198worker2 # [ 0.425478] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1199worker1 # [ 0.431550] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1200client # [ 0.421400] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1201lb # [ 0.429466] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1202worker1 # [ 0.432397] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1203worker2 # [ 0.427021] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1204lb # [ 0.430897] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1205client # [ 0.422889] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1206worker1 # [ 0.433285] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1207worker2 # [ 0.427853] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1208worker2 # [ 0.428725] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1209worker1 # [ 0.434860] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1210client # [ 0.424490] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1211worker2 # [ 0.429020] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1212lb # [ 0.433301] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1213client # [ 0.425353] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1214worker1 # [ 0.436517] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1215lb # [ 0.433529] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1216client # [ 0.426215] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1217worker2 # [ 0.431051] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1218worker1 # [ 0.437461] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1219lb # [ 0.434544] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1220client # [ 0.426464] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1221worker1 # [ 0.438252] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1222worker2 # [ 0.432758] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1223lb # [ 0.436074] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1224worker2 # [ 0.433037] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1225client # [ 0.428023] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1226worker1 # [ 0.439096] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1227lb # [ 0.437552] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1228worker2 # [ 0.434594] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1229client # [ 0.429470] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1230worker1 # [ 0.440537] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1231lb # [ 0.438415] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1232client # [ 0.430427] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1233worker1 # [ 0.441558] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1234lb # [ 0.439278] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1235worker2 # [ 0.436068] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1236client # [ 0.431866] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1237worker1 # [ 0.443038] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1238worker2 # [ 0.437020] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1239lb # [ 0.440076] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1240worker2 # [ 0.437955] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1241client # [ 0.434217] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1242lb # [ 0.441875] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1243worker2 # [ 0.439472] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1244client # [ 0.434466] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1245lb # [ 0.442739] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1246client # [ 0.435480] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1247worker2 # [ 0.441046] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1248lb # [ 0.444142] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1249worker2 # [ 0.441913] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1250client # [ 0.437034] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1251worker2 # [ 0.442745] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1252client # [ 0.438489] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1253worker2 # [ 0.443597] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1254client # [ 0.439384] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1255client # [ 0.440265] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1256worker2 # [ 0.444694] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1257worker1 # [ 0.443714] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1258worker2 # [ 0.445229] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1259client # [ 0.441905] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1260worker1 # [ 0.446084] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1261worker2 # [ 0.446557] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1262client # [ 0.443181] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1263worker1 # [ 0.446759] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1264client # [ 0.443668] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1265worker1 # [ 0.448275] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1266client # [ 0.445105] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1267lb # [ 0.444731] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1268worker1 # [ 0.448771] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1269lb # [ 0.447206] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1270worker1 # [ 0.449795] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1271lb # [ 0.447817] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1272worker1 # [ 0.450764] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1273lb # [ 0.449421] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1274worker1 # [ 0.452360] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1275worker1 # [ 0.453211] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1276lb # [ 0.450504] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1277lb # [ 0.451821] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1278worker2 # [ 0.447213] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1279worker1 # [ 0.453838] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1280lb # [ 0.452785] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1281worker1 # [ 0.455037] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1282worker2 # [ 0.449597] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1283worker2 # [ 0.450251] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1284lb # [ 0.454486] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1285lb # [ 0.455175] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1286client # [ 0.445659] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1287worker2 # [ 0.451551] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1288worker1 # [ 0.458542] ACPI: PCI: Interrupt link LNKA configured for IRQ 101289client # [ 0.448093] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1290lb # [ 0.455855] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1291worker2 # [ 0.452282] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1292worker1 # [ 0.459620] ACPI: PCI: Interrupt link LNKB configured for IRQ 101293client # [ 0.448727] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1294worker1 # [ 0.460608] ACPI: PCI: Interrupt link LNKC configured for IRQ 111295lb # [ 0.457180] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1296worker2 # [ 0.453312] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1297worker1 # [ 0.461611] ACPI: PCI: Interrupt link LNKD configured for IRQ 111298client # [ 0.450058] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1299worker2 # [ 0.454262] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1300worker1 # [ 0.462614] ACPI: PCI: Interrupt link LNKE configured for IRQ 101301client # [ 0.450723] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1302worker1 # [ 0.463606] ACPI: PCI: Interrupt link LNKF configured for IRQ 101303worker2 # [ 0.455869] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1304lb # [ 0.460463] ACPI: PCI: Interrupt link LNKA configured for IRQ 101305worker1 # [ 0.464609] ACPI: PCI: Interrupt link LNKG configured for IRQ 111306client # [ 0.451778] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1307worker2 # [ 0.456703] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1308lb # [ 0.461387] ACPI: PCI: Interrupt link LNKB configured for IRQ 101309worker1 # [ 0.465608] ACPI: PCI: Interrupt link LNKH configured for IRQ 111310client # [ 0.452711] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1311lb # [ 0.462361] ACPI: PCI: Interrupt link LNKC configured for IRQ 111312worker2 # [ 0.457342] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1313worker1 # [ 0.466547] ACPI: PCI: Interrupt link GSIA configured for IRQ 161314lb # [ 0.463299] ACPI: PCI: Interrupt link LNKD configured for IRQ 111315worker1 # [ 0.467480] ACPI: PCI: Interrupt link GSIB configured for IRQ 171316client # [ 0.454486] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1317worker2 # [ 0.458603] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1318worker1 # [ 0.468282] ACPI: PCI: Interrupt link GSIC configured for IRQ 181319lb # [ 0.464382] ACPI: PCI: Interrupt link LNKE configured for IRQ 101320client # [ 0.455314] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1321worker1 # [ 0.468525] ACPI: PCI: Interrupt link GSID configured for IRQ 191322lb # [ 0.465550] ACPI: PCI: Interrupt link LNKF configured for IRQ 101323worker1 # [ 0.469513] ACPI: PCI: Interrupt link GSIE configured for IRQ 201324lb # [ 0.466610] ACPI: PCI: Interrupt link LNKG configured for IRQ 111325client # [ 0.456545] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1326worker1 # [ 0.470494] ACPI: PCI: Interrupt link GSIF configured for IRQ 211327lb # [ 0.467622] ACPI: PCI: Interrupt link LNKH configured for IRQ 111328worker1 # [ 0.471259] ACPI: PCI: Interrupt link GSIG configured for IRQ 221329client # [ 0.458130] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1330worker2 # [ 0.462360] ACPI: PCI: Interrupt link LNKA configured for IRQ 101331lb # [ 0.468556] ACPI: PCI: Interrupt link GSIA configured for IRQ 161332worker1 # [ 0.471529] ACPI: PCI: Interrupt link GSIH configured for IRQ 231333worker2 # [ 0.463116] ACPI: PCI: Interrupt link LNKB configured for IRQ 101334lb # [ 0.469489] ACPI: PCI: Interrupt link GSIB configured for IRQ 171335worker2 # [ 0.464113] ACPI: PCI: Interrupt link LNKC configured for IRQ 111336lb # [ 0.470260] ACPI: PCI: Interrupt link GSIC configured for IRQ 181337worker1 # [ 0.473539] iommu: Default domain type: Translated1338worker2 # [ 0.465112] ACPI: PCI: Interrupt link LNKD configured for IRQ 111339lb # [ 0.470534] ACPI: PCI: Interrupt link GSID configured for IRQ 191340worker1 # [ 0.474318] iommu: DMA domain TLB invalidation policy: lazy mode1341client # [ 0.461446] ACPI: PCI: Interrupt link LNKA configured for IRQ 101342lb # [ 0.471487] ACPI: PCI: Interrupt link GSIE configured for IRQ 201343worker2 # [ 0.466111] ACPI: PCI: Interrupt link LNKE configured for IRQ 101344worker1 # [ 0.475497] ACPI: bus type USB registered1345lb # [ 0.472242] ACPI: PCI: Interrupt link GSIF configured for IRQ 211346client # [ 0.462488] ACPI: PCI: Interrupt link LNKB configured for IRQ 101347worker2 # [ 0.467107] ACPI: PCI: Interrupt link LNKF configured for IRQ 101348worker1 # [ 0.476023] usbcore: registered new interface driver usbfs1349lb # [ 0.472533] ACPI: PCI: Interrupt link GSIG configured for IRQ 221350client # [ 0.463516] ACPI: PCI: Interrupt link LNKC configured for IRQ 111351worker2 # [ 0.468100] ACPI: PCI: Interrupt link LNKG configured for IRQ 111352lb # [ 0.473467] ACPI: PCI: Interrupt link GSIH configured for IRQ 231353client # [ 0.464543] ACPI: PCI: Interrupt link LNKD configured for IRQ 111354worker2 # [ 0.469124] ACPI: PCI: Interrupt link LNKH configured for IRQ 111355worker1 # [ 0.476523] usbcore: registered new interface driver hub1356client # [ 0.465560] ACPI: PCI: Interrupt link LNKE configured for IRQ 101357worker2 # [ 0.470055] ACPI: PCI: Interrupt link GSIA configured for IRQ 161358lb # [ 0.475378] iommu: Default domain type: Translated1359worker1 # [ 0.478204] usbcore: registered new device driver usb1360client # [ 0.466558] ACPI: PCI: Interrupt link LNKF configured for IRQ 101361worker2 # [ 0.471019] ACPI: PCI: Interrupt link GSIB configured for IRQ 171362lb # [ 0.476101] iommu: DMA domain TLB invalidation policy: lazy mode1363client # [ 0.467559] ACPI: PCI: Interrupt link LNKG configured for IRQ 111364worker2 # [ 0.471969] ACPI: PCI: Interrupt link GSIC configured for IRQ 181365worker1 # [ 0.479340] NetLabel: Initializing1366lb # [ 0.476816] ACPI: bus type USB registered1367worker2 # [ 0.472770] ACPI: PCI: Interrupt link GSID configured for IRQ 191368client # [ 0.468555] ACPI: PCI: Interrupt link LNKH configured for IRQ 111369worker1 # [ 0.479950] NetLabel: domain hash size = 1281370lb # [ 0.477536] usbcore: registered new interface driver usbfs1371worker2 # [ 0.473029] ACPI: PCI: Interrupt link GSIE configured for IRQ 201372client # [ 0.469496] ACPI: PCI: Interrupt link GSIA configured for IRQ 161373worker1 # [ 0.480509] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1374lb # [ 0.478371] usbcore: registered new interface driver hub1375worker2 # [ 0.473940] ACPI: PCI: Interrupt link GSIF configured for IRQ 211376client # [ 0.470432] ACPI: PCI: Interrupt link GSIB configured for IRQ 171377worker1 # [ 0.481452] NetLabel: unlabeled traffic allowed by default1378worker1 # [ 0.482226] PCI: Using ACPI for IRQ routing1379client # [ 0.471242] ACPI: PCI: Interrupt link GSIC configured for IRQ 181380lb # [ 0.479153] usbcore: registered new device driver usb1381client # [ 0.472276] ACPI: PCI: Interrupt link GSID configured for IRQ 191382worker2 # [ 0.474741] ACPI: PCI: Interrupt link GSIG configured for IRQ 221383client # [ 0.473256] ACPI: PCI: Interrupt link GSIE configured for IRQ 201384worker2 # [ 0.475036] ACPI: PCI: Interrupt link GSIH configured for IRQ 231385lb # [ 0.481110] NetLabel: Initializing1386lb # [ 0.481523] NetLabel: domain hash size = 1281387lb # [ 0.482299] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1388worker2 # [ 0.477087] iommu: Default domain type: Translated1389worker2 # [ 0.477896] iommu: DMA domain TLB invalidation policy: lazy mode1390lb # [ 0.483306] NetLabel: unlabeled traffic allowed by default1391client # [ 0.474217] ACPI: PCI: Interrupt link GSIF configured for IRQ 211392lb # [ 0.484234] PCI: Using ACPI for IRQ routing1393worker2 # [ 0.479004] ACPI: bus type USB registered1394client # [ 0.474479] ACPI: PCI: Interrupt link GSIG configured for IRQ 221395worker2 # [ 0.479772] usbcore: registered new interface driver usbfs1396client # [ 0.475471] ACPI: PCI: Interrupt link GSIH configured for IRQ 231397worker2 # [ 0.480030] usbcore: registered new interface driver hub1398worker2 # [ 0.480940] usbcore: registered new device driver usb1399client # [ 0.477634] iommu: Default domain type: Translated1400worker2 # [ 0.482541] NetLabel: Initializing1401client # [ 0.478465] iommu: DMA domain TLB invalidation policy: lazy mode1402worker2 # [ 0.483015] NetLabel: domain hash size = 1281403client # [ 0.479711] ACPI: bus type USB registered1404worker2 # [ 0.483785] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1405client # [ 0.480507] usbcore: registered new interface driver usbfs1406worker2 # [ 0.484048] NetLabel: unlabeled traffic allowed by default1407client # [ 0.481439] usbcore: registered new interface driver hub1408worker2 # [ 0.484992] PCI: Using ACPI for IRQ routing1409client # [ 0.482152] usbcore: registered new device driver usb1410client # [ 0.483308] NetLabel: Initializing1411client # [ 0.483904] NetLabel: domain hash size = 1281412client # [ 0.484458] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413client # [ 0.485488] NetLabel: unlabeled traffic allowed by default1414client # [ 0.486446] PCI: Using ACPI for IRQ routing1415worker1 # [ 0.525655] pci 0000:00:01.0: vgaarb: setting as boot VGA device1416worker1 # [ 0.526505] pci 0000:00:01.0: vgaarb: bridge control possible1417worker1 # [ 0.526505] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1418worker1 # [ 0.526513] vgaarb: loaded1419worker1 # [ 0.527269] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01420worker1 # [ 0.527511] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1421lb # [ 0.527606] pci 0000:00:01.0: vgaarb: setting as boot VGA device1422lb # [ 0.528518] pci 0000:00:01.0: vgaarb: bridge control possible1423lb # [ 0.528518] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1424lb # [ 0.528526] vgaarb: loaded1425worker1 # [ 0.532607] clocksource: Switched to clocksource kvm-clock1426lb # [ 0.529290] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01427lb # [ 0.529523] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1428worker1 # [ 0.536117] VFS: Disk quotas dquot_6.6.01429worker2 # [ 0.528793] pci 0000:00:01.0: vgaarb: setting as boot VGA device1430worker1 # [ 0.536876] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1431worker2 # [ 0.529010] pci 0000:00:01.0: vgaarb: bridge control possible1432lb # [ 0.533611] clocksource: Switched to clocksource kvm-clock1433worker1 # [ 0.538283] pnp: PnP ACPI init1434worker2 # [ 0.529010] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1435worker1 # [ 0.539080] ACPI: IRQ 4 override to edge(!), high(!)1436worker2 # [ 0.529019] vgaarb: loaded1437worker1 # [ 0.540120] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1438worker2 # [ 0.529799] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01439lb # [ 0.537116] VFS: Disk quotas dquot_6.6.01440worker2 # [ 0.530016] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1441worker1 # [ 0.541688] pnp: PnP ACPI: found 6 devices1442lb # [ 0.537863] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1443client # [ 0.530314] pci 0000:00:01.0: vgaarb: setting as boot VGA device1444lb # [ 0.539221] pnp: PnP ACPI init1445client # [ 0.530454] pci 0000:00:01.0: vgaarb: bridge control possible1446lb # [ 0.540084] ACPI: IRQ 4 override to edge(!), high(!)1447client # [ 0.530454] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1448client # [ 0.530463] vgaarb: loaded1449lb # [ 0.541099] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1450worker2 # [ 0.533112] clocksource: Switched to clocksource kvm-clock1451client # [ 0.531190] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01452lb # [ 0.542638] pnp: PnP ACPI: found 6 devices1453client # [ 0.531458] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1454worker2 # [ 0.536682] VFS: Disk quotas dquot_6.6.01455worker1 # [ 0.549304] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1456worker2 # [ 0.537414] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1457worker1 # [ 0.550766] clocksource: Switched to clocksource acpi_pm1458worker2 # [ 0.538775] pnp: PnP ACPI init1459client # [ 0.534579] clocksource: Switched to clocksource kvm-clock1460worker1 # [ 0.551766] NET: Registered PF_INET protocol family1461worker2 # [ 0.539542] ACPI: IRQ 4 override to edge(!), high(!)1462worker2 # [ 0.540521] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1463worker1 # [ 0.553000] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1464worker2 # [ 0.542105] pnp: PnP ACPI: found 6 devices1465client # [ 0.538152] VFS: Disk quotas dquot_6.6.01466lb # [ 0.550321] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1467client # [ 0.538842] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1468lb # [ 0.551792] clocksource: Switched to clocksource acpi_pm1469client # [ 0.540219] pnp: PnP ACPI init1470lb # [ 0.552829] NET: Registered PF_INET protocol family1471client # [ 0.541067] ACPI: IRQ 4 override to edge(!), high(!)1472lb # [ 0.554006] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1473client # [ 0.542088] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1474client # [ 0.543655] pnp: PnP ACPI: found 6 devices1475worker2 # [ 0.549770] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1476worker2 # [ 0.551275] clocksource: Switched to clocksource acpi_pm1477worker2 # [ 0.552301] NET: Registered PF_INET protocol family1478worker2 # [ 0.553545] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1479client # [ 0.551309] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1480client # [ 0.552817] clocksource: Switched to clocksource acpi_pm1481worker1 # [ 0.569741] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1482client # [ 0.553835] NET: Registered PF_INET protocol family1483worker1 # [ 0.571315] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1484client # [ 0.555343] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1485worker1 # [ 0.572670] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1486worker1 # [ 0.574047] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1487lb # [ 0.570719] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1488worker1 # [ 0.575350] TCP: Hash tables configured (established 16384 bind 16384)1489lb # [ 0.572281] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1490worker1 # [ 0.576489] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1491worker1 # [ 0.577767] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1492lb # [ 0.573620] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1493worker1 # [ 0.578878] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1494lb # [ 0.574989] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1495worker1 # [ 0.580101] NET: Registered PF_UNIX/PF_LOCAL protocol family1496lb # [ 0.576262] TCP: Hash tables configured (established 16384 bind 16384)1497worker1 # [ 0.581095] NET: Registered PF_XDP protocol family1498lb # [ 0.577399] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1499worker1 # [ 0.581979] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1500lb # [ 0.578659] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1501worker1 # [ 0.583020] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1502worker2 # [ 0.570336] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1503lb # [ 0.579738] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1504worker1 # [ 0.583988] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1505worker2 # [ 0.571850] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1506lb # [ 0.580935] NET: Registered PF_UNIX/PF_LOCAL protocol family1507worker1 # [ 0.585067] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1508lb # [ 0.581898] NET: Registered PF_XDP protocol family1509worker2 # [ 0.573145] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1510worker1 # [ 0.586155] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1511lb # [ 0.582705] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1512worker2 # [ 0.574465] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1513worker1 # [ 0.587242] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1514lb # [ 0.583670] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1515worker2 # [ 0.575743] TCP: Hash tables configured (established 16384 bind 16384)1516worker1 # [ 0.588933] ACPI: \_SB_.GSIA: Enabled at IRQ 161517lb # [ 0.584656] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1518client # [ 0.571942] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1519worker2 # [ 0.576865] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1520lb # [ 0.585725] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1521client # [ 0.573410] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1522worker2 # [ 0.578114] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1523worker1 # [ 0.591056] ACPI: \_SB_.GSIB: Enabled at IRQ 171524lb # [ 0.586816] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1525worker2 # [ 0.579169] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1526client # [ 0.574729] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1527lb # [ 0.587881] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1528worker2 # [ 0.580330] NET: Registered PF_UNIX/PF_LOCAL protocol family1529worker1 # [ 0.593004] ACPI: \_SB_.GSIC: Enabled at IRQ 181530client # [ 0.576044] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1531lb # [ 0.589560] ACPI: \_SB_.GSIA: Enabled at IRQ 161532worker2 # [ 0.581288] NET: Registered PF_XDP protocol family1533client # [ 0.577279] TCP: Hash tables configured (established 16384 bind 16384)1534worker2 # [ 0.582147] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1535worker1 # [ 0.594958] ACPI: \_SB_.GSID: Enabled at IRQ 191536client # [ 0.578408] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1537lb # [ 0.591691] ACPI: \_SB_.GSIB: Enabled at IRQ 171538worker2 # [ 0.583165] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1539worker1 # [ 0.596709] PCI: CLS 0 bytes, default 641540client # [ 0.579648] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1541worker2 # [ 0.584159] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1542client # [ 0.580747] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1543lb # [ 0.593842] ACPI: \_SB_.GSIC: Enabled at IRQ 181544worker2 # [ 0.585261] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1545worker1 # [ 0.597598] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1546client # [ 0.581897] NET: Registered PF_UNIX/PF_LOCAL protocol family1547worker2 # [ 0.586311] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1548worker1 # [ 0.599255] Trying to unpack rootfs image as initramfs...1549client # [ 0.582851] NET: Registered PF_XDP protocol family1550lb # [ 0.595825] ACPI: \_SB_.GSID: Enabled at IRQ 191551worker2 # [ 0.587399] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1552client # [ 0.583629] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1553worker2 # [ 0.589097] ACPI: \_SB_.GSIA: Enabled at IRQ 161554client # [ 0.584663] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1555lb # [ 0.597629] PCI: CLS 0 bytes, default 641556client # [ 0.585646] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1557lb # [ 0.598587] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1558client # [ 0.586776] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1559worker2 # [ 0.591474] ACPI: \_SB_.GSIB: Enabled at IRQ 171560lb # [ 0.600339] Trying to unpack rootfs image as initramfs...1561client # [ 0.587863] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1562client # [ 0.588987] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1563worker2 # [ 0.593776] ACPI: \_SB_.GSIC: Enabled at IRQ 181564client # [ 0.590725] ACPI: \_SB_.GSIA: Enabled at IRQ 161565worker2 # [ 0.595734] ACPI: \_SB_.GSID: Enabled at IRQ 191566client # [ 0.592934] ACPI: \_SB_.GSIB: Enabled at IRQ 171567worker2 # [ 0.597456] PCI: CLS 0 bytes, default 641568client # [ 0.594993] ACPI: \_SB_.GSIC: Enabled at IRQ 181569worker2 # [ 0.598344] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1570worker2 # [ 0.600212] Trying to unpack rootfs image as initramfs...1571client # [ 0.597010] ACPI: \_SB_.GSID: Enabled at IRQ 191572client # [ 0.598897] PCI: CLS 0 bytes, default 641573client # [ 0.599875] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574client # [ 0.601670] Trying to unpack rootfs image as initramfs...1575worker1 # [ 0.643208] Initialise system trusted keyrings1576worker1 # [ 0.646332] workingset: timestamp_bits=40 max_order=19 bucket_order=01577lb # [ 0.644490] Initialise system trusted keyrings1578lb # [ 0.647386] workingset: timestamp_bits=40 max_order=19 bucket_order=01579worker2 # [ 0.643114] Initialise system trusted keyrings1580worker2 # [ 0.644150] workingset: timestamp_bits=40 max_order=19 bucket_order=01581client # [ 0.646189] Initialise system trusted keyrings1582client # [ 0.649773] workingset: timestamp_bits=40 max_order=19 bucket_order=01583worker1 # [ 0.669402] Key type asymmetric registered1584worker1 # [ 0.670120] Asymmetric key parser 'x509' registered1585worker1 # [ 0.671028] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1586worker1 # [ 0.674446] io scheduler mq-deadline registered1587worker1 # [ 0.675230] io scheduler kyber registered1588lb # [ 0.671489] Key type asymmetric registered1589lb # [ 0.672245] Asymmetric key parser 'x509' registered1590lb # [ 0.673156] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1591worker1 # [ 0.678859] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1592worker2 # [ 0.667886] Key type asymmetric registered1593lb # [ 0.676387] io scheduler mq-deadline registered1594worker1 # [ 0.680174] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1595worker2 # [ 0.668605] Asymmetric key parser 'x509' registered1596lb # [ 0.677145] io scheduler kyber registered1597worker2 # [ 0.669475] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1598lb # [ 0.680912] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1599worker1 # [ 0.685433] Linux agpgart interface v0.1031600worker1 # [ 0.686166] ACPI: bus type drm_connector registered1601lb # [ 0.682269] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1602worker2 # [ 0.674837] io scheduler mq-deadline registered1603client # [ 0.671042] Key type asymmetric registered1604worker2 # [ 0.675616] io scheduler kyber registered1605client # [ 0.671778] Asymmetric key parser 'x509' registered1606worker1 # [ 0.689457] usbcore: registered new interface driver usbserial_generic1607client # [ 0.672627] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1608lb # [ 0.685852] Linux agpgart interface v0.1031609worker1 # [ 0.690475] usbserial: USB Serial support registered for generic1610lb # [ 0.686629] ACPI: bus type drm_connector registered1611worker2 # [ 0.678249] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1612worker2 # [ 0.679615] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1613worker1 # [ 0.693301] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1614lb # [ 0.689718] usbcore: registered new interface driver usbserial_generic1615client # [ 0.677851] io scheduler mq-deadline registered1616worker1 # [ 0.693301] Please enable it if your BIOS has the CPPC option.1617lb # [ 0.690817] usbserial: USB Serial support registered for generic1618client # [ 0.678641] io scheduler kyber registered1619worker1 # [ 0.695543] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1620worker2 # [ 0.685518] Linux agpgart interface v0.1031621client # [ 0.681316] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1622worker2 # [ 0.686323] ACPI: bus type drm_connector registered1623lb # [ 0.694248] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1624worker1 # [ 0.699445] drop_monitor: Initializing network drop monitor service1625lb # [ 0.694248] Please enable it if your BIOS has the CPPC option.1626client # [ 0.682595] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1627worker1 # [ 0.700580] NET: Registered PF_INET6 protocol family1628lb # [ 0.696448] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1629worker2 # [ 0.689161] usbcore: registered new interface driver usbserial_generic1630worker2 # [ 0.690238] usbserial: USB Serial support registered for generic1631worker1 # [ 0.703776] Segment Routing with IPv61632lb # [ 0.700362] drop_monitor: Initializing network drop monitor service1633client # [ 0.688121] Linux agpgart interface v0.1031634client # [ 0.688888] ACPI: bus type drm_connector registered1635lb # [ 0.701586] NET: Registered PF_INET6 protocol family1636worker2 # [ 0.692694] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1637worker1 # [ 0.706310] In-situ OAM (IOAM) with IPv61638worker2 # [ 0.692694] Please enable it if your BIOS has the CPPC option.1639worker1 # [ 0.707370] IPI shorthand broadcast: enabled1640worker2 # [ 0.694891] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1641lb # [ 0.704776] Segment Routing with IPv61642client # [ 0.692129] usbcore: registered new interface driver usbserial_generic1643client # [ 0.693229] usbserial: USB Serial support registered for generic1644worker2 # [ 0.698818] drop_monitor: Initializing network drop monitor service1645lb # [ 0.708253] In-situ OAM (IOAM) with IPv61646worker2 # [ 0.700052] NET: Registered PF_INET6 protocol family1647client # [ 0.695728] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1648lb # [ 0.709332] IPI shorthand broadcast: enabled1649client # [ 0.695728] Please enable it if your BIOS has the CPPC option.1650client # [ 0.697994] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1651worker1 # [ 0.715704] sched_clock: Marking stable (583013985, 132262023)->(792227609, -76951601)1652worker2 # [ 0.705163] Segment Routing with IPv61653client # [ 0.701901] drop_monitor: Initializing network drop monitor service1654client # [ 0.703145] NET: Registered PF_INET6 protocol family1655worker2 # [ 0.707703] In-situ OAM (IOAM) with IPv61656worker2 # [ 0.708755] IPI shorthand broadcast: enabled1657worker1 # [ 0.721477] registered taskstats version 11658lb # [ 0.717690] sched_clock: Marking stable (584014837, 133209144)->(795018274, -77794293)1659worker1 # [ 0.722427] Loading compiled-in X.509 certificates1660client # [ 0.707213] Segment Routing with IPv61661lb # [ 0.721308] registered taskstats version 11662client # [ 0.709737] In-situ OAM (IOAM) with IPv61663lb # [ 0.722324] Loading compiled-in X.509 certificates1664client # [ 0.710777] IPI shorthand broadcast: enabled1665worker2 # [ 0.717054] sched_clock: Marking stable (583014653, 133655571)->(795005595, -78335371)1666worker2 # [ 0.721909] registered taskstats version 11667worker2 # [ 0.722898] Loading compiled-in X.509 certificates1668client # [ 0.718998] sched_clock: Marking stable (585013888, 133689642)->(798017780, -79314250)1669client # [ 0.723949] registered taskstats version 11670worker1 # [ 0.741294] Demotion targets for Node 0: null1671client # [ 0.724924] Loading compiled-in X.509 certificates1672worker1 # [ 0.742199] Key type .fscrypt registered1673worker1 # [ 0.742930] Key type fscrypt-provisioning registered1674worker1 # [ 0.743918] ima: No TPM chip found, activating TPM-bypass!1675lb # [ 0.742240] Demotion targets for Node 0: null1676lb # [ 0.743184] Key type .fscrypt registered1677lb # [ 0.743927] Key type fscrypt-provisioning registered1678worker1 # [ 0.748304] ima: Allocated hash algorithm: sha11679worker1 # [ 0.749121] ima: No architecture policies found1680lb # [ 0.747364] ima: No TPM chip found, activating TPM-bypass!1681lb # [ 0.748323] ima: Allocated hash algorithm: sha11682lb # [ 0.749137] ima: No architecture policies found1683worker1 # [ 0.753538] PM: Magic number: 10:209:8861684worker2 # [ 0.741693] Demotion targets for Node 0: null1685worker2 # [ 0.742596] Key type .fscrypt registered1686lb # [ 0.751404] PM: Magic number: 10:209:8861687worker2 # [ 0.743298] Key type fscrypt-provisioning registered1688worker2 # [ 0.744302] ima: No TPM chip found, activating TPM-bypass!1689lb # [ 0.753093] RAS: Correctable Errors collector initialized.1690worker1 # [ 0.755256] RAS: Correctable Errors collector initialized.1691client # [ 0.742720] Demotion targets for Node 0: null1692client # [ 0.743624] Key type .fscrypt registered1693worker2 # [ 0.748695] ima: Allocated hash algorithm: sha11694worker2 # [ 0.749472] ima: No architecture policies found1695client # [ 0.745712] Key type fscrypt-provisioning registered1696worker2 # [ 0.750459] PM: Magic number: 10:209:8861697client # [ 0.746631] ima: No TPM chip found, activating TPM-bypass!1698client # [ 0.747597] ima: Allocated hash algorithm: sha11699worker1 # [ 0.765913] clk: Disabling unused clocks1700lb # [ 0.761919] clk: Disabling unused clocks1701worker1 # [ 0.766633] PM: genpd: Disabling unused power domains1702worker2 # [ 0.754125] RAS: Correctable Errors collector initialized.1703client # [ 0.751743] ima: No architecture policies found1704client # [ 0.752769] PM: Magic number: 10:209:8861705lb # [ 0.765244] PM: genpd: Disabling unused power domains1706client # [ 0.754405] RAS: Correctable Errors collector initialized.1707worker2 # [ 0.762767] clk: Disabling unused clocks1708worker2 # [ 0.763476] PM: genpd: Disabling unused power domains1709client # [ 0.764388] clk: Disabling unused clocks1710client # [ 0.766722] PM: genpd: Disabling unused power domains1711lb # [ 0.935992] Freeing initrd memory: 29092K1712worker1 # [ 0.941761] Freeing initrd memory: 29092K1713lb # [ 0.939253] Freeing unused decrypted memory: 2028K1714worker1 # [ 0.944947] Freeing unused decrypted memory: 2028K1715worker2 # [ 0.933511] Freeing initrd memory: 29092K1716lb # [ 0.941952] Freeing unused kernel image (initmem) memory: 3644K1717lb # [ 0.943010] Write protecting the kernel read-only data: 32768k1718worker1 # [ 0.947677] Freeing unused kernel image (initmem) memory: 3644K1719worker1 # [ 0.948790] Write protecting the kernel read-only data: 32768k1720worker2 # [ 0.936710] Freeing unused decrypted memory: 2028K1721lb # [ 0.944945] Freeing unused kernel image (text/rodata gap) memory: 1208K1722lb # [ 0.946476] Freeing unused kernel image (rodata/data gap) memory: 752K1723worker1 # [ 0.950702] Freeing unused kernel image (text/rodata gap) memory: 1208K1724worker2 # [ 0.939342] Freeing unused kernel image (initmem) memory: 3644K1725worker1 # [ 0.952255] Freeing unused kernel image (rodata/data gap) memory: 752K1726client # [ 0.936068] Freeing initrd memory: 29096K1727worker2 # [ 0.940430] Write protecting the kernel read-only data: 32768k1728worker2 # [ 0.942311] Freeing unused kernel image (text/rodata gap) memory: 1208K1729client # [ 0.939308] Freeing unused decrypted memory: 2028K1730worker2 # [ 0.943899] Freeing unused kernel image (rodata/data gap) memory: 752K1731client # [ 0.942014] Freeing unused kernel image (initmem) memory: 3644K1732client # [ 0.943125] Write protecting the kernel read-only data: 32768k1733client # [ 0.945063] Freeing unused kernel image (text/rodata gap) memory: 1208K1734client # [ 0.946616] Freeing unused kernel image (rodata/data gap) memory: 752K1735lb # [ 0.997760] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736lb # [ 0.998873] Run /init as init process1737worker1 # [ 1.003523] x86/mm: Checked W+X mappings: passed, no W+X pages found.1738worker1 # [ 1.004629] Run /init as init process1739worker2 # [ 0.995185] x86/mm: Checked W+X mappings: passed, no W+X pages found.1740worker2 # [ 0.996285] Run /init as init process1741lb # [ 1.009606] systemd[1]: Inserted module 'autofs4'1742client # [ 0.997885] x86/mm: Checked W+X mappings: passed, no W+X pages found.1743worker1 # [ 1.014921] systemd[1]: Inserted module 'autofs4'1744client # [ 0.998960] Run /init as init process1745worker2 # [ 1.007095] systemd[1]: Inserted module 'autofs4'1746client # [ 1.009485] systemd[1]: Inserted module 'autofs4'1747lb # [ 1.026136] fuse: init (API version 7.45)1748worker1 # [ 1.031150] fuse: init (API version 7.45)1749worker2 # [ 1.023446] fuse: init (API version 7.45)1750lb # [ 1.031984] ACPI: \_SB_.GSIG: Enabled at IRQ 221751worker1 # [ 1.037028] ACPI: \_SB_.GSIG: Enabled at IRQ 221752lb # [ 1.034366] ACPI: \_SB_.GSIH: Enabled at IRQ 231753worker1 # [ 1.039553] ACPI: \_SB_.GSIH: Enabled at IRQ 231754lb # [ 1.037449] ACPI: \_SB_.GSIE: Enabled at IRQ 201755worker2 # [ 1.029281] ACPI: \_SB_.GSIG: Enabled at IRQ 221756worker1 # [ 1.042459] ACPI: \_SB_.GSIE: Enabled at IRQ 201757client # [ 1.026123] fuse: init (API version 7.45)1758lb # [ 1.039637] ACPI: \_SB_.GSIF: Enabled at IRQ 211759worker2 # [ 1.031745] ACPI: \_SB_.GSIH: Enabled at IRQ 231760worker1 # [ 1.045030] ACPI: \_SB_.GSIF: Enabled at IRQ 211761worker2 # [ 1.034604] ACPI: \_SB_.GSIE: Enabled at IRQ 201762client # [ 1.032117] ACPI: \_SB_.GSIG: Enabled at IRQ 221763worker2 # [ 1.036921] ACPI: \_SB_.GSIF: Enabled at IRQ 211764client # [ 1.034590] ACPI: \_SB_.GSIH: Enabled at IRQ 231765client # [ 1.038121] ACPI: \_SB_.GSIE: Enabled at IRQ 201766client # [ 1.040423] ACPI: \_SB_.GSIF: Enabled at IRQ 211767lb # [ 1.066379] systemd[1]: Successfully made /usr/ read-only.1768worker1 # [ 1.073155] systemd[1]: Successfully made /usr/ read-only.1769worker2 # [ 1.062364] systemd[1]: Successfully made /usr/ read-only.1770client # [ 1.065484] systemd[1]: Successfully made /usr/ read-only.1771lb # [ 1.403042] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772lb # [ 1.415047] systemd[1]: Detected virtualization kvm.1773worker1 # [ 1.409895] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1774worker2 # [ 1.398768] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1775lb # [ 1.417185] systemd[1]: Detected architecture x86-64.1776worker1 # [ 1.421847] systemd[1]: Detected virtualization kvm.1777lb # [ 1.419305] systemd[1]: Running in initrd.1778worker2 # [ 1.410608] systemd[1]: Detected virtualization kvm.1779worker1 # [ 1.423972] systemd[1]: Detected architecture x86-64.1780worker2 # [ 1.412747] systemd[1]: Detected architecture x86-64.1781worker1 # [ 1.426111] systemd[1]: Running in initrd.1782lb # [ 1.421914] systemd[1]: Initializing machine ID from random generator.1783worker2 # [ 1.414861] systemd[1]: Running in initrd.1784lb # [ 1.424872] systemd[1]: Hostname set to <lb>.1785worker1 # [ 1.428781] systemd[1]: Initializing machine ID from random generator.1786client # [ 1.402066] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1787worker2 # [ 1.417346] systemd[1]: Initializing machine ID from random generator.1788worker1 # [ 1.431647] systemd[1]: Hostname set to <worker1>.1789client # [ 1.414893] systemd[1]: Detected virtualization kvm.1790worker2 # [ 1.420179] systemd[1]: Hostname set to <worker2>.1791client # [ 1.417004] systemd[1]: Detected architecture x86-64.1792client # [ 1.419127] systemd[1]: Running in initrd.1793client # [ 1.421587] systemd[1]: Initializing machine ID from random generator.1794client # [ 1.424561] systemd[1]: Hostname set to <client>.1795worker1 # [ 1.638513] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796worker2 # [ 1.631910] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797lb # [ 1.645362] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798client # [ 1.644654] systemd[1]: bpf-restrict-fs: LSM BPF program attached1799worker1 # [ 1.677098] systemd[1]: Queued start job for default target Initrd Default Target.1800worker1 # [ 1.681620] systemd[1]: Created slice Slice /system/modprobe.1801worker1 # [ 1.682820] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1802worker2 # [ 1.670392] systemd[1]: Queued start job for default target Initrd Default Target.1803worker1 # [ 1.684205] systemd[1]: Expecting device /dev/disk/by-label/nixos...1804worker1 # [ 1.685377] systemd[1]: Reached target Path Units.1805worker1 # [ 1.686238] systemd[1]: Reached target Slice Units.1806worker1 # [ 1.687123] systemd[1]: Reached target Swaps.1807worker2 # [ 1.674994] systemd[1]: Created slice Slice /system/modprobe.1808worker1 # [ 1.687941] systemd[1]: Reached target Timer Units.1809worker1 # [ 1.688907] systemd[1]: Listening on D-Bus System Message Bus Socket.1810worker2 # [ 1.676168] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1811worker1 # [ 1.690108] systemd[1]: Listening on Journal Socket (/dev/log).1812worker2 # [ 1.677562] systemd[1]: Expecting device /dev/disk/by-label/nixos...1813worker2 # [ 1.678635] systemd[1]: Reached target Path Units.1814worker1 # [ 1.691285] systemd[1]: Listening on Journal Sockets.1815worker2 # [ 1.679501] systemd[1]: Reached target Slice Units.1816worker1 # [ 1.692251] systemd[1]: Listening on udev Control Socket.1817worker2 # [ 1.680437] systemd[1]: Reached target Swaps.1818worker1 # [ 1.693332] systemd[1]: Listening on udev Kernel Socket.1819worker2 # [ 1.681236] systemd[1]: Reached target Timer Units.1820worker1 # [ 1.694289] systemd[1]: Reached target Socket Units.1821worker2 # [ 1.682212] systemd[1]: Listening on D-Bus System Message Bus Socket.1822worker2 # [ 1.683428] systemd[1]: Listening on Journal Socket (/dev/log).1823worker1 # [ 1.696093] systemd[1]: Starting Create List of Static Device Nodes...1824worker2 # [ 1.684568] systemd[1]: Listening on Journal Sockets.1825worker2 # [ 1.685581] systemd[1]: Listening on udev Control Socket.1826worker2 # [ 1.686646] systemd[1]: Listening on udev Kernel Socket.1827worker2 # [ 1.687630] systemd[1]: Reached target Socket Units.1828worker1 # [ 1.700347] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1829worker2 # [ 1.689396] systemd[1]: Starting Create List of Static Device Nodes...1830lb # [ 1.700609] systemd[1]: Queued start job for default target Initrd Default Target.1831worker2 # [ 1.694748] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1832worker1 # [ 1.708344] systemd[1]: Starting Load Kernel Module configfs...1833lb # [ 1.704617] systemd[1]: Created slice Slice /system/modprobe.1834lb # [ 1.705804] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1835lb # [ 1.707264] systemd[1]: Expecting device /dev/disk/by-label/nixos...1836lb # [ 1.708399] systemd[1]: Reached target Path Units.1837worker2 # [ 1.700373] systemd[1]: Starting Load Kernel Module configfs...1838lb # [ 1.709286] systemd[1]: Reached target Slice Units.1839lb # [ 1.710152] systemd[1]: Reached target Swaps.1840lb # [ 1.711020] systemd[1]: Reached target Timer Units.1841client # [ 1.698193] systemd[1]: Queued start job for default target Initrd Default Target.1842lb # [ 1.712041] systemd[1]: Listening on D-Bus System Message Bus Socket.1843lb # [ 1.713311] systemd[1]: Listening on Journal Socket (/dev/log).1844lb # [ 1.714519] systemd[1]: Listening on Journal Sockets.1845client # [ 1.702074] systemd[1]: Created slice Slice /system/modprobe.1846lb # [ 1.715570] systemd[1]: Listening on udev Control Socket.1847client # [ 1.703235] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1848lb # [ 1.716599] systemd[1]: Listening on udev Kernel Socket.1849worker1 # [ 1.721573] systemd[1]: Starting Journal Service...1850lb # [ 1.717558] systemd[1]: Reached target Socket Units.1851client # [ 1.704641] systemd[1]: Expecting device /dev/disk/by-label/nixos...1852client # [ 1.705716] systemd[1]: Reached target Path Units.1853client # [ 1.706584] systemd[1]: Reached target Slice Units.1854lb # [ 1.719291] systemd[1]: Starting Create List of Static Device Nodes...1855client # [ 1.707489] systemd[1]: Reached target Swaps.1856client # [ 1.708309] systemd[1]: Reached target Timer Units.1857client # [ 1.709304] systemd[1]: Listening on D-Bus System Message Bus Socket.1858lb # [ 1.723066] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1859client # [ 1.710548] systemd[1]: Listening on Journal Socket (/dev/log).1860client # [ 1.711706] systemd[1]: Listening on Journal Sockets.1861worker2 # [ 1.716301] systemd[1]: Starting Journal Service...1862client # [ 1.712715] systemd[1]: Listening on udev Control Socket.1863client # [ 1.713725] systemd[1]: Listening on udev Kernel Socket.1864client # [ 1.714661] systemd[1]: Reached target Socket Units.1865client # [ 1.716442] systemd[1]: Starting Create List of Static Device Nodes...1866lb # [ 1.730302] systemd[1]: Starting Load Kernel Module configfs...1867worker1 # [ 1.735356] systemd[1]: Starting Load Kernel Modules...1868client # [ 1.720780] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1869worker2 # [ 1.728746] systemd[1]: Starting Load Kernel Modules...1870client # [ 1.727821] systemd[1]: Starting Load Kernel Module configfs...1871lb # [ 1.742468] systemd[1]: Starting Journal Service...1872worker1 # [ 1.739352] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1873worker1 # [ 1.757236] netfs: FS-Cache loaded1874lb # [ 1.753006] systemd[1]: Starting Load Kernel Modules...1875client # [ 1.742894] systemd[1]: Starting Journal Service...1876worker2 # [ 1.735747] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1877worker1 # [ 1.761963] 9pnet: Installing 9P2000 support1878worker2 # [ 1.750939] netfs: FS-Cache loaded1879worker2 # [ 1.755642] 9pnet: Installing 9P2000 support1880worker1 # [ 1.771051] systemd[1]: Starting Coldplug All udev Devices...1881client # [ 1.755781] systemd[1]: Starting Load Kernel Modules...1882lb # [ 1.760312] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1883worker2 # [ 1.763924] systemd[1]: Starting Coldplug All udev Devices...1884lb # [ 1.781039] netfs: FS-Cache loaded1885worker1 # [ 1.784876] systemd[1]: Finished Create List of Static Device Nodes.1886client # [ 1.760797] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1887lb # [ 1.785803] 9pnet: Installing 9P2000 support1888worker2 # [ 1.777196] systemd[1]: Finished Create List of Static Device Nodes.1889worker1 # [ 1.790856] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1890client # [ 1.778137] netfs: FS-Cache loaded1891worker2 # [ 1.782266] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1892client # [ 1.782836] 9pnet: Installing 9P2000 support1893lb # [ 1.795274] systemd[1]: Starting Coldplug All udev Devices...1894worker1 # [ 1.799704] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1895worker2 # [ 1.790741] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1896worker1 # [ 1.805695] systemd[1]: modprobe@configfs.service: Deactivated successfully.1897worker1 # [ 1.807050] systemd-journald[67]: Collecting audit messages is disabled.1898client # [ 1.791484] systemd[1]: Starting Coldplug All udev Devices...1899worker2 # [ 1.798158] systemd[1]: modprobe@configfs.service: Deactivated successfully.1900worker1 # [ 1.812651] systemd[1]: Finished Load Kernel Module configfs.1901worker2 # [ 1.801096] systemd-journald[67]: Collecting audit messages is disabled.1902lb # [ 1.810650] systemd[1]: Finished Create List of Static Device Nodes.1903worker2 # [ 1.803736] systemd[1]: Finished Load Kernel Module configfs.1904worker1 # [ 1.816729] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1905worker1 # [ 1.819299] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1906client # [ 1.805249] systemd[1]: Finished Create List of Static Device Nodes.1907worker2 # [ 1.810122] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1908lb # [ 1.819433] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1909worker2 # [ 1.813477] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1910client # [ 1.810304] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1911worker1 # [ 1.828430] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1912lb # [ 1.824985] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1913worker1 # [ 1.830057] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1914lb # [ 1.829663] systemd[1]: modprobe@configfs.service: Deactivated successfully.1915worker2 # [ 1.821749] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1916client # [ 1.819111] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1917worker2 # [ 1.823289] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1918lb # [ 1.835123] systemd-journald[67]: Collecting audit messages is disabled.1919lb # [ 1.837598] systemd[1]: Finished Load Kernel Module configfs.1920client # [ 1.825151] systemd[1]: modprobe@configfs.service: Deactivated successfully.1921client # [ 1.826935] systemd-journald[67]: Collecting audit messages is disabled.1922lb # [ 1.843678] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1923client # [ 1.832065] systemd[1]: Finished Load Kernel Module configfs.1924lb # [ 1.848273] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1925client # [ 1.837169] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1926worker1 # [ 1.858180] systemd[1]: Finished Load Kernel Modules.1927client # [ 1.841100] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1928lb # [ 1.855303] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1929lb # [ 1.857381] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1930worker2 # [ 1.852242] systemd[1]: Finished Load Kernel Modules.1931client # [ 1.847507] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1932worker1 # [ 1.865486] systemd[1]: Starting Apply Kernel Variables...1933client # [ 1.849785] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1934worker2 # [ 1.860797] systemd[1]: Starting Apply Kernel Variables...1935worker1 # [ 1.877625] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1936worker2 # [ 1.872017] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1937worker1 # [ 1.886466] systemd[1]: Starting Create Static Device Nodes in /dev...1938lb # [ 1.888872] systemd[1]: Finished Load Kernel Modules.1939worker2 # [ 1.881915] systemd[1]: Starting Create Static Device Nodes in /dev...1940client # [ 1.878202] systemd[1]: Finished Load Kernel Modules.1941lb # [ 1.895436] systemd[1]: Starting Apply Kernel Variables...1942worker1 # [ 1.901649] systemd[1]: Finished Apply Kernel Variables.1943client # [ 1.884942] systemd[1]: Starting Apply Kernel Variables...1944worker2 # [ 1.894775] systemd[1]: Finished Apply Kernel Variables.1945lb # [ 1.905334] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1946client # [ 1.898808] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1947lb # [ 1.914668] systemd[1]: Starting Create Static Device Nodes in /dev...1948worker1 # [ 1.919939] systemd[1]: Started Journal Service.1949worker1 # [ 1.788698] systemd-modules-load[68]: Inserted module 'dm_mod'1950worker1 # [ 1.791572] systemd-modules-load[68]: Inserted module 'virtio_balloon'1951worker1 # [ 1.792638] systemd-modules-load[68]: Inserted module 'virtio_gpu'1952worker2 # [ 1.912541] systemd[1]: Started Journal Service.1953client # [ 1.908919] systemd[1]: Starting Create Static Device Nodes in /dev...1954worker1 # [ 1.795293] systemd[1]: Finished Create Static Device Nodes in /dev.1955worker1 # [ 1.797367] systemd[1]: Reached target Preparation for Local File Systems.1956worker2 # [ 1.782693] systemd-modules-load[68]: Inserted module 'dm_mod'1957worker1 # [ 1.798470] systemd[1]: Reached target Local File Systems.1958worker2 # [ 1.786111] systemd-modules-load[68]: Inserted module 'virtio_balloon'1959worker2 # [ 1.787157] systemd-modules-load[68]: Inserted module 'virtio_gpu'1960worker1 # [ 1.802081] systemd[1]: Starting Create System Files and Directories...1961worker2 # [ 1.789248] systemd[1]: Finished Create Static Device Nodes in /dev.1962worker2 # [ 1.791332] systemd[1]: Reached target Preparation for Local File Systems.1963worker2 # [ 1.792477] systemd[1]: Reached target Local File Systems.1964client # [ 1.921832] systemd[1]: Finished Apply Kernel Variables.1965lb # [ 1.935340] systemd[1]: Finished Apply Kernel Variables.1966worker1 # [ 1.808781] systemd[1]: Starting Rule-based Manager for Device Events and Files...1967worker2 # [ 1.795255] systemd[1]: Starting Create System Files and Directories...1968worker2 # [ 1.798947] systemd[1]: Starting Rule-based Manager for Device Events and Files...1969lb # [ 1.948283] systemd[1]: Started Journal Service.1970lb # [ 1.813847] systemd-modules-load[68]: Inserted module 'dm_mod'1971lb # [ 1.818059] systemd-modules-load[68]: Inserted module 'virtio_balloon'1972client # [ 1.940145] systemd[1]: Started Journal Service.1973lb # [ 1.819559] systemd-modules-load[68]: Inserted module 'virtio_gpu'1974lb # [ 1.820690] systemd[1]: Finished Create Static Device Nodes in /dev.1975client # [ 1.805913] systemd-modules-load[68]: Inserted module 'dm_mod'1976client # [ 1.810259] systemd-modules-load[68]: Inserted module 'virtio_balloon'1977client # [ 1.811316] systemd-modules-load[68]: Inserted module 'virtio_gpu'1978client # [ 1.814374] systemd[1]: Finished Create Static Device Nodes in /dev.1979client # [ 1.815569] systemd[1]: Reached target Preparation for Local File Systems.1980worker1 # [ 1.834819] systemd[1]: Finished Create System Files and Directories.1981client # [ 1.817331] systemd[1]: Reached target Local File Systems.1982lb # [ 1.830224] systemd[1]: Reached target Preparation for Local File Systems.1983lb # [ 1.832089] systemd[1]: Reached target Local File Systems.1984lb # [ 1.835067] systemd[1]: Starting Create System Files and Directories...1985client # [ 1.822083] systemd[1]: Starting Create System Files and Directories...1986client # [ 1.824903] systemd[1]: Starting Rule-based Manager for Device Events and Files...1987worker2 # [ 1.830062] systemd[1]: Finished Create System Files and Directories.1988lb # [ 1.840075] systemd[1]: Starting Rule-based Manager for Device Events and Files...1989worker1 # [ 1.861368] systemd-udevd[76]: Using default interface naming scheme 'v261'.1990worker2 # [ 1.852894] systemd-udevd[76]: Using default interface naming scheme 'v261'.1991lb # [ 1.867477] systemd[1]: Finished Create System Files and Directories.1992client # [ 1.855657] systemd[1]: Finished Create System Files and Directories.1993worker1 # [ 1.885347] systemd[1]: Started Rule-based Manager for Device Events and Files.1994worker2 # [ 1.877849] systemd[1]: Started Rule-based Manager for Device Events and Files.1995lb # [ 1.888619] systemd-udevd[83]: Using default interface naming scheme 'v261'.1996client # [ 1.879364] systemd-udevd[77]: Using default interface naming scheme 'v261'.1997lb # [ 1.912789] systemd[1]: Started Rule-based Manager for Device Events and Files.1998client # [ 1.903083] systemd[1]: Started Rule-based Manager for Device Events and Files.1999worker1 # [ 1.929452] systemd[1]: Finished Coldplug All udev Devices.2000worker1 # [ 1.930638] systemd[1]: Reached target System Initialization.
loading 2,202 hidden lines…
4203worker2 # [ 22.207592] nix-grpc-daemon[786]: ts=2026-09-21T12:53:48Z level=info event=scheduler_connected addr=lb:500514204worker2 # [ 22.211321] nix-grpc-daemon[786]: ts=2026-09-21T12:53:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker1 # [ 22.379487] nix-grpc-daemon[785]: ts=2026-09-21T12:53:48Z level=info event=scheduler_connected addr=lb:500514206worker1 # [ 22.383172] nix-grpc-daemon[785]: ts=2026-09-21T12:53:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4207worker2 # [ 22.411802] nix-grpc-daemon[786]: ts=2026-09-21T12:53:48Z level=info event=scheduler_connected addr=lb:500514208worker2 # [ 22.413670] nix-grpc-daemon[786]: ts=2026-09-21T12:53:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker1 # [ 22.583623] nix-grpc-daemon[785]: ts=2026-09-21T12:53:48Z level=info event=scheduler_connected addr=lb:500514210worker1 # [ 22.585358] nix-grpc-daemon[785]: ts=2026-09-21T12:53:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4211worker2 # [ 22.614415] nix-grpc-daemon[786]: ts=2026-09-21T12:53:48Z level=info event=scheduler_connected addr=lb:500514212worker2 # [ 22.616188] nix-grpc-daemon[786]: ts=2026-09-21T12:53:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker1 # [ 22.787144] nix-grpc-daemon[785]: ts=2026-09-21T12:53:48Z level=info event=scheduler_connected addr=lb:500514214worker1 # [ 22.790741] nix-grpc-daemon[785]: ts=2026-09-21T12:53:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4215worker2 # [ 22.816892] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514216worker2 # [ 22.818586] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker1 # [ 22.988281] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 22.989987] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4219worker2 # [ 23.019690] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514220worker2 # [ 23.021310] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker1 # [ 23.190842] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514222worker1 # [ 23.194671] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4223worker2 # [ 23.222720] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514224worker2 # [ 23.226798] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker1 # [ 23.395450] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514226worker1 # [ 23.399257] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4227worker2 # [ 23.427254] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514228worker2 # [ 23.428992] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker1 # [ 23.599897] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514230worker1 # [ 23.602969] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4231worker2 # [ 23.630437] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514232worker2 # [ 23.634235] nix-grpc-daemon[786]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker1 # [ 23.804667] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_connected addr=lb:500514234worker1 # [ 23.808395] nix-grpc-daemon[785]: ts=2026-09-21T12:53:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4235worker2 # [ 23.834900] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514236worker2 # [ 23.838238] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker1 # [ 24.009287] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514238worker1 # [ 24.013168] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4239worker2 # [ 24.040201] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514240worker2 # [ 24.043853] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker1 # [ 24.213968] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514242worker1 # [ 24.217832] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4243worker2 # [ 24.244818] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514244worker2 # [ 24.248549] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker1 # [ 24.418616] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514246worker1 # [ 24.422467] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4247worker2 # [ 24.449170] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514248worker2 # [ 24.450973] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker1 # [ 24.623249] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514250worker1 # [ 24.627109] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4251worker2 # [ 24.651690] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514252worker2 # [ 24.653445] nix-grpc-daemon[786]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker1 # [ 24.829436] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_connected addr=lb:500514254worker1 # [ 24.832492] nix-grpc-daemon[785]: ts=2026-09-21T12:53:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4255worker2 # [ 24.854552] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514256worker2 # [ 24.858845] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker1 # [ 25.030381] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514258worker1 # [ 25.034233] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker2 # [ 25.061448] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514260worker2 # [ 25.064576] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker1 # [ 25.234527] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514262worker1 # [ 25.236245] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4263worker2 # [ 25.262149] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514264worker2 # [ 25.263885] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker1 # [ 25.437157] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514266worker1 # [ 25.441180] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4267worker2 # [ 25.465248] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514268worker2 # [ 25.469079] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker1 # [ 25.641568] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514270worker1 # [ 25.643254] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4271worker2 # [ 25.670185] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514272worker2 # [ 25.674120] nix-grpc-daemon[786]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker1 # [ 25.844887] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_connected addr=lb:500514274worker1 # [ 25.848740] nix-grpc-daemon[785]: ts=2026-09-21T12:53:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4275worker2 # [ 25.874977] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514276worker2 # [ 25.878829] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker1 # [ 26.049120] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514278worker1 # [ 26.050917] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4279worker2 # [ 26.081433] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514280worker2 # [ 26.084511] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker1 # [ 26.251938] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514282worker1 # [ 26.255673] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4283worker2 # [ 26.281995] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514284worker2 # [ 26.285854] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker1 # [ 26.456558] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514286worker1 # [ 26.460267] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4287worker2 # [ 26.486805] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514288worker2 # [ 26.490584] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker1 # [ 26.660939] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514290worker1 # [ 26.664869] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4291worker2 # [ 26.691788] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514292worker2 # [ 26.695613] nix-grpc-daemon[786]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker1 # [ 26.866558] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_connected addr=lb:500514294worker1 # [ 26.870203] nix-grpc-daemon[785]: ts=2026-09-21T12:53:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4295worker2 # [ 26.896568] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514296worker2 # [ 26.900496] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker1 # [ 27.070907] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514298worker1 # [ 27.074819] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4299worker2 # [ 27.102735] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514300worker2 # [ 27.105854] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker1 # [ 27.275605] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514302worker1 # [ 27.279469] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4303worker2 # [ 27.304516] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514304worker2 # [ 27.307973] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker1 # [ 27.480453] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 27.484269] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 27.508572] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 27.510339] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 27.684857] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 27.688602] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker2 # [ 27.711764] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514312worker2 # [ 27.715817] nix-grpc-daemon[786]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313worker1 # [ 27.890408] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_connected addr=lb:500514314worker1 # [ 27.893453] nix-grpc-daemon[785]: ts=2026-09-21T12:53:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4315worker2 # [ 27.917061] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514316worker2 # [ 27.920846] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4317worker1 # [ 28.091304] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514318worker1 # [ 28.095239] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4319worker2 # [ 28.123314] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514320worker2 # [ 28.126491] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4321worker1 # [ 28.296061] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514322worker1 # [ 28.299834] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4323worker2 # [ 28.324642] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514324worker2 # [ 28.328631] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4325worker1 # [ 28.500676] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514326worker1 # [ 28.504494] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker2 # [ 28.529790] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514328worker2 # [ 28.533613] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329worker1 # [ 28.705318] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514330worker1 # [ 28.709141] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4331worker2 # [ 28.734719] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514332worker2 # [ 28.738510] nix-grpc-daemon[786]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4333worker1 # [ 28.910848] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_connected addr=lb:500514334worker1 # [ 28.913987] nix-grpc-daemon[785]: ts=2026-09-21T12:53:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4335worker2 # [ 28.939314] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_connected addr=lb:500514336worker2 # [ 28.942945] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4337worker1 # [ 29.111578] nix-grpc-daemon[785]: ts=2026-09-21T12:53:55Z level=info event=scheduler_connected addr=lb:500514338worker1 # [ 29.115348] nix-grpc-daemon[785]: ts=2026-09-21T12:53:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4339worker2 # [ 29.144934] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_connected addr=lb:500514340worker2 # [ 29.148304] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4341worker1 # [ 29.316724] nix-grpc-daemon[785]: ts=2026-09-21T12:53:55Z level=info event=scheduler_connected addr=lb:500514342worker1 # [ 29.321187] nix-grpc-daemon[785]: ts=2026-09-21T12:53:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4343worker2 # [ 29.349433] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_connected addr=lb:500514344worker2 # [ 29.353134] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4345worker1 # [ 29.521095] nix-grpc-daemon[785]: ts=2026-09-21T12:53:55Z level=info event=scheduler_connected addr=lb:500514346worker1 # [ 29.524869] nix-grpc-daemon[785]: ts=2026-09-21T12:53:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4347worker2 # [ 29.554312] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_connected addr=lb:500514348worker2 # [ 29.558129] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4349worker1 # [ 29.725729] nix-grpc-daemon[785]: ts=2026-09-21T12:53:55Z level=info event=scheduler_connected addr=lb:500514350worker1 # [ 29.729677] nix-grpc-daemon[785]: ts=2026-09-21T12:53:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4351worker2 # [ 29.759387] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_connected addr=lb:500514352worker2 # [ 29.763094] nix-grpc-daemon[786]: ts=2026-09-21T12:53:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4353worker1 # [ 29.931514] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514354worker1 # [ 29.934516] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4355worker2 # [ 29.963937] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514356worker2 # [ 29.967713] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4357worker1 # [ 30.132470] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514358worker1 # [ 30.136312] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4359worker2 # [ 30.170234] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514360worker2 # [ 30.174318] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4361lb # [ 30.307484] rustfs-bucket-start[1037]: mb s3://farm4362lb # [ 30.313482] systemd[1]: Finished rustfs-bucket.service.4363lb # [ 30.321471] systemd[1]: Starting niks3 server...4364worker1 # [ 30.337088] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514365worker1 # [ 30.340919] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4366worker2 # [ 30.371383] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514367worker2 # [ 30.375439] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4368worker1 # [ 30.542101] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514369worker1 # [ 30.546620] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4370lb # [ 30.564969] postgres[1056]: [1056] ERROR: relation "goose_db_version" does not exist at character 364371lb # [ 30.566445] postgres[1056]: [1056] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4372worker2 # [ 30.576528] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514373worker2 # [ 30.580355] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4374lb # [ 30.612222] niks3-server[1047]: 2026/09/21 12:53:56 OK 20241026095416_initial_model.sql (21.26ms)4375lb # [ 30.615995] niks3-server[1047]: 2026/09/21 12:53:56 OK 20251210153512_drop_unused_gin_index.sql (3.85ms)4376lb # [ 30.621153] niks3-server[1047]: 2026/09/21 12:53:56 OK 20251218171726_add_pins.sql (3.38ms)4377lb # [ 30.626067] niks3-server[1047]: 2026/09/21 12:53:56 OK 20260628120000_add_object_size_and_stats.sql (5.21ms)4378lb # [ 30.630696] niks3-server[1047]: 2026/09/21 12:53:56 OK 20260905000000_add_claims.sql (4.59ms)4379lb # [ 30.635421] niks3-server[1047]: 2026/09/21 12:53:56 OK 20260920000000_drop_claims.sql (4.59ms)4380lb # [ 30.636643] niks3-server[1047]: 2026/09/21 12:53:56 goose: successfully migrated database to version: 202609200000004381lb # [ 30.642397] niks3-server[1047]: 2026/09/21 12:53:56 OK 1_commit_pending_closure.sql (6.93ms)4382lb # [ 30.644966] niks3-server[1047]: 2026/09/21 12:53:56 OK 2_object_stats_trigger.sql (2.58ms)4383lb # [ 30.646115] niks3-server[1047]: 2026/09/21 12:53:56 goose: up to current file version: 24384lb # [ 30.652480] niks3-server[1047]: 2026/09/21 12:53:56 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4385lb # [ 30.728322] niks3-server[1047]: 2026/09/21 12:53:56 INFO Read proxy enabled — serving cache objects from S34386lb # [ 30.730296] niks3-server[1047]: 2026/09/21 12:53:56 INFO Using socket-activated listener address=0.0.0.0:57514387lb # [ 30.735262] systemd[1]: Started niks3 server.4388lb # [ 30.735925] niks3-server[1047]: 2026/09/21 12:53:56 INFO systemd watchdog enabled interval=15s4389lb # [ 30.737061] systemd[1]: Reached target Multi-User System.4390lb # [ 30.739944] systemd[1]: Startup finished in 869ms (kernel) + 5.054s (initrd) + 24.815s (userspace) = 30.739s.4391lb # [ 30.742972] niks3-server[1047]: 2026/09/21 12:53:56 INFO Starting HTTP server address=0.0.0.0:57514392worker1 # [ 30.747403] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514393worker1 # [ 30.751779] nix-grpc-daemon[785]: ts=2026-09-21T12:53:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4394lb # [ 30.751337] niks3-server[1047]: 2026/09/21 12:53:56 INFO lead: acquired remote=192.168.1.4:417304395worker2 # [ 30.748382] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_take_over4396worker2 # [ 30.781400] nix-grpc-daemon[786]: ts=2026-09-21T12:53:56Z level=info event=scheduler_connected addr=lb:500514397worker2 # [ 30.782970] nix-grpc-daemon[786]: ts=2026-09-21T12:53:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4398worker1 # [ 30.953394] nix-grpc-daemon[785]: ts=2026-09-21T12:53:57Z level=info event=scheduler_connected addr=lb:500514399worker2 # [ 30.951791] nix-grpc-daemon[786]: ts=2026-09-21T12:53:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:349604400worker2 # [ 30.956233] nix-grpc-daemon[786]: ts=2026-09-21T12:53:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04401worker2 # [ 30.983903] nix-grpc-daemon[786]: ts=2026-09-21T12:53:57Z level=info event=scheduler_connected addr=lb:500514402worker2 # [ 31.005523] nix-grpc-daemon[786]: ts=2026-09-21T12:53:57Z level=info event=oidc_jwks provider=mock keys=14403worker2 # [ 31.009797] nix-grpc-daemon[786]: ts=2026-09-21T12:53:57Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:349604404worker2 # [ 31.013977] nix-grpc-daemon[786]: ts=2026-09-21T12:53:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04405lb: (finished: waiting for unit niks3.service, in 19.59 seconds)4406lb: waiting for TCP port 5751 on localhost4407lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4408lb: (finished: waiting for TCP port 5751 on localhost, in 0.05 seconds)4409lb: waiting for unit envoy.service4410lb: (finished: waiting for unit envoy.service, in 0.05 seconds)4411lb: waiting for TCP port 50051 on localhost4412lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4413lb # [ 31.747105] envoy[909]: conn peer=[::1]:34262 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=24414lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4415worker2 # [ 31.878957] systemd[1]: Starting scheduler-token.service...4416worker2 # [ 31.952259] systemd[1]: scheduler-token.service: Deactivated successfully.4417worker2 # [ 31.953258] systemd[1]: Finished scheduler-token.service.4418worker2 # [ 31.954387] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 74ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4419lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4420lb: must succeed: curl -sf localhost:9901/clusters4421lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4422lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.10 seconds)4423subtest: one node schedules and both builders hold a WorkerSession on it4424lb: must succeed: curl -sf localhost:9901/clusters4425lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4426worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4427worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4428lb: must succeed: curl -sf localhost:9901/clusters4429lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4430lb: must succeed: curl -sf localhost:9901/clusters4431lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4432lb: must succeed: curl -sf localhost:9901/clusters4433lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4434lb: must succeed: curl -sf localhost:9901/clusters4435lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4436worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4437worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds)4438worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4439worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4440lb: must succeed: curl -sf localhost:9901/clusters4441lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4442lb: must succeed: curl -sf localhost:9901/clusters4443lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4444worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4445worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4446worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4447worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4448worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4449worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4450worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4451worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4452(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.69 seconds)4453subtest: balancer auth: client cert, bearer token, nothing, unknown CN4454worker1 # [ 33.096521] nix-grpc-daemon[785]: ts=2026-09-21T12:53:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494444455worker1 # [ 33.102974] systemd[1]: Started Nix Daemon.4456worker1 # [ 33.300603] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4457worker1 # [ 33.323846] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 1/5)4458worker1 # [ 33.353273] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 2/5)4459worker1 # [ 33.383585] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 250 ms (attempt 3/5)4460worker1 # [ 33.636693] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 208 ms (attempt 4/5)4461worker1 # [ 33.852397] nix-grpc-daemon[785]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4462worker1 # [ 33.854489] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4463worker2 # [ 33.880770] nix-grpc-daemon[786]: ts=2026-09-21T12:54:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:360284464worker2 # [ 33.894103] systemd[1]: Started Nix Daemon.4465worker2 # [ 34.103745] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4466worker2 # [ 34.127482] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)4467worker2 # [ 34.166085] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 2/5)4468worker2 # [ 34.267942] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 3/5)4469worker2 # [ 34.468956] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 4/5)4470worker2 # [ 34.539596] nix-grpc-daemon[786]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4471worker2 # [ 34.541717] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4472worker2 # [ 34.552361] nix-grpc-daemon[786]: ts=2026-09-21T12:54:00Z level=debug event=path_info_miss path=iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test error="error: path '/nix/store/iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test' is required, but there is no substituter that can build it"4473worker1 # [ 34.582843] nix-grpc-daemon[785]: ts=2026-09-21T12:54:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494444474worker1 # [ 34.593877] nix-grpc-daemon[785]: ts=2026-09-21T12:54:00Z level=debug event=path_info_miss path=iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test error="error: path '/nix/store/iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test' is required, but there is no substituter that can build it"4475client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4476client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)4477client: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4478client: (finished: must succeed: install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4479worker2 # [ 34.951800] nix-grpc-daemon[786]: ts=2026-09-21T12:54:01Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:360284480worker2 # [ 34.967838] nix-grpc-daemon[786]: ts=2026-09-21T12:54:01Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:360284481worker2 # [ 34.976522] nix-grpc-daemon[786]: ts=2026-09-21T12:54:01Z level=debug event=path_info_miss path=iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test error="error: path '/nix/store/iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test' is required, but there is no substituter that can build it"4482worker1 # [ 35.003614] nix-grpc-daemon[785]: ts=2026-09-21T12:54:01Z level=info event=oidc_jwks provider=mock keys=14483worker1 # [ 35.005620] nix-grpc-daemon[785]: ts=2026-09-21T12:54:01Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:494444484worker1 # [ 35.010689] nix-grpc-daemon[785]: ts=2026-09-21T12:54:01Z level=debug event=path_info_miss path=iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test error="error: path '/nix/store/iwfnwmr510zyiiapzic7q3y408r3ypyx-nixos-system-client-test' is required, but there is no substituter that can build it"4485client: must succeed: rm -r /var/lib/nix-grpc-store4486client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.02 seconds)4487worker2 # [ 35.214149] nix-grpc-daemon[786]: ts=2026-09-21T12:54:01Z level=info event=denied method=QueryMissing cn=- role=none4488worker2 # [ 35.440773] nix-grpc-daemon[786]: ts=2026-09-21T12:54:01Z level=info event=denied method=QueryMissing cn=stranger role=none4489(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.75 seconds)4490subtest: DAG build is scheduled across workers and lands in the cache4491client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24492worker2 # [ 35.714577] nix-grpc-daemon[786]: ts=2026-09-21T12:54:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:360284493client # don't know how to build these paths:4494client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4495worker1 # [ 35.743619] nix-grpc-daemon[785]: ts=2026-09-21T12:54:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444496worker2 # [ 35.763808] nix-grpc-daemon[786]: ts=2026-09-21T12:54:01Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04497worker2 # [ 35.765461] nix-grpc-daemon[786]: ts=2026-09-21T12:54:01Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14498worker1 # [ 35.842161] nix-grpc-daemon[785]: ts=2026-09-21T12:54:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444499worker1 # [ 35.860627] nix-grpc-daemon[785]: ts=2026-09-21T12:54:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444500client # copying 1 paths to 'lb:50051'...4501worker1 # [ 35.874763] nix-grpc-daemon[785]: ts=2026-09-21T12:54:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444502client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4503worker1 # [ 35.892498] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4504worker1 # [ 35.906743] nix-grpc-daemon[785]: ts=2026-09-21T12:54:01Z level=info event=niks3_push_started pid=8634505worker1 # [ 36.166564] nix-grpc-daemon[863]: time=2026-09-21T12:54:02.246Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""4506worker1 # [ 36.356327] nix-daemon[850]: accepted connection from pid 875, user nix-grpc-daemon (trusted)4507worker1 # [ 36.371513] nix-daemon[850]: reaped child process 878, status = succeeded4508worker1 # [ 36.532557] nix-daemon[850]: accepted connection from pid 880, user nix-grpc-daemon (trusted)4509worker1 # [ 36.544752] nix-daemon[850]: reaped child process 883, status = succeeded4510lb # [ 36.554922] niks3-server[1047]: 2026/09/21 12:54:02 INFO Received uploads request method=POST path=/api/pending_closures4511worker1 # [ 36.583732] nix-grpc-daemon[863]: time=2026-09-21T12:54:02.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4512worker1 # [ 36.586305] nix-grpc-daemon[863]: time=2026-09-21T12:54:02.669Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4513lb # [ 36.624727] niks3-server[1047]: 2026/09/21 12:54:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4514lb # [ 36.637162] niks3-server[1047]: 2026/09/21 12:54:02 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4515lb # [ 36.646776] niks3-server[1047]: 2026/09/21 12:54:02 INFO Signed narinfos id=1 count=14516worker1 # [ 36.653689] nix-grpc-daemon[863]: time=2026-09-21T12:54:02.736Z level=INFO msg="Uploading 1 narinfos"4517lb # [ 36.655256] niks3-server[1047]: 2026/09/21 12:54:02 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4518lb # [ 36.663565] niks3-server[1047]: 2026/09/21 12:54:02 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4519lb # [ 36.674495] niks3-server[1047]: 2026/09/21 12:54:02 INFO Completed upload id=14520worker1 # [ 36.680086] nix-grpc-daemon[863]: time=2026-09-21T12:54:02.763Z level=INFO msg="Upload complete. (496ms)"4521worker1 # [ 36.684731] nix-grpc-daemon[785]: ts=2026-09-21T12:54:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=5044522lb # [ 36.683517] niks3-server[1047]: 2026/09/21 12:54:02 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4523worker1 # [ 36.692324] nix-grpc-daemon[785]: ts=2026-09-21T12:54:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444524worker1 # [ 36.701567] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4525worker1 # [ 36.712446] nix-daemon[850]: reaped child process 861, status = succeeded4526client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4527worker1 # [ 36.719940] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4528client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4529client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4530client # farm-a> LOG-farm-a-t14531worker1 # [ 37.945672] nix-daemon[850]: reaped child process 887, status = succeeded4532worker1 # [ 38.085330] nix-daemon[850]: accepted connection from pid 903, user nix-grpc-daemon (trusted)4533worker1 # [ 38.096321] nix-daemon[850]: reaped child process 906, status = succeeded4534lb # [ 38.102946] niks3-server[1047]: 2026/09/21 12:54:04 INFO Received uploads request method=POST path=/api/pending_closures4535worker1 # [ 38.119410] nix-grpc-daemon[863]: time=2026-09-21T12:54:04.202Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4536worker1 # [ 38.121483] nix-grpc-daemon[863]: time=2026-09-21T12:54:04.204Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4537lb # [ 38.147536] niks3-server[1047]: 2026/09/21 12:54:04 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4538lb # [ 38.158559] niks3-server[1047]: 2026/09/21 12:54:04 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4539lb # [ 38.169070] niks3-server[1047]: 2026/09/21 12:54:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4540worker1 # [ 38.178786] nix-grpc-daemon[863]: time=2026-09-21T12:54:04.261Z level=INFO msg="Uploading 1 narinfos"4541lb # [ 38.173329] niks3-server[1047]: 2026/09/21 12:54:04 INFO Signed narinfos id=2 count=14542lb # [ 38.179133] niks3-server[1047]: 2026/09/21 12:54:04 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4543lb # [ 38.188190] niks3-server[1047]: 2026/09/21 12:54:04 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4544lb # [ 38.190928] niks3-server[1047]: 2026/09/21 12:54:04 INFO Completed upload id=24545worker1 # [ 38.196350] nix-grpc-daemon[863]: time=2026-09-21T12:54:04.279Z level=INFO msg="Upload complete. (263ms)"4546lb # [ 38.192570] niks3-server[1047]: 2026/09/21 12:54:04 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4547worker1 # [ 38.201490] nix-grpc-daemon[785]: ts=2026-09-21T12:54:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14548worker1 # [ 38.212613] nix-daemon[850]: reaped child process 885, status = succeeded4549worker2 # [ 38.207436] nix-grpc-daemon[786]: ts=2026-09-21T12:54:04Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04550worker2 # [ 38.211514] nix-grpc-daemon[786]: ts=2026-09-21T12:54:04Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14551worker1 # [ 38.265864] nix-grpc-daemon[785]: ts=2026-09-21T12:54:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444552worker1 # [ 38.278776] nix-grpc-daemon[785]: ts=2026-09-21T12:54:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444553worker1 # [ 38.292989] nix-grpc-daemon[863]: time=2026-09-21T12:54:04.376Z level=INFO msg="All 1 paths already cached"4554client # copying 1 paths to 'lb:50051'...4555worker1 # [ 38.300737] nix-grpc-daemon[785]: ts=2026-09-21T12:54:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444556client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4557worker1 # [ 38.319292] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4558worker1 # [ 38.471940] nix-daemon[850]: accepted connection from pid 910, user nix-grpc-daemon (trusted)4559worker1 # [ 38.485765] nix-daemon[850]: reaped child process 913, status = succeeded4560lb # [ 38.508522] envoy[909]: conn peer=[2001:db8:1::1]:46090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=15414561lb # [ 38.512242] envoy[909]: conn peer=[2001:db8:1::1]:46094 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=584562lb # [ 38.515561] envoy[909]: conn peer=[2001:db8:1::1]:46100 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=74563lb # [ 38.518780] envoy[909]: conn peer=[2001:db8:1::1]:46110 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=94564worker1 # [ 38.623826] nix-daemon[850]: accepted connection from pid 916, user nix-grpc-daemon (trusted)4565worker1 # [ 38.636228] nix-daemon[850]: reaped child process 919, status = succeeded4566lb # [ 38.642891] niks3-server[1047]: 2026/09/21 12:54:04 INFO Received uploads request method=POST path=/api/pending_closures4567worker1 # [ 38.657797] nix-grpc-daemon[863]: time=2026-09-21T12:54:04.740Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4568worker1 # [ 38.659316] nix-grpc-daemon[863]: time=2026-09-21T12:54:04.742Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4569lb # [ 38.674878] niks3-server[1047]: 2026/09/21 12:54:04 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4570lb # [ 38.684896] niks3-server[1047]: 2026/09/21 12:54:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4571worker1 # [ 38.695483] nix-grpc-daemon[863]: time=2026-09-21T12:54:04.778Z level=INFO msg="Uploading 1 narinfos"4572lb # [ 38.690468] niks3-server[1047]: 2026/09/21 12:54:04 INFO Signed narinfos id=3 count=14573lb # [ 38.697662] niks3-server[1047]: 2026/09/21 12:54:04 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4574lb # [ 38.705427] niks3-server[1047]: 2026/09/21 12:54:04 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4575lb # [ 38.714695] niks3-server[1047]: 2026/09/21 12:54:04 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4576lb # [ 38.717843] niks3-server[1047]: 2026/09/21 12:54:04 INFO Completed upload id=34577worker1 # [ 38.723148] nix-grpc-daemon[863]: time=2026-09-21T12:54:04.806Z level=INFO msg="Upload complete. (389ms)"4578worker1 # [ 38.727137] nix-grpc-daemon[785]: ts=2026-09-21T12:54:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=6244579worker1 # [ 38.731570] nix-grpc-daemon[785]: ts=2026-09-21T12:54:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444580worker1 # [ 38.740189] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4581worker1 # [ 38.749149] nix-daemon[850]: reaped child process 908, status = succeeded4582worker1 # [ 38.753143] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4583client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4584client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4585client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4586client # farm-b> LOG-farm-b-t14587worker1 # [ 39.164509] nix-daemon[850]: reaped child process 923, status = succeeded4588worker1 # [ 39.316143] nix-daemon[850]: accepted connection from pid 940, user nix-grpc-daemon (trusted)4589worker1 # [ 39.328529] nix-daemon[850]: reaped child process 943, status = succeeded4590lb # [ 39.342281] niks3-server[1047]: 2026/09/21 12:54:05 INFO Received uploads request method=POST path=/api/pending_closures4591worker1 # [ 39.355169] nix-grpc-daemon[863]: time=2026-09-21T12:54:05.438Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4592worker1 # [ 39.356707] nix-grpc-daemon[863]: time=2026-09-21T12:54:05.439Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4593lb # [ 39.378961] niks3-server[1047]: 2026/09/21 12:54:05 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4594lb # [ 39.390930] niks3-server[1047]: 2026/09/21 12:54:05 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4595lb # [ 39.398305] niks3-server[1047]: 2026/09/21 12:54:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4596lb # [ 39.403250] niks3-server[1047]: 2026/09/21 12:54:05 INFO Signed narinfos id=4 count=14597worker1 # [ 39.411695] nix-grpc-daemon[863]: time=2026-09-21T12:54:05.494Z level=INFO msg="Uploading 1 narinfos"4598lb # [ 39.411310] niks3-server[1047]: 2026/09/21 12:54:05 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4599lb # [ 39.420326] niks3-server[1047]: 2026/09/21 12:54:05 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4600lb # [ 39.424859] niks3-server[1047]: 2026/09/21 12:54:05 INFO Completed upload id=44601lb # [ 39.426991] niks3-server[1047]: 2026/09/21 12:54:05 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4602worker1 # [ 39.431581] nix-grpc-daemon[863]: time=2026-09-21T12:54:05.514Z level=INFO msg="Upload complete. (273ms)"4603worker1 # [ 39.438283] nix-grpc-daemon[785]: ts=2026-09-21T12:54:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14604worker2 # [ 39.430654] nix-grpc-daemon[786]: ts=2026-09-21T12:54:05Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04605worker2 # [ 39.434696] nix-grpc-daemon[786]: ts=2026-09-21T12:54:05Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14606worker1 # [ 39.451908] nix-daemon[850]: reaped child process 921, status = succeeded4607worker1 # [ 39.488275] nix-grpc-daemon[785]: ts=2026-09-21T12:54:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444608worker1 # [ 39.501063] nix-grpc-daemon[785]: ts=2026-09-21T12:54:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444609worker1 # [ 39.515942] nix-grpc-daemon[863]: time=2026-09-21T12:54:05.598Z level=INFO msg="All 2 paths already cached"4610client # copying 1 paths to 'lb:50051'...4611worker1 # [ 39.524331] nix-grpc-daemon[785]: ts=2026-09-21T12:54:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444612client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4613worker1 # [ 39.542327] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4614worker1 # [ 39.724155] nix-daemon[850]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4615worker1 # [ 39.736841] nix-daemon[850]: reaped child process 950, status = succeeded4616worker1 # [ 39.901375] nix-daemon[850]: accepted connection from pid 952, user nix-grpc-daemon (trusted)4617worker1 # [ 39.913621] nix-daemon[850]: reaped child process 955, status = succeeded4618lb # [ 39.923564] niks3-server[1047]: 2026/09/21 12:54:05 INFO Received uploads request method=POST path=/api/pending_closures4619worker1 # [ 39.934656] nix-grpc-daemon[863]: time=2026-09-21T12:54:06.017Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4620worker1 # [ 39.936046] nix-grpc-daemon[863]: time=2026-09-21T12:54:06.017Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4621lb # [ 39.937774] niks3-server[1047]: 2026/09/21 12:54:05 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4622lb # [ 39.942684] niks3-server[1047]: 2026/09/21 12:54:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4623lb # [ 39.945521] niks3-server[1047]: 2026/09/21 12:54:05 INFO Signed narinfos id=5 count=14624worker1 # [ 39.950762] nix-grpc-daemon[863]: time=2026-09-21T12:54:06.033Z level=INFO msg="Uploading 1 narinfos"4625lb # [ 39.949214] niks3-server[1047]: 2026/09/21 12:54:05 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4626lb # [ 39.953524] niks3-server[1047]: 2026/09/21 12:54:05 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4627lb # [ 39.957587] niks3-server[1047]: 2026/09/21 12:54:05 INFO Completed upload id=54628worker1 # [ 39.964284] nix-grpc-daemon[863]: time=2026-09-21T12:54:06.047Z level=INFO msg="Upload complete. (407ms)"4629lb # [ 39.960724] niks3-server[1047]: 2026/09/21 12:54:05 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4630worker1 # [ 39.967078] nix-grpc-daemon[785]: ts=2026-09-21T12:54:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=7524631worker1 # [ 39.970453] nix-grpc-daemon[785]: ts=2026-09-21T12:54:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444632worker1 # [ 39.976088] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4633worker1 # [ 39.978162] nix-daemon[850]: reaped child process 945, status = succeeded4634worker1 # [ 39.983544] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4635client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4636client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4637client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4638client # farm-top> LOG-farm-top-t14639worker1 # [ 40.797132] nix-daemon[850]: reaped child process 959, status = succeeded4640worker1 # [ 40.947563] nix-daemon[850]: accepted connection from pid 975, user nix-grpc-daemon (trusted)4641worker1 # [ 40.960319] nix-daemon[850]: reaped child process 978, status = succeeded4642lb # [ 40.972922] niks3-server[1047]: 2026/09/21 12:54:06 INFO Received uploads request method=POST path=/api/pending_closures4643worker1 # [ 40.989357] nix-grpc-daemon[863]: time=2026-09-21T12:54:07.070Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4644worker1 # [ 40.990713] nix-grpc-daemon[863]: time=2026-09-21T12:54:07.071Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4645lb # [ 41.012534] niks3-server[1047]: 2026/09/21 12:54:06 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4646lb # [ 41.017124] niks3-server[1047]: 2026/09/21 12:54:06 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4647lb # [ 41.025714] niks3-server[1047]: 2026/09/21 12:54:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4648lb # [ 41.029047] niks3-server[1047]: 2026/09/21 12:54:06 INFO Signed narinfos id=6 count=14649worker1 # [ 41.036780] nix-grpc-daemon[863]: time=2026-09-21T12:54:07.119Z level=INFO msg="Uploading 1 narinfos"4650lb # [ 41.035283] niks3-server[1047]: 2026/09/21 12:54:06 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4651lb # [ 41.043991] niks3-server[1047]: 2026/09/21 12:54:06 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4652lb # [ 41.046981] niks3-server[1047]: 2026/09/21 12:54:06 INFO Completed upload id=64653lb # [ 41.048902] niks3-server[1047]: 2026/09/21 12:54:06 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4654worker1 # [ 41.053589] nix-grpc-daemon[863]: time=2026-09-21T12:54:07.136Z level=INFO msg="Upload complete. (263ms)"4655worker1 # [ 41.058403] nix-grpc-daemon[785]: ts=2026-09-21T12:54:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14656worker1 # [ 41.069767] nix-daemon[850]: reaped child process 957, status = succeeded4657client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 5.60 seconds)4658client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4659client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.23 seconds)4660client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4661client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4662client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4663client # copying 3 paths...4664client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4665client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4666client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4667client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.31 seconds)4668(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.25 seconds)4669subtest: repeat is answered Cached by the scheduler without building4670worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4671worker1 # finding garbage collector roots...4672worker1 # deleting specified paths...4673worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4674worker1 # deleting unused links...4675worker1 # note: hard linking is currently saving 0.0 KiB4676worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.25 seconds)4677worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4678worker2 # finding garbage collector roots...4679worker2 # deleting specified paths...4680worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4681worker2 # deleting unused links...4682worker2 # note: hard linking is currently saving 0.0 KiB4683worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.28 seconds)4684client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24685worker1 # [ 42.530886] nix-grpc-daemon[785]: ts=2026-09-21T12:54:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494444686client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4687client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4688worker1 # [ 42.549764] nix-grpc-daemon[785]: ts=2026-09-21T12:54:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444689worker1 # [ 42.559341] nix-grpc-daemon[863]: time=2026-09-21T12:54:08.641Z level=INFO msg="All 2 paths already cached"4690worker2 # [ 42.561749] nix-grpc-daemon[786]: ts=2026-09-21T12:54:08Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14691client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.31 seconds)4692client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4693client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.21 seconds)4694lb: must succeed: curl -sf localhost:9901/clusters4695lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4696lb: must succeed: curl -sf localhost:9901/clusters4697lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4698worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4699worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4700(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.30 seconds)4701subtest: inputs already in the cache are not sent to the scheduler4702lb: must succeed: curl -sf localhost:9901/clusters4703lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4704lb: must succeed: curl -sf localhost:9901/clusters4705lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4706worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4707worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4708client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24709client # don't know how to build these paths:4710client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4711worker1 # [ 43.478293] nix-grpc-daemon[785]: ts=2026-09-21T12:54:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494444712worker1 # [ 43.484258] nix-grpc-daemon[785]: ts=2026-09-21T12:54:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444713worker1 # [ 43.489293] nix-grpc-daemon[863]: time=2026-09-21T12:54:09.571Z level=INFO msg="All 2 paths already cached"4714worker2 # [ 43.484175] nix-grpc-daemon[786]: ts=2026-09-21T12:54:09Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04715worker2 # [ 43.488673] nix-grpc-daemon[786]: ts=2026-09-21T12:54:09Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14716worker1 # [ 43.543317] nix-grpc-daemon[785]: ts=2026-09-21T12:54:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444717worker1 # [ 43.556315] nix-grpc-daemon[785]: ts=2026-09-21T12:54:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444718worker1 # [ 43.569283] nix-grpc-daemon[863]: time=2026-09-21T12:54:09.651Z level=INFO msg="All 2 paths already cached"4719client # copying 1 paths to 'lb:50051'...4720client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4721worker1 # [ 43.577547] nix-grpc-daemon[785]: ts=2026-09-21T12:54:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444722worker1 # [ 43.589357] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4723worker1 # [ 43.772154] nix-daemon[850]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4724worker1 # [ 43.785399] nix-daemon[850]: reaped child process 999, status = succeeded4725worker1 # [ 43.953634] nix-daemon[850]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4726worker1 # [ 43.965703] nix-daemon[850]: reaped child process 1004, status = succeeded4727lb # [ 43.981499] niks3-server[1047]: 2026/09/21 12:54:09 INFO Received uploads request method=POST path=/api/pending_closures4728worker1 # [ 43.994426] nix-grpc-daemon[863]: time=2026-09-21T12:54:10.077Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4729worker1 # [ 43.995786] nix-grpc-daemon[863]: time=2026-09-21T12:54:10.077Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4730lb # [ 44.008941] niks3-server[1047]: 2026/09/21 12:54:09 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4731lb # [ 44.022117] niks3-server[1047]: 2026/09/21 12:54:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4732worker1 # [ 44.033331] nix-grpc-daemon[863]: time=2026-09-21T12:54:10.116Z level=INFO msg="Uploading 1 narinfos"4733lb # [ 44.027810] niks3-server[1047]: 2026/09/21 12:54:09 INFO Signed narinfos id=7 count=14734lb # [ 44.034352] niks3-server[1047]: 2026/09/21 12:54:09 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4735lb # [ 44.046362] niks3-server[1047]: 2026/09/21 12:54:09 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4736lb # [ 44.051335] niks3-server[1047]: 2026/09/21 12:54:09 INFO Completed upload id=74737worker1 # [ 44.059296] nix-grpc-daemon[863]: time=2026-09-21T12:54:10.142Z level=INFO msg="Upload complete. (458ms)"4738lb # [ 44.055287] niks3-server[1047]: 2026/09/21 12:54:09 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4739worker1 # [ 44.063420] nix-grpc-daemon[785]: ts=2026-09-21T12:54:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=7524740worker1 # [ 44.069566] nix-grpc-daemon[785]: ts=2026-09-21T12:54:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444741worker1 # [ 44.079139] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4742worker1 # [ 44.086959] nix-daemon[850]: reaped child process 994, status = succeeded4743worker1 # [ 44.092348] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4744client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4745client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4746client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4747client # farm-top> LOG-farm-top-t1b4748worker1 # [ 45.306520] nix-daemon[850]: reaped child process 1008, status = succeeded4749worker1 # [ 45.457853] nix-daemon[850]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4750worker1 # [ 45.470152] nix-daemon[850]: reaped child process 1027, status = succeeded4751lb # [ 45.482322] niks3-server[1047]: 2026/09/21 12:54:11 INFO Received uploads request method=POST path=/api/pending_closures4752worker1 # [ 45.498099] nix-grpc-daemon[863]: time=2026-09-21T12:54:11.580Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4753worker1 # [ 45.500623] nix-grpc-daemon[863]: time=2026-09-21T12:54:11.582Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4754lb # [ 45.523453] niks3-server[1047]: 2026/09/21 12:54:11 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4755lb # [ 45.532116] niks3-server[1047]: 2026/09/21 12:54:11 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4756lb # [ 45.543709] niks3-server[1047]: 2026/09/21 12:54:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4757worker1 # [ 45.552967] nix-grpc-daemon[863]: time=2026-09-21T12:54:11.635Z level=INFO msg="Uploading 1 narinfos"4758lb # [ 45.547654] niks3-server[1047]: 2026/09/21 12:54:11 INFO Signed narinfos id=8 count=14759lb # [ 45.552371] niks3-server[1047]: 2026/09/21 12:54:11 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4760lb # [ 45.561370] niks3-server[1047]: 2026/09/21 12:54:11 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4761lb # [ 45.566130] niks3-server[1047]: 2026/09/21 12:54:11 INFO Completed upload id=84762worker1 # [ 45.572895] nix-grpc-daemon[863]: time=2026-09-21T12:54:11.655Z level=INFO msg="Upload complete. (273ms)"4763lb # [ 45.568871] niks3-server[1047]: 2026/09/21 12:54:11 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4764worker1 # [ 45.577884] nix-grpc-daemon[785]: ts=2026-09-21T12:54:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14765worker1 # [ 45.590967] nix-daemon[850]: reaped child process 1006, status = succeeded4766client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.41 seconds)4767client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4768client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.22 seconds)4769lb: must succeed: curl -sf localhost:9901/clusters4770lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4771lb: must succeed: curl -sf localhost:9901/clusters4772lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4773worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4774worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4775(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.97 seconds)4776subtest: two clients wanting the same drv build it once4777client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4778client # [ 46.064828] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4779client # Running as unit: dedup1.service; invocation ID: 4bd94658768a48e0a016b4b4fdef0b7e4780client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.07 seconds)4781client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4782worker1 # [ 46.320350] nix-grpc-daemon[785]: ts=2026-09-21T12:54:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494444783client # [ 46.308528] nix[884]: don't know how to build these paths:4784client # [ 46.309428] nix[884]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4785worker2 # [ 46.322879] nix-grpc-daemon[786]: ts=2026-09-21T12:54:12Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04786worker2 # [ 46.328533] nix-grpc-daemon[786]: ts=2026-09-21T12:54:12Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14787worker2 # [ 46.333242] nix-grpc-daemon[786]: ts=2026-09-21T12:54:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360284788worker2 # [ 46.347861] nix-grpc-daemon[786]: ts=2026-09-21T12:54:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360284789client # [ 46.353672] nix[884]: copying 1 paths to 'lb:50051'...4790client # [ 46.354538] nix[884]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4791worker2 # [ 46.358662] nix-grpc-daemon[786]: ts=2026-09-21T12:54:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:360284792worker2 # [ 46.367816] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4793worker2 # [ 46.391114] nix-grpc-daemon[786]: ts=2026-09-21T12:54:12Z level=info event=niks3_push_started pid=9244794worker2 # [ 46.681376] nix-grpc-daemon[924]: time=2026-09-21T12:54:12.895Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""4795worker2 # [ 46.910093] nix-daemon[871]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4796worker2 # [ 46.924456] nix-daemon[871]: reaped child process 940, status = succeeded4797worker2 # [ 47.118648] nix-daemon[871]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4798worker2 # [ 47.130650] nix-daemon[871]: reaped child process 945, status = succeeded4799lb # [ 47.159537] niks3-server[1047]: 2026/09/21 12:54:13 INFO Received uploads request method=POST path=/api/pending_closures4800client # [ 47.146577] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4801client # Running as unit: dedup2.service; invocation ID: f198ce2b22d742be9210b8af442a0c0b4802worker2 # [ 47.162945] nix-grpc-daemon[924]: time=2026-09-21T12:54:13.380Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4803worker2 # [ 47.165643] nix-grpc-daemon[924]: time=2026-09-21T12:54:13.383Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4804client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.09 seconds)4805lb # [ 47.211403] niks3-server[1047]: 2026/09/21 12:54:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4806lb # [ 47.219550] niks3-server[1047]: 2026/09/21 12:54:13 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4807lb # [ 47.226633] niks3-server[1047]: 2026/09/21 12:54:13 INFO Signed narinfos id=9 count=14808worker2 # [ 47.221291] nix-grpc-daemon[924]: time=2026-09-21T12:54:13.439Z level=INFO msg="Uploading 1 narinfos"4809lb # [ 47.234291] niks3-server[1047]: 2026/09/21 12:54:13 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4810lb # [ 47.247576] niks3-server[1047]: 2026/09/21 12:54:13 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4811lb # [ 47.255961] niks3-server[1047]: 2026/09/21 12:54:13 INFO Completed upload id=94812worker2 # [ 47.247349] nix-grpc-daemon[924]: time=2026-09-21T12:54:13.465Z level=INFO msg="Upload complete. (540ms)"4813lb # [ 47.259086] niks3-server[1047]: 2026/09/21 12:54:13 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4814worker2 # [ 47.251488] nix-grpc-daemon[786]: ts=2026-09-21T12:54:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=0 paths=1 nar_bytes_in=4324815worker2 # [ 47.257924] nix-grpc-daemon[786]: ts=2026-09-21T12:54:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360284816worker2 # [ 47.272461] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4817worker2 # [ 47.283552] nix-daemon[871]: reaped child process 922, status = succeeded4818client # [ 47.282638] nix[884]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4819worker2 # [ 47.289991] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4820client # [ 47.444460] nix[900]: don't know how to build these paths:4821client # [ 47.445637] nix[900]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4822worker1 # [ 47.461880] nix-grpc-daemon[785]: ts=2026-09-21T12:54:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494444823worker2 # [ 47.459638] nix-grpc-daemon[786]: ts=2026-09-21T12:54:13Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04824worker2 # [ 47.468943] nix-grpc-daemon[786]: ts=2026-09-21T12:54:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360284825client # [ 47.469862] nix[900]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4826client: must succeed: sleep 24827lb # [ 48.508463] envoy[909]: conn peer=[2001:db8:1::1]:46116 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4054 ms=53654828lb # [ 48.512434] envoy[909]: conn peer=[2001:db8:1::1]:46124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=594829lb # [ 48.515826] envoy[909]: conn peer=[2001:db8:1::1]:54656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1805 ms=21114830client: (finished: must succeed: sleep 2, in 2.06 seconds)4831client: waiting for success: ! systemctl is-active dedup1 dedup24832worker2 # [ 50.705659] nix-daemon[871]: reaped child process 952, status = succeeded4833worker2 # [ 50.876857] nix-daemon[871]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4834worker2 # [ 50.889117] nix-daemon[871]: reaped child process 984, status = succeeded4835lb # [ 50.918760] niks3-server[1047]: 2026/09/21 12:54:16 INFO Received uploads request method=POST path=/api/pending_closures4836worker2 # [ 50.916640] nix-grpc-daemon[924]: time=2026-09-21T12:54:17.134Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4837worker2 # [ 50.918181] nix-grpc-daemon[924]: time=2026-09-21T12:54:17.136Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4838lb # [ 50.957165] niks3-server[1047]: 2026/09/21 12:54:16 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4839lb # [ 50.973130] niks3-server[1047]: 2026/09/21 12:54:16 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4840lb # [ 50.985573] niks3-server[1047]: 2026/09/21 12:54:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4841worker2 # [ 50.981425] nix-grpc-daemon[924]: time=2026-09-21T12:54:17.199Z level=INFO msg="Uploading 1 narinfos"4842lb # [ 50.990308] niks3-server[1047]: 2026/09/21 12:54:16 INFO Signed narinfos id=10 count=14843lb # [ 50.995092] niks3-server[1047]: 2026/09/21 12:54:16 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4844lb # [ 51.002533] niks3-server[1047]: 2026/09/21 12:54:16 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4845lb # [ 51.006293] niks3-server[1047]: 2026/09/21 12:54:16 INFO Completed upload id=104846worker2 # [ 50.997531] nix-grpc-daemon[924]: time=2026-09-21T12:54:17.215Z level=INFO msg="Upload complete. (297ms)"4847lb # [ 51.008149] niks3-server[1047]: 2026/09/21 12:54:16 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4848worker2 # [ 51.005319] nix-grpc-daemon[786]: ts=2026-09-21T12:54:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14849worker2 # [ 51.011737] nix-grpc-daemon[786]: ts=2026-09-21T12:54:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14850worker2 # [ 51.021168] nix-daemon[871]: reaped child process 950, status = succeeded4851client # [ 51.049493] systemd[1]: dedup2.service: Deactivated successfully.4852client # [ 51.055455] systemd[1]: dedup2.service: Consumed 144ms CPU time over 3.906s wall clock time, 35.2M memory peak, 5.8K incoming IP traffic, 5.6K outgoing IP traffic.4853client # [ 51.093830] systemd[1]: dedup1.service: Deactivated successfully.4854client # [ 51.095370] systemd[1]: dedup1.service: Consumed 120ms CPU time over 5.031s wall clock time, 43.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.4855client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4856client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4857client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4858worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4859worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4860worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4861worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4862worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4863worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4864worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4865worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4866(finished: subtest: two clients wanting the same drv build it once, in 6.07 seconds)4867subtest: build hook: nix-daemon with builders = grpc://lb4868client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4869client # [ 52.296745] systemd[1]: Started Nix Daemon.4870client # [ 52.514268] nix-daemon[942]: accepted connection from pid 939, user root (trusted)4871worker2 # [ 52.753852] nix-grpc-daemon[786]: ts=2026-09-21T12:54:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:360284872worker2 # [ 52.759139] nix-grpc-daemon[786]: ts=2026-09-21T12:54:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360284873worker2 # [ 52.770251] nix-grpc-daemon[786]: ts=2026-09-21T12:54:18Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04874worker2 # [ 52.774115] nix-grpc-daemon[786]: ts=2026-09-21T12:54:18Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14875worker1 # [ 52.838118] nix-grpc-daemon[785]: ts=2026-09-21T12:54:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444876worker1 # [ 52.851123] nix-grpc-daemon[785]: ts=2026-09-21T12:54:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444877worker1 # [ 52.863283] nix-grpc-daemon[785]: ts=2026-09-21T12:54:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444878worker1 # [ 52.874448] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4879worker1 # [ 53.059655] nix-daemon[850]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)4880worker1 # [ 53.072620] nix-daemon[850]: reaped child process 1055, status = succeeded4881worker1 # [ 53.234363] nix-daemon[850]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)4882worker1 # [ 53.246182] nix-daemon[850]: reaped child process 1060, status = succeeded4883lb # [ 53.257920] niks3-server[1047]: 2026/09/21 12:54:19 INFO Received uploads request method=POST path=/api/pending_closures4884worker1 # [ 53.271401] nix-grpc-daemon[863]: time=2026-09-21T12:54:19.354Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4885worker1 # [ 53.272917] nix-grpc-daemon[863]: time=2026-09-21T12:54:19.356Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4886lb # [ 53.285507] niks3-server[1047]: 2026/09/21 12:54:19 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4887lb # [ 53.297296] niks3-server[1047]: 2026/09/21 12:54:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4888lb # [ 53.302775] niks3-server[1047]: 2026/09/21 12:54:19 INFO Signed narinfos id=11 count=14889worker1 # [ 53.310731] nix-grpc-daemon[863]: time=2026-09-21T12:54:19.393Z level=INFO msg="Uploading 1 narinfos"4890lb # [ 53.309997] niks3-server[1047]: 2026/09/21 12:54:19 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4891lb # [ 53.320976] niks3-server[1047]: 2026/09/21 12:54:19 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4892worker1 # [ 53.332486] nix-grpc-daemon[863]: time=2026-09-21T12:54:19.415Z level=INFO msg="Upload complete. (445ms)"4893lb # [ 53.327538] niks3-server[1047]: 2026/09/21 12:54:19 INFO Completed upload id=114894worker1 # [ 53.334963] nix-grpc-daemon[785]: ts=2026-09-21T12:54:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=5124895lb # [ 53.330800] niks3-server[1047]: 2026/09/21 12:54:19 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4896worker1 # [ 53.340142] nix-grpc-daemon[785]: ts=2026-09-21T12:54:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444897worker1 # [ 53.350141] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4898worker1 # [ 53.354749] nix-daemon[850]: reaped child process 1050, status = succeeded4899worker1 # [ 53.359104] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4900worker1 # [ 54.989844] nix-daemon[850]: reaped child process 1064, status = succeeded4901worker1 # [ 55.141300] nix-daemon[850]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)4902worker1 # [ 55.153268] nix-daemon[850]: reaped child process 1083, status = succeeded4903lb # [ 55.164958] niks3-server[1047]: 2026/09/21 12:54:21 INFO Received uploads request method=POST path=/api/pending_closures4904worker1 # [ 55.179148] nix-grpc-daemon[863]: time=2026-09-21T12:54:21.262Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4905worker1 # [ 55.180742] nix-grpc-daemon[863]: time=2026-09-21T12:54:21.263Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4906lb # [ 55.206369] niks3-server[1047]: 2026/09/21 12:54:21 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4907lb # [ 55.217128] niks3-server[1047]: 2026/09/21 12:54:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4908worker1 # [ 55.226848] nix-grpc-daemon[863]: time=2026-09-21T12:54:21.309Z level=INFO msg="Uploading 1 narinfos"4909lb # [ 55.222124] niks3-server[1047]: 2026/09/21 12:54:21 INFO Signed narinfos id=12 count=14910lb # [ 55.227614] niks3-server[1047]: 2026/09/21 12:54:21 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4911lb # [ 55.234381] niks3-server[1047]: 2026/09/21 12:54:21 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4912lb # [ 55.243478] niks3-server[1047]: 2026/09/21 12:54:21 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4913lb # [ 55.249628] niks3-server[1047]: 2026/09/21 12:54:21 INFO Completed upload id=124914worker1 # [ 55.254911] nix-grpc-daemon[863]: time=2026-09-21T12:54:21.337Z level=INFO msg="Upload complete. (285ms)"4915lb # [ 55.251296] niks3-server[1047]: 2026/09/21 12:54:21 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4916worker1 # [ 55.257370] nix-grpc-daemon[785]: ts=2026-09-21T12:54:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14917worker1 # [ 55.264461] nix-grpc-daemon[785]: ts=2026-09-21T12:54:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494444918worker1 # [ 55.266472] nix-daemon[850]: reaped child process 1062, status = succeeded4919worker1 # [ 55.279592] nix-grpc-daemon[785]: ts=2026-09-21T12:54:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:494444920worker1 # [ 55.282274] nix-grpc-daemon[785]: ts=2026-09-21T12:54:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_out=1284921worker1 # [ 55.577075] nix-grpc-daemon[785]: ts=2026-09-21T12:54:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:494444922worker2 # [ 55.568860] nix-grpc-daemon[786]: ts=2026-09-21T12:54:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360284923worker2 # [ 55.591174] nix-grpc-daemon[786]: ts=2026-09-21T12:54:21Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04924worker1 # [ 55.611652] nix-grpc-daemon[785]: ts=2026-09-21T12:54:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444925worker2 # [ 55.595824] nix-grpc-daemon[786]: ts=2026-09-21T12:54:21Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14926worker1 # [ 55.618897] nix-grpc-daemon[785]: ts=2026-09-21T12:54:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444927worker1 # [ 55.627501] nix-grpc-daemon[863]: time=2026-09-21T12:54:21.710Z level=INFO msg="All 1 paths already cached"4928worker1 # [ 55.631420] nix-grpc-daemon[785]: ts=2026-09-21T12:54:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444929worker1 # [ 55.636469] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4930worker1 # [ 55.794629] nix-daemon[850]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)4931worker1 # [ 55.808253] nix-daemon[850]: reaped child process 1090, status = succeeded4932worker1 # [ 55.965756] nix-daemon[850]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)4933worker1 # [ 55.977778] nix-daemon[850]: reaped child process 1095, status = succeeded4934lb # [ 55.988388] niks3-server[1047]: 2026/09/21 12:54:21 INFO Received uploads request method=POST path=/api/pending_closures4935worker1 # [ 56.003209] nix-grpc-daemon[863]: time=2026-09-21T12:54:22.086Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4936worker1 # [ 56.004710] nix-grpc-daemon[863]: time=2026-09-21T12:54:22.087Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4937lb # [ 56.018142] niks3-server[1047]: 2026/09/21 12:54:21 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4938lb # [ 56.029605] niks3-server[1047]: 2026/09/21 12:54:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4939worker1 # [ 56.040584] nix-grpc-daemon[863]: time=2026-09-21T12:54:22.123Z level=INFO msg="Uploading 1 narinfos"4940lb # [ 56.034869] niks3-server[1047]: 2026/09/21 12:54:21 INFO Signed narinfos id=13 count=14941lb # [ 56.041311] niks3-server[1047]: 2026/09/21 12:54:21 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4942lb # [ 56.051940] niks3-server[1047]: 2026/09/21 12:54:21 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4943worker1 # [ 56.063822] nix-grpc-daemon[863]: time=2026-09-21T12:54:22.146Z level=INFO msg="Upload complete. (422ms)"4944lb # [ 56.058776] niks3-server[1047]: 2026/09/21 12:54:21 INFO Completed upload id=134945worker1 # [ 56.065798] nix-grpc-daemon[785]: ts=2026-09-21T12:54:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=6244946lb # [ 56.061953] niks3-server[1047]: 2026/09/21 12:54:21 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4947worker1 # [ 56.071422] nix-grpc-daemon[785]: ts=2026-09-21T12:54:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444948worker1 # [ 56.076707] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4949worker1 # [ 56.078987] nix-daemon[850]: reaped child process 1085, status = succeeded4950worker1 # [ 56.084392] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4951worker1 # [ 57.109900] nix-daemon[850]: reaped child process 1099, status = succeeded4952worker1 # [ 57.273482] nix-daemon[850]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4953worker1 # [ 57.285699] nix-daemon[850]: reaped child process 1118, status = succeeded4954lb # [ 57.296496] niks3-server[1047]: 2026/09/21 12:54:23 INFO Received uploads request method=POST path=/api/pending_closures4955worker1 # [ 57.311319] nix-grpc-daemon[863]: time=2026-09-21T12:54:23.394Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4956worker1 # [ 57.312957] nix-grpc-daemon[863]: time=2026-09-21T12:54:23.395Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4957lb # [ 57.333170] niks3-server[1047]: 2026/09/21 12:54:23 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4958lb # [ 57.347543] niks3-server[1047]: 2026/09/21 12:54:23 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4959lb # [ 57.359675] niks3-server[1047]: 2026/09/21 12:54:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4960lb # [ 57.363091] niks3-server[1047]: 2026/09/21 12:54:23 INFO Signed narinfos id=14 count=14961worker1 # [ 57.371422] nix-grpc-daemon[863]: time=2026-09-21T12:54:23.454Z level=INFO msg="Uploading 1 narinfos"4962lb # [ 57.368327] niks3-server[1047]: 2026/09/21 12:54:23 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4963lb # [ 57.377064] niks3-server[1047]: 2026/09/21 12:54:23 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4964lb # [ 57.379949] niks3-server[1047]: 2026/09/21 12:54:23 INFO Completed upload id=144965worker1 # [ 57.385406] nix-grpc-daemon[863]: time=2026-09-21T12:54:23.468Z level=INFO msg="Upload complete. (293ms)"4966lb # [ 57.382689] niks3-server[1047]: 2026/09/21 12:54:23 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4967worker1 # [ 57.390760] nix-grpc-daemon[785]: ts=2026-09-21T12:54:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14968worker1 # [ 57.404263] nix-daemon[850]: reaped child process 1097, status = succeeded4969worker2 # [ 57.423540] nix-grpc-daemon[786]: ts=2026-09-21T12:54:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:360284970worker2 # [ 57.437797] nix-grpc-daemon[786]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4971worker2 # [ 57.455196] nix-grpc-daemon[786]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4972worker2 # [ 57.485860] nix-grpc-daemon[786]: ts=2026-09-21T12:54:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:360284973worker2 # [ 57.491717] nix-grpc-daemon[786]: ts=2026-09-21T12:54:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=0 paths=1 nar_bytes_out=1764974worker2 # [ 57.787607] nix-grpc-daemon[786]: ts=2026-09-21T12:54:24Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:360284975worker1 # [ 57.804638] nix-grpc-daemon[785]: ts=2026-09-21T12:54:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444976worker1 # [ 57.815109] nix-grpc-daemon[863]: time=2026-09-21T12:54:23.898Z level=INFO msg="All 2 paths already cached"4977worker2 # [ 57.816133] nix-grpc-daemon[786]: ts=2026-09-21T12:54:24Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04978worker2 # [ 57.820445] nix-grpc-daemon[786]: ts=2026-09-21T12:54:24Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14979worker1 # [ 57.874841] nix-grpc-daemon[785]: ts=2026-09-21T12:54:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444980worker1 # [ 57.889286] nix-grpc-daemon[785]: ts=2026-09-21T12:54:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444981worker1 # [ 57.902253] nix-grpc-daemon[863]: time=2026-09-21T12:54:23.985Z level=INFO msg="All 2 paths already cached"4982worker1 # [ 57.909502] nix-grpc-daemon[785]: ts=2026-09-21T12:54:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444983worker1 # [ 57.917726] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4984worker1 # [ 58.096404] nix-daemon[850]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4985worker1 # [ 58.109293] nix-daemon[850]: reaped child process 1125, status = succeeded4986worker1 # [ 58.268106] nix-daemon[850]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)4987worker1 # [ 58.280076] nix-daemon[850]: reaped child process 1130, status = succeeded4988lb # [ 58.291340] niks3-server[1047]: 2026/09/21 12:54:24 INFO Received uploads request method=POST path=/api/pending_closures4989worker1 # [ 58.304719] nix-grpc-daemon[863]: time=2026-09-21T12:54:24.387Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4990worker1 # [ 58.306529] nix-grpc-daemon[863]: time=2026-09-21T12:54:24.389Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4991lb # [ 58.318980] niks3-server[1047]: 2026/09/21 12:54:24 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4992lb # [ 58.330653] niks3-server[1047]: 2026/09/21 12:54:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4993lb # [ 58.335529] niks3-server[1047]: 2026/09/21 12:54:24 INFO Signed narinfos id=15 count=14994worker1 # [ 58.344038] nix-grpc-daemon[863]: time=2026-09-21T12:54:24.426Z level=INFO msg="Uploading 1 narinfos"4995lb # [ 58.342585] niks3-server[1047]: 2026/09/21 12:54:24 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4996lb # [ 58.354197] niks3-server[1047]: 2026/09/21 12:54:24 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4997worker1 # [ 58.365731] nix-grpc-daemon[863]: time=2026-09-21T12:54:24.448Z level=INFO msg="Upload complete. (432ms)"4998lb # [ 58.360536] niks3-server[1047]: 2026/09/21 12:54:24 INFO Completed upload id=154999worker1 # [ 58.369087] nix-grpc-daemon[785]: ts=2026-09-21T12:54:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=7525000lb # [ 58.364570] niks3-server[1047]: 2026/09/21 12:54:24 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5001worker1 # [ 58.374523] nix-grpc-daemon[785]: ts=2026-09-21T12:54:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494445002worker1 # [ 58.376253] nix-daemon[850]: reaped child process 1120, status = succeeded5003worker1 # [ 58.379307] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5004worker1 # [ 58.385907] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5005lb # [ 58.508612] envoy[909]: conn peer=[2001:db8:1::1]:54686 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=35785006lb # [ 58.512325] envoy[909]: conn peer=[2001:db8:1::1]:54670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3011 tx=1451 ms=47595007lb # [ 58.515857] envoy[909]: conn peer=[2001:db8:1::1]:54696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3427 tx=1713 ms=25685008lb # [ 58.519351] envoy[909]: conn peer=[2001:db8:1::1]:58718 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=545009lb # [ 58.522738] envoy[909]: conn peer=[2001:db8:1::1]:58732 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=595010lb # [ 58.526133] envoy[909]: conn peer=[2001:db8:1::1]:58730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=2045 ms=19825011worker1 # [ 59.812903] nix-daemon[850]: reaped child process 1134, status = succeeded5012worker1 # [ 59.980148] nix-daemon[850]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)5013worker1 # [ 59.992602] nix-daemon[850]: reaped child process 1153, status = succeeded5014lb # [ 60.004253] niks3-server[1047]: 2026/09/21 12:54:25 INFO Received uploads request method=POST path=/api/pending_closures5015worker1 # [ 60.018056] nix-grpc-daemon[863]: time=2026-09-21T12:54:26.100Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5016worker1 # [ 60.019697] nix-grpc-daemon[863]: time=2026-09-21T12:54:26.102Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5017lb # [ 60.040553] niks3-server[1047]: 2026/09/21 12:54:25 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5018lb # [ 60.051282] niks3-server[1047]: 2026/09/21 12:54:25 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5019lb # [ 60.061966] niks3-server[1047]: 2026/09/21 12:54:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5020lb # [ 60.066832] niks3-server[1047]: 2026/09/21 12:54:25 INFO Signed narinfos id=16 count=15021worker1 # [ 60.074722] nix-grpc-daemon[863]: time=2026-09-21T12:54:26.157Z level=INFO msg="Uploading 1 narinfos"5022lb # [ 60.071593] niks3-server[1047]: 2026/09/21 12:54:25 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5023lb # [ 60.080610] niks3-server[1047]: 2026/09/21 12:54:25 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5024lb # [ 60.085529] niks3-server[1047]: 2026/09/21 12:54:25 INFO Completed upload id=165025worker1 # [ 60.090680] nix-grpc-daemon[863]: time=2026-09-21T12:54:26.173Z level=INFO msg="Upload complete. (295ms)"5026lb # [ 60.088164] niks3-server[1047]: 2026/09/21 12:54:25 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5027worker1 # [ 60.096482] nix-grpc-daemon[785]: ts=2026-09-21T12:54:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15028worker2 # [ 60.087134] nix-grpc-daemon[786]: ts=2026-09-21T12:54:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:360285029worker1 # [ 60.109283] nix-daemon[850]: reaped child process 1132, status = succeeded5030worker2 # [ 60.095571] nix-grpc-daemon[786]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5031worker2 # [ 60.121514] nix-grpc-daemon[786]: ts=2026-09-21T12:54:26Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:360285032worker2 # [ 60.126526] nix-grpc-daemon[786]: ts=2026-09-21T12:54:26Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=0 paths=1 nar_bytes_out=2325033client # [ 60.232719] nix-daemon[942]: reaped child process 944, status = succeeded5034client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.17 seconds)5035client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5036client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)5037client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5038client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)5039client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5040client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5041(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.27 seconds)5042subtest: an input only one worker has still reaches the builder5043client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only5044client # this derivation will be built:5045client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5046client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5047client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.43 seconds)5048client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5049client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)5050worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5051worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds)5052worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5053worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.03 seconds)5054client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&25055client # [ 61.452155] nix-daemon[942]: accepted connection from pid 1025, user root (trusted)5056client # this derivation will be built:5057client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5058worker2 # [ 61.683674] nix-grpc-daemon[786]: ts=2026-09-21T12:54:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:360285059client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5060worker1 # [ 61.702610] nix-grpc-daemon[785]: ts=2026-09-21T12:54:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494445061worker1 # [ 61.884599] nix-daemon[850]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5062worker1 # [ 61.895600] nix-daemon[850]: reaped child process 1163, status = succeeded5063lb # [ 61.907308] niks3-server[1047]: 2026/09/21 12:54:27 INFO Received uploads request method=POST path=/api/pending_closures5064worker1 # [ 61.919621] nix-grpc-daemon[863]: time=2026-09-21T12:54:28.002Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5065worker1 # [ 61.921059] nix-grpc-daemon[863]: time=2026-09-21T12:54:28.002Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5066lb # [ 61.932988] niks3-server[1047]: 2026/09/21 12:54:27 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5067lb # [ 61.944241] niks3-server[1047]: 2026/09/21 12:54:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5068lb # [ 61.949258] niks3-server[1047]: 2026/09/21 12:54:27 INFO Signed narinfos id=17 count=15069worker1 # [ 61.957596] nix-grpc-daemon[863]: time=2026-09-21T12:54:28.040Z level=INFO msg="Uploading 1 narinfos"5070lb # [ 61.956073] niks3-server[1047]: 2026/09/21 12:54:27 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5071lb # [ 61.964452] niks3-server[1047]: 2026/09/21 12:54:27 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5072lb # [ 61.972170] niks3-server[1047]: 2026/09/21 12:54:27 INFO Completed upload id=175073worker1 # [ 61.977270] nix-grpc-daemon[863]: time=2026-09-21T12:54:28.060Z level=INFO msg="Upload complete. (275ms)"5074lb # [ 61.974972] niks3-server[1047]: 2026/09/21 12:54:27 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5075worker2 # [ 61.985523] nix-grpc-daemon[786]: ts=2026-09-21T12:54:28Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05076worker2 # [ 61.989739] nix-grpc-daemon[786]: ts=2026-09-21T12:54:28Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15077worker2 # [ 62.028373] nix-grpc-daemon[786]: ts=2026-09-21T12:54:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360285078worker2 # [ 62.041234] nix-grpc-daemon[786]: ts=2026-09-21T12:54:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360285079client # copying 2 paths to 'lb:50051'...5080client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5081worker2 # [ 62.058617] nix-grpc-daemon[786]: ts=2026-09-21T12:54:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:360285082client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5083worker2 # [ 62.067934] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5084worker2 # [ 62.257654] nix-daemon[871]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)5085worker2 # [ 62.270847] nix-daemon[871]: reaped child process 1014, status = succeeded5086worker2 # [ 62.421073] nix-daemon[871]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)5087worker2 # [ 62.433585] nix-daemon[871]: reaped child process 1019, status = succeeded5088lb # [ 62.454969] niks3-server[1047]: 2026/09/21 12:54:28 INFO Received uploads request method=POST path=/api/pending_closures5089lb # [ 62.463260] niks3-server[1047]: 2026/09/21 12:54:28 INFO Received uploads request method=POST path=/api/pending_closures5090worker2 # [ 62.461841] nix-grpc-daemon[924]: time=2026-09-21T12:54:28.679Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5091worker2 # [ 62.463426] nix-grpc-daemon[924]: time=2026-09-21T12:54:28.681Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5092worker2 # [ 62.465741] nix-grpc-daemon[924]: time=2026-09-21T12:54:28.682Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5093lb # [ 62.503044] niks3-server[1047]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5094lb # [ 62.510753] niks3-server[1047]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5095lb # [ 62.522579] niks3-server[1047]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5096lb # [ 62.526322] niks3-server[1047]: 2026/09/21 12:54:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5097lb # [ 62.528953] niks3-server[1047]: 2026/09/21 12:54:28 INFO Signed narinfos id=18 count=15098lb # [ 62.531215] niks3-server[1047]: 2026/09/21 12:54:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5099lb # [ 62.533323] niks3-server[1047]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5100lb # [ 62.535145] niks3-server[1047]: 2026/09/21 12:54:28 INFO Signed narinfos id=19 count=25101worker2 # [ 62.527689] nix-grpc-daemon[924]: time=2026-09-21T12:54:28.744Z level=INFO msg="Uploading 3 narinfos"5102lb # [ 62.547811] niks3-server[1047]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5103lb # [ 62.550257] niks3-server[1047]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5104lb # [ 62.553668] niks3-server[1047]: 2026/09/21 12:54:28 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5105lb # [ 62.557149] niks3-server[1047]: 2026/09/21 12:54:28 INFO Completed upload id=185106lb # [ 62.559501] niks3-server[1047]: 2026/09/21 12:54:28 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5107lb # [ 62.561044] niks3-server[1047]: 2026/09/21 12:54:28 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5108lb # [ 62.563417] niks3-server[1047]: 2026/09/21 12:54:28 INFO Completed upload id=195109worker2 # [ 62.554903] nix-grpc-daemon[924]: time=2026-09-21T12:54:28.772Z level=INFO msg="Upload complete. (470ms)"5110worker2 # [ 62.560162] nix-grpc-daemon[786]: ts=2026-09-21T12:54:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=0 paths=2 nar_bytes_in=9765111worker2 # [ 62.565705] nix-grpc-daemon[786]: ts=2026-09-21T12:54:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360285112worker2 # [ 62.577112] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5113worker2 # [ 62.585371] nix-daemon[871]: reaped child process 1009, status = succeeded5114worker2 # [ 62.594378] nix-grpc-daemon[786]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5115worker2 # [ 62.601988] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5116client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5117client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5118client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5119worker2 # [ 64.028144] nix-daemon[871]: reaped child process 1024, status = succeeded5120worker2 # [ 64.227498] nix-daemon[871]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)5121worker2 # [ 64.239774] nix-daemon[871]: reaped child process 1043, status = succeeded5122lb # [ 64.270975] niks3-server[1047]: 2026/09/21 12:54:30 INFO Received uploads request method=POST path=/api/pending_closures5123worker2 # [ 64.271469] nix-grpc-daemon[924]: time=2026-09-21T12:54:30.489Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5124lb # [ 64.303652] niks3-server[1047]: 2026/09/21 12:54:30 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5125lb # [ 64.311534] niks3-server[1047]: 2026/09/21 12:54:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5126lb # [ 64.317446] niks3-server[1047]: 2026/09/21 12:54:30 INFO Signed narinfos id=20 count=15127worker2 # [ 64.311703] nix-grpc-daemon[924]: time=2026-09-21T12:54:30.529Z level=INFO msg="Uploading 1 narinfos"5128lb # [ 64.320598] niks3-server[1047]: 2026/09/21 12:54:30 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5129lb # [ 64.335111] niks3-server[1047]: 2026/09/21 12:54:30 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5130lb # [ 64.339976] niks3-server[1047]: 2026/09/21 12:54:30 INFO Completed upload id=205131worker2 # [ 64.331243] nix-grpc-daemon[924]: time=2026-09-21T12:54:30.549Z level=INFO msg="Upload complete. (318ms)"5132lb # [ 64.342874] niks3-server[1047]: 2026/09/21 12:54:30 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5133worker2 # [ 64.335509] nix-grpc-daemon[786]: ts=2026-09-21T12:54:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15134client # copying 1 paths...5135worker2 # [ 64.348883] nix-daemon[871]: reaped child process 1021, status = succeeded5136worker1 # [ 64.361583] nix-grpc-daemon[785]: ts=2026-09-21T12:54:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494445137worker1 # [ 64.375200] nix-grpc-daemon[785]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5138client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5139worker2 # [ 64.398148] nix-grpc-daemon[786]: ts=2026-09-21T12:54:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:360285140worker2 # [ 64.402530] nix-grpc-daemon[786]: ts=2026-09-21T12:54:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=0 paths=1 nar_bytes_out=1205141client # [ 64.459084] nix-daemon[942]: reaped child process 1028, status = succeeded5142client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.25 seconds)5143client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25144client # [ 64.687859] nix-daemon[942]: accepted connection from pid 1047, user root (trusted)5145client # this derivation will be built:5146client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5147worker2 # [ 64.929859] nix-grpc-daemon[786]: ts=2026-09-21T12:54:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:360285148client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5149worker2 # [ 64.935563] nix-grpc-daemon[786]: ts=2026-09-21T12:54:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360285150worker2 # [ 64.940496] nix-grpc-daemon[924]: time=2026-09-21T12:54:31.157Z level=INFO msg="All 1 paths already cached"5151worker2 # [ 64.950901] nix-grpc-daemon[786]: ts=2026-09-21T12:54:31Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05152worker1 # [ 64.967765] nix-grpc-daemon[785]: ts=2026-09-21T12:54:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494445153worker2 # [ 64.955767] nix-grpc-daemon[786]: ts=2026-09-21T12:54:31Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15154worker1 # [ 64.980063] nix-grpc-daemon[785]: ts=2026-09-21T12:54:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494445155client # copying 1 paths to 'lb:50051'...5156client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5157worker1 # [ 64.997168] nix-grpc-daemon[785]: ts=2026-09-21T12:54:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494445158worker1 # [ 65.008411] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5159worker1 # [ 65.025313] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)5160worker1 # [ 65.085185] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)5161worker1 # [ 65.099876] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 3/5)5162worker1 # [ 65.267878] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 344 ms (attempt 4/5)5163worker1 # [ 65.618741] nix-grpc-daemon[785]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5164worker1 # [ 65.622894] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5165worker1 # [ 65.627937] nix-grpc-daemon[785]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5166worker1 # [ 65.830579] nix-daemon[850]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5167worker1 # [ 65.843741] nix-daemon[850]: reaped child process 1176, status = succeeded5168worker1 # [ 66.002107] nix-daemon[850]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5169worker1 # [ 66.014303] nix-daemon[850]: reaped child process 1181, status = succeeded5170lb # [ 66.025955] niks3-server[1047]: 2026/09/21 12:54:31 INFO Received uploads request method=POST path=/api/pending_closures5171worker1 # [ 66.041158] nix-grpc-daemon[863]: time=2026-09-21T12:54:32.123Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5172worker1 # [ 66.042542] nix-grpc-daemon[863]: time=2026-09-21T12:54:32.123Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5173lb # [ 66.051251] niks3-server[1047]: 2026/09/21 12:54:31 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5174lb # [ 66.063531] niks3-server[1047]: 2026/09/21 12:54:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5175worker1 # [ 66.073513] nix-grpc-daemon[863]: time=2026-09-21T12:54:32.156Z level=INFO msg="Uploading 1 narinfos"5176lb # [ 66.068670] niks3-server[1047]: 2026/09/21 12:54:31 INFO Signed narinfos id=21 count=15177lb # [ 66.075099] niks3-server[1047]: 2026/09/21 12:54:31 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5178lb # [ 66.084354] niks3-server[1047]: 2026/09/21 12:54:31 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5179worker1 # [ 66.096091] nix-grpc-daemon[863]: time=2026-09-21T12:54:32.179Z level=INFO msg="Upload complete. (447ms)"5180lb # [ 66.090960] niks3-server[1047]: 2026/09/21 12:54:31 INFO Completed upload id=215181worker1 # [ 66.098592] nix-grpc-daemon[785]: ts=2026-09-21T12:54:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 paths=1 nar_bytes_in=5605182lb # [ 66.094188] niks3-server[1047]: 2026/09/21 12:54:31 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5183worker1 # [ 66.102917] nix-grpc-daemon[785]: ts=2026-09-21T12:54:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494445184worker1 # [ 66.108668] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5185worker1 # [ 66.111445] nix-daemon[850]: reaped child process 1166, status = succeeded5186worker1 # [ 66.116500] nix-daemon[850]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5187client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5188client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5189client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5190worker1 # [ 67.542196] nix-daemon[850]: reaped child process 1185, status = succeeded5191worker1 # [ 67.717785] nix-daemon[850]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5192worker1 # [ 67.729949] nix-daemon[850]: reaped child process 1204, status = succeeded5193lb # [ 67.742881] niks3-server[1047]: 2026/09/21 12:54:33 INFO Received uploads request method=POST path=/api/pending_closures5194worker1 # [ 67.757379] nix-grpc-daemon[863]: time=2026-09-21T12:54:33.840Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5195lb # [ 67.774546] niks3-server[1047]: 2026/09/21 12:54:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5196lb # [ 67.779240] niks3-server[1047]: 2026/09/21 12:54:33 INFO Signed narinfos id=22 count=15197worker1 # [ 67.787512] nix-grpc-daemon[863]: time=2026-09-21T12:54:33.870Z level=INFO msg="Uploading 1 narinfos"5198lb # [ 67.787553] niks3-server[1047]: 2026/09/21 12:54:33 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5199lb # [ 67.796805] niks3-server[1047]: 2026/09/21 12:54:33 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5200lb # [ 67.804883] niks3-server[1047]: 2026/09/21 12:54:33 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5201lb # [ 67.812862] niks3-server[1047]: 2026/09/21 12:54:33 INFO Completed upload id=225202worker1 # [ 67.820764] nix-grpc-daemon[863]: time=2026-09-21T12:54:33.903Z level=INFO msg="Upload complete. (295ms)"5203lb # [ 67.815745] niks3-server[1047]: 2026/09/21 12:54:33 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5204client # copying 1 paths...5205worker1 # [ 67.826544] nix-grpc-daemon[785]: ts=2026-09-21T12:54:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15206worker2 # [ 67.819261] nix-grpc-daemon[786]: ts=2026-09-21T12:54:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:360285207worker1 # [ 67.838623] nix-daemon[850]: reaped child process 1183, status = succeeded5208worker2 # [ 67.828883] nix-grpc-daemon[786]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5209client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5210worker1 # [ 67.872160] nix-grpc-daemon[785]: ts=2026-09-21T12:54:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:494445211worker1 # [ 67.876637] nix-grpc-daemon[785]: ts=2026-09-21T12:54:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_out=1205212client # [ 67.950095] nix-daemon[942]: reaped child process 1050, status = succeeded5213client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.49 seconds)5214client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25215client # [ 68.182705] nix-daemon[942]: accepted connection from pid 1069, user root (trusted)5216client # this derivation will be built:5217client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5218worker1 # [ 68.431167] nix-grpc-daemon[785]: ts=2026-09-21T12:54:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:494445219client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux'...5220worker1 # [ 68.438501] nix-grpc-daemon[785]: ts=2026-09-21T12:54:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494445221worker1 # [ 68.447339] nix-grpc-daemon[863]: time=2026-09-21T12:54:34.530Z level=INFO msg="All 1 paths already cached"5222worker2 # [ 68.447686] nix-grpc-daemon[786]: ts=2026-09-21T12:54:34Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05223worker2 # [ 68.453413] nix-grpc-daemon[786]: ts=2026-09-21T12:54:34Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15224worker2 # [ 68.492244] nix-grpc-daemon[786]: ts=2026-09-21T12:54:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360285225lb # [ 68.510832] envoy[909]: conn peer=[2001:db8:1::1]:58748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=575226lb # [ 68.515219] envoy[909]: conn peer=[2001:db8:1::1]:58734 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4327 tx=2277 ms=23945227worker2 # [ 68.506647] nix-grpc-daemon[786]: ts=2026-09-21T12:54:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360285228lb # [ 68.518724] envoy[909]: conn peer=[2001:db8:1::1]:58762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3879 tx=1975 ms=27315229lb # [ 68.523254] envoy[909]: conn peer=[2001:db8:1::1]:43236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=175230lb # [ 68.526665] envoy[909]: conn peer=[2001:db8:1::1]:43244 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2012 ms=29455231lb # [ 68.530150] envoy[909]: conn peer=[2001:db8:1::1]:43248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=165232worker2 # [ 68.532532] nix-grpc-daemon[924]: time=2026-09-21T12:54:34.749Z level=INFO msg="All 1 paths already cached"5233client # copying 1 paths to 'lb:50051'...5234client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5235worker2 # [ 68.541629] nix-grpc-daemon[786]: ts=2026-09-21T12:54:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:360285236worker2 # [ 68.552690] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5237worker2 # [ 68.756830] nix-daemon[871]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)5238worker2 # [ 68.769718] nix-daemon[871]: reaped child process 1055, status = succeeded5239worker2 # [ 68.946283] nix-daemon[871]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)5240worker2 # [ 68.959128] nix-daemon[871]: reaped child process 1060, status = succeeded5241lb # [ 68.984677] niks3-server[1047]: 2026/09/21 12:54:34 INFO Received uploads request method=POST path=/api/pending_closures5242worker2 # [ 68.983854] nix-grpc-daemon[924]: time=2026-09-21T12:54:35.201Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5243worker2 # [ 68.985267] nix-grpc-daemon[924]: time=2026-09-21T12:54:35.201Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5244lb # [ 69.009560] niks3-server[1047]: 2026/09/21 12:54:34 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5245lb # [ 69.022325] niks3-server[1047]: 2026/09/21 12:54:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5246worker2 # [ 69.019611] nix-grpc-daemon[924]: time=2026-09-21T12:54:35.237Z level=INFO msg="Uploading 1 narinfos"5247lb # [ 69.028707] niks3-server[1047]: 2026/09/21 12:54:34 INFO Signed narinfos id=23 count=15248lb # [ 69.033417] niks3-server[1047]: 2026/09/21 12:54:34 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5249lb # [ 69.045140] niks3-server[1047]: 2026/09/21 12:54:34 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5250worker2 # [ 69.042763] nix-grpc-daemon[924]: time=2026-09-21T12:54:35.260Z level=INFO msg="Upload complete. (478ms)"5251lb # [ 69.051555] niks3-server[1047]: 2026/09/21 12:54:34 INFO Completed upload id=235252lb # [ 69.053852] niks3-server[1047]: 2026/09/21 12:54:34 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5253worker2 # [ 69.047140] nix-grpc-daemon[786]: ts=2026-09-21T12:54:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=0 paths=1 nar_bytes_in=5605254worker2 # [ 69.052107] nix-grpc-daemon[786]: ts=2026-09-21T12:54:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360285255worker2 # [ 69.061926] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5256worker2 # [ 69.064161] nix-daemon[871]: reaped child process 1050, status = succeeded5257worker2 # [ 69.069587] nix-daemon[871]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5258client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5259client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5260client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5261worker2 # [ 70.093502] nix-daemon[871]: reaped child process 1064, status = succeeded5262worker2 # [ 70.276314] nix-daemon[871]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)5263worker2 # [ 70.288245] nix-daemon[871]: reaped child process 1083, status = succeeded5264lb # [ 70.314903] niks3-server[1047]: 2026/09/21 12:54:36 INFO Received uploads request method=POST path=/api/pending_closures5265worker2 # [ 70.314590] nix-grpc-daemon[924]: time=2026-09-21T12:54:36.532Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5266lb # [ 70.343159] niks3-server[1047]: 2026/09/21 12:54:36 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5267lb # [ 70.352178] niks3-server[1047]: 2026/09/21 12:54:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5268worker2 # [ 70.350551] nix-grpc-daemon[924]: time=2026-09-21T12:54:36.568Z level=INFO msg="Uploading 1 narinfos"5269lb # [ 70.359125] niks3-server[1047]: 2026/09/21 12:54:36 INFO Signed narinfos id=24 count=15270lb # [ 70.363786] niks3-server[1047]: 2026/09/21 12:54:36 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5271lb # [ 70.376828] niks3-server[1047]: 2026/09/21 12:54:36 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5272lb # [ 70.382402] niks3-server[1047]: 2026/09/21 12:54:36 INFO Completed upload id=245273worker2 # [ 70.373576] nix-grpc-daemon[924]: time=2026-09-21T12:54:36.591Z level=INFO msg="Upload complete. (297ms)"5274lb # [ 70.385166] niks3-server[1047]: 2026/09/21 12:54:36 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5275worker2 # [ 70.379515] nix-grpc-daemon[786]: ts=2026-09-21T12:54:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36028 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15276worker2 # [ 70.392068] nix-daemon[871]: reaped child process 1062, status = succeeded5277client # copying 1 paths...5278worker1 # [ 70.445580] nix-grpc-daemon[785]: ts=2026-09-21T12:54:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494445279worker1 # [ 70.457199] nix-grpc-daemon[785]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5280client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5281worker1 # [ 70.491367] nix-grpc-daemon[785]: ts=2026-09-21T12:54:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:494445282worker1 # [ 70.496100] nix-grpc-daemon[785]: ts=2026-09-21T12:54:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_out=1205283client # [ 70.610790] nix-daemon[942]: reaped child process 1072, status = succeeded5284client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.66 seconds)5285(finished: subtest: an input only one worker has still reaches the builder, in 10.29 seconds)5286subtest: upload whose reference the worker lacks is completed from the cache5287client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5288client # this derivation will be built:5289client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5290client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5291client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.48 seconds)5292client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5293client # this derivation will be built:5294client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5295client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5296client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.48 seconds)5297client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5298client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.23 seconds)5299worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5300worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds)5301worker2 # [ 72.118589] systemd[1]: Stopping Nix worker-protocol over gRPC...5302worker2 # [ 72.867801] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=signal_exit5303worker2 # [ 73.125963] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=scheduler_yield5304worker2 # [ 73.130252] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=scheduler_restarting addr=lb:500515305lb # [ 73.138128] niks3-server[1047]: 2026/09/21 12:54:39 INFO lead: released remote=192.168.1.4:417305306worker2 # [ 73.137227] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:349605307worker1 # [ 73.152556] nix-grpc-daemon[785]: ts=2026-09-21T12:54:39Z level=info event=scheduler_restarting addr=lb:500515308worker2 # [ 73.141906] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=worker_gone addr=192.168.1.4:500515309worker2 # [ 73.144921] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:349605310worker2 # [ 73.148243] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=worker_gone addr=192.168.1.3:500515311worker2 # [ 73.331360] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=scheduler_connected addr=lb:500515312worker2 # [ 73.335160] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5313worker1 # [ 73.352634] nix-grpc-daemon[785]: ts=2026-09-21T12:54:39Z level=info event=scheduler_connected addr=lb:500515314worker1 # [ 73.356596] nix-grpc-daemon[785]: ts=2026-09-21T12:54:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5315worker2 # [ 73.536397] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=scheduler_connected addr=lb:500515316worker2 # [ 73.540235] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5317worker1 # [ 73.557556] nix-grpc-daemon[785]: ts=2026-09-21T12:54:39Z level=info event=scheduler_connected addr=lb:500515318worker1 # [ 73.561478] nix-grpc-daemon[785]: ts=2026-09-21T12:54:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5319worker2 # [ 73.740773] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=scheduler_connected addr=lb:500515320worker2 # [ 73.744596] nix-grpc-daemon[786]: ts=2026-09-21T12:54:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5321worker1 # [ 73.762387] nix-grpc-daemon[785]: ts=2026-09-21T12:54:39Z level=info event=scheduler_connected addr=lb:500515322worker1 # [ 73.766254] nix-grpc-daemon[785]: ts=2026-09-21T12:54:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5323worker2 # [ 73.946936] nix-grpc-daemon[786]: ts=2026-09-21T12:54:40Z level=info event=scheduler_connected addr=lb:500515324worker2 # [ 73.950196] nix-grpc-daemon[786]: ts=2026-09-21T12:54:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5325worker1 # [ 73.966809] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_connected addr=lb:500515326worker1 # [ 73.970701] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5327worker2 # [ 74.127167] nix-grpc-daemon[786]: ts=2026-09-21T12:54:40Z level=info event=scheduler_restarting5328worker1 # [ 74.172391] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_connected addr=lb:500515329worker1 # [ 74.176407] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5330worker2 # [ 74.163374] nix-grpc-daemon[786]: ts=2026-09-21T12:54:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5331worker2 # [ 74.167921] nix-daemon[871]: reaped child process 873, status = succeeded5332worker1 # [ 74.377408] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_connected addr=lb:500515333worker1 # [ 74.381297] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5334worker1 # [ 74.582177] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_connected addr=lb:500515335worker1 # [ 74.586233] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5336worker1 # [ 74.787116] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_connected addr=lb:500515337worker1 # [ 74.791107] nix-grpc-daemon[785]: ts=2026-09-21T12:54:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5338worker1 # [ 74.991813] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_connected addr=lb:500515339worker1 # [ 74.995790] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5340worker2 # [ 75.022872] nix-grpc-daemon[786]: E0000 00:00:1789995281.239892 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5341worker2 # [ 75.027759] nix-grpc-daemon[786]: E0000 00:00:1789995281.240074 831 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5342worker1 # [ 75.197470] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_connected addr=lb:500515343worker1 # [ 75.201230] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5344worker1 # [ 75.402096] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_connected addr=lb:500515345worker1 # [ 75.405992] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5346worker1 # [ 75.606868] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_connected addr=lb:500515347worker1 # [ 75.610837] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5348worker1 # [ 75.756093] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_take_over5349lb # [ 75.750993] niks3-server[1047]: 2026/09/21 12:54:41 INFO lead: acquired remote=192.168.1.3:479125350worker1 # [ 75.811683] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_connected addr=lb:500515351worker1 # [ 75.815549] nix-grpc-daemon[785]: ts=2026-09-21T12:54:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5352worker1 # [ 76.016484] nix-grpc-daemon[785]: ts=2026-09-21T12:54:42Z level=info event=scheduler_connected addr=lb:500515353worker1 # [ 76.027386] nix-grpc-daemon[785]: ts=2026-09-21T12:54:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:565105354worker1 # [ 76.032176] nix-grpc-daemon[785]: ts=2026-09-21T12:54:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05355worker2 # [ 76.136071] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5356worker2 # [ 76.139731] systemd[1]: Stopped Nix worker-protocol over gRPC.5357worker2 # [ 76.143424] systemd[1]: nix-grpc-daemon.service: Consumed 1.486s CPU time over 57.518s wall clock time, 53.1M memory peak, 57.1K incoming IP traffic, 175.2K outgoing IP traffic.5358worker2 # [ 76.154636] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5359worker2 # [ 76.157947] systemd[1]: Closed Nix worker-protocol over gRPC.5360worker2 # [ 76.160591] systemd[1]: nix-grpc-daemon.socket: Consumed 295us CPU time over 1min 5.900s wall clock time, 256K memory peak, 74.8K incoming IP traffic, 63.9K outgoing IP traffic.5361lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5362lb: must succeed: curl -sf localhost:9901/clusters5363lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5364lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5365client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25366worker1 # [ 76.519987] nix-grpc-daemon[785]: ts=2026-09-21T12:54:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494445367client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5368worker1 # [ 76.526343] nix-grpc-daemon[785]: ts=2026-09-21T12:54:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494445369client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.29 seconds)5370worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5371worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5372worker2 # [ 76.628177] systemd[1]: Listening on Nix worker-protocol over gRPC.5373worker1 # [ 76.704679] systemd[1]: Stopping Nix worker-protocol over gRPC...5374worker1 # [ 76.836528] nix-grpc-daemon[785]: ts=2026-09-21T12:54:42Z level=info event=signal_exit5375worker1 # [ 77.090725] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_yield5376lb # [ 77.088575] niks3-server[1047]: 2026/09/21 12:54:42 INFO lead: released remote=192.168.1.3:479125377worker1 # [ 77.094746] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_restarting addr=lb:500515378worker1 # [ 77.099353] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:565105379worker1 # [ 77.102582] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=worker_gone addr=192.168.1.3:500515380worker2 # [ 77.123831] systemd[1]: Starting scheduler-token.service...5381worker2 # [ 77.227142] systemd[1]: scheduler-token.service: Deactivated successfully.5382worker2 # [ 77.229312] systemd[1]: Finished scheduler-token.service.5383worker2 # [ 77.231976] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 103ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5384worker2 # [ 77.238379] systemd[1]: Starting Nix worker-protocol over gRPC...5385worker1 # [ 77.295596] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_connected addr=lb:500515386worker1 # [ 77.299442] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5387worker2 # [ 77.407461] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5388worker2 # [ 77.409269] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5389worker2 # [ 77.413237] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15390worker2 # [ 77.420078] nix-grpc-daemon[1107]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5391worker2 # [ 77.421622] nix-grpc-daemon[1107]: E0000 00:00:1789995283.637866 1107 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5392worker2 # [ 77.428108] nix-grpc-daemon[1107]: E0000 00:00:1789995283.643448 1107 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5393worker2 # [ 77.431962] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5394worker2 # [ 77.434976] systemd[1]: Started Nix worker-protocol over gRPC.5395lb # [ 77.451342] niks3-server[1047]: 2026/09/21 12:54:43 INFO lead: acquired remote=192.168.1.4:371405396worker2 # [ 77.443069] nix-grpc-daemon[1107]: E0000 00:00:1789995283.660148 1147 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5397worker2 # [ 77.447058] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:43Z level=info event=scheduler_take_over5398worker2 # [ 77.455809] nix-grpc-daemon[1107]: E0000 00:00:1789995283.673700 1147 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5399worker2 # [ 77.460055] nix-grpc-daemon[1107]: E0000 00:00:1789995283.675997 1147 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5400worker2 # [ 77.470047] nix-grpc-daemon[1107]: E0000 00:00:1789995283.687922 1147 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5401worker2 # [ 77.479151] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:43Z level=info event=scheduler_connected addr=lb:500515402worker2 # [ 77.480755] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5403worker1 # [ 77.500182] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_connected addr=lb:500515404worker1 # [ 77.504059] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5405worker2 # [ 77.682422] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:43Z level=info event=scheduler_connected addr=lb:500515406worker2 # [ 77.686375] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker1 # [ 77.706385] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_connected addr=lb:500515408worker1 # [ 77.709485] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5409worker2 # [ 77.887515] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=scheduler_connected addr=lb:500515410worker2 # [ 77.891498] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5411worker1 # [ 77.907321] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_connected addr=lb:500515412worker1 # [ 77.911064] nix-grpc-daemon[785]: ts=2026-09-21T12:54:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5413worker1 # [ 78.095212] nix-grpc-daemon[785]: ts=2026-09-21T12:54:44Z level=info event=scheduler_restarting5414worker2 # [ 78.092747] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=scheduler_connected addr=lb:500515415worker2 # [ 78.096586] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5416worker1 # [ 78.127772] nix-grpc-daemon[785]: ts=2026-09-21T12:54:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5417worker1 # [ 78.139167] nix-daemon[850]: reaped child process 852, status = succeeded5418worker2 # [ 78.297732] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=scheduler_connected addr=lb:500515419worker2 # [ 78.301619] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5420lb # [ 78.510300] envoy[909]: conn peer=[2001:db8:1::1]:43264 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3742 tx=2015 ms=21175421worker2 # [ 78.503366] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=scheduler_connected addr=lb:500515422lb # [ 78.515578] envoy[909]: conn peer=[2001:db8:1::1]:43274 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575423lb # [ 78.518978] envoy[909]: conn peer=[2001:db8:1::4]:57056 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=53647 tx=2689 ms=552225424lb # [ 78.523722] envoy[909]: conn peer=[2001:db8:1::1]:48106 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=185425lb # [ 78.527987] envoy[909]: conn peer=192.168.1.3:50486 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=17495 tx=3502 ms=592235426worker2 # [ 78.527453] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=oidc_jwks provider=mock keys=15427worker2 # [ 78.530517] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:434405428worker2 # [ 78.534464] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:44Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05429worker1 # [ 78.968626] nix-grpc-daemon[785]: E0000 00:00:1789995285.050738 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5430worker1 # [ 78.973525] nix-grpc-daemon[785]: E0000 00:00:1789995285.050974 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5431worker1 # [ 80.082670] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5432worker1 # [ 80.085744] systemd[1]: Stopped Nix worker-protocol over gRPC.5433worker1 # [ 80.089561] systemd[1]: nix-grpc-daemon.service: Consumed 2.699s CPU time over 1min 1.571s wall clock time, 61.7M memory peak, 89.7K incoming IP traffic, 169.7K outgoing IP traffic.5434worker1 # [ 80.101432] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5435worker1 # [ 80.105657] systemd[1]: Closed Nix worker-protocol over gRPC.5436worker1 # [ 80.109721] systemd[1]: nix-grpc-daemon.socket: Consumed 302us CPU time over 1min 10.716s wall clock time, 256K memory peak, 82K incoming IP traffic, 68.9K outgoing IP traffic.5437lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5438lb: must succeed: curl -sf localhost:9901/clusters5439lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5440lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5441client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25442worker2 # [ 80.441105] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434525443worker2 # [ 80.454389] nix-daemon[871]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5444worker2 # [ 80.471598] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5)5445worker2 # [ 80.529391] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 127 ms (attempt 2/5)5446worker2 # [ 80.662304] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 308 ms (attempt 3/5)5447worker2 # [ 80.976147] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 715 ms (attempt 4/5)5448worker2 # [ 81.698263] nix-grpc-daemon[1107]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5449worker2 # [ 81.702343] nix-grpc-daemon[1107]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5450client # copying 2 paths...5451client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5452worker2 # [ 81.725145] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:434525453client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5454worker2 # [ 81.735890] nix-daemon[871]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5455worker2 # [ 81.757249] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:47Z level=info event=niks3_push_started pid=11675456worker2 # [ 81.965137] nix-grpc-daemon[1167]: time=2026-09-21T12:54:48.182Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5457worker2 # [ 82.153092] nix-daemon[871]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)5458worker2 # [ 82.165479] nix-daemon[871]: reaped child process 1183, status = succeeded5459lb # [ 82.196157] niks3-server[1047]: 2026/09/21 12:54:48 INFO Received uploads request method=POST path=/api/pending_closures5460lb # [ 82.203978] niks3-server[1047]: 2026/09/21 12:54:48 INFO Received uploads request method=POST path=/api/pending_closures5461worker2 # [ 82.202762] nix-grpc-daemon[1167]: time=2026-09-21T12:54:48.420Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5462worker2 # [ 82.204329] nix-grpc-daemon[1167]: time=2026-09-21T12:54:48.422Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5463worker2 # [ 82.206183] nix-grpc-daemon[1167]: time=2026-09-21T12:54:48.424Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5464lb # [ 82.252729] niks3-server[1047]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5465lb # [ 82.267403] niks3-server[1047]: 2026/09/21 12:54:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5466lb # [ 82.274935] niks3-server[1047]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5467lb # [ 82.280317] niks3-server[1047]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5468lb # [ 82.286852] niks3-server[1047]: 2026/09/21 12:54:48 INFO Signed narinfos id=25 count=15469lb # [ 82.289459] niks3-server[1047]: 2026/09/21 12:54:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5470lb # [ 82.291539] niks3-server[1047]: 2026/09/21 12:54:48 INFO Signed narinfos id=26 count=25471worker2 # [ 82.284447] nix-grpc-daemon[1167]: time=2026-09-21T12:54:48.502Z level=INFO msg="Uploading 3 narinfos"5472lb # [ 82.297510] niks3-server[1047]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5473lb # [ 82.309898] niks3-server[1047]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5474lb # [ 82.316129] niks3-server[1047]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5475lb # [ 82.318903] niks3-server[1047]: 2026/09/21 12:54:48 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5476lb # [ 82.323727] niks3-server[1047]: 2026/09/21 12:54:48 INFO Completed upload id=255477lb # [ 82.325141] niks3-server[1047]: 2026/09/21 12:54:48 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5478lb # [ 82.327474] niks3-server[1047]: 2026/09/21 12:54:48 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5479lb # [ 82.330668] niks3-server[1047]: 2026/09/21 12:54:48 INFO Completed upload id=265480worker2 # [ 82.322375] nix-grpc-daemon[1167]: time=2026-09-21T12:54:48.540Z level=INFO msg="Upload complete. (353ms)"5481worker2 # [ 82.328249] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43452 duration_s=0 paths=2 nar_bytes_in=3125482worker2 # [ 82.340306] nix-daemon[871]: reaped child process 1165, status = succeeded5483client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 2.18 seconds)5484worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5485worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5486worker1 # [ 82.482053] systemd[1]: Listening on Nix worker-protocol over gRPC.5487worker1 # [ 82.486911] systemd[1]: Starting Nix worker-protocol over gRPC...5488worker1 # [ 82.682037] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5489worker1 # [ 82.683981] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5490worker1 # [ 82.688210] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15491worker1 # [ 82.696065] nix-grpc-daemon[1228]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5492worker1 # [ 82.697516] nix-grpc-daemon[1228]: E0000 00:00:1789995288.778629 1228 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5493worker1 # [ 82.703974] nix-grpc-daemon[1228]: E0000 00:00:1789995288.784077 1228 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5494worker1 # [ 82.707173] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5495worker1 # [ 82.709941] systemd[1]: Started Nix worker-protocol over gRPC.5496lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5497lb: must succeed: curl -sf localhost:9901/clusters5498worker1 # [ 82.717107] nix-grpc-daemon[1228]: E0000 00:00:1789995288.798946 1267 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5499worker1 # [ 82.729213] nix-grpc-daemon[1228]: E0000 00:00:1789995288.812306 1267 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5500worker1 # [ 82.734103] nix-grpc-daemon[1228]: E0000 00:00:1789995288.814717 1267 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5501worker1 # [ 82.744941] nix-grpc-daemon[1228]: E0000 00:00:1789995288.828018 1267 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5502worker1 # [ 82.755559] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:48Z level=info event=scheduler_connected addr=lb:500515503worker2 # [ 82.744399] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:434405504worker2 # [ 82.748670] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05505lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5506lb: must succeed: curl -sf localhost:9901/clusters5507lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5508lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.17 seconds)5509(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 13.23 seconds)5510subtest: scheduler down: the other node takes the lock and keeps it5511lb: must succeed: curl -sf localhost:9901/clusters5512lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5513lb: must succeed: curl -sf localhost:9901/clusters5514lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5515lb: must succeed: curl -sf localhost:9901/clusters5516lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5517lb: must succeed: curl -sf localhost:9901/clusters5518lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5519worker2 # [ 84.134726] systemd[1]: Stopping Nix worker-protocol over gRPC...5520worker2 # [ 84.430744] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:50Z level=info event=signal_exit5521worker2 # [ 84.649199] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:50Z level=info event=scheduler_yield5522lb # [ 84.661792] niks3-server[1047]: 2026/09/21 12:54:50 INFO lead: released remote=192.168.1.4:371405523worker1 # [ 84.667410] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:50Z level=info event=scheduler_restarting addr=lb:500515524worker2 # [ 84.653599] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:434405525worker2 # [ 84.656871] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:50Z level=info event=worker_gone addr=192.168.1.3:500515526worker2 # [ 84.662502] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:50Z level=info event=scheduler_restarting addr=lb:500515527worker2 # [ 84.665513] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:434405528worker2 # [ 84.668716] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:50Z level=info event=worker_gone addr=192.168.1.4:500515529worker1 # [ 84.867283] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:50Z level=info event=scheduler_connected addr=lb:500515530worker1 # [ 84.871642] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5531worker2 # [ 84.862862] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=scheduler_connected addr=lb:500515532worker2 # [ 84.866746] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5533worker1 # [ 85.072665] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_connected addr=lb:500515534worker1 # [ 85.076962] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5535worker2 # [ 85.067988] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=scheduler_connected addr=lb:500515536worker2 # [ 85.072091] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5537worker1 # [ 85.277911] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_connected addr=lb:500515538worker1 # [ 85.282369] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5539worker2 # [ 85.273268] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=scheduler_connected addr=lb:500515540worker2 # [ 85.277130] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5541worker1 # [ 85.483123] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_connected addr=lb:500515542worker1 # [ 85.487483] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5543worker2 # [ 85.479277] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=scheduler_connected addr=lb:500515544worker2 # [ 85.482532] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5545worker2 # [ 85.654114] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=scheduler_restarting5546worker2 # [ 85.676247] nix-daemon[871]: reaped child process 1159, status = succeeded5547worker1 # [ 85.689849] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_connected addr=lb:500515548worker1 # [ 85.693504] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5549worker2 # [ 85.690783] nix-grpc-daemon[1107]: ts=2026-09-21T12:54:51Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5550worker1 # [ 85.894560] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_connected addr=lb:500515551worker1 # [ 85.898832] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5552worker1 # [ 86.099480] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:52Z level=info event=scheduler_connected addr=lb:500515553worker1 # [ 86.103806] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5554worker1 # [ 86.304657] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:52Z level=info event=scheduler_connected addr=lb:500515555worker1 # [ 86.308929] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5556worker2 # [ 86.457267] nix-grpc-daemon[1107]: E0000 00:00:1789995292.674919 1152 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5557worker2 # [ 86.462524] nix-grpc-daemon[1107]: E0000 00:00:1789995292.675445 1152 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5558worker1 # [ 86.509736] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:52Z level=info event=scheduler_connected addr=lb:500515559worker1 # [ 86.513977] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5560worker1 # [ 86.715568] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:52Z level=info event=scheduler_connected addr=lb:500515561worker1 # [ 86.719516] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5562worker1 # [ 86.920427] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_connected addr=lb:500515563worker1 # [ 86.924600] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5564worker1 # [ 87.125396] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_connected addr=lb:500515565worker1 # [ 87.129757] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5566worker1 # [ 87.330544] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_connected addr=lb:500515567worker1 # [ 87.334856] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5568worker2 # [ 87.456644] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5569worker2 # [ 87.461369] systemd[1]: Stopped Nix worker-protocol over gRPC.5570worker2 # [ 87.463455] systemd[1]: nix-grpc-daemon.service: Consumed 383ms CPU time over 10.216s wall clock time, 33.1M memory peak, 20.3K incoming IP traffic, 31.5K outgoing IP traffic.5571worker2 # [ 87.474261] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5572worker2 # [ 87.477633] systemd[1]: Closed Nix worker-protocol over gRPC.5573worker2 # [ 87.480250] systemd[1]: nix-grpc-daemon.socket: Consumed 291us CPU time over 10.852s wall clock time, 256K memory peak, 21K incoming IP traffic, 15.8K outgoing IP traffic.5574worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5575worker1 # [ 87.535734] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_connected addr=lb:500515576worker1 # [ 87.540176] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5577worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5578worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5579worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5580lb # [ 87.710563] niks3-server[1047]: 2026/09/21 12:54:53 INFO lead: acquired remote=192.168.1.3:596265581worker1 # [ 87.719852] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_take_over5582worker1 # [ 87.740743] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_connected addr=lb:500515583worker1 # [ 87.744842] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5584worker1 # [ 87.945774] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:54Z level=info event=scheduler_connected addr=lb:500515585worker1 # [ 87.950068] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5586worker1 # [ 88.150981] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:54Z level=info event=scheduler_connected addr=lb:500515587worker1 # [ 88.158896] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:491645588worker1 # [ 88.163288] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05589lb # [ 88.511515] envoy[909]: conn peer=[2001:db8:1::1]:48110 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1004 tx=282 ms=19005590lb # [ 88.516387] envoy[909]: conn peer=[2001:db8:1::4]:40458 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8603 tx=512 ms=82065591worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5592worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5593client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25594worker1 # [ 88.988094] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:491745595worker1 # [ 89.001323] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5596client # don't know how to build these paths:5597client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5598worker1 # [ 89.017684] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491745599worker1 # [ 89.030749] nix-grpc-daemon[1228]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5)5600worker1 # [ 89.100125] nix-grpc-daemon[1228]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 196 ms (attempt 2/5)5601worker1 # [ 89.302631] nix-grpc-daemon[1228]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 178 ms (attempt 3/5)5602worker1 # [ 89.487135] nix-grpc-daemon[1228]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 4/5)5603worker1 # [ 89.512794] nix-grpc-daemon[1228]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5604worker1 # [ 89.517058] nix-grpc-daemon[1228]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5605worker1 # [ 89.549559] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:55Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05606worker1 # [ 89.555908] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:55Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15607worker1 # [ 89.600438] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491745608worker1 # [ 89.614622] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491745609client # copying 1 paths to 'lb:50051'...5610client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5611worker1 # [ 89.628317] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491745612worker1 # [ 89.640291] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5613worker1 # [ 89.662572] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:55Z level=info event=niks3_push_started pid=13025614worker1 # [ 89.900307] nix-grpc-daemon[1302]: time=2026-09-21T12:54:55.982Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""5615worker1 # [ 90.108947] nix-daemon[850]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5616worker1 # [ 90.119956] nix-daemon[850]: reaped child process 1318, status = succeeded5617worker1 # [ 90.343036] nix-daemon[850]: accepted connection from pid 1320, user nix-grpc-daemon (trusted)5618worker1 # [ 90.355067] nix-daemon[850]: reaped child process 1323, status = succeeded5619lb # [ 90.371501] niks3-server[1047]: 2026/09/21 12:54:56 INFO Received uploads request method=POST path=/api/pending_closures5620worker1 # [ 90.391202] nix-grpc-daemon[1302]: time=2026-09-21T12:54:56.474Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5621worker1 # [ 90.392776] nix-grpc-daemon[1302]: time=2026-09-21T12:54:56.475Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5622lb # [ 90.418644] niks3-server[1047]: 2026/09/21 12:54:56 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5623lb # [ 90.423810] niks3-server[1047]: 2026/09/21 12:54:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5624lb # [ 90.429812] niks3-server[1047]: 2026/09/21 12:54:56 INFO Signed narinfos id=27 count=15625worker1 # [ 90.439462] nix-grpc-daemon[1302]: time=2026-09-21T12:54:56.522Z level=INFO msg="Uploading 1 narinfos"5626lb # [ 90.442546] niks3-server[1047]: 2026/09/21 12:54:56 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5627lb # [ 90.448882] niks3-server[1047]: 2026/09/21 12:54:56 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5628lb # [ 90.460285] niks3-server[1047]: 2026/09/21 12:54:56 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5629lb # [ 90.468052] niks3-server[1047]: 2026/09/21 12:54:56 INFO Completed upload id=275630worker1 # [ 90.473318] nix-grpc-daemon[1302]: time=2026-09-21T12:54:56.556Z level=INFO msg="Upload complete. (568ms)"5631worker1 # [ 90.479216] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49174 duration_s=0 paths=1 nar_bytes_in=5205632worker1 # [ 90.485491] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491745633worker1 # [ 90.497114] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5634worker1 # [ 90.504360] nix-daemon[850]: reaped child process 1300, status = succeeded5635worker1 # [ 90.512719] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5636client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5637client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5638client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5639client # farm-a> LOG-farm-a-on-standby5640worker1 # [ 91.526254] nix-daemon[850]: reaped child process 1327, status = succeeded5641worker1 # [ 91.718361] nix-daemon[850]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5642worker1 # [ 91.730848] nix-daemon[850]: reaped child process 1346, status = succeeded5643lb # [ 91.744812] niks3-server[1047]: 2026/09/21 12:54:57 INFO Received uploads request method=POST path=/api/pending_closures5644worker1 # [ 91.792828] nix-grpc-daemon[1302]: time=2026-09-21T12:54:57.875Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5645worker1 # [ 91.795676] nix-grpc-daemon[1302]: time=2026-09-21T12:54:57.877Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5646lb # [ 91.811167] niks3-server[1047]: 2026/09/21 12:54:57 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5647lb # [ 91.829217] niks3-server[1047]: 2026/09/21 12:54:57 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5648lb # [ 91.833854] niks3-server[1047]: 2026/09/21 12:54:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5649lb # [ 91.838259] niks3-server[1047]: 2026/09/21 12:54:57 INFO Signed narinfos id=28 count=15650worker1 # [ 91.847161] nix-grpc-daemon[1302]: time=2026-09-21T12:54:57.929Z level=INFO msg="Uploading 1 narinfos"5651lb # [ 91.847267] niks3-server[1047]: 2026/09/21 12:54:57 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5652lb # [ 91.857100] niks3-server[1047]: 2026/09/21 12:54:57 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5653lb # [ 91.862735] niks3-server[1047]: 2026/09/21 12:54:57 INFO Completed upload id=285654worker1 # [ 91.868170] nix-grpc-daemon[1302]: time=2026-09-21T12:54:57.951Z level=INFO msg="Upload complete. (348ms)"5655lb # [ 91.865642] niks3-server[1047]: 2026/09/21 12:54:57 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5656worker1 # [ 91.882856] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49174 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15657worker1 # [ 91.898282] nix-daemon[850]: reaped child process 1325, status = succeeded5658worker1 # [ 91.903899] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:57Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05659worker1 # [ 91.907710] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:57Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15660worker1 # [ 91.948658] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491745661worker1 # [ 91.961398] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491745662worker1 # [ 91.973785] nix-grpc-daemon[1302]: time=2026-09-21T12:54:58.056Z level=INFO msg="All 1 paths already cached"5663client # copying 1 paths to 'lb:50051'...5664client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5665worker1 # [ 91.981824] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491745666worker1 # [ 91.990340] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5667worker1 # [ 92.195602] nix-daemon[850]: accepted connection from pid 1352, user nix-grpc-daemon (trusted)5668worker1 # [ 92.208356] nix-daemon[850]: reaped child process 1355, status = succeeded5669worker1 # [ 92.390397] nix-daemon[850]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5670worker1 # [ 92.401290] nix-daemon[850]: reaped child process 1360, status = succeeded5671lb # [ 92.413881] niks3-server[1047]: 2026/09/21 12:54:58 INFO Received uploads request method=POST path=/api/pending_closures5672worker1 # [ 92.426763] nix-grpc-daemon[1302]: time=2026-09-21T12:54:58.509Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5673worker1 # [ 92.428328] nix-grpc-daemon[1302]: time=2026-09-21T12:54:58.511Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5674lb # [ 92.443622] niks3-server[1047]: 2026/09/21 12:54:58 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5675lb # [ 92.454821] niks3-server[1047]: 2026/09/21 12:54:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5676worker1 # [ 92.465068] nix-grpc-daemon[1302]: time=2026-09-21T12:54:58.548Z level=INFO msg="Uploading 1 narinfos"5677lb # [ 92.460064] niks3-server[1047]: 2026/09/21 12:54:58 INFO Signed narinfos id=29 count=15678lb # [ 92.465248] niks3-server[1047]: 2026/09/21 12:54:58 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5679lb # [ 92.475237] niks3-server[1047]: 2026/09/21 12:54:58 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5680worker1 # [ 92.487912] nix-grpc-daemon[1302]: time=2026-09-21T12:54:58.571Z level=INFO msg="Upload complete. (483ms)"5681lb # [ 92.482941] niks3-server[1047]: 2026/09/21 12:54:58 INFO Completed upload id=295682lb # [ 92.486406] niks3-server[1047]: 2026/09/21 12:54:58 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5683worker1 # [ 92.490698] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49174 duration_s=0 paths=1 nar_bytes_in=6405684worker1 # [ 92.495825] nix-grpc-daemon[1228]: ts=2026-09-21T12:54:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491745685worker1 # [ 92.505765] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5686worker1 # [ 92.510667] nix-daemon[850]: reaped child process 1350, status = succeeded5687worker1 # [ 92.514944] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5688client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5689client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5690client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5691client # farm-b> LOG-farm-b-on-standby5692worker1 # [ 93.942404] nix-daemon[850]: reaped child process 1364, status = succeeded5693worker1 # [ 94.110655] nix-daemon[850]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5694worker1 # [ 94.123544] nix-daemon[850]: reaped child process 1383, status = succeeded5695lb # [ 94.131669] niks3-server[1047]: 2026/09/21 12:55:00 INFO Received uploads request method=POST path=/api/pending_closures5696worker1 # [ 94.145059] nix-grpc-daemon[1302]: time=2026-09-21T12:55:00.227Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5697worker1 # [ 94.146549] nix-grpc-daemon[1302]: time=2026-09-21T12:55:00.227Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5698lb # [ 94.164104] niks3-server[1047]: 2026/09/21 12:55:00 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5699lb # [ 94.173246] niks3-server[1047]: 2026/09/21 12:55:00 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5700lb # [ 94.181184] niks3-server[1047]: 2026/09/21 12:55:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5701worker1 # [ 94.192362] nix-grpc-daemon[1302]: time=2026-09-21T12:55:00.275Z level=INFO msg="Uploading 1 narinfos"5702lb # [ 94.187559] niks3-server[1047]: 2026/09/21 12:55:00 INFO Signed narinfos id=30 count=15703lb # [ 94.192925] niks3-server[1047]: 2026/09/21 12:55:00 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5704lb # [ 94.203061] niks3-server[1047]: 2026/09/21 12:55:00 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5705lb # [ 94.208190] niks3-server[1047]: 2026/09/21 12:55:00 INFO Completed upload id=305706worker1 # [ 94.213564] nix-grpc-daemon[1302]: time=2026-09-21T12:55:00.296Z level=INFO msg="Upload complete. (290ms)"5707lb # [ 94.210981] niks3-server[1047]: 2026/09/21 12:55:00 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5708worker1 # [ 94.221846] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49174 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15709worker1 # [ 94.228808] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:00Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05710worker1 # [ 94.232680] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:00Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15711worker1 # [ 94.240617] nix-daemon[850]: reaped child process 1362, status = succeeded5712worker1 # [ 94.272426] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491745713worker1 # [ 94.284401] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491745714worker1 # [ 94.296264] nix-grpc-daemon[1302]: time=2026-09-21T12:55:00.379Z level=INFO msg="All 2 paths already cached"5715client # copying 1 paths to 'lb:50051'...5716client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5717worker1 # [ 94.304580] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491745718worker1 # [ 94.315168] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5719worker1 # [ 94.491970] nix-daemon[850]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)5720worker1 # [ 94.504401] nix-daemon[850]: reaped child process 1392, status = succeeded5721worker1 # [ 94.685772] nix-daemon[850]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5722worker1 # [ 94.698949] nix-daemon[850]: reaped child process 1397, status = succeeded5723lb # [ 94.710973] niks3-server[1047]: 2026/09/21 12:55:00 INFO Received uploads request method=POST path=/api/pending_closures5724worker1 # [ 94.725568] nix-grpc-daemon[1302]: time=2026-09-21T12:55:00.808Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5725worker1 # [ 94.727615] nix-grpc-daemon[1302]: time=2026-09-21T12:55:00.810Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5726lb # [ 94.743459] niks3-server[1047]: 2026/09/21 12:55:00 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5727lb # [ 94.754480] niks3-server[1047]: 2026/09/21 12:55:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5728lb # [ 94.759376] niks3-server[1047]: 2026/09/21 12:55:00 INFO Signed narinfos id=31 count=15729worker1 # [ 94.767775] nix-grpc-daemon[1302]: time=2026-09-21T12:55:00.850Z level=INFO msg="Uploading 1 narinfos"5730lb # [ 94.765973] niks3-server[1047]: 2026/09/21 12:55:00 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5731lb # [ 94.774479] niks3-server[1047]: 2026/09/21 12:55:00 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5732lb # [ 94.783173] niks3-server[1047]: 2026/09/21 12:55:00 INFO Completed upload id=315733worker1 # [ 94.788608] nix-grpc-daemon[1302]: time=2026-09-21T12:55:00.871Z level=INFO msg="Upload complete. (463ms)"5734lb # [ 94.787835] niks3-server[1047]: 2026/09/21 12:55:00 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5735worker1 # [ 94.792067] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49174 duration_s=0 paths=1 nar_bytes_in=7685736worker1 # [ 94.798593] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491745737worker1 # [ 94.807166] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5738worker1 # [ 94.816160] nix-daemon[850]: reaped child process 1387, status = succeeded5739worker1 # [ 94.820097] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5740client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5741client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5742client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5743client # farm-top> LOG-farm-top-on-standby5744worker1 # [ 96.046491] nix-daemon[850]: reaped child process 1401, status = succeeded5745worker1 # [ 96.225274] nix-daemon[850]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)5746worker1 # [ 96.237559] nix-daemon[850]: reaped child process 1420, status = succeeded5747lb # [ 96.251971] niks3-server[1047]: 2026/09/21 12:55:02 INFO Received uploads request method=POST path=/api/pending_closures5748worker1 # [ 96.290695] nix-grpc-daemon[1302]: time=2026-09-21T12:55:02.373Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5749worker1 # [ 96.292260] nix-grpc-daemon[1302]: time=2026-09-21T12:55:02.375Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5750lb # [ 96.315498] niks3-server[1047]: 2026/09/21 12:55:02 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5751lb # [ 96.321294] niks3-server[1047]: 2026/09/21 12:55:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5752lb # [ 96.327559] niks3-server[1047]: 2026/09/21 12:55:02 INFO Signed narinfos id=32 count=15753worker1 # [ 96.336066] nix-grpc-daemon[1302]: time=2026-09-21T12:55:02.418Z level=INFO msg="Uploading 1 narinfos"5754lb # [ 96.340634] niks3-server[1047]: 2026/09/21 12:55:02 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5755lb # [ 96.343589] niks3-server[1047]: 2026/09/21 12:55:02 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5756lb # [ 96.348465] niks3-server[1047]: 2026/09/21 12:55:02 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5757lb # [ 96.351678] niks3-server[1047]: 2026/09/21 12:55:02 INFO Completed upload id=325758lb # [ 96.353324] niks3-server[1047]: 2026/09/21 12:55:02 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5759worker1 # [ 96.357154] nix-grpc-daemon[1302]: time=2026-09-21T12:55:02.440Z level=INFO msg="Upload complete. (329ms)"5760worker1 # [ 96.364611] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49174 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15761worker1 # [ 96.378469] nix-daemon[850]: reaped child process 1399, status = succeeded5762client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 7.69 seconds)5763client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5764client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5765worker2 # [ 96.705478] systemd[1]: Listening on Nix worker-protocol over gRPC.5766worker2 # [ 96.712717] systemd[1]: Starting scheduler-token.service...5767worker2 # [ 96.789894] systemd[1]: scheduler-token.service: Deactivated successfully.5768worker2 # [ 96.791506] systemd[1]: Finished scheduler-token.service.5769worker2 # [ 96.793177] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 79ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5770worker2 # [ 96.798751] systemd[1]: Starting Nix worker-protocol over gRPC...5771worker2 # [ 96.979580] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:03Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5772worker2 # [ 96.981436] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5773worker2 # [ 96.985454] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15774worker2 # [ 96.993063] nix-grpc-daemon[1207]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5775worker2 # [ 96.995093] nix-grpc-daemon[1207]: E0000 00:00:1789995303.210481 1207 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5776worker2 # [ 97.000090] nix-grpc-daemon[1207]: E0000 00:00:1789995303.215695 1207 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5777worker2 # [ 97.003886] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:03Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5778worker2 # [ 97.006900] systemd[1]: Started Nix worker-protocol over gRPC.5779lb: must succeed: curl -sf localhost:9901/clusters5780worker2 # [ 97.013218] nix-grpc-daemon[1207]: E0000 00:00:1789995303.230103 1249 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5781worker2 # [ 97.024366] nix-grpc-daemon[1207]: E0000 00:00:1789995303.242259 1249 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5782worker2 # [ 97.028854] nix-grpc-daemon[1207]: E0000 00:00:1789995303.244585 1249 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5783worker2 # [ 97.038704] nix-grpc-daemon[1207]: E0000 00:00:1789995303.256609 1249 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5784worker2 # [ 97.049899] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:03Z level=info event=scheduler_connected addr=lb:500515785worker1 # [ 97.073327] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:03Z level=info event=oidc_jwks provider=mock keys=15786worker1 # [ 97.077703] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:491645787worker1 # [ 97.081614] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05788lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5789worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5790worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5791lb: must succeed: curl -sf localhost:9901/clusters5792lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5793lb: must succeed: curl -sf localhost:9901/clusters5794lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5795worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5796worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5797(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.49 seconds)5798subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5799client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5800client # [ 97.411437] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5801client # Running as unit: rs.service; invocation ID: bb0df61afc8e4f98a15693fb43b2dad05802client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5803worker1 # [ 97.676831] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:491745804client # [ 97.664583] nix[1173]: don't know how to build these paths:5805client # [ 97.666120] nix[1173]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5806worker1 # [ 97.692166] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:03Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05807worker1 # [ 97.697438] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:03Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15808worker1 # [ 97.736559] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491745809worker1 # [ 97.748856] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491745810client # [ 97.741170] nix[1173]: copying 1 paths to 'lb:50051'...5811worker1 # [ 97.760150] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491745812client # [ 97.742874] nix[1173]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5813worker1 # [ 97.769775] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5814worker1 # [ 97.953626] nix-daemon[850]: accepted connection from pid 1433, user nix-grpc-daemon (trusted)5815worker1 # [ 97.965945] nix-daemon[850]: reaped child process 1436, status = succeeded5816worker1 # [ 98.138159] nix-daemon[850]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)5817worker1 # [ 98.149955] nix-daemon[850]: reaped child process 1441, status = succeeded5818lb # [ 98.162658] niks3-server[1047]: 2026/09/21 12:55:04 INFO Received uploads request method=POST path=/api/pending_closures5819worker1 # [ 98.177146] nix-grpc-daemon[1302]: time=2026-09-21T12:55:04.260Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5820worker1 # [ 98.178666] nix-grpc-daemon[1302]: time=2026-09-21T12:55:04.261Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5821lb # [ 98.191296] niks3-server[1047]: 2026/09/21 12:55:04 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5822lb # [ 98.202229] niks3-server[1047]: 2026/09/21 12:55:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5823lb # [ 98.206896] niks3-server[1047]: 2026/09/21 12:55:04 INFO Signed narinfos id=33 count=15824worker1 # [ 98.215152] nix-grpc-daemon[1302]: time=2026-09-21T12:55:04.298Z level=INFO msg="Uploading 1 narinfos"5825lb # [ 98.216641] niks3-server[1047]: 2026/09/21 12:55:04 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5826lb # [ 98.226308] niks3-server[1047]: 2026/09/21 12:55:04 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5827lb # [ 98.233364] niks3-server[1047]: 2026/09/21 12:55:04 INFO Completed upload id=335828worker1 # [ 98.238590] nix-grpc-daemon[1302]: time=2026-09-21T12:55:04.321Z level=INFO msg="Upload complete. (454ms)"5829lb # [ 98.236335] niks3-server[1047]: 2026/09/21 12:55:04 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5830worker1 # [ 98.242759] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49174 duration_s=0 paths=1 nar_bytes_in=4165831worker1 # [ 98.248078] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491745832worker1 # [ 98.256874] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5833worker1 # [ 98.263092] nix-daemon[850]: reaped child process 1431, status = succeeded5834worker1 # [ 98.267970] nix-daemon[850]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5835client # [ 98.249427] nix[1173]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5836lb # [ 98.513396] envoy[909]: conn peer=[2001:db8:1::1]:41272 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9650 tx=4117 ms=73955837lb: must succeed: curl -sf localhost:9901/clusters5838lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5839lb: must succeed: curl -sf localhost:9901/clusters5840lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5841lb: must succeed: curl -sf localhost:9901/clusters5842lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5843lb: must succeed: curl -sf localhost:9901/clusters5844lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5845worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5846worker2 # [ 100.366479] systemd-journald[289]: Received client request to rotate journal, rotating.5847worker2 # [ 100.387571] systemd-journald[289]: Vacuuming done, freed 0B of archived journals from /var/log/journal/c19a115525614b5eadf2b3cc89a520f2.5848worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5849worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5850worker1 # [ 100.319898] systemd[1]: Stopping Nix worker-protocol over gRPC...5851worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5852client: must succeed: sleep 5.05853worker1 # [ 100.707677] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:06Z level=info event=draining5854client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5855worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5856worker1 # [ 105.687830] nix-daemon[850]: reaped child process 1445, status = succeeded5857worker1 # [ 105.838609] nix-daemon[850]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)5858worker1 # [ 105.850551] nix-daemon[850]: reaped child process 1489, status = succeeded5859lb # [ 105.863172] niks3-server[1047]: 2026/09/21 12:55:11 INFO Received uploads request method=POST path=/api/pending_closures5860worker1 # [ 105.878310] nix-grpc-daemon[1302]: time=2026-09-21T12:55:11.959Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5861worker1 # [ 105.879670] nix-grpc-daemon[1302]: time=2026-09-21T12:55:11.959Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5862lb # [ 105.902640] niks3-server[1047]: 2026/09/21 12:55:11 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5863lb # [ 105.911933] niks3-server[1047]: 2026/09/21 12:55:11 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5864lb # [ 105.921936] niks3-server[1047]: 2026/09/21 12:55:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5865lb # [ 105.926362] niks3-server[1047]: 2026/09/21 12:55:11 INFO Signed narinfos id=34 count=15866worker1 # [ 105.934736] nix-grpc-daemon[1302]: time=2026-09-21T12:55:12.017Z level=INFO msg="Uploading 1 narinfos"5867lb # [ 105.932236] niks3-server[1047]: 2026/09/21 12:55:11 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5868lb # [ 105.940465] niks3-server[1047]: 2026/09/21 12:55:11 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5869lb # [ 105.947112] niks3-server[1047]: 2026/09/21 12:55:11 INFO Completed upload id=345870worker1 # [ 105.952508] nix-grpc-daemon[1302]: time=2026-09-21T12:55:12.035Z level=INFO msg="Upload complete. (272ms)"5871lb # [ 105.948857] niks3-server[1047]: 2026/09/21 12:55:11 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5872worker1 # [ 105.960658] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49174 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15873worker1 # [ 105.973203] nix-daemon[850]: reaped child process 1443, status = succeeded5874client # [ 105.982445] systemd[1]: rs.service: Deactivated successfully.5875client # [ 105.986235] systemd[1]: rs.service: Consumed 124ms CPU time over 8.573s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.5876worker1 # [ 106.709007] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:12Z level=info event=signal_exit5877worker1 # [ 106.935384] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_yield5878worker2 # [ 106.925538] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_restarting addr=lb:500515879lb # [ 106.932459] niks3-server[1047]: 2026/09/21 12:55:12 INFO lead: released remote=192.168.1.3:596265880worker1 # [ 106.942857] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_restarting addr=lb:500515881worker1 # [ 106.947310] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:491645882worker1 # [ 106.950571] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=worker_gone addr=192.168.1.4:500515883worker1 # [ 106.953524] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:491645884worker1 # [ 106.956715] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=worker_gone addr=192.168.1.3:500515885lb # [ 107.020111] niks3-server[1047]: 2026/09/21 12:55:12 INFO lead: acquired remote=192.168.1.4:406145886worker2 # [ 107.011900] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_take_over5887worker2 # [ 107.126505] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_connected addr=lb:500515888worker1 # [ 107.144718] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_connected addr=lb:500515889worker2 # [ 107.130560] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5890worker1 # [ 107.148417] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5891worker2 # [ 107.332219] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_connected addr=lb:500515892worker1 # [ 107.349313] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_connected addr=lb:500515893worker2 # [ 107.336263] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5894worker1 # [ 107.353269] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5895worker2 # [ 107.537288] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_connected addr=lb:500515896worker1 # [ 107.554091] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_connected addr=lb:500515897worker2 # [ 107.541169] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5898worker1 # [ 107.558111] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5899worker2 # [ 107.742544] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_connected addr=lb:500515900worker1 # [ 107.758894] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_connected addr=lb:500515901worker2 # [ 107.746939] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5902worker1 # [ 107.762891] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5903worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.18 seconds)5904client: waiting for success: ! systemctl is-active rs5905client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5906client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5907client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5908client: must fail: journalctl -u rs | grep -q 'reconnecting'5909worker1 # [ 107.939931] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:14Z level=info event=scheduler_restarting5910client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5911lb: must succeed: curl -sf localhost:9901/clusters5912worker2 # [ 107.949293] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=scheduler_connected addr=lb:500515913worker2 # [ 107.953384] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5914worker1 # [ 107.976907] nix-daemon[850]: reaped child process 1292, status = succeeded5915worker1 # [ 107.979478] nix-grpc-daemon[1228]: ts=2026-09-21T12:55:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5916lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5917worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5918worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5919worker1 # [ 108.128211] nix-grpc-daemon[1228]: E0000 00:00:1789995314.210986 1273 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5920worker1 # [ 108.133260] nix-grpc-daemon[1228]: E0000 00:00:1789995314.215980 1273 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5921worker2 # [ 108.154834] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=scheduler_connected addr=lb:500515922worker2 # [ 108.158804] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5923worker2 # [ 108.360235] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=scheduler_connected addr=lb:500515924worker2 # [ 108.364303] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5925lb # [ 108.512776] envoy[909]: conn peer=[2001:db8:1::1]:33220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1403 ms=82935926lb # [ 108.516674] envoy[909]: conn peer=[2001:db8:1::3]:32864 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5568 tx=1292 ms=252155927worker2 # [ 108.565632] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=scheduler_connected addr=lb:500515928worker2 # [ 108.579153] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=oidc_jwks provider=mock keys=15929worker2 # [ 108.582778] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:359665930worker2 # [ 108.586667] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:14Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05931lb: must succeed: curl -sf localhost:9901/clusters5932lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5933worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5934worker1 # [ 109.216558] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5935worker1 # [ 109.221588] systemd[1]: Stopped Nix worker-protocol over gRPC.5936worker1 # [ 109.223782] systemd[1]: nix-grpc-daemon.service: Consumed 1.343s CPU time over 26.729s wall clock time, 35.6M memory peak, 49K incoming IP traffic, 96.4K outgoing IP traffic.5937worker1 # [ 109.239204] systemd[1]: Starting Nix worker-protocol over gRPC...5938worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5939worker1 # [ 109.407262] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:15Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5940worker1 # [ 109.409186] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5941worker1 # [ 109.412654] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15942worker1 # [ 109.420067] nix-grpc-daemon[1496]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5943worker1 # [ 109.421447] nix-grpc-daemon[1496]: E0000 00:00:1789995315.502605 1496 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5944worker1 # [ 109.426399] nix-grpc-daemon[1496]: E0000 00:00:1789995315.507667 1496 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5945worker1 # [ 109.429441] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5946worker1 # [ 109.432085] systemd[1]: Started Nix worker-protocol over gRPC.5947worker1 # [ 109.438069] nix-grpc-daemon[1496]: E0000 00:00:1789995315.520036 1536 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5948worker1 # [ 109.448682] nix-grpc-daemon[1496]: E0000 00:00:1789995315.531781 1536 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5949worker1 # [ 109.453054] nix-grpc-daemon[1496]: E0000 00:00:1789995315.534035 1536 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5950worker1 # [ 109.462801] nix-grpc-daemon[1496]: E0000 00:00:1789995315.545904 1536 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5951worker1 # [ 109.472476] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:15Z level=info event=scheduler_connected addr=lb:500515952worker2 # [ 109.461652] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:15Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:359665953worker2 # [ 109.465661] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:15Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05954lb: must succeed: curl -sf localhost:9901/clusters5955lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5956worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5957worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5958(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.04 seconds)5959subtest: scheduler restart mid-build: build finishes, no double build5960client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5961client # [ 110.451082] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5962client # Running as unit: mid.service; invocation ID: 1b2d4d77f564459ba9aa83924959e02a5963client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5964worker2 # [ 110.688785] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:359785965worker2 # [ 110.700802] nix-daemon[871]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)5966client # [ 110.709175] nix[1209]: don't know how to build these paths:5967client # [ 110.709993] nix[1209]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5968worker2 # [ 110.722760] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:16Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05969worker2 # [ 110.727566] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:16Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15970worker2 # [ 110.771671] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359785971worker2 # [ 110.784196] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5)5972worker2 # [ 110.848133] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 157 ms (attempt 2/5)5973worker2 # [ 111.011095] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 190 ms (attempt 3/5)5974worker2 # [ 111.207321] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 547 ms (attempt 4/5)5975worker2 # [ 111.755422] nix-grpc-daemon[1207]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5976worker2 # [ 111.756836] nix-grpc-daemon[1207]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5977worker2 # [ 111.770528] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359785978client # [ 111.773727] nix[1209]: copying 1 paths to 'lb:50051'...5979worker2 # [ 111.781283] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359785980client # [ 111.776981] nix[1209]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5981worker2 # [ 111.791210] nix-daemon[871]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)5982worker2 # [ 111.803131] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:18Z level=info event=niks3_push_started pid=13365983worker2 # [ 112.015847] nix-grpc-daemon[1336]: time=2026-09-21T12:55:18.233Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""5984worker2 # [ 112.209794] nix-daemon[871]: accepted connection from pid 1349, user nix-grpc-daemon (trusted)5985worker2 # [ 112.222432] nix-daemon[871]: reaped child process 1352, status = succeeded5986worker2 # [ 112.422187] nix-daemon[871]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)5987worker2 # [ 112.435781] nix-daemon[871]: reaped child process 1357, status = succeeded5988lb # [ 112.463607] niks3-server[1047]: 2026/09/21 12:55:18 INFO Received uploads request method=POST path=/api/pending_closures5989worker2 # [ 112.463646] nix-grpc-daemon[1336]: time=2026-09-21T12:55:18.681Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5990worker2 # [ 112.465213] nix-grpc-daemon[1336]: time=2026-09-21T12:55:18.683Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5991lb # [ 112.502451] niks3-server[1047]: 2026/09/21 12:55:18 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5992lb # [ 112.513082] niks3-server[1047]: 2026/09/21 12:55:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5993lb # [ 112.519401] niks3-server[1047]: 2026/09/21 12:55:18 INFO Signed narinfos id=35 count=15994worker2 # [ 112.513537] nix-grpc-daemon[1336]: time=2026-09-21T12:55:18.731Z level=INFO msg="Uploading 1 narinfos"5995lb # [ 112.529403] niks3-server[1047]: 2026/09/21 12:55:18 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5996lb # [ 112.537456] niks3-server[1047]: 2026/09/21 12:55:18 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5997lb # [ 112.547410] niks3-server[1047]: 2026/09/21 12:55:18 INFO Completed upload id=355998worker2 # [ 112.538914] nix-grpc-daemon[1336]: time=2026-09-21T12:55:18.756Z level=INFO msg="Upload complete. (518ms)"5999lb # [ 112.549269] niks3-server[1047]: 2026/09/21 12:55:18 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6000worker2 # [ 112.544165] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35978 duration_s=0 paths=1 nar_bytes_in=4246001worker2 # [ 112.549256] nix-grpc-daemon[1207]: ts=2026-09-21T12:55:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359786002worker2 # [ 112.559141] nix-daemon[871]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)6003worker2 # [ 112.564567] nix-daemon[871]: reaped child process 1334, status = succeeded6004worker2 # [ 112.569950] nix-daemon[871]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)6005client # [ 112.565194] nix[1209]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6006lb: must succeed: curl -sf localhost:9901/clusters6007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6008lb: must succeed: curl -sf localhost:9901/clusters6009lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6010worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6011worker2 # [ 114.342375] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1207 (nix-grpc-daemon) on client request.6012worker2 # [ 114.345363] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6013lb # [ 114.359737] niks3-server[1047]: 2026/09/21 12:55:20 INFO lead: released remote=192.168.1.4:406146014client # [ 114.356642] nix[1209]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6015worker1 # [ 114.373665] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:20Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6016worker2 # [ 114.367077] systemd[1]: Starting scheduler-token.service...6017client # [ 114.360616] nix[1209]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6018worker2 # [ 114.381114] nix-daemon[871]: reaped child process 1359, status = succeeded6019worker2 # [ 114.382479] nix-daemon[871]: reaped child process 1322, status = succeeded6020worker2 # [ 114.386540] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6021worker2 # [ 114.387991] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6022worker2 # [ 114.390200] systemd[1]: nix-grpc-daemon.service: Consumed 490ms CPU time over 17.587s wall clock time, 33.6M memory peak, 16.8K incoming IP traffic, 36.4K outgoing IP traffic.6023worker2 # [ 114.398243] nix-daemon[1361]: unexpected Nix daemon error: error: interrupted by the user6024worker2 # [ 114.402153] nix-daemon[871]: reaped child process 1361, status = failed with exit code 16025lb # [ 114.430488] niks3-server[1047]: 2026/09/21 12:55:20 INFO lead: acquired remote=192.168.1.3:529246026worker1 # [ 114.439205] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:20Z level=info event=scheduler_take_over6027worker2 # [ 114.464082] systemd[1]: scheduler-token.service: Deactivated successfully.6028worker2 # [ 114.465771] systemd[1]: Finished scheduler-token.service.6029worker2 # [ 114.466610] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 97ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6030worker2 # [ 114.469330] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6031worker2 # [ 114.474440] systemd[1]: Starting scheduler-token.service...6032worker2 # [ 114.546663] systemd[1]: scheduler-token.service: Deactivated successfully.6033worker2 # [ 114.548347] systemd[1]: Finished scheduler-token.service.6034worker2 # [ 114.549601] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 73ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6035worker2 # [ 114.555475] systemd[1]: Starting Nix worker-protocol over gRPC...6036worker1 # [ 114.574088] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:20Z level=info event=scheduler_connected addr=lb:500516037worker2 # [ 114.738861] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6038worker2 # [ 114.740676] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6039worker2 # [ 114.744531] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16040worker2 # [ 114.752063] nix-grpc-daemon[1404]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6041worker2 # [ 114.754098] nix-grpc-daemon[1404]: E0000 00:00:1789995320.969665 1404 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6042worker2 # [ 114.760213] nix-grpc-daemon[1404]: E0000 00:00:1789995320.974936 1404 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6043worker2 # [ 114.762343] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:20Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6044worker2 # [ 114.763867] systemd[1]: Started Nix worker-protocol over gRPC.6045worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.48 seconds)6046client: must succeed: sleep 5.06047worker2 # [ 114.776131] nix-grpc-daemon[1404]: E0000 00:00:1789995320.993547 1445 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6048worker1 # [ 114.796185] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6049worker2 # [ 114.789808] nix-grpc-daemon[1404]: E0000 00:00:1789995321.007679 1445 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6050worker2 # [ 114.794288] nix-grpc-daemon[1404]: E0000 00:00:1789995321.009978 1445 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6051worker2 # [ 114.805747] nix-grpc-daemon[1404]: E0000 00:00:1789995321.023559 1445 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6052worker2 # [ 114.814948] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516053worker2 # [ 114.816463] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6054client # [ 114.858541] nix[1209]: scheduler at lb:50051: no healthy upstream, reconnecting6055worker1 # [ 114.997312] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516056worker1 # [ 115.001264] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6057worker2 # [ 115.017073] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516058worker2 # [ 115.020852] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6059worker1 # [ 115.202058] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516060worker1 # [ 115.206134] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6061worker2 # [ 115.221948] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516062worker2 # [ 115.225888] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6063worker1 # [ 115.407735] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516064worker1 # [ 115.411429] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6065worker2 # [ 115.427121] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516066worker2 # [ 115.430939] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6067worker1 # [ 115.609565] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516068worker1 # [ 115.613516] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6069worker2 # [ 115.632478] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516070worker2 # [ 115.636339] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6071worker1 # [ 115.814446] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_connected addr=lb:500516072worker1 # [ 115.818402] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6073worker2 # [ 115.838709] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:22Z level=info event=scheduler_connected addr=lb:500516074worker2 # [ 115.841870] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6075client # [ 115.861383] nix[1209]: scheduler at lb:50051: no healthy upstream, reconnecting6076worker1 # [ 116.019376] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:22Z level=info event=scheduler_connected addr=lb:500516077worker1 # [ 116.023231] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6078worker2 # [ 116.039496] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:22Z level=info event=scheduler_connected addr=lb:500516079worker2 # [ 116.043437] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6080worker1 # [ 116.224371] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:22Z level=info event=scheduler_connected addr=lb:500516081worker1 # [ 116.236340] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:414966082worker1 # [ 116.240955] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06083worker2 # [ 116.244598] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:22Z level=info event=scheduler_connected addr=lb:500516084worker1 # [ 116.265155] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:22Z level=info event=oidc_jwks provider=mock keys=16085worker1 # [ 116.268743] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:414966086worker1 # [ 116.272725] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06087worker1 # [ 117.892985] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:23Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06088worker1 # [ 117.897117] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:23Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16089worker1 # [ 117.942819] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415066090worker1 # [ 117.953988] nix-daemon[850]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6091worker1 # [ 117.970979] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5)6092worker1 # [ 118.005795] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 2/5)6093worker1 # [ 118.043378] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 3/5)6094worker1 # [ 118.128239] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 788 ms (attempt 4/5)6095lb # [ 118.513907] envoy[909]: conn peer=[2001:db8:1::4]:50352 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7821 tx=567 ms=173156096worker1 # [ 118.923445] nix-grpc-daemon[1496]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6097worker1 # [ 118.927418] nix-grpc-daemon[1496]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6098worker1 # [ 118.947780] nix-grpc-daemon[1496]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6099worker1 # [ 118.969988] nix-daemon[850]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6100worker1 # [ 118.985531] nix-daemon[850]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6101client # [ 118.967279] nix[1209]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6102client: (finished: must succeed: sleep 5.0, in 5.06 seconds)6103client: waiting for success: ! systemctl is-active mid6104worker1 # [ 120.205571] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:26Z level=info event=niks3_push_started pid=16046105worker1 # [ 120.237949] nix-daemon[850]: reaped child process 1579, status = succeeded6106worker1 # [ 120.432740] nix-grpc-daemon[1604]: time=2026-09-21T12:55:26.515Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6107worker1 # [ 120.603354] nix-daemon[850]: accepted connection from pid 1617, user nix-grpc-daemon (trusted)6108worker1 # [ 120.615435] nix-daemon[850]: reaped child process 1620, status = succeeded6109lb # [ 120.629270] niks3-server[1047]: 2026/09/21 12:55:26 INFO Received uploads request method=POST path=/api/pending_closures6110worker1 # [ 120.644931] nix-grpc-daemon[1604]: time=2026-09-21T12:55:26.728Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6111worker1 # [ 120.646651] nix-grpc-daemon[1604]: time=2026-09-21T12:55:26.729Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6112lb # [ 120.676967] niks3-server[1047]: 2026/09/21 12:55:26 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6113lb # [ 120.687768] niks3-server[1047]: 2026/09/21 12:55:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6114worker1 # [ 120.701487] nix-grpc-daemon[1604]: time=2026-09-21T12:55:26.784Z level=INFO msg="Uploading 1 narinfos"6115lb # [ 120.696969] niks3-server[1047]: 2026/09/21 12:55:26 INFO Signed narinfos id=36 count=16116lb # [ 120.717475] niks3-server[1047]: 2026/09/21 12:55:26 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6117lb # [ 120.724754] niks3-server[1047]: 2026/09/21 12:55:26 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6118lb # [ 120.728609] niks3-server[1047]: 2026/09/21 12:55:26 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6119lb # [ 120.731994] niks3-server[1047]: 2026/09/21 12:55:26 INFO Completed upload id=366120lb # [ 120.733989] niks3-server[1047]: 2026/09/21 12:55:26 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6121worker1 # [ 120.738705] nix-grpc-daemon[1604]: time=2026-09-21T12:55:26.821Z level=INFO msg="Upload complete. (302ms)"6122worker1 # [ 120.745708] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41506 duration_s=2 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16123worker1 # [ 120.760459] nix-daemon[850]: reaped child process 1577, status = succeeded6124client # [ 120.777988] systemd[1]: mid.service: Deactivated successfully.6125client # [ 120.781436] systemd[1]: mid.service: Consumed 144ms CPU time over 10.330s wall clock time, 41.3M memory peak, 9.1K incoming IP traffic, 11.2K outgoing IP traffic.6126client: (finished: waiting for success: ! systemctl is-active mid, in 1.13 seconds)6127client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6128client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6129(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.87 seconds)6130subtest: interrupting the client stops the build on the worker6131client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr6132client # [ 121.293729] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.6133client # Running as unit: intr.service; invocation ID: aefb2cfb78fe4a59a7568edea7b4ef516134client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)6135worker2 # [ 121.532508] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487666136worker2 # [ 121.543854] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6137client # [ 121.552126] nix[1245]: don't know how to build these paths:6138client # [ 121.552955] nix[1245]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6139worker1 # [ 121.580863] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:27Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06140worker1 # [ 121.585432] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:27Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16141worker2 # [ 121.611629] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487666142worker2 # [ 121.624308] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 1/5)6143worker2 # [ 121.661990] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 2/5)6144worker2 # [ 121.754766] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 3/5)6145worker2 # [ 121.895582] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 693 ms (attempt 4/5)6146worker2 # [ 122.594887] nix-grpc-daemon[1404]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6147worker2 # [ 122.597953] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6148worker2 # [ 122.614265] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487666149client # [ 122.619693] nix[1245]: copying 1 paths to 'lb:50051'...6150client # [ 122.623235] nix[1245]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6151worker2 # [ 122.627644] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487666152worker2 # [ 122.637774] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6153worker2 # [ 122.650818] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:28Z level=info event=niks3_push_started pid=14806154worker2 # [ 122.848698] nix-grpc-daemon[1480]: time=2026-09-21T12:55:29.066Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker2.key ca=""6155worker2 # [ 123.034405] nix-daemon[871]: accepted connection from pid 1493, user nix-grpc-daemon (trusted)6156worker2 # [ 123.045102] nix-daemon[871]: reaped child process 1496, status = succeeded6157worker2 # [ 123.230351] nix-daemon[871]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6158worker2 # [ 123.242438] nix-daemon[871]: reaped child process 1501, status = succeeded6159lb # [ 123.268583] niks3-server[1047]: 2026/09/21 12:55:29 INFO Received uploads request method=POST path=/api/pending_closures6160worker2 # [ 123.270124] nix-grpc-daemon[1480]: time=2026-09-21T12:55:29.488Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6161worker2 # [ 123.272604] nix-grpc-daemon[1480]: time=2026-09-21T12:55:29.489Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6162lb # [ 123.303675] niks3-server[1047]: 2026/09/21 12:55:29 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6163lb # [ 123.312665] niks3-server[1047]: 2026/09/21 12:55:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6164lb # [ 123.318725] niks3-server[1047]: 2026/09/21 12:55:29 INFO Signed narinfos id=37 count=16165worker2 # [ 123.312942] nix-grpc-daemon[1480]: time=2026-09-21T12:55:29.530Z level=INFO msg="Uploading 1 narinfos"6166lb # [ 123.327573] niks3-server[1047]: 2026/09/21 12:55:29 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6167lb # [ 123.337537] niks3-server[1047]: 2026/09/21 12:55:29 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6168worker2 # [ 123.335856] nix-grpc-daemon[1480]: time=2026-09-21T12:55:29.553Z level=INFO msg="Upload complete. (482ms)"6169lb # [ 123.344522] niks3-server[1047]: 2026/09/21 12:55:29 INFO Completed upload id=376170lb # [ 123.347367] niks3-server[1047]: 2026/09/21 12:55:29 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6171worker2 # [ 123.338203] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=0 paths=1 nar_bytes_in=4246172worker2 # [ 123.343525] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487666173worker2 # [ 123.352421] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6174worker2 # [ 123.357369] nix-daemon[871]: reaped child process 1478, status = succeeded6175worker2 # [ 123.362578] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6176client # [ 123.357726] nix[1245]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6177client: must succeed: systemctl kill -s INT intr6178worker2 # [ 124.898528] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:31Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6179client # [ 124.893121] systemd[1]: intr.service: Sent signal SIGINT to main process 1245 (nix) on client request.6180client # [ 124.897887] nix[1245]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6181client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds)6182client # [ 124.904940] nix[1245]: error: interrupted by the user6183client # [ 124.940095] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6184client # [ 124.941990] systemd[1]: intr.service: Failed with result 'exit-code'.6185client # [ 124.943614] systemd[1]: intr.service: Consumed 119ms CPU time over 3.647s wall clock time, 39.4M memory peak, 6.5K incoming IP traffic, 8.2K outgoing IP traffic.6186worker2 # [ 124.983260] nix-daemon[1505]: unexpected Nix daemon error: error: interrupted by the user6187worker2 # [ 124.985089] nix-daemon[871]: reaped child process 1503, status = succeeded6188worker2 # [ 124.989282] nix-daemon[871]: reaped child process 1505, status = failed with exit code 16189(finished: subtest: interrupting the client stops the build on the worker, in 3.76 seconds)6190subtest: a deploy mid-build drains: the build finishes, the new generation takes over6191client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw6192client # [ 125.053253] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.6193client # Running as unit: sw.service; invocation ID: eb1c964e4de745f0a1af53e91348bb386194client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.04 seconds)6195client # [ 125.288143] nix[1266]: don't know how to build these paths:6196client # [ 125.289081] nix[1266]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6197worker1 # [ 125.305270] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:415066198worker1 # [ 125.315703] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:31Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06199worker1 # [ 125.320934] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:31Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16200worker1 # [ 125.359813] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415066201worker1 # [ 125.372956] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:415066202client # [ 125.365213] nix[1266]: copying 1 paths to 'lb:50051'...6203worker1 # [ 125.384912] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:415066204client # [ 125.368189] nix[1266]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6205worker1 # [ 125.397925] nix-daemon[850]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6206worker1 # [ 125.574483] nix-daemon[850]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6207worker1 # [ 125.586884] nix-daemon[850]: reaped child process 1655, status = succeeded6208worker1 # [ 125.749511] nix-daemon[850]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6209worker1 # [ 125.761302] nix-daemon[850]: reaped child process 1660, status = succeeded6210lb # [ 125.773259] niks3-server[1047]: 2026/09/21 12:55:31 INFO Received uploads request method=POST path=/api/pending_closures6211worker1 # [ 125.787513] nix-grpc-daemon[1604]: time=2026-09-21T12:55:31.870Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6212worker1 # [ 125.789045] nix-grpc-daemon[1604]: time=2026-09-21T12:55:31.872Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6213lb # [ 125.804306] niks3-server[1047]: 2026/09/21 12:55:31 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6214lb # [ 125.818149] niks3-server[1047]: 2026/09/21 12:55:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6215worker1 # [ 125.828499] nix-grpc-daemon[1604]: time=2026-09-21T12:55:31.911Z level=INFO msg="Uploading 1 narinfos"6216lb # [ 125.824142] niks3-server[1047]: 2026/09/21 12:55:31 INFO Signed narinfos id=38 count=16217lb # [ 125.836212] niks3-server[1047]: 2026/09/21 12:55:31 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6218lb # [ 125.867620] niks3-server[1047]: 2026/09/21 12:55:31 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6219lb # [ 125.874250] niks3-server[1047]: 2026/09/21 12:55:31 INFO Completed upload id=386220worker1 # [ 125.880833] nix-grpc-daemon[1604]: time=2026-09-21T12:55:31.962Z level=INFO msg="Upload complete. (469ms)"6221lb # [ 125.879751] niks3-server[1047]: 2026/09/21 12:55:31 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6222worker1 # [ 125.884313] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41506 duration_s=0 paths=1 nar_bytes_in=4166223worker1 # [ 125.891633] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:415066224worker1 # [ 125.903170] nix-daemon[850]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6225worker1 # [ 125.912581] nix-daemon[850]: reaped child process 1650, status = succeeded6226client # [ 125.902287] nix[1266]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6227worker1 # [ 125.921820] nix-daemon[850]: accepted connection from pid 1496, user nix-grpc-daemon (trusted)6228worker1: must succeed: systemctl reload nix-grpc-daemon.service6229worker1 # [ 127.594736] systemd[1]: Reloading Nix worker-protocol over gRPC...6230worker1 # [ 127.635067] systemd[1]: Reloaded Nix worker-protocol over gRPC.6231worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6232worker1: must succeed: pgrep -f 'read -t [9]0 x'6233worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6234client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26235worker1 # [ 127.948754] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:415066236client # don't know how to build these paths:6237client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6238worker2 # [ 127.942487] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487666239worker1 # [ 127.978151] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:34Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06240worker1 # [ 127.980068] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:34Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16241worker2 # [ 128.013763] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487666242worker2 # [ 128.026481] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487666243client # copying 1 paths to 'lb:50051'...6244worker2 # [ 128.041799] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487666245client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6246worker2 # [ 128.064699] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6247worker2 # [ 128.267871] nix-daemon[871]: accepted connection from pid 1552, user nix-grpc-daemon (trusted)6248worker2 # [ 128.279965] nix-daemon[871]: reaped child process 1555, status = succeeded6249worker1 # [ 128.432271] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:34Z level=info event=draining6250worker2 # [ 128.451815] nix-daemon[871]: accepted connection from pid 1557, user nix-grpc-daemon (trusted)6251worker2 # [ 128.463697] nix-daemon[871]: reaped child process 1560, status = succeeded6252lb # [ 128.489165] niks3-server[1047]: 2026/09/21 12:55:34 INFO Received uploads request method=POST path=/api/pending_closures6253worker2 # [ 128.491219] nix-grpc-daemon[1480]: time=2026-09-21T12:55:34.708Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6254worker2 # [ 128.492967] nix-grpc-daemon[1480]: time=2026-09-21T12:55:34.710Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6255lb # [ 128.515727] envoy[909]: conn peer=[2001:db8:1::1]:40788 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4278 tx=1975 ms=100666256lb # [ 128.520548] envoy[909]: conn peer=[2001:db8:1::1]:36138 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2967 tx=853 ms=33986257lb # [ 128.529086] niks3-server[1047]: 2026/09/21 12:55:34 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6258lb # [ 128.542551] niks3-server[1047]: 2026/09/21 12:55:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6259lb # [ 128.547426] niks3-server[1047]: 2026/09/21 12:55:34 INFO Signed narinfos id=39 count=16260worker2 # [ 128.541405] nix-grpc-daemon[1480]: time=2026-09-21T12:55:34.759Z level=INFO msg="Uploading 1 narinfos"6261lb # [ 128.559176] niks3-server[1047]: 2026/09/21 12:55:34 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6262lb # [ 128.564352] niks3-server[1047]: 2026/09/21 12:55:34 INFO Completed upload id=396263lb # [ 128.566229] niks3-server[1047]: 2026/09/21 12:55:34 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6264worker2 # [ 128.556980] nix-grpc-daemon[1480]: time=2026-09-21T12:55:34.774Z level=INFO msg="Upload complete. (480ms)"6265lb # [ 128.568515] niks3-server[1047]: 2026/09/21 12:55:34 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6266worker2 # [ 128.561753] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=0 paths=1 nar_bytes_in=5286267worker2 # [ 128.567283] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487666268worker2 # [ 128.575493] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6269worker2 # [ 128.582992] nix-daemon[871]: reaped child process 1550, status = succeeded6270client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6271worker2 # [ 128.588743] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6272client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6273client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6274client # farm-a> LOG-farm-a-during-drain6275worker2 # [ 130.013947] nix-daemon[871]: reaped child process 1564, status = succeeded6276worker2 # [ 130.206363] nix-daemon[871]: accepted connection from pid 1580, user nix-grpc-daemon (trusted)6277worker2 # [ 130.218271] nix-daemon[871]: reaped child process 1583, status = succeeded6278lb # [ 130.245788] niks3-server[1047]: 2026/09/21 12:55:36 INFO Received uploads request method=POST path=/api/pending_closures6279worker2 # [ 130.246745] nix-grpc-daemon[1480]: time=2026-09-21T12:55:36.464Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6280worker2 # [ 130.250973] nix-grpc-daemon[1480]: time=2026-09-21T12:55:36.466Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6281lb # [ 130.278129] niks3-server[1047]: 2026/09/21 12:55:36 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6282lb # [ 130.292481] niks3-server[1047]: 2026/09/21 12:55:36 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6283lb # [ 130.305542] niks3-server[1047]: 2026/09/21 12:55:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6284lb # [ 130.310674] niks3-server[1047]: 2026/09/21 12:55:36 INFO Signed narinfos id=40 count=16285worker2 # [ 130.302543] nix-grpc-daemon[1480]: time=2026-09-21T12:55:36.519Z level=INFO msg="Uploading 1 narinfos"6286lb # [ 130.315653] niks3-server[1047]: 2026/09/21 12:55:36 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6287lb # [ 130.323635] niks3-server[1047]: 2026/09/21 12:55:36 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6288lb # [ 130.328862] niks3-server[1047]: 2026/09/21 12:55:36 INFO Completed upload id=406289worker2 # [ 130.320355] nix-grpc-daemon[1480]: time=2026-09-21T12:55:36.538Z level=INFO msg="Upload complete. (324ms)"6290lb # [ 130.331320] niks3-server[1047]: 2026/09/21 12:55:36 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6291worker2 # [ 130.325885] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16292worker1 # [ 130.344400] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:36Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06293worker1 # [ 130.346163] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:36Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16294worker2 # [ 130.340195] nix-daemon[871]: reaped child process 1562, status = succeeded6295worker2 # [ 130.374791] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487666296worker2 # [ 130.387431] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487666297worker2 # [ 130.400881] nix-grpc-daemon[1480]: time=2026-09-21T12:55:36.618Z level=INFO msg="All 1 paths already cached"6298client # copying 1 paths to 'lb:50051'...6299worker2 # [ 130.408918] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487666300client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6301worker2 # [ 130.427577] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6302worker2 # [ 130.627124] nix-daemon[871]: accepted connection from pid 1587, user nix-grpc-daemon (trusted)6303worker2 # [ 130.639301] nix-daemon[871]: reaped child process 1590, status = succeeded6304worker2 # [ 130.821452] nix-daemon[871]: accepted connection from pid 1592, user nix-grpc-daemon (trusted)6305worker2 # [ 130.833737] nix-daemon[871]: reaped child process 1595, status = succeeded6306lb # [ 130.859352] niks3-server[1047]: 2026/09/21 12:55:36 INFO Received uploads request method=POST path=/api/pending_closures6307worker2 # [ 130.859245] nix-grpc-daemon[1480]: time=2026-09-21T12:55:37.077Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6308worker2 # [ 130.860794] nix-grpc-daemon[1480]: time=2026-09-21T12:55:37.078Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6309lb # [ 130.889381] niks3-server[1047]: 2026/09/21 12:55:36 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6310lb # [ 130.901780] niks3-server[1047]: 2026/09/21 12:55:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6311worker2 # [ 130.897961] nix-grpc-daemon[1480]: time=2026-09-21T12:55:37.115Z level=INFO msg="Uploading 1 narinfos"6312lb # [ 130.907984] niks3-server[1047]: 2026/09/21 12:55:36 INFO Signed narinfos id=41 count=16313lb # [ 130.913319] niks3-server[1047]: 2026/09/21 12:55:36 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6314lb # [ 130.924114] niks3-server[1047]: 2026/09/21 12:55:36 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6315lb # [ 130.931369] niks3-server[1047]: 2026/09/21 12:55:36 INFO Completed upload id=416316worker2 # [ 130.922814] nix-grpc-daemon[1480]: time=2026-09-21T12:55:37.140Z level=INFO msg="Upload complete. (480ms)"6317lb # [ 130.935979] niks3-server[1047]: 2026/09/21 12:55:36 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6318worker2 # [ 130.926362] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=0 paths=1 nar_bytes_in=6406319worker2 # [ 130.931765] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487666320worker2 # [ 130.943114] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6321worker2 # [ 130.947470] nix-daemon[871]: reaped child process 1585, status = succeeded6322worker2 # [ 130.952533] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6323client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6324client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6325client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6326client # farm-b> LOG-farm-b-during-drain6327worker2 # [ 132.377600] nix-daemon[871]: reaped child process 1599, status = succeeded6328worker2 # [ 132.525938] nix-daemon[871]: accepted connection from pid 1615, user nix-grpc-daemon (trusted)6329worker2 # [ 132.538602] nix-daemon[871]: reaped child process 1618, status = succeeded6330lb # [ 132.560760] niks3-server[1047]: 2026/09/21 12:55:38 INFO Received uploads request method=POST path=/api/pending_closures6331worker2 # [ 132.556947] nix-grpc-daemon[1480]: time=2026-09-21T12:55:38.774Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6332worker2 # [ 132.558484] nix-grpc-daemon[1480]: time=2026-09-21T12:55:38.776Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6333lb # [ 132.578492] niks3-server[1047]: 2026/09/21 12:55:38 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6334lb # [ 132.583627] niks3-server[1047]: 2026/09/21 12:55:38 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6335lb # [ 132.589889] niks3-server[1047]: 2026/09/21 12:55:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6336lb # [ 132.592059] niks3-server[1047]: 2026/09/21 12:55:38 INFO Signed narinfos id=42 count=16337worker2 # [ 132.583424] nix-grpc-daemon[1480]: time=2026-09-21T12:55:38.801Z level=INFO msg="Uploading 1 narinfos"6338lb # [ 132.595438] niks3-server[1047]: 2026/09/21 12:55:38 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6339lb # [ 132.600155] niks3-server[1047]: 2026/09/21 12:55:38 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6340lb # [ 132.603990] niks3-server[1047]: 2026/09/21 12:55:38 INFO Completed upload id=426341worker2 # [ 132.595049] nix-grpc-daemon[1480]: time=2026-09-21T12:55:38.812Z level=INFO msg="Upload complete. (232ms)"6342lb # [ 132.605454] niks3-server[1047]: 2026/09/21 12:55:38 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6343worker2 # [ 132.597466] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16344worker2 # [ 132.604318] nix-daemon[871]: reaped child process 1597, status = succeeded6345worker1 # [ 132.619046] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:38Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06346worker1 # [ 132.621035] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:38Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16347worker2 # [ 132.647740] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487666348worker2 # [ 132.655205] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487666349worker2 # [ 132.662934] nix-grpc-daemon[1480]: time=2026-09-21T12:55:38.880Z level=INFO msg="All 2 paths already cached"6350client # copying 1 paths to 'lb:50051'...6351worker2 # [ 132.666465] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487666352client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6353worker2 # [ 132.671931] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6354worker2 # [ 132.799577] nix-daemon[871]: accepted connection from pid 1622, user nix-grpc-daemon (trusted)6355worker2 # [ 132.812053] nix-daemon[871]: reaped child process 1625, status = succeeded6356worker2 # [ 132.961144] nix-daemon[871]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6357worker2 # [ 132.973857] nix-daemon[871]: reaped child process 1630, status = succeeded6358lb # [ 132.993107] niks3-server[1047]: 2026/09/21 12:55:38 INFO Received uploads request method=POST path=/api/pending_closures6359worker2 # [ 132.991113] nix-grpc-daemon[1480]: time=2026-09-21T12:55:39.208Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6360worker2 # [ 132.992459] nix-grpc-daemon[1480]: time=2026-09-21T12:55:39.208Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6361lb # [ 133.012417] niks3-server[1047]: 2026/09/21 12:55:38 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6362lb # [ 133.021510] niks3-server[1047]: 2026/09/21 12:55:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6363lb # [ 133.025920] niks3-server[1047]: 2026/09/21 12:55:38 INFO Signed narinfos id=43 count=16364worker2 # [ 133.019571] nix-grpc-daemon[1480]: time=2026-09-21T12:55:39.237Z level=INFO msg="Uploading 1 narinfos"6365lb # [ 133.029082] niks3-server[1047]: 2026/09/21 12:55:38 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6366lb # [ 133.036805] niks3-server[1047]: 2026/09/21 12:55:38 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6367lb # [ 133.040698] niks3-server[1047]: 2026/09/21 12:55:38 INFO Completed upload id=436368worker2 # [ 133.033200] nix-grpc-daemon[1480]: time=2026-09-21T12:55:39.251Z level=INFO msg="Upload complete. (355ms)"6369lb # [ 133.042370] niks3-server[1047]: 2026/09/21 12:55:38 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6370worker2 # [ 133.035097] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=0 paths=1 nar_bytes_in=7686371worker2 # [ 133.038086] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487666372worker2 # [ 133.044945] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6373worker2 # [ 133.047263] nix-daemon[871]: reaped child process 1620, status = succeeded6374client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6375worker2 # [ 133.052971] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6376client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6377client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6378client # farm-top> LOG-farm-top-during-drain6379worker2 # [ 133.665216] nix-daemon[871]: reaped child process 1634, status = succeeded6380worker2 # [ 133.790319] nix-daemon[871]: accepted connection from pid 1650, user nix-grpc-daemon (trusted)6381worker2 # [ 133.801983] nix-daemon[871]: reaped child process 1653, status = succeeded6382lb # [ 133.820756] niks3-server[1047]: 2026/09/21 12:55:39 INFO Received uploads request method=POST path=/api/pending_closures6383worker2 # [ 133.820359] nix-grpc-daemon[1480]: time=2026-09-21T12:55:40.037Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6384worker2 # [ 133.823823] nix-grpc-daemon[1480]: time=2026-09-21T12:55:40.040Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6385lb # [ 133.856285] niks3-server[1047]: 2026/09/21 12:55:39 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6386lb # [ 133.868298] niks3-server[1047]: 2026/09/21 12:55:39 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6387lb # [ 133.872481] niks3-server[1047]: 2026/09/21 12:55:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6388lb # [ 133.877308] niks3-server[1047]: 2026/09/21 12:55:39 INFO Signed narinfos id=44 count=16389worker2 # [ 133.871525] nix-grpc-daemon[1480]: time=2026-09-21T12:55:40.089Z level=INFO msg="Uploading 1 narinfos"6390lb # [ 133.889486] niks3-server[1047]: 2026/09/21 12:55:39 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6391lb # [ 133.894759] niks3-server[1047]: 2026/09/21 12:55:39 INFO Completed upload id=446392worker2 # [ 133.887277] nix-grpc-daemon[1480]: time=2026-09-21T12:55:40.105Z level=INFO msg="Upload complete. (228ms)"6393lb # [ 133.896468] niks3-server[1047]: 2026/09/21 12:55:39 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6394lb # [ 133.899051] niks3-server[1047]: 2026/09/21 12:55:39 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6395worker2 # [ 133.889990] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=0 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16396worker2 # [ 133.896700] nix-daemon[871]: reaped child process 1632, status = succeeded6397client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.26 seconds)6398client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6399client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6400worker1: must succeed: pkill -f 'read -t [9]0 x'6401worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6402client: waiting for success: ! systemctl is-active sw6403worker1 # [ 134.356397] nix-daemon[850]: reaped child process 1664, status = succeeded6404worker1 # [ 134.484075] nix-daemon[850]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6405worker1 # [ 134.496588] nix-daemon[850]: reaped child process 1715, status = succeeded6406lb # [ 134.505591] niks3-server[1047]: 2026/09/21 12:55:40 INFO Received uploads request method=POST path=/api/pending_closures6407worker1 # [ 134.519498] nix-grpc-daemon[1604]: time=2026-09-21T12:55:40.602Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6408worker1 # [ 134.521757] nix-grpc-daemon[1604]: time=2026-09-21T12:55:40.604Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6409lb # [ 134.541975] niks3-server[1047]: 2026/09/21 12:55:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6410worker1 # [ 134.552053] nix-grpc-daemon[1604]: time=2026-09-21T12:55:40.634Z level=INFO msg="Uploading 1 narinfos"6411lb # [ 134.547584] niks3-server[1047]: 2026/09/21 12:55:40 INFO Signed narinfos id=45 count=16412lb # [ 134.549850] niks3-server[1047]: 2026/09/21 12:55:40 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6413lb # [ 134.557409] niks3-server[1047]: 2026/09/21 12:55:40 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6414lb # [ 134.561639] niks3-server[1047]: 2026/09/21 12:55:40 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6415lb # [ 134.568146] niks3-server[1047]: 2026/09/21 12:55:40 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6416lb # [ 134.571062] niks3-server[1047]: 2026/09/21 12:55:40 INFO Completed upload id=456417worker1 # [ 134.577524] nix-grpc-daemon[1604]: time=2026-09-21T12:55:40.660Z level=INFO msg="Upload complete. (237ms)"6418lb # [ 134.573991] niks3-server[1047]: 2026/09/21 12:55:40 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6419worker1 # [ 134.579933] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41506 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16420worker1 # [ 134.587776] nix-daemon[850]: reaped child process 1662, status = succeeded6421client # [ 134.590333] systemd[1]: sw.service: Deactivated successfully.6422client # [ 134.593707] systemd[1]: sw.service: Consumed 115ms CPU time over 9.539s wall clock time, 43.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6423client: (finished: waiting for success: ! systemctl is-active sw, in 1.07 seconds)6424client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6425client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6426worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6427worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.03 seconds)6428lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6429lb: must succeed: curl -sf localhost:9901/clusters6430lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6431worker1 # [ 135.434464] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:41Z level=info event=signal_exit6432worker1 # [ 135.663178] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:41Z level=info event=scheduler_yield6433lb # [ 135.659432] niks3-server[1047]: 2026/09/21 12:55:41 INFO lead: released remote=192.168.1.3:529246434worker2 # [ 135.650616] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:41Z level=info event=scheduler_restarting addr=lb:500516435worker1 # [ 135.666403] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:414966436worker1 # [ 135.667723] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:41Z level=info event=worker_gone addr=192.168.1.4:500516437worker1 # [ 135.669857] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:41Z level=info event=scheduler_restarting addr=lb:500516438worker1 # [ 135.671137] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:414966439worker1 # [ 135.672524] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:41Z level=info event=worker_gone addr=192.168.1.3:500516440worker2 # [ 135.850992] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_connected addr=lb:500516441worker2 # [ 135.853027] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6442worker1 # [ 135.870042] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:41Z level=info event=scheduler_connected addr=lb:500516443worker1 # [ 135.872151] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6444worker2 # [ 136.053954] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_connected addr=lb:500516445worker2 # [ 136.056182] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6446worker1 # [ 136.072547] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:42Z level=info event=scheduler_connected addr=lb:500516447worker1 # [ 136.074585] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6448worker2 # [ 136.257155] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_connected addr=lb:500516449worker2 # [ 136.259341] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6450worker1 # [ 136.275319] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:42Z level=info event=scheduler_connected addr=lb:500516451worker1 # [ 136.277254] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6452lb: must succeed: curl -sf localhost:9901/clusters6453lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6454worker2 # [ 136.460173] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_connected addr=lb:500516455worker2 # [ 136.462172] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6456worker1 # [ 136.477688] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:42Z level=info event=scheduler_connected addr=lb:500516457worker1 # [ 136.479865] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6458worker1 # [ 136.664177] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:42Z level=info event=scheduler_restarting6459worker2 # [ 136.663771] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_connected addr=lb:500516460worker2 # [ 136.666072] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6461worker1 # [ 136.688115] nix-grpc-daemon[1496]: ts=2026-09-21T12:55:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6462worker1 # [ 136.690786] nix-daemon[850]: reaped child process 1570, status = succeeded6463worker2 # [ 136.866798] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_connected addr=lb:500516464worker2 # [ 136.869936] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6465worker2 # [ 137.070836] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_connected addr=lb:500516466worker2 # [ 137.073690] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6467worker1 # [ 137.265504] nix-grpc-daemon[1496]: E0000 00:00:1789995343.348436 1541 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6468worker1 # [ 137.268502] nix-grpc-daemon[1496]: E0000 00:00:1789995343.348598 1541 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6469worker2 # [ 137.274511] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_connected addr=lb:500516470worker2 # [ 137.276543] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6471lb: must succeed: curl -sf localhost:9901/clusters6472lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6473worker1 # [ 137.465355] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6474worker1 # [ 137.466463] systemd[1]: nix-grpc-daemon.service: Consumed 719ms CPU time over 28.224s wall clock time, 33.2M memory peak, 28.1K incoming IP traffic, 37.4K outgoing IP traffic.6475worker2 # [ 137.477408] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_connected addr=lb:500516476worker2 # [ 137.479429] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6477worker2 # [ 137.681090] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_connected addr=lb:500516478worker2 # [ 137.683448] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6479worker2 # [ 137.884253] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_connected addr=lb:500516480worker2 # [ 137.886554] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6481worker2 # [ 138.087470] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_connected addr=lb:500516482worker2 # [ 138.089747] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6483worker2 # [ 138.291310] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_connected addr=lb:500516484worker2 # [ 138.293617] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6485worker1 # [ 138.459910] systemd[1]: Starting Nix worker-protocol over gRPC...6486lb: must succeed: curl -sf localhost:9901/clusters6487worker2 # [ 138.494817] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_connected addr=lb:500516488worker2 # [ 138.497181] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6489lb # [ 138.515429] envoy[909]: conn peer=[2001:db8:1::1]:48214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9585 tx=4066 ms=59776490lb # [ 138.517533] envoy[909]: conn peer=[2001:db8:1::1]:48204 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2962 tx=1404 ms=92886491lb # [ 138.519310] envoy[909]: conn peer=[2001:db8:1::3]:39648 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3338 tx=928 ms=272146492lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6493worker1 # [ 138.646539] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6494worker1 # [ 138.648576] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6495worker1 # [ 138.653060] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16496worker1 # [ 138.660082] nix-grpc-daemon[1727]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6497worker1 # [ 138.661935] nix-grpc-daemon[1727]: E0000 00:00:1789995344.742641 1727 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6498worker1 # [ 138.666876] nix-grpc-daemon[1727]: E0000 00:00:1789995344.748864 1727 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6499worker1 # [ 138.673137] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6500worker1 # [ 138.678253] systemd[1]: Started Nix worker-protocol over gRPC.6501lb # [ 138.678410] niks3-server[1047]: 2026/09/21 12:55:44 INFO lead: acquired remote=192.168.1.3:546186502worker1 # [ 138.686079] nix-grpc-daemon[1727]: E0000 00:00:1789995344.768321 1769 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6503worker1 # [ 138.691086] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:44Z level=info event=scheduler_take_over6504worker1 # [ 138.706090] nix-grpc-daemon[1727]: E0000 00:00:1789995344.789060 1769 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6505worker1 # [ 138.711128] nix-grpc-daemon[1727]: E0000 00:00:1789995344.792264 1769 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6506worker2 # [ 138.698626] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_connected addr=lb:500516507worker2 # [ 138.701137] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6508worker1 # [ 138.722736] nix-grpc-daemon[1727]: E0000 00:00:1789995344.805790 1769 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6509worker1 # [ 138.732476] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:44Z level=info event=scheduler_connected addr=lb:500516510worker1 # [ 138.734755] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6511worker2 # [ 138.902023] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516512worker2 # [ 138.903973] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6513worker1 # [ 138.935281] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516514worker1 # [ 138.937607] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6515worker2 # [ 139.104759] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516516worker2 # [ 139.106518] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6517worker1 # [ 139.138431] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516518worker1 # [ 139.140739] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6519worker2 # [ 139.307476] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516520worker2 # [ 139.309550] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6521worker1 # [ 139.341185] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516522worker1 # [ 139.343322] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6523worker2 # [ 139.511247] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516524worker2 # [ 139.515989] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6525lb: must succeed: curl -sf localhost:9901/clusters6526worker1 # [ 139.544265] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516527worker1 # [ 139.547609] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6528lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6529lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 4.29 seconds)6530lb: must succeed: curl -sf localhost:9901/clusters6531lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6532worker2 # [ 139.717349] nix-grpc-daemon[1404]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516533worker1 # [ 139.736803] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=oidc_jwks provider=mock keys=16534worker1 # [ 139.738379] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:579006535worker1 # [ 139.739989] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06536worker1 # [ 139.747794] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=scheduler_connected addr=lb:500516537worker1 # [ 139.749528] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:579006538worker1 # [ 139.751068] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06539lb: must succeed: curl -sf localhost:9901/clusters6540lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6541worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6542worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6543(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.67 seconds)6544subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6545client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16546worker1 # [ 140.895482] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:579086547worker1 # [ 140.907473] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6548worker1 # [ 140.924378] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086549worker1 # [ 140.936696] nix-grpc-daemon[1727]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)6550worker1 # [ 140.965253] nix-grpc-daemon[1727]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 2/5)6551worker1 # [ 141.090204] nix-grpc-daemon[1727]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 3/5)6552worker1 # [ 141.187735] nix-grpc-daemon[1727]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 663 ms (attempt 4/5)6553worker1 # [ 141.857806] nix-grpc-daemon[1727]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6554worker1 # [ 141.860063] nix-grpc-daemon[1727]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6555worker1 # [ 141.886379] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:47Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06556worker1 # [ 141.888875] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:47Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16557worker1 # [ 141.890767] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:47Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06558worker1 # [ 141.892378] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:47Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06559worker1 # [ 141.930486] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086560worker1 # [ 141.936982] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086561worker1 # [ 141.942667] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579086562worker1 # [ 141.948464] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6563worker1 # [ 141.958532] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:48Z level=info event=niks3_push_started pid=17926564worker1 # [ 142.096737] nix-grpc-daemon[1792]: time=2026-09-21T12:55:48.179Z level=INFO msg="Configuring client TLS" cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.pem key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/worker1.key ca=""6565worker1 # [ 142.221475] nix-daemon[850]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)6566worker1 # [ 142.233826] nix-daemon[850]: reaped child process 1808, status = succeeded6567worker1 # [ 142.388171] nix-daemon[850]: accepted connection from pid 1810, user nix-grpc-daemon (trusted)6568worker1 # [ 142.401526] nix-daemon[850]: reaped child process 1813, status = succeeded6569lb # [ 142.410445] niks3-server[1047]: 2026/09/21 12:55:48 INFO Received uploads request method=POST path=/api/pending_closures6570worker1 # [ 142.431417] nix-grpc-daemon[1792]: time=2026-09-21T12:55:48.514Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6571worker1 # [ 142.434051] nix-grpc-daemon[1792]: time=2026-09-21T12:55:48.516Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6572lb # [ 142.452875] niks3-server[1047]: 2026/09/21 12:55:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6573lb # [ 142.457087] niks3-server[1047]: 2026/09/21 12:55:48 INFO Signed narinfos id=46 count=16574worker1 # [ 142.462353] nix-grpc-daemon[1792]: time=2026-09-21T12:55:48.545Z level=INFO msg="Uploading 1 narinfos"6575lb # [ 142.461162] niks3-server[1047]: 2026/09/21 12:55:48 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6576lb # [ 142.469139] niks3-server[1047]: 2026/09/21 12:55:48 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6577lb # [ 142.476638] niks3-server[1047]: 2026/09/21 12:55:48 INFO Completed upload id=466578worker1 # [ 142.482160] nix-grpc-daemon[1792]: time=2026-09-21T12:55:48.565Z level=INFO msg="Upload complete. (381ms)"6579lb # [ 142.479504] niks3-server[1047]: 2026/09/21 12:55:48 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6580worker1 # [ 142.486078] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 paths=1 nar_bytes_in=5046581worker1 # [ 142.489122] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086582lb # [ 142.485314] niks3-server[1047]: 2026/09/21 12:55:48 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6583worker1 # [ 142.496258] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6584worker1 # [ 142.498585] nix-daemon[850]: reaped child process 1790, status = succeeded6585worker1 # [ 142.503741] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6586worker1 # [ 143.916414] nix-daemon[850]: reaped child process 1817, status = succeeded6587worker1 # [ 144.051412] nix-daemon[850]: accepted connection from pid 1833, user nix-grpc-daemon (trusted)6588worker1 # [ 144.063642] nix-daemon[850]: reaped child process 1836, status = succeeded6589lb # [ 144.072557] niks3-server[1047]: 2026/09/21 12:55:49 INFO Received uploads request method=POST path=/api/pending_closures6590worker1 # [ 144.089237] nix-grpc-daemon[1792]: time=2026-09-21T12:55:50.172Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6591worker1 # [ 144.090852] nix-grpc-daemon[1792]: time=2026-09-21T12:55:50.173Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6592lb # [ 144.112978] niks3-server[1047]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6593lb # [ 144.121929] niks3-server[1047]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6594lb # [ 144.133977] niks3-server[1047]: 2026/09/21 12:55:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6595worker1 # [ 144.143353] nix-grpc-daemon[1792]: time=2026-09-21T12:55:50.226Z level=INFO msg="Uploading 1 narinfos"6596lb # [ 144.138446] niks3-server[1047]: 2026/09/21 12:55:50 INFO Signed narinfos id=47 count=16597lb # [ 144.143392] niks3-server[1047]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6598lb # [ 144.151770] niks3-server[1047]: 2026/09/21 12:55:50 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6599lb # [ 144.159641] niks3-server[1047]: 2026/09/21 12:55:50 INFO Completed upload id=476600lb # [ 144.160852] niks3-server[1047]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6601worker1 # [ 144.165066] nix-grpc-daemon[1792]: time=2026-09-21T12:55:50.248Z level=INFO msg="Upload complete. (254ms)"6602worker1 # [ 144.172865] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16603worker1 # [ 144.183300] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:50Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16604worker1 # [ 144.187189] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:50Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06605worker1 # [ 144.194966] nix-daemon[850]: reaped child process 1815, status = succeeded6606worker1 # [ 144.224872] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086607worker1 # [ 144.237082] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086608worker1 # [ 144.248557] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579086609worker1 # [ 144.258157] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6610worker1 # [ 144.418122] nix-daemon[850]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6611worker1 # [ 144.430188] nix-daemon[850]: reaped child process 1845, status = succeeded6612worker1 # [ 144.581197] nix-daemon[850]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)6613worker1 # [ 144.593166] nix-daemon[850]: reaped child process 1850, status = succeeded6614lb # [ 144.603764] niks3-server[1047]: 2026/09/21 12:55:50 INFO Received uploads request method=POST path=/api/pending_closures6615worker1 # [ 144.616880] nix-grpc-daemon[1792]: time=2026-09-21T12:55:50.700Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6616worker1 # [ 144.618782] nix-grpc-daemon[1792]: time=2026-09-21T12:55:50.701Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6617lb # [ 144.632156] niks3-server[1047]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6618lb # [ 144.640458] niks3-server[1047]: 2026/09/21 12:55:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6619lb # [ 144.645132] niks3-server[1047]: 2026/09/21 12:55:50 INFO Signed narinfos id=48 count=16620worker1 # [ 144.653337] nix-grpc-daemon[1792]: time=2026-09-21T12:55:50.736Z level=INFO msg="Uploading 1 narinfos"6621lb # [ 144.651946] niks3-server[1047]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6622lb # [ 144.664423] niks3-server[1047]: 2026/09/21 12:55:50 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6623lb # [ 144.669273] niks3-server[1047]: 2026/09/21 12:55:50 INFO Completed upload id=486624worker1 # [ 144.676959] nix-grpc-daemon[1792]: time=2026-09-21T12:55:50.760Z level=INFO msg="Upload complete. (408ms)"6625lb # [ 144.672316] niks3-server[1047]: 2026/09/21 12:55:50 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6626worker1 # [ 144.680774] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 paths=1 nar_bytes_in=5046627worker1 # [ 144.686294] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086628worker1 # [ 144.695274] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6629worker1 # [ 144.700230] nix-daemon[850]: reaped child process 1840, status = succeeded6630worker1 # [ 144.705255] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6631worker1 # [ 145.718812] nix-daemon[850]: reaped child process 1854, status = succeeded6632worker1 # [ 145.859371] nix-daemon[850]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6633worker1 # [ 145.870724] nix-daemon[850]: reaped child process 1873, status = succeeded6634lb # [ 145.881165] niks3-server[1047]: 2026/09/21 12:55:51 INFO Received uploads request method=POST path=/api/pending_closures6635worker1 # [ 145.895491] nix-grpc-daemon[1792]: time=2026-09-21T12:55:51.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6636worker1 # [ 145.896965] nix-grpc-daemon[1792]: time=2026-09-21T12:55:51.980Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6637lb # [ 145.917676] niks3-server[1047]: 2026/09/21 12:55:51 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6638lb # [ 145.923956] niks3-server[1047]: 2026/09/21 12:55:51 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6639lb # [ 145.933216] niks3-server[1047]: 2026/09/21 12:55:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6640lb # [ 145.938232] niks3-server[1047]: 2026/09/21 12:55:51 INFO Signed narinfos id=49 count=16641worker1 # [ 145.946165] nix-grpc-daemon[1792]: time=2026-09-21T12:55:52.028Z level=INFO msg="Uploading 1 narinfos"6642lb # [ 145.943124] niks3-server[1047]: 2026/09/21 12:55:51 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6643lb # [ 145.948910] niks3-server[1047]: 2026/09/21 12:55:51 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6644lb # [ 145.951723] niks3-server[1047]: 2026/09/21 12:55:51 INFO Completed upload id=496645lb # [ 145.953584] niks3-server[1047]: 2026/09/21 12:55:51 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6646worker1 # [ 145.958126] nix-grpc-daemon[1792]: time=2026-09-21T12:55:52.041Z level=INFO msg="Upload complete. (246ms)"6647worker1 # [ 145.962894] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16648worker1 # [ 145.972655] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:52Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16649worker1 # [ 145.976249] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:52Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06650worker1 # [ 145.983364] nix-daemon[850]: reaped child process 1852, status = succeeded6651worker1 # [ 146.012116] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086652worker1 # [ 146.022439] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086653worker1 # [ 146.033388] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579086654worker1 # [ 146.041457] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6655worker1 # [ 146.204314] nix-daemon[850]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)6656worker1 # [ 146.216610] nix-daemon[850]: reaped child process 1882, status = succeeded6657worker1 # [ 146.359578] nix-daemon[850]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6658worker1 # [ 146.371441] nix-daemon[850]: reaped child process 1887, status = succeeded6659lb # [ 146.376714] niks3-server[1047]: 2026/09/21 12:55:52 INFO Received uploads request method=POST path=/api/pending_closures6660worker1 # [ 146.391637] nix-grpc-daemon[1792]: time=2026-09-21T12:55:52.474Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6661worker1 # [ 146.393426] nix-grpc-daemon[1792]: time=2026-09-21T12:55:52.476Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6662lb # [ 146.407333] niks3-server[1047]: 2026/09/21 12:55:52 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6663lb # [ 146.418276] niks3-server[1047]: 2026/09/21 12:55:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6664worker1 # [ 146.428564] nix-grpc-daemon[1792]: time=2026-09-21T12:55:52.511Z level=INFO msg="Uploading 1 narinfos"6665lb # [ 146.423475] niks3-server[1047]: 2026/09/21 12:55:52 INFO Signed narinfos id=50 count=16666lb # [ 146.428804] niks3-server[1047]: 2026/09/21 12:55:52 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6667lb # [ 146.442173] niks3-server[1047]: 2026/09/21 12:55:52 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6668lb # [ 146.447802] niks3-server[1047]: 2026/09/21 12:55:52 INFO Completed upload id=506669worker1 # [ 146.452920] nix-grpc-daemon[1792]: time=2026-09-21T12:55:52.536Z level=INFO msg="Upload complete. (400ms)"6670lb # [ 146.451613] niks3-server[1047]: 2026/09/21 12:55:52 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6671worker1 # [ 146.456335] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 paths=1 nar_bytes_in=5046672worker1 # [ 146.462265] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086673worker1 # [ 146.470523] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6674worker1 # [ 146.476357] nix-daemon[850]: reaped child process 1877, status = succeeded6675worker1 # [ 146.481981] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6676worker1 # [ 147.505349] nix-daemon[850]: reaped child process 1891, status = succeeded6677worker1 # [ 147.668374] nix-daemon[850]: accepted connection from pid 1907, user nix-grpc-daemon (trusted)6678worker1 # [ 147.680925] nix-daemon[850]: reaped child process 1910, status = succeeded6679lb # [ 147.693109] niks3-server[1047]: 2026/09/21 12:55:53 INFO Received uploads request method=POST path=/api/pending_closures6680worker1 # [ 147.706576] nix-grpc-daemon[1792]: time=2026-09-21T12:55:53.789Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6681worker1 # [ 147.708176] nix-grpc-daemon[1792]: time=2026-09-21T12:55:53.791Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6682lb # [ 147.729994] niks3-server[1047]: 2026/09/21 12:55:53 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6683lb # [ 147.735186] niks3-server[1047]: 2026/09/21 12:55:53 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6684lb # [ 147.748062] niks3-server[1047]: 2026/09/21 12:55:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6685lb # [ 147.752429] niks3-server[1047]: 2026/09/21 12:55:53 INFO Signed narinfos id=51 count=16686worker1 # [ 147.759986] nix-grpc-daemon[1792]: time=2026-09-21T12:55:53.843Z level=INFO msg="Uploading 1 narinfos"6687lb # [ 147.758909] niks3-server[1047]: 2026/09/21 12:55:53 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6688lb # [ 147.768520] niks3-server[1047]: 2026/09/21 12:55:53 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6689lb # [ 147.773081] niks3-server[1047]: 2026/09/21 12:55:53 INFO Completed upload id=516690lb # [ 147.776365] niks3-server[1047]: 2026/09/21 12:55:53 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6691worker1 # [ 147.780450] nix-grpc-daemon[1792]: time=2026-09-21T12:55:53.863Z level=INFO msg="Upload complete. (292ms)"6692worker1 # [ 147.792814] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:53Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16693worker1 # [ 147.798352] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16694worker1 # [ 147.802756] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086695worker1 # [ 147.806074] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:53Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06696worker1 # [ 147.816372] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086697worker1 # [ 147.818909] nix-daemon[850]: reaped child process 1889, status = succeeded6698worker1 # [ 147.824997] nix-grpc-daemon[1792]: time=2026-09-21T12:55:53.908Z level=INFO msg="All 1 paths already cached"6699worker1 # [ 147.829033] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579086700worker1 # [ 147.833986] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6701worker1 # [ 147.980518] nix-daemon[850]: accepted connection from pid 1916, user nix-grpc-daemon (trusted)6702worker1 # [ 147.992641] nix-daemon[850]: reaped child process 1919, status = succeeded6703worker1 # [ 148.147916] nix-daemon[850]: accepted connection from pid 1921, user nix-grpc-daemon (trusted)6704worker1 # [ 148.159842] nix-daemon[850]: reaped child process 1924, status = succeeded6705lb # [ 148.167879] niks3-server[1047]: 2026/09/21 12:55:54 INFO Received uploads request method=POST path=/api/pending_closures6706worker1 # [ 148.181131] nix-grpc-daemon[1792]: time=2026-09-21T12:55:54.264Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6707worker1 # [ 148.182510] nix-grpc-daemon[1792]: time=2026-09-21T12:55:54.264Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6708lb # [ 148.194882] niks3-server[1047]: 2026/09/21 12:55:54 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6709lb # [ 148.201299] niks3-server[1047]: 2026/09/21 12:55:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6710lb # [ 148.205345] niks3-server[1047]: 2026/09/21 12:55:54 INFO Signed narinfos id=52 count=16711worker1 # [ 148.213526] nix-grpc-daemon[1792]: time=2026-09-21T12:55:54.296Z level=INFO msg="Uploading 1 narinfos"6712lb # [ 148.216905] niks3-server[1047]: 2026/09/21 12:55:54 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6713lb # [ 148.224104] niks3-server[1047]: 2026/09/21 12:55:54 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6714lb # [ 148.232915] niks3-server[1047]: 2026/09/21 12:55:54 INFO Completed upload id=526715worker1 # [ 148.240109] nix-grpc-daemon[1792]: time=2026-09-21T12:55:54.323Z level=INFO msg="Upload complete. (401ms)"6716lb # [ 148.236984] niks3-server[1047]: 2026/09/21 12:55:54 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6717worker1 # [ 148.243501] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 paths=1 nar_bytes_in=6246718worker1 # [ 148.248290] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086719worker1 # [ 148.256499] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6720worker1 # [ 148.262061] nix-daemon[850]: reaped child process 1914, status = succeeded6721worker1 # [ 148.266990] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6722worker1 # [ 149.692474] nix-daemon[850]: reaped child process 1928, status = succeeded6723worker1 # [ 149.855196] nix-daemon[850]: accepted connection from pid 1944, user nix-grpc-daemon (trusted)6724worker1 # [ 149.867480] nix-daemon[850]: reaped child process 1947, status = succeeded6725lb # [ 149.873733] niks3-server[1047]: 2026/09/21 12:55:55 INFO Received uploads request method=POST path=/api/pending_closures6726worker1 # [ 149.887660] nix-grpc-daemon[1792]: time=2026-09-21T12:55:55.970Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6727worker1 # [ 149.889180] nix-grpc-daemon[1792]: time=2026-09-21T12:55:55.972Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6728lb # [ 149.911371] niks3-server[1047]: 2026/09/21 12:55:55 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6729lb # [ 149.921153] niks3-server[1047]: 2026/09/21 12:55:55 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6730lb # [ 149.929722] niks3-server[1047]: 2026/09/21 12:55:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6731lb # [ 149.936247] niks3-server[1047]: 2026/09/21 12:55:55 INFO Signed narinfos id=53 count=16732worker1 # [ 149.944200] nix-grpc-daemon[1792]: time=2026-09-21T12:55:56.027Z level=INFO msg="Uploading 1 narinfos"6733lb # [ 149.939243] niks3-server[1047]: 2026/09/21 12:55:55 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6734lb # [ 149.950550] niks3-server[1047]: 2026/09/21 12:55:55 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6735lb # [ 149.956612] niks3-server[1047]: 2026/09/21 12:55:55 INFO Completed upload id=536736worker1 # [ 149.961835] nix-grpc-daemon[1792]: time=2026-09-21T12:55:56.044Z level=INFO msg="Upload complete. (286ms)"6737lb # [ 149.959809] niks3-server[1047]: 2026/09/21 12:55:55 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6738worker1 # [ 149.966864] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16739worker1 # [ 149.979525] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:56Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16740worker1 # [ 149.982933] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:56Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06741worker1 # [ 149.991669] nix-daemon[850]: reaped child process 1926, status = succeeded6742worker1 # [ 150.019807] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086743worker1 # [ 150.032138] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086744worker1 # [ 150.044774] nix-grpc-daemon[1792]: time=2026-09-21T12:55:56.127Z level=INFO msg="All 1 paths already cached"6745worker1 # [ 150.052331] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579086746worker1 # [ 150.061162] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6747worker1 # [ 150.202290] nix-daemon[850]: accepted connection from pid 1953, user nix-grpc-daemon (trusted)6748worker1 # [ 150.214981] nix-daemon[850]: reaped child process 1956, status = succeeded6749worker1 # [ 150.345830] nix-daemon[850]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6750worker1 # [ 150.357747] nix-daemon[850]: reaped child process 1961, status = succeeded6751lb # [ 150.363613] niks3-server[1047]: 2026/09/21 12:55:56 INFO Received uploads request method=POST path=/api/pending_closures6752worker1 # [ 150.376470] nix-grpc-daemon[1792]: time=2026-09-21T12:55:56.459Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6753worker1 # [ 150.377913] nix-grpc-daemon[1792]: time=2026-09-21T12:55:56.461Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6754lb # [ 150.390092] niks3-server[1047]: 2026/09/21 12:55:56 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6755lb # [ 150.402689] niks3-server[1047]: 2026/09/21 12:55:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6756lb # [ 150.409146] niks3-server[1047]: 2026/09/21 12:55:56 INFO Signed narinfos id=54 count=16757worker1 # [ 150.417237] nix-grpc-daemon[1792]: time=2026-09-21T12:55:56.500Z level=INFO msg="Uploading 1 narinfos"6758lb # [ 150.412779] niks3-server[1047]: 2026/09/21 12:55:56 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6759lb # [ 150.426335] niks3-server[1047]: 2026/09/21 12:55:56 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6760lb # [ 150.432357] niks3-server[1047]: 2026/09/21 12:55:56 INFO Completed upload id=546761worker1 # [ 150.440251] nix-grpc-daemon[1792]: time=2026-09-21T12:55:56.523Z level=INFO msg="Upload complete. (368ms)"6762lb # [ 150.435406] niks3-server[1047]: 2026/09/21 12:55:56 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6763worker1 # [ 150.444118] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 paths=1 nar_bytes_in=6246764worker1 # [ 150.449291] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086765worker1 # [ 150.457555] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6766worker1 # [ 150.464978] nix-daemon[850]: reaped child process 1951, status = succeeded6767worker1 # [ 150.470162] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6768worker1 # [ 151.295609] nix-daemon[850]: reaped child process 1965, status = succeeded6769worker1 # [ 151.453954] nix-daemon[850]: accepted connection from pid 1981, user nix-grpc-daemon (trusted)6770worker1 # [ 151.465276] nix-daemon[850]: reaped child process 1984, status = succeeded6771lb # [ 151.475579] niks3-server[1047]: 2026/09/21 12:55:57 INFO Received uploads request method=POST path=/api/pending_closures6772worker1 # [ 151.491944] nix-grpc-daemon[1792]: time=2026-09-21T12:55:57.574Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6773worker1 # [ 151.493413] nix-grpc-daemon[1792]: time=2026-09-21T12:55:57.576Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6774lb # [ 151.522350] niks3-server[1047]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6775lb # [ 151.529893] niks3-server[1047]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6776lb # [ 151.537225] niks3-server[1047]: 2026/09/21 12:55:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6777lb # [ 151.542357] niks3-server[1047]: 2026/09/21 12:55:57 INFO Signed narinfos id=55 count=16778worker1 # [ 151.550471] nix-grpc-daemon[1792]: time=2026-09-21T12:55:57.633Z level=INFO msg="Uploading 1 narinfos"6779lb # [ 151.548944] niks3-server[1047]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6780lb # [ 151.557478] niks3-server[1047]: 2026/09/21 12:55:57 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6781lb # [ 151.560616] niks3-server[1047]: 2026/09/21 12:55:57 INFO Completed upload id=556782lb # [ 151.562177] niks3-server[1047]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6783worker1 # [ 151.567189] nix-grpc-daemon[1792]: time=2026-09-21T12:55:57.650Z level=INFO msg="Upload complete. (289ms)"6784worker1 # [ 151.577242] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16785worker1 # [ 151.583578] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:57Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16786worker1 # [ 151.588852] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086787worker1 # [ 151.593424] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:57Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06788worker1 # [ 151.603713] nix-daemon[850]: reaped child process 1963, status = succeeded6789worker1 # [ 151.605105] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086790worker1 # [ 151.613583] nix-grpc-daemon[1792]: time=2026-09-21T12:55:57.696Z level=INFO msg="All 1 paths already cached"6791worker1 # [ 151.617659] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579086792worker1 # [ 151.621587] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6793worker1 # [ 151.780079] nix-daemon[850]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)6794worker1 # [ 151.792416] nix-daemon[850]: reaped child process 1993, status = succeeded6795worker1 # [ 151.956361] nix-daemon[850]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)6796worker1 # [ 151.967948] nix-daemon[850]: reaped child process 1998, status = succeeded6797lb # [ 151.978713] niks3-server[1047]: 2026/09/21 12:55:57 INFO Received uploads request method=POST path=/api/pending_closures6798worker1 # [ 151.992269] nix-grpc-daemon[1792]: time=2026-09-21T12:55:58.075Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6799worker1 # [ 151.993734] nix-grpc-daemon[1792]: time=2026-09-21T12:55:58.076Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6800lb # [ 152.012458] niks3-server[1047]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6801lb # [ 152.022809] niks3-server[1047]: 2026/09/21 12:55:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6802worker1 # [ 152.032024] nix-grpc-daemon[1792]: time=2026-09-21T12:55:58.115Z level=INFO msg="Uploading 1 narinfos"6803lb # [ 152.027067] niks3-server[1047]: 2026/09/21 12:55:57 INFO Signed narinfos id=56 count=16804lb # [ 152.034493] niks3-server[1047]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6805lb # [ 152.043187] niks3-server[1047]: 2026/09/21 12:55:57 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6806lb # [ 152.048829] niks3-server[1047]: 2026/09/21 12:55:57 INFO Completed upload id=566807worker1 # [ 152.053790] nix-grpc-daemon[1792]: time=2026-09-21T12:55:58.136Z level=INFO msg="Upload complete. (426ms)"6808lb # [ 152.051586] niks3-server[1047]: 2026/09/21 12:55:57 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6809worker1 # [ 152.057326] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 paths=1 nar_bytes_in=6246810worker1 # [ 152.062906] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086811worker1 # [ 152.071658] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6812worker1 # [ 152.077456] nix-daemon[850]: reaped child process 1988, status = succeeded6813worker1 # [ 152.081431] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6814worker1 # [ 152.693650] nix-daemon[850]: reaped child process 2002, status = succeeded6815worker1 # [ 152.824825] nix-daemon[850]: accepted connection from pid 2018, user nix-grpc-daemon (trusted)6816worker1 # [ 152.836890] nix-daemon[850]: reaped child process 2021, status = succeeded6817lb # [ 152.846340] niks3-server[1047]: 2026/09/21 12:55:58 INFO Received uploads request method=POST path=/api/pending_closures6818worker1 # [ 152.856209] nix-grpc-daemon[1792]: time=2026-09-21T12:55:58.939Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6819worker1 # [ 152.858159] nix-grpc-daemon[1792]: time=2026-09-21T12:55:58.940Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6820lb # [ 152.867148] niks3-server[1047]: 2026/09/21 12:55:58 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6821lb # [ 152.869891] niks3-server[1047]: 2026/09/21 12:55:58 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6822lb # [ 152.874148] niks3-server[1047]: 2026/09/21 12:55:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6823lb # [ 152.876261] niks3-server[1047]: 2026/09/21 12:55:58 INFO Signed narinfos id=57 count=16824worker1 # [ 152.882741] nix-grpc-daemon[1792]: time=2026-09-21T12:55:58.965Z level=INFO msg="Uploading 1 narinfos"6825lb # [ 152.881155] niks3-server[1047]: 2026/09/21 12:55:58 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6826lb # [ 152.884788] niks3-server[1047]: 2026/09/21 12:55:58 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6827lb # [ 152.887569] niks3-server[1047]: 2026/09/21 12:55:58 INFO Completed upload id=576828worker1 # [ 152.892518] nix-grpc-daemon[1792]: time=2026-09-21T12:55:58.975Z level=INFO msg="Upload complete. (205ms)"6829lb # [ 152.890313] niks3-server[1047]: 2026/09/21 12:55:58 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6830worker1 # [ 152.895885] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16831worker1 # [ 152.899542] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:58Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16832worker1 # [ 152.901811] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086833worker1 # [ 152.904685] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:58Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06834worker1 # [ 152.910562] nix-daemon[850]: reaped child process 2000, status = succeeded6835worker1 # [ 152.911981] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086836worker1 # [ 152.917729] nix-grpc-daemon[1792]: time=2026-09-21T12:55:59.000Z level=INFO msg="All 2 paths already cached"6837worker1 # [ 152.921687] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579086838worker1 # [ 152.926215] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6839worker1 # [ 153.070240] nix-daemon[850]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)6840worker1 # [ 153.082985] nix-daemon[850]: reaped child process 2030, status = succeeded6841worker1 # [ 153.237390] nix-daemon[850]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)6842worker1 # [ 153.250343] nix-daemon[850]: reaped child process 2035, status = succeeded6843lb # [ 153.254414] niks3-server[1047]: 2026/09/21 12:55:59 INFO Received uploads request method=POST path=/api/pending_closures6844worker1 # [ 153.269035] nix-grpc-daemon[1792]: time=2026-09-21T12:55:59.352Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6845worker1 # [ 153.270557] nix-grpc-daemon[1792]: time=2026-09-21T12:55:59.353Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6846lb # [ 153.279076] niks3-server[1047]: 2026/09/21 12:55:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6847lb # [ 153.281036] niks3-server[1047]: 2026/09/21 12:55:59 INFO Signed narinfos id=58 count=16848worker1 # [ 153.286256] nix-grpc-daemon[1792]: time=2026-09-21T12:55:59.369Z level=INFO msg="Uploading 1 narinfos"6849lb # [ 153.283448] niks3-server[1047]: 2026/09/21 12:55:59 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6850lb # [ 153.289435] niks3-server[1047]: 2026/09/21 12:55:59 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6851lb # [ 153.293722] niks3-server[1047]: 2026/09/21 12:55:59 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6852lb # [ 153.300574] niks3-server[1047]: 2026/09/21 12:55:59 INFO Completed upload id=586853worker1 # [ 153.305546] nix-grpc-daemon[1792]: time=2026-09-21T12:55:59.388Z level=INFO msg="Upload complete. (374ms)"6854worker1 # [ 153.307461] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 paths=1 nar_bytes_in=7526855worker1 # [ 153.310493] nix-grpc-daemon[1727]: ts=2026-09-21T12:55:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086856lb # [ 153.306176] niks3-server[1047]: 2026/09/21 12:55:59 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6857worker1 # [ 153.317101] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6858worker1 # [ 153.320417] nix-daemon[850]: reaped child process 2025, status = succeeded6859worker1 # [ 153.325644] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6860worker1 # [ 154.138250] nix-daemon[850]: reaped child process 2039, status = succeeded6861worker1 # [ 154.276090] nix-daemon[850]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)6862worker1 # [ 154.286922] nix-daemon[850]: reaped child process 2058, status = succeeded6863lb # [ 154.296721] niks3-server[1047]: 2026/09/21 12:56:00 INFO Received uploads request method=POST path=/api/pending_closures6864worker1 # [ 154.310423] nix-grpc-daemon[1792]: time=2026-09-21T12:56:00.393Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6865worker1 # [ 154.313061] nix-grpc-daemon[1792]: time=2026-09-21T12:56:00.395Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6866lb # [ 154.335867] niks3-server[1047]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6867lb # [ 154.341743] niks3-server[1047]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6868lb # [ 154.350365] niks3-server[1047]: 2026/09/21 12:56:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6869worker1 # [ 154.359770] nix-grpc-daemon[1792]: time=2026-09-21T12:56:00.442Z level=INFO msg="Uploading 1 narinfos"6870lb # [ 154.354313] niks3-server[1047]: 2026/09/21 12:56:00 INFO Signed narinfos id=59 count=16871lb # [ 154.359616] niks3-server[1047]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6872lb # [ 154.368850] niks3-server[1047]: 2026/09/21 12:56:00 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6873lb # [ 154.374822] niks3-server[1047]: 2026/09/21 12:56:00 INFO Completed upload id=596874worker1 # [ 154.380057] nix-grpc-daemon[1792]: time=2026-09-21T12:56:00.463Z level=INFO msg="Upload complete. (249ms)"6875lb # [ 154.377706] niks3-server[1047]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6876worker1 # [ 154.382902] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16877worker1 # [ 154.385973] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:00Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16878worker1 # [ 154.391606] nix-daemon[850]: reaped child process 2037, status = succeeded6879worker1 # [ 154.425824] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086880worker1 # [ 154.432839] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086881worker1 # [ 154.440856] nix-grpc-daemon[1792]: time=2026-09-21T12:56:00.523Z level=INFO msg="All 2 paths already cached"6882worker1 # [ 154.445840] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579086883worker1 # [ 154.450680] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6884worker1 # [ 154.589636] nix-daemon[850]: accepted connection from pid 2062, user nix-grpc-daemon (trusted)6885worker1 # [ 154.603521] nix-daemon[850]: reaped child process 2065, status = succeeded6886worker1 # [ 154.731836] nix-daemon[850]: accepted connection from pid 2067, user nix-grpc-daemon (trusted)6887worker1 # [ 154.743936] nix-daemon[850]: reaped child process 2070, status = succeeded6888lb # [ 154.748912] niks3-server[1047]: 2026/09/21 12:56:00 INFO Received uploads request method=POST path=/api/pending_closures6889worker1 # [ 154.764236] nix-grpc-daemon[1792]: time=2026-09-21T12:56:00.847Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6890worker1 # [ 154.765683] nix-grpc-daemon[1792]: time=2026-09-21T12:56:00.847Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6891lb # [ 154.778547] niks3-server[1047]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6892lb # [ 154.787545] niks3-server[1047]: 2026/09/21 12:56:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6893lb # [ 154.792514] niks3-server[1047]: 2026/09/21 12:56:00 INFO Signed narinfos id=60 count=16894worker1 # [ 154.800685] nix-grpc-daemon[1792]: time=2026-09-21T12:56:00.883Z level=INFO msg="Uploading 1 narinfos"6895lb # [ 154.804117] niks3-server[1047]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6896lb # [ 154.810933] niks3-server[1047]: 2026/09/21 12:56:00 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6897lb # [ 154.818279] niks3-server[1047]: 2026/09/21 12:56:00 INFO Completed upload id=606898worker1 # [ 154.823337] nix-grpc-daemon[1792]: time=2026-09-21T12:56:00.906Z level=INFO msg="Upload complete. (367ms)"6899lb # [ 154.822818] niks3-server[1047]: 2026/09/21 12:56:00 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6900worker1 # [ 154.826753] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 paths=1 nar_bytes_in=7526901worker1 # [ 154.832299] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086902worker1 # [ 154.839470] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6903worker1 # [ 154.846912] nix-daemon[850]: reaped child process 2060, status = succeeded6904worker1 # [ 154.850057] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6905worker1 # [ 156.263516] nix-daemon[850]: reaped child process 2074, status = succeeded6906worker1 # [ 156.401283] nix-daemon[850]: accepted connection from pid 2090, user nix-grpc-daemon (trusted)6907worker1 # [ 156.413499] nix-daemon[850]: reaped child process 2093, status = succeeded6908lb # [ 156.423479] niks3-server[1047]: 2026/09/21 12:56:02 INFO Received uploads request method=POST path=/api/pending_closures6909worker1 # [ 156.438405] nix-grpc-daemon[1792]: time=2026-09-21T12:56:02.521Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6910worker1 # [ 156.439860] nix-grpc-daemon[1792]: time=2026-09-21T12:56:02.523Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6911lb # [ 156.459690] niks3-server[1047]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6912lb # [ 156.474880] niks3-server[1047]: 2026/09/21 12:56:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6913worker1 # [ 156.490120] nix-grpc-daemon[1792]: time=2026-09-21T12:56:02.573Z level=INFO msg="Uploading 1 narinfos"6914lb # [ 156.484708] niks3-server[1047]: 2026/09/21 12:56:02 INFO Signed narinfos id=61 count=16915lb # [ 156.489924] niks3-server[1047]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6916lb # [ 156.493741] niks3-server[1047]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6917lb # [ 156.503726] niks3-server[1047]: 2026/09/21 12:56:02 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6918lb # [ 156.509618] niks3-server[1047]: 2026/09/21 12:56:02 INFO Completed upload id=616919worker1 # [ 156.516096] nix-grpc-daemon[1792]: time=2026-09-21T12:56:02.599Z level=INFO msg="Upload complete. (259ms)"6920lb # [ 156.511442] niks3-server[1047]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6921worker1 # [ 156.519630] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16922worker1 # [ 156.522576] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:02Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16923worker1 # [ 156.525126] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086924worker1 # [ 156.530950] nix-daemon[850]: reaped child process 2072, status = succeeded6925worker1 # [ 156.565241] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579086926worker1 # [ 156.576165] nix-grpc-daemon[1792]: time=2026-09-21T12:56:02.659Z level=INFO msg="All 2 paths already cached"6927worker1 # [ 156.580709] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579086928worker1 # [ 156.587270] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6929worker1 # [ 156.742536] nix-daemon[850]: accepted connection from pid 2097, user nix-grpc-daemon (trusted)6930worker1 # [ 156.756241] nix-daemon[850]: reaped child process 2100, status = succeeded6931worker1 # [ 156.915303] nix-daemon[850]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)6932worker1 # [ 156.927552] nix-daemon[850]: reaped child process 2105, status = succeeded6933lb # [ 156.936892] niks3-server[1047]: 2026/09/21 12:56:02 INFO Received uploads request method=POST path=/api/pending_closures6934worker1 # [ 156.948985] nix-grpc-daemon[1792]: time=2026-09-21T12:56:03.031Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6935worker1 # [ 156.950574] nix-grpc-daemon[1792]: time=2026-09-21T12:56:03.031Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6936lb # [ 156.954306] niks3-server[1047]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6937lb # [ 156.959640] niks3-server[1047]: 2026/09/21 12:56:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6938lb # [ 156.961913] niks3-server[1047]: 2026/09/21 12:56:02 INFO Signed narinfos id=62 count=16939worker1 # [ 156.967254] nix-grpc-daemon[1792]: time=2026-09-21T12:56:03.050Z level=INFO msg="Uploading 1 narinfos"6940lb # [ 156.967143] niks3-server[1047]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6941lb # [ 156.972740] niks3-server[1047]: 2026/09/21 12:56:02 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6942lb # [ 156.976462] niks3-server[1047]: 2026/09/21 12:56:02 INFO Completed upload id=626943worker1 # [ 156.983028] nix-grpc-daemon[1792]: time=2026-09-21T12:56:03.066Z level=INFO msg="Upload complete. (390ms)"6944lb # [ 156.979681] niks3-server[1047]: 2026/09/21 12:56:02 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6945worker1 # [ 156.985441] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=0 paths=1 nar_bytes_in=7526946worker1 # [ 156.989139] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579086947worker1 # [ 156.996547] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6948worker1 # [ 156.999531] nix-daemon[850]: reaped child process 2095, status = succeeded6949worker1 # [ 157.005113] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6950worker1 # [ 158.029107] nix-daemon[850]: reaped child process 2109, status = succeeded6951worker1 # [ 158.159225] nix-daemon[850]: accepted connection from pid 2125, user nix-grpc-daemon (trusted)6952worker1 # [ 158.175433] nix-daemon[850]: reaped child process 2128, status = succeeded6953lb # [ 158.181605] niks3-server[1047]: 2026/09/21 12:56:04 INFO Received uploads request method=POST path=/api/pending_closures6954worker1 # [ 158.195439] nix-grpc-daemon[1792]: time=2026-09-21T12:56:04.278Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6955worker1 # [ 158.199556] nix-grpc-daemon[1792]: time=2026-09-21T12:56:04.280Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6956lb # [ 158.217957] niks3-server[1047]: 2026/09/21 12:56:04 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6957lb # [ 158.226312] niks3-server[1047]: 2026/09/21 12:56:04 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6958lb # [ 158.238953] niks3-server[1047]: 2026/09/21 12:56:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6959lb # [ 158.243323] niks3-server[1047]: 2026/09/21 12:56:04 INFO Signed narinfos id=63 count=16960worker1 # [ 158.251458] nix-grpc-daemon[1792]: time=2026-09-21T12:56:04.334Z level=INFO msg="Uploading 1 narinfos"6961lb # [ 158.248531] niks3-server[1047]: 2026/09/21 12:56:04 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6962lb # [ 158.259654] niks3-server[1047]: 2026/09/21 12:56:04 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6963lb # [ 158.264300] niks3-server[1047]: 2026/09/21 12:56:04 INFO Completed upload id=636964worker1 # [ 158.269632] nix-grpc-daemon[1792]: time=2026-09-21T12:56:04.352Z level=INFO msg="Upload complete. (257ms)"6965lb # [ 158.266443] niks3-server[1047]: 2026/09/21 12:56:04 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6966worker1 # [ 158.272711] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57908 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16967worker1 # [ 158.281841] nix-daemon[850]: reaped child process 2107, status = succeeded6968client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.59 seconds)6969worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6970worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6971client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]'6972client # [ 158.363863] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6973client # Running as unit: gpu.service; invocation ID: 6298dd09750640928cfb7c114fea23be6974client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.04 seconds)6975lb: must succeed: curl -sf localhost:9901/clusters6976lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6977lb: must succeed: curl -sf localhost:9901/clusters6978lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6979worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6980lb # [ 158.516526] envoy[909]: conn peer=[2001:db8:1::1]:48792 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27111 tx=11417 ms=174016981worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6982worker1 # [ 158.602248] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:579086983client # [ 158.590743] nix[1346]: don't know how to build these paths:6984client # [ 158.591818] nix[1346]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv6985worker2 # [ 158.598166] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487666986worker1 # [ 158.632183] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:04Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06987client # [ 158.619204] nix[1346]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker6988lb: must succeed: curl -sf localhost:9901/clusters6989lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6990lb: must succeed: curl -sf localhost:9901/clusters6991lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6992worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6993worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6994client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6995client # [ 188.620459] nix[1346]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}6996client # [ 188.629836] nix[1346]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed6997client # [ 188.670169] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6998client # [ 188.674386] systemd[1]: gpu.service: Failed with result 'exit-code'.6999client # [ 188.678252] systemd[1]: gpu.service: Consumed 134ms CPU time over 30.309s wall clock time, 37.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7000client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.53 seconds)7001client: must succeed: journalctl -u gpu -o cat7002client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7003lb: must succeed: curl -sf localhost:9901/clusters7004lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7005lb: must succeed: curl -sf localhost:9901/clusters7006lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7007worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7008worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7009(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.76 seconds)7010subtest: low disk drains a worker and builds go to the other7011worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7012worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.98 seconds)7013worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7014worker1 # [ 190.691685] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:36Z level=info event=unhealthy reason=min_free7015worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)7016client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&27017client # don't know how to build these paths:7018client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7019worker2 # [ 191.751805] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:487667020worker1 # [ 191.769887] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579087021worker1 # [ 191.786247] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:37Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07022worker1 # [ 191.788242] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:37Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17023worker2 # [ 191.816781] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487667024worker2 # [ 191.828834] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487667025client # copying 1 paths to 'lb:50051'...7026client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7027worker2 # [ 191.841444] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487667028worker2 # [ 191.853110] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)7029worker2 # [ 192.040685] nix-daemon[871]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)7030worker2 # [ 192.052655] nix-daemon[871]: reaped child process 1660, status = succeeded7031worker2 # [ 192.216155] nix-daemon[871]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)7032worker2 # [ 192.227715] nix-daemon[871]: reaped child process 1665, status = succeeded7033lb # [ 192.253544] niks3-server[1047]: 2026/09/21 12:56:38 INFO Received uploads request method=POST path=/api/pending_closures7034worker2 # [ 192.254149] nix-grpc-daemon[1480]: time=2026-09-21T12:56:38.471Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7035worker2 # [ 192.255595] nix-grpc-daemon[1480]: time=2026-09-21T12:56:38.471Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7036lb # [ 192.288652] niks3-server[1047]: 2026/09/21 12:56:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7037lb # [ 192.293598] niks3-server[1047]: 2026/09/21 12:56:38 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7038lb # [ 192.299516] niks3-server[1047]: 2026/09/21 12:56:38 INFO Signed narinfos id=64 count=17039worker2 # [ 192.293843] nix-grpc-daemon[1480]: time=2026-09-21T12:56:38.511Z level=INFO msg="Uploading 1 narinfos"7040lb # [ 192.307832] niks3-server[1047]: 2026/09/21 12:56:38 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7041lb # [ 192.319441] niks3-server[1047]: 2026/09/21 12:56:38 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7042worker2 # [ 192.317618] nix-grpc-daemon[1480]: time=2026-09-21T12:56:38.535Z level=INFO msg="Upload complete. (453ms)"7043lb # [ 192.325965] niks3-server[1047]: 2026/09/21 12:56:38 INFO Completed upload id=647044worker2 # [ 192.320067] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=0 paths=1 nar_bytes_in=5127045lb # [ 192.329943] niks3-server[1047]: 2026/09/21 12:56:38 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7046worker2 # [ 192.325439] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487667047worker2 # [ 192.330285] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)7048worker2 # [ 192.332526] nix-daemon[871]: reaped child process 1655, status = succeeded7049worker2 # [ 192.337761] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)7050client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7051client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7052client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7053client # farm-a> LOG-farm-a-drain7054worker2 # [ 193.552915] nix-daemon[871]: reaped child process 1669, status = succeeded7055worker2 # [ 193.704624] nix-daemon[871]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)7056worker2 # [ 193.717635] nix-daemon[871]: reaped child process 1688, status = succeeded7057lb # [ 193.740505] niks3-server[1047]: 2026/09/21 12:56:39 INFO Received uploads request method=POST path=/api/pending_closures7058worker2 # [ 193.739503] nix-grpc-daemon[1480]: time=2026-09-21T12:56:39.957Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7059worker2 # [ 193.741243] nix-grpc-daemon[1480]: time=2026-09-21T12:56:39.958Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7060lb # [ 193.789185] niks3-server[1047]: 2026/09/21 12:56:39 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7061lb # [ 193.796761] niks3-server[1047]: 2026/09/21 12:56:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7062lb # [ 193.801607] niks3-server[1047]: 2026/09/21 12:56:39 INFO Signed narinfos id=65 count=17063worker2 # [ 193.795632] nix-grpc-daemon[1480]: time=2026-09-21T12:56:40.013Z level=INFO msg="Uploading 1 narinfos"7064lb # [ 193.809271] niks3-server[1047]: 2026/09/21 12:56:39 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7065lb # [ 193.816206] niks3-server[1047]: 2026/09/21 12:56:39 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7066lb # [ 193.828296] niks3-server[1047]: 2026/09/21 12:56:39 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7067lb # [ 193.832581] niks3-server[1047]: 2026/09/21 12:56:39 INFO Completed upload id=657068lb # [ 193.834963] niks3-server[1047]: 2026/09/21 12:56:39 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7069worker2 # [ 193.825292] nix-grpc-daemon[1480]: time=2026-09-21T12:56:40.042Z level=INFO msg="Upload complete. (280ms)"7070worker2 # [ 193.830841] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17071worker1 # [ 193.850801] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:39Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07072worker1 # [ 193.854526] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:39Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17073worker2 # [ 193.845737] nix-daemon[871]: reaped child process 1667, status = succeeded7074worker2 # [ 193.880946] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487667075worker2 # [ 193.892432] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487667076worker2 # [ 193.906079] nix-grpc-daemon[1480]: time=2026-09-21T12:56:40.123Z level=INFO msg="All 1 paths already cached"7077client # copying 1 paths to 'lb:50051'...7078client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7079worker2 # [ 193.914600] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487667080worker2 # [ 193.924383] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)7081worker2 # [ 194.112760] nix-daemon[871]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)7082worker2 # [ 194.126514] nix-daemon[871]: reaped child process 1695, status = succeeded7083worker2 # [ 194.294254] nix-daemon[871]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)7084worker2 # [ 194.305934] nix-daemon[871]: reaped child process 1700, status = succeeded7085lb # [ 194.331526] niks3-server[1047]: 2026/09/21 12:56:40 INFO Received uploads request method=POST path=/api/pending_closures7086worker2 # [ 194.330047] nix-grpc-daemon[1480]: time=2026-09-21T12:56:40.547Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7087worker2 # [ 194.331546] nix-grpc-daemon[1480]: time=2026-09-21T12:56:40.549Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7088lb # [ 194.361513] niks3-server[1047]: 2026/09/21 12:56:40 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7089lb # [ 194.371374] niks3-server[1047]: 2026/09/21 12:56:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7090lb # [ 194.376519] niks3-server[1047]: 2026/09/21 12:56:40 INFO Signed narinfos id=66 count=17091worker2 # [ 194.370895] nix-grpc-daemon[1480]: time=2026-09-21T12:56:40.588Z level=INFO msg="Uploading 1 narinfos"7092lb # [ 194.383082] niks3-server[1047]: 2026/09/21 12:56:40 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7093lb # [ 194.394388] niks3-server[1047]: 2026/09/21 12:56:40 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7094lb # [ 194.402244] niks3-server[1047]: 2026/09/21 12:56:40 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7095worker2 # [ 194.397510] nix-grpc-daemon[1480]: time=2026-09-21T12:56:40.615Z level=INFO msg="Upload complete. (459ms)"7096lb # [ 194.406310] niks3-server[1047]: 2026/09/21 12:56:40 INFO Completed upload id=667097worker2 # [ 194.399510] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=0 paths=1 nar_bytes_in=6247098worker2 # [ 194.402503] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487667099worker2 # [ 194.409255] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)7100worker2 # [ 194.411944] nix-daemon[871]: reaped child process 1690, status = succeeded7101client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7102worker2 # [ 194.418047] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)7103client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7104client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7105client # farm-b> LOG-farm-b-drain7106worker2 # [ 195.832306] nix-daemon[871]: reaped child process 1704, status = succeeded7107worker2 # [ 195.954531] nix-daemon[871]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7108worker2 # [ 195.966904] nix-daemon[871]: reaped child process 1723, status = succeeded7109lb # [ 195.991324] niks3-server[1047]: 2026/09/21 12:56:41 INFO Received uploads request method=POST path=/api/pending_closures7110worker2 # [ 195.987730] nix-grpc-daemon[1480]: time=2026-09-21T12:56:42.205Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7111worker2 # [ 195.989246] nix-grpc-daemon[1480]: time=2026-09-21T12:56:42.207Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7112lb # [ 196.014092] niks3-server[1047]: 2026/09/21 12:56:41 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7113lb # [ 196.017503] niks3-server[1047]: 2026/09/21 12:56:41 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7114lb # [ 196.021428] niks3-server[1047]: 2026/09/21 12:56:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7115lb # [ 196.023142] niks3-server[1047]: 2026/09/21 12:56:41 INFO Signed narinfos id=67 count=17116worker2 # [ 196.014562] nix-grpc-daemon[1480]: time=2026-09-21T12:56:42.232Z level=INFO msg="Uploading 1 narinfos"7117lb # [ 196.027324] niks3-server[1047]: 2026/09/21 12:56:41 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7118lb # [ 196.031051] niks3-server[1047]: 2026/09/21 12:56:41 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7119lb # [ 196.034371] niks3-server[1047]: 2026/09/21 12:56:41 INFO Completed upload id=677120worker2 # [ 196.025459] nix-grpc-daemon[1480]: time=2026-09-21T12:56:42.243Z level=INFO msg="Upload complete. (201ms)"7121lb # [ 196.036296] niks3-server[1047]: 2026/09/21 12:56:41 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7122worker2 # [ 196.028211] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17123worker2 # [ 196.034230] nix-daemon[871]: reaped child process 1702, status = succeeded7124worker1 # [ 196.049924] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:42Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07125worker1 # [ 196.054312] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:42Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17126worker2 # [ 196.077221] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487667127worker2 # [ 196.083959] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:487667128worker2 # [ 196.090918] nix-grpc-daemon[1480]: time=2026-09-21T12:56:42.308Z level=INFO msg="All 2 paths already cached"7129client # copying 1 paths to 'lb:50051'...7130client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7131worker2 # [ 196.095511] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:487667132worker2 # [ 196.099935] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)7133worker2 # [ 196.230656] nix-daemon[871]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)7134worker2 # [ 196.244171] nix-daemon[871]: reaped child process 1730, status = succeeded7135worker2 # [ 196.389129] nix-daemon[871]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)7136worker2 # [ 196.401186] nix-daemon[871]: reaped child process 1735, status = succeeded7137lb # [ 196.424175] niks3-server[1047]: 2026/09/21 12:56:42 INFO Received uploads request method=POST path=/api/pending_closures7138worker2 # [ 196.422557] nix-grpc-daemon[1480]: time=2026-09-21T12:56:42.640Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7139worker2 # [ 196.424042] nix-grpc-daemon[1480]: time=2026-09-21T12:56:42.641Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7140lb # [ 196.450136] niks3-server[1047]: 2026/09/21 12:56:42 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7141lb # [ 196.461674] niks3-server[1047]: 2026/09/21 12:56:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7142worker2 # [ 196.457968] nix-grpc-daemon[1480]: time=2026-09-21T12:56:42.675Z level=INFO msg="Uploading 1 narinfos"7143lb # [ 196.467376] niks3-server[1047]: 2026/09/21 12:56:42 INFO Signed narinfos id=68 count=17144lb # [ 196.471211] niks3-server[1047]: 2026/09/21 12:56:42 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7145lb # [ 196.482649] niks3-server[1047]: 2026/09/21 12:56:42 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7146worker2 # [ 196.479745] nix-grpc-daemon[1480]: time=2026-09-21T12:56:42.697Z level=INFO msg="Upload complete. (373ms)"7147lb # [ 196.488727] niks3-server[1047]: 2026/09/21 12:56:42 INFO Completed upload id=687148worker2 # [ 196.481767] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=0 paths=1 nar_bytes_in=7527149lb # [ 196.492270] niks3-server[1047]: 2026/09/21 12:56:42 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7150worker2 # [ 196.488410] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:487667151worker2 # [ 196.490183] nix-daemon[871]: reaped child process 1725, status = succeeded7152worker2 # [ 196.493132] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)7153worker2 # [ 196.499601] nix-daemon[871]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)7154client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7155client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7156client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7157client # farm-top> LOG-farm-top-drain7158worker2 # [ 198.124741] nix-daemon[871]: reaped child process 1739, status = succeeded7159worker2 # [ 198.304274] nix-daemon[871]: accepted connection from pid 1755, user nix-grpc-daemon (trusted)7160worker2 # [ 198.317174] nix-daemon[871]: reaped child process 1758, status = succeeded7161lb # [ 198.343852] niks3-server[1047]: 2026/09/21 12:56:44 INFO Received uploads request method=POST path=/api/pending_closures7162worker2 # [ 198.343435] nix-grpc-daemon[1480]: time=2026-09-21T12:56:44.561Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7163worker2 # [ 198.344985] nix-grpc-daemon[1480]: time=2026-09-21T12:56:44.562Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7164lb # [ 198.383282] niks3-server[1047]: 2026/09/21 12:56:44 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7165lb # [ 198.390920] niks3-server[1047]: 2026/09/21 12:56:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7166lb # [ 198.397245] niks3-server[1047]: 2026/09/21 12:56:44 INFO Signed narinfos id=69 count=17167worker2 # [ 198.391655] nix-grpc-daemon[1480]: time=2026-09-21T12:56:44.609Z level=INFO msg="Uploading 1 narinfos"7168lb # [ 198.404280] niks3-server[1047]: 2026/09/21 12:56:44 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7169lb # [ 198.409335] niks3-server[1047]: 2026/09/21 12:56:44 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7170lb # [ 198.418959] niks3-server[1047]: 2026/09/21 12:56:44 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7171lb # [ 198.424722] niks3-server[1047]: 2026/09/21 12:56:44 INFO Completed upload id=697172lb # [ 198.427741] niks3-server[1047]: 2026/09/21 12:56:44 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7173worker2 # [ 198.418420] nix-grpc-daemon[1480]: time=2026-09-21T12:56:44.636Z level=INFO msg="Upload complete. (309ms)"7174worker2 # [ 198.423788] nix-grpc-daemon[1404]: ts=2026-09-21T12:56:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48766 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17175worker2 # [ 198.437732] nix-daemon[871]: reaped child process 1737, status = succeeded7176client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.pem&client-key=/nix/store/gg5dnjqvsbzhmrc2lb82r8kx0z2r1xcp-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 6.93 seconds)7177client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath7178lb # [ 198.518398] envoy[909]: conn peer=[2001:db8:1::1]:33714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300567179lb # [ 198.522619] envoy[909]: conn peer=[2001:db8:1::1]:35096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9559 tx=4102 ms=66887180client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.21 seconds)7181worker1: must succeed: rm /nix/.rw-store/fill7182worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.13 seconds)7183worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7184worker1 # [ 199.697166] nix-grpc-daemon[1727]: ts=2026-09-21T12:56:45Z level=info event=healthy reason=min_free7185worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)7186(finished: subtest: low disk drains a worker and builds go to the other, in 10.54 seconds)7187(finished: run the VM test script, in 200.96 seconds)7188test script finished in 201.22s7189cleanup7190kill QemuMachine (pid 49)7191client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7192kill QemuMachine (pid 50)7193lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7194kill QemuMachine (pid 52)7195worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7196kill QemuMachine (pid 51)7197worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)7198(finished: cleanup, in 0.97 seconds)7199additionally exposed symbols:7200 client, lb, worker1, worker2,7201 vlan1,7202 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh