vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #517
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.01 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2: QEMU running (pid 52)14worker1: QEMU running (pid 51)15worker1 # Disk image does not exist, creating the virtualisation disk image...16worker2 # Formatting '/build/vm-state-worker2/tmp.HWLkhLL1VS', fmt=raw size=107374182417worker1 # Formatting '/build/vm-state-worker1/tmp.We9H7PbDqt', fmt=raw size=107374182418worker2 # mke2fs 1.47.4 (6-Mar-2025)19worker1 # mke2fs 1.47.4 (6-Mar-2025)20worker2 # Discarding device blocks: 0/262144 done21worker1 # Discarding device blocks: 0/262144 done22worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes23worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes24client # Disk image does not exist, creating the virtualisation disk image...25worker1 # Filesystem UUID: 280d9be3-c2e4-4209-9178-28d86de898d326client # Formatting '/build/vm-state-client/tmp.CEZYHtAHNz', fmt=raw size=107374182427worker1 # Superblock backups stored on blocks:28worker2 # Filesystem UUID: c071cae7-5824-45f9-ae84-8b3bf15bcaf629client: QEMU running (pid 50)30client # mke2fs 1.47.4 (6-Mar-2025)31worker1 # 32768, 98304, 163840, 22937632client # Discarding device blocks: 0/262144 done33worker1 # 34client # Creating filesystem with 262144 4k blocks and 65536 inodes35worker1 # Allocating group tables: 0/8 done36client # Filesystem UUID: 1de6ff50-6551-4a52-a3be-a5fc55a9d9d637worker1 # Writing inode tables: 0/8 done38client # Superblock backups stored on blocks:39worker1 # Creating journal (8192 blocks): done40client # 32768, 98304, 163840, 22937641worker1 # Writing superblocks and filesystem accounting information: 0/8 done42client # 43worker1 # 44client # Allocating group tables: 0/8 done45worker1 # Virtualisation disk image created.46client # Writing inode tables: 0/8 done47worker1 # Starting virtiofs daemons...48client # Creating journal (8192 blocks): done49worker1 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)50client # Writing superblocks and filesystem accounting information: 0/8 done51worker1 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether52client # 53worker1 # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...54client # Virtualisation disk image created.55worker1 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)56client # Starting virtiofs daemons...57worker1 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether58client # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)59worker1 # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...60client # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether61worker1 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)62client # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...63worker1 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether64client # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)65worker1 # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...66client # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether67worker1 # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests68client # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...69worker1 # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests70client # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)71worker1 # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests72client # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73worker2 # Superblock backups stored on blocks:74client # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...75worker2 # 32768, 98304, 163840, 22937676client # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests77worker2 # 78client # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests79worker2 # Allocating group tables: 0/8 done80client # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests81worker2 # Writing inode tables: 0/8 done82worker2 # Creating journal (8192 blocks): done83worker2 # Writing superblocks and filesystem accounting information: 0/8 done84worker2 # 85worker2 # Virtualisation disk image created.86worker2 # Starting virtiofs daemons...87worker2 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88worker2 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90worker2 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether91worker2 # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...92worker2 # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker2 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94worker2 # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95worker2 # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...96worker2 # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests97worker2 # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests98worker2 # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests99lb # Disk image does not exist, creating the virtualisation disk image...100lb # Formatting '/build/vm-state-lb/tmp.b39WLxk1IY', fmt=raw size=1073741824101lb # mke2fs 1.47.4 (6-Mar-2025)102lb # Discarding device blocks: 0/262144 done103lb # Creating filesystem with 262144 4k blocks and 65536 inodes104lb # Filesystem UUID: df7cbbfc-89ee-4645-8544-acc1445d568e105lb # Superblock backups stored on blocks:106lb # 32768, 98304, 163840, 229376107lb # 108lb # Allocating group tables: 0/8 done109lb # Writing inode tables: 0/8 done110lb # Creating journal (8192 blocks): done111lb # Writing superblocks and filesystem accounting information: 0/8 done112lb # 113lb: QEMU running (pid 49)114lb # Virtualisation disk image created.115lb # Starting virtiofs daemons...116lb # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)117lb # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether118lb # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...119lb # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...122lb # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123(finished: start all VMs, in 0.29 seconds)124lb # [2026-09-23T05:11:12Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether125client: waiting for unit network-addresses-eth1.service126lb # [2026-09-23T05:11:12Z INFO virtiofsd] Waiting for vhost-user socket connection...127client: waiting for the VM to finish booting128lb # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-23T05:11:12Z INFO virtiofsd] Client connected, servicing requests131worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker1 # 136worker1 # 137worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139worker1 # 140worker1 # 141worker1 # 142worker1 # 143worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00144worker2 # 145worker2 # 146worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00147worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...148worker1 # 149worker1 # 150worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...151worker2 # 152worker2 # 153worker2 # 154worker2 # 155worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...157worker2 # 158worker2 # 159lb # 160lb # 161client # 162client # 163lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00164client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00165lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...166client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...167lb # 168client # 169lb # 170client # 171lb # 172client # 173lb # 174client # 175client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00176lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00177client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178client # 179client # 180lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181lb # 182lb # 183worker1 # Booting from ROM...184worker2 # Booting from ROM...185client # Booting from ROM...186lb # Booting from ROM...187worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0ym0wc85dbhvnvj6k69526s1svi4p9xg-nixos-system-worker1-test/init regInfo=/nix/store/wzqcqchf19qr3m5ir14zak7vlf2q4qzb-closure-info/registration console=ttyS0,115200n8 console=tty0189worker1 # [ 0.000000] BIOS-provided physical RAM map:190worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026199worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved200worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved201worker1 # [ 0.000000] NX (Execute Disable) protection: active202worker1 # [ 0.000000] APIC: Static calls initialized203worker1 # [ 0.000000] SMBIOS 2.8 present.204worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014205worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7s2vi9ijv1kw0jlp1wf8jikzg2pn58zv-nixos-system-worker2-test/init regInfo=/nix/store/js6liz6pjckfvss1w0yp5w89jwr0k6dq-closure-info/registration console=ttyS0,115200n8 console=tty0206worker1 # [ 0.000000] DMI: Memory slots populated: 1/1207worker2 # [ 0.000000] BIOS-provided physical RAM map:208worker1 # [ 0.000000] Hypervisor detected: KVM209worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable210worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000211worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved212worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00213worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved214worker1 # [ 0.000000] kvm-clock: using sched offset of 459051725 cycles215worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable216worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns217worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved218worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor219worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved220worker1 # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000221worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved222worker1 # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs223worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved224worker1 # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT225worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved226worker1 # [ 0.002734] found SMP MP-table at [mem 0x000f5450-0x000f545f]227worker1 # [ 0.002745] Using GB pages for direct mapping228worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved229worker1 # [ 0.002882] RAMDISK: [mem 0x5e370000-0x5ffcffff]230worker2 # [ 0.000000] NX (Execute Disable) protection: active231worker2 # [ 0.000000] APIC: Static calls initialized232worker1 # [ 0.002890] ACPI: Early table checksum verification disabled233worker2 # [ 0.000000] SMBIOS 2.8 present.234worker1 # [ 0.002893] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )235client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026236worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014237worker1 # [ 0.002897] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)238worker2 # [ 0.000000] DMI: Memory slots populated: 1/1239worker2 # [ 0.000000] Hypervisor detected: KVM240worker1 # [ 0.002901] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)241worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000242worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00243worker1 # [ 0.002908] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)244worker1 # [ 0.002910] ACPI: FACS 0x000000005FFE0000 000040245worker2 # [ 0.000001] kvm-clock: using sched offset of 468035159 cycles246client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/givaa4v6hqgmrc2v49zb17w5k6jj3j7r-nixos-system-client-test/init regInfo=/nix/store/44ly9xw1plzr89mxxy7qmaqhksmlpcj4-closure-info/registration console=ttyS0,115200n8 console=tty0247worker1 # [ 0.002912] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)248client # [ 0.000000] BIOS-provided physical RAM map:249worker2 # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns250worker2 # [ 0.000007] tsc: Detected 2400.012 MHz processor251worker1 # [ 0.002913] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)252client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable253worker2 # [ 0.000808] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000254client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved255worker1 # [ 0.002915] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)256worker2 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs257client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved258worker1 # [ 0.002916] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)259worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT260client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable261worker1 # [ 0.002918] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]262worker2 # [ 0.002736] found SMP MP-table at [mem 0x000f5450-0x000f545f]263client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved264worker2 # [ 0.002753] Using GB pages for direct mapping265worker1 # [ 0.002919] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]266client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved267worker2 # [ 0.002867] RAMDISK: [mem 0x5e36f000-0x5ffcffff]268worker1 # [ 0.002919] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]269worker2 # [ 0.002879] ACPI: Early table checksum verification disabled270client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved271worker1 # [ 0.002920] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]272worker2 # [ 0.002883] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )273client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved274lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026275worker1 # [ 0.002920] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]276client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved277worker2 # [ 0.002888] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)278worker1 # [ 0.002921] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]279client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved280worker2 # [ 0.002893] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)281worker1 # [ 0.002921] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]282client # [ 0.000000] NX (Execute Disable) protection: active283worker1 # [ 0.003153] No NUMA configuration found284client # [ 0.000000] APIC: Static calls initialized285worker2 # [ 0.002902] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)286client # [ 0.000000] SMBIOS 2.8 present.287worker1 # [ 0.003154] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]288worker2 # [ 0.002904] ACPI: FACS 0x000000005FFE0000 000040289lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0290worker1 # [ 0.003156] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]291lb # [ 0.000000] BIOS-provided physical RAM map:292client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014293worker2 # [ 0.002906] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)294worker1 # [ 0.005520] Zone ranges:295client # [ 0.000000] DMI: Memory slots populated: 1/1296lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable297worker1 # [ 0.005521] DMA [mem 0x0000000000001000-0x0000000000ffffff]298client # [ 0.000000] Hypervisor detected: KVM299worker2 # [ 0.002907] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)300lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved301client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000302worker1 # [ 0.005523] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]303worker1 # [ 0.005524] Normal empty304worker2 # [ 0.002909] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)305client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00306worker1 # [ 0.005525] Device empty307lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved308worker1 # [ 0.005526] Movable zone start for each node309client # [ 0.000000] kvm-clock: using sched offset of 486605767 cycles310worker2 # [ 0.002910] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)311lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable312worker1 # [ 0.005527] Early memory node ranges313worker2 # [ 0.002911] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]314lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved315worker1 # [ 0.005527] node 0: [mem 0x0000000000001000-0x000000000009efff]316client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns317client # [ 0.000005] tsc: Detected 2400.012 MHz processor318worker1 # [ 0.005528] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]319worker2 # [ 0.002913] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]320lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved321client # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000322worker1 # [ 0.005529] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]323lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved324worker2 # [ 0.002913] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]325worker1 # [ 0.005549] On node 0, zone DMA: 1 pages in unavailable ranges326client # [ 0.000833] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs327lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved328worker2 # [ 0.002914] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]329worker1 # [ 0.005824] On node 0, zone DMA: 97 pages in unavailable ranges330client # [ 0.000836] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT331lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved332worker2 # [ 0.002914] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]333worker1 # [ 0.034145] On node 0, zone DMA32: 40 pages in unavailable ranges334client # [ 0.002732] found SMP MP-table at [mem 0x000f5450-0x000f545f]335lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved336worker2 # [ 0.002915] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]337worker1 # [ 0.034611] ACPI: PM-Timer IO Port: 0x608338client # [ 0.002744] Using GB pages for direct mapping339lb # [ 0.000000] NX (Execute Disable) protection: active340client # [ 0.002895] RAMDISK: [mem 0x5e370000-0x5ffcffff]341worker1 # [ 0.034622] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])342worker2 # [ 0.002915] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]343lb # [ 0.000000] APIC: Static calls initialized344worker2 # [ 0.003170] No NUMA configuration found345lb # [ 0.000000] SMBIOS 2.8 present.346client # [ 0.002904] ACPI: Early table checksum verification disabled347worker1 # [ 0.034653] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23348client # [ 0.002907] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )349worker2 # [ 0.003171] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]350worker1 # [ 0.034655] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)351lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014352worker2 # [ 0.003174] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]353client # [ 0.002911] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)354lb # [ 0.000000] DMI: Memory slots populated: 1/1355worker2 # [ 0.005592] Zone ranges:356worker1 # [ 0.034657] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)357lb # [ 0.000000] Hypervisor detected: KVM358worker2 # [ 0.005594] DMA [mem 0x0000000000001000-0x0000000000ffffff]359worker1 # [ 0.034658] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)360client # [ 0.002916] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)361lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000362worker2 # [ 0.005595] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]363worker1 # [ 0.034659] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)364lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00365worker2 # [ 0.005596] Normal empty366client # [ 0.002923] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)367worker2 # [ 0.005598] Device empty368worker1 # [ 0.034660] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)369lb # [ 0.000000] kvm-clock: using sched offset of 475725290 cycles370client # [ 0.002925] ACPI: FACS 0x000000005FFE0000 000040371worker2 # [ 0.005598] Movable zone start for each node372worker2 # [ 0.005599] Early memory node ranges373worker1 # [ 0.034663] ACPI: Using ACPI (MADT) for SMP configuration information374client # [ 0.002927] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)375lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns376worker1 # [ 0.034664] ACPI: HPET id: 0x8086a201 base: 0xfed00000377worker2 # [ 0.005599] node 0: [mem 0x0000000000001000-0x000000000009efff]378lb # [ 0.000005] tsc: Detected 2400.012 MHz processor379worker1 # [ 0.034667] TSC deadline timer available380client # [ 0.002928] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)381worker2 # [ 0.005601] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]382worker1 # [ 0.034671] CPU topo: Max. logical packages: 1383lb # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000384worker1 # [ 0.034672] CPU topo: Max. logical dies: 1385client # [ 0.002930] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)386worker2 # [ 0.005602] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]387lb # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs388worker1 # [ 0.034672] CPU topo: Max. dies per package: 1389worker2 # [ 0.005627] On node 0, zone DMA: 1 pages in unavailable ranges390worker1 # [ 0.034676] CPU topo: Max. threads per core: 1391client # [ 0.002931] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)392lb # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT393worker2 # [ 0.005902] On node 0, zone DMA: 97 pages in unavailable ranges394worker1 # [ 0.034676] CPU topo: Num. cores per package: 1395lb # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]396client # [ 0.002933] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]397worker1 # [ 0.034677] CPU topo: Num. threads per package: 1398worker2 # [ 0.034191] On node 0, zone DMA32: 40 pages in unavailable ranges399lb # [ 0.002743] Using GB pages for direct mapping400worker2 # [ 0.034656] ACPI: PM-Timer IO Port: 0x608401client # [ 0.002934] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]402worker1 # [ 0.034677] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs403lb # [ 0.002843] RAMDISK: [mem 0x5e36e000-0x5ffcffff]404worker2 # [ 0.034669] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])405client # [ 0.002934] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]406lb # [ 0.002852] ACPI: Early table checksum verification disabled407worker1 # [ 0.034696] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()408worker2 # [ 0.034698] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23409lb # [ 0.002856] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )410client # [ 0.002935] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]411worker1 # [ 0.034728] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]412worker2 # [ 0.034701] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)413client # [ 0.002935] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]414lb # [ 0.002860] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)415worker1 # [ 0.034730] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]416worker2 # [ 0.034703] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)417client # [ 0.002936] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]418worker1 # [ 0.034731] [mem 0x60000000-0xafffffff] available for PCI devices419lb # [ 0.002864] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)420worker2 # [ 0.034704] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)421worker1 # [ 0.034733] Booting paravirtualized kernel on KVM422client # [ 0.002936] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]423lb # [ 0.002872] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)424worker2 # [ 0.034705] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)425client # [ 0.003176] No NUMA configuration found426lb # [ 0.002875] ACPI: FACS 0x000000005FFE0000 000040427worker1 # [ 0.034736] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns428worker2 # [ 0.034706] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)429client # [ 0.003177] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]430client # [ 0.003179] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]431worker2 # [ 0.034709] ACPI: Using ACPI (MADT) for SMP configuration information432lb # [ 0.002876] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)433worker1 # [ 0.039234] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1434client # [ 0.005560] Zone ranges:435worker2 # [ 0.034710] ACPI: HPET id: 0x8086a201 base: 0xfed00000436worker1 # [ 0.041539] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152437lb # [ 0.002878] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)438worker2 # [ 0.034714] TSC deadline timer available439client # [ 0.005561] DMA [mem 0x0000000000001000-0x0000000000ffffff]440worker1 # [ 0.041584] kvm-guest: PV spinlocks disabled, single CPU441worker2 # [ 0.034718] CPU topo: Max. logical packages: 1442client # [ 0.005563] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]443lb # [ 0.002879] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)444client # [ 0.005564] Normal empty445worker2 # [ 0.034718] CPU topo: Max. logical dies: 1446client # [ 0.005565] Device empty447worker2 # [ 0.034719] CPU topo: Max. dies per package: 1448client # [ 0.005565] Movable zone start for each node449lb # [ 0.002881] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)450worker2 # [ 0.034722] CPU topo: Max. threads per core: 1451client # [ 0.005566] Early memory node ranges452lb # [ 0.002882] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]453worker2 # [ 0.034723] CPU topo: Num. cores per package: 1454client # [ 0.005567] node 0: [mem 0x0000000000001000-0x000000000009efff]455worker2 # [ 0.034723] CPU topo: Num. threads per package: 1456lb # [ 0.002883] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]457client # [ 0.005568] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]458worker1 # [ 0.041585] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0ym0wc85dbhvnvj6k69526s1svi4p9xg-nixos-system-worker1-test/init regInfo=/nix/store/wzqcqchf19qr3m5ir14zak7vlf2q4qzb-closure-info/registration console=ttyS0,115200n8 console=tty0459worker2 # [ 0.034723] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs460lb # [ 0.002884] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]461client # [ 0.005569] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]462worker2 # [ 0.034747] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()463lb # [ 0.002884] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]464client # [ 0.005591] On node 0, zone DMA: 1 pages in unavailable ranges465worker1 # [ 0.041683] Unknown kernel command line parameters "regInfo=/nix/store/wzqcqchf19qr3m5ir14zak7vlf2q4qzb-closure-info/registration", will be passed to user space.466worker2 # [ 0.034780] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]467client # [ 0.005874] On node 0, zone DMA: 97 pages in unavailable ranges468lb # [ 0.002885] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]469worker1 # [ 0.041695] random: crng init done470client # [ 0.033873] On node 0, zone DMA32: 40 pages in unavailable ranges471worker2 # [ 0.034782] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]472lb # [ 0.002885] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]473worker1 # [ 0.041695] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes474client # [ 0.034348] ACPI: PM-Timer IO Port: 0x608475worker2 # [ 0.034784] [mem 0x60000000-0xafffffff] available for PCI devices476lb # [ 0.002886] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]477client # [ 0.034361] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])478worker1 # [ 0.043946] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)479worker2 # [ 0.034785] Booting paravirtualized kernel on KVM480lb # [ 0.003125] No NUMA configuration found481client # [ 0.034390] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23482worker1 # [ 0.045355] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)483lb # [ 0.003126] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]484worker2 # [ 0.034789] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns485worker1 # [ 0.045394] Fallback order for Node 0: 0486client # [ 0.034393] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)487lb # [ 0.003129] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]488lb # [ 0.005517] Zone ranges:489worker2 # [ 0.039293] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1490client # [ 0.034395] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)491worker1 # [ 0.045397] Built 1 zonelists, mobility grouping on. Total pages: 393078492worker1 # [ 0.045398] Policy zone: DMA32493lb # [ 0.005518] DMA [mem 0x0000000000001000-0x0000000000ffffff]494client # [ 0.034396] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)495worker2 # [ 0.041590] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152496worker1 # [ 0.048036] mem auto-init: stack:all(zero), heap alloc:on, heap free:off497lb # [ 0.005520] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]498worker2 # [ 0.041639] kvm-guest: PV spinlocks disabled, single CPU499client # [ 0.034397] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)500lb # [ 0.005521] Normal empty501worker1 # [ 0.051381] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1502lb # [ 0.005522] Device empty503client # [ 0.034398] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)504worker1 # [ 0.054911] allocated 3145728 bytes of page_ext505lb # [ 0.005523] Movable zone start for each node506lb # [ 0.005523] Early memory node ranges507client # [ 0.034400] ACPI: Using ACPI (MADT) for SMP configuration information508worker1 # [ 0.065416] ftrace: allocating 48787 entries in 192 pages509lb # [ 0.005524] node 0: [mem 0x0000000000001000-0x000000000009efff]510worker1 # [ 0.065417] ftrace: allocated 192 pages with 2 groups511client # [ 0.034402] ACPI: HPET id: 0x8086a201 base: 0xfed00000512worker1 # [ 0.066292] Dynamic Preempt: lazy513client # [ 0.034406] TSC deadline timer available514lb # [ 0.005525] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]515worker1 # [ 0.066433] rcu: Preemptible hierarchical RCU implementation.516client # [ 0.034410] CPU topo: Max. logical packages: 1517worker2 # [ 0.041640] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7s2vi9ijv1kw0jlp1wf8jikzg2pn58zv-nixos-system-worker2-test/init regInfo=/nix/store/js6liz6pjckfvss1w0yp5w89jwr0k6dq-closure-info/registration console=ttyS0,115200n8 console=tty0518lb # [ 0.005526] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]519worker1 # [ 0.066433] rcu: RCU event tracing is enabled.520client # [ 0.034411] CPU topo: Max. logical dies: 1521client # [ 0.034411] CPU topo: Max. dies per package: 1522lb # [ 0.005546] On node 0, zone DMA: 1 pages in unavailable ranges523worker1 # [ 0.066434] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.524client # [ 0.034415] CPU topo: Max. threads per core: 1525lb # [ 0.005822] On node 0, zone DMA: 97 pages in unavailable ranges526worker1 # [ 0.066436] Trampoline variant of Tasks RCU enabled.527worker2 # [ 0.041741] Unknown kernel command line parameters "regInfo=/nix/store/js6liz6pjckfvss1w0yp5w89jwr0k6dq-closure-info/registration", will be passed to user space.528client # [ 0.034415] CPU topo: Num. cores per package: 1529lb # [ 0.034088] On node 0, zone DMA32: 40 pages in unavailable ranges530worker2 # [ 0.041754] random: crng init done531worker1 # [ 0.066436] Rude variant of Tasks RCU enabled.532client # [ 0.034416] CPU topo: Num. threads per package: 1533lb # [ 0.034560] ACPI: PM-Timer IO Port: 0x608534worker1 # [ 0.066437] Tracing variant of Tasks RCU enabled.535worker2 # [ 0.041755] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes536client # [ 0.034416] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs537lb # [ 0.034574] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])538worker1 # [ 0.066437] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.539worker2 # [ 0.044016] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)540client # [ 0.034439] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()541lb # [ 0.034610] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23542worker1 # [ 0.066438] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1543client # [ 0.034476] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]544worker2 # [ 0.045394] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)545lb # [ 0.034613] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)546worker2 # [ 0.045427] Fallback order for Node 0: 0547worker1 # [ 0.066504] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.548lb # [ 0.034614] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)549client # [ 0.034478] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]550worker2 # [ 0.045430] Built 1 zonelists, mobility grouping on. Total pages: 393078551client # [ 0.034479] [mem 0x60000000-0xafffffff] available for PCI devices552lb # [ 0.034616] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)553worker2 # [ 0.045431] Policy zone: DMA32554worker1 # [ 0.066505] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.555client # [ 0.034481] Booting paravirtualized kernel on KVM556lb # [ 0.034617] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)557worker2 # [ 0.048059] mem auto-init: stack:all(zero), heap alloc:on, heap free:off558worker1 # [ 0.066506] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.559lb # [ 0.034617] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)560worker2 # [ 0.051542] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1561client # [ 0.034485] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns562worker1 # [ 0.071333] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16563worker2 # [ 0.054985] allocated 3145728 bytes of page_ext564lb # [ 0.034620] ACPI: Using ACPI (MADT) for SMP configuration information565worker1 # [ 0.071620] rcu: srcu_init: Setting srcu_struct sizes based on contention.566client # [ 0.038984] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1567worker2 # [ 0.065538] ftrace: allocating 48787 entries in 192 pages568lb # [ 0.034621] ACPI: HPET id: 0x8086a201 base: 0xfed00000569client # [ 0.041270] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152570worker2 # [ 0.065541] ftrace: allocated 192 pages with 2 groups571lb # [ 0.034625] TSC deadline timer available572worker1 # [ 0.071628] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns573worker2 # [ 0.066501] Dynamic Preempt: lazy574lb # [ 0.034629] CPU topo: Max. logical packages: 1575client # [ 0.041312] kvm-guest: PV spinlocks disabled, single CPU576lb # [ 0.034630] CPU topo: Max. logical dies: 1577worker2 # [ 0.066643] rcu: Preemptible hierarchical RCU implementation.578worker1 # [ 0.071738] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)579worker2 # [ 0.066644] rcu: RCU event tracing is enabled.580lb # [ 0.034630] CPU topo: Max. dies per package: 1581worker1 # [ 0.075286] Console: colour VGA+ 80x25582lb # [ 0.034634] CPU topo: Max. threads per core: 1583worker2 # [ 0.066645] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.584worker1 # [ 0.075288] printk: legacy console [tty0] enabled585lb # [ 0.034635] CPU topo: Num. cores per package: 1586worker2 # [ 0.066646] Trampoline variant of Tasks RCU enabled.587worker1 # [ 0.104766] printk: legacy console [ttyS0] enabled588lb # [ 0.034635] CPU topo: Num. threads per package: 1589worker2 # [ 0.066647] Rude variant of Tasks RCU enabled.590worker1 # [ 0.203109] ACPI: Core revision 20250807591lb # [ 0.034635] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs592worker2 # [ 0.066647] Tracing variant of Tasks RCU enabled.593client # [ 0.041314] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/givaa4v6hqgmrc2v49zb17w5k6jj3j7r-nixos-system-client-test/init regInfo=/nix/store/44ly9xw1plzr89mxxy7qmaqhksmlpcj4-closure-info/registration console=ttyS0,115200n8 console=tty0594lb # [ 0.034660] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()595worker2 # [ 0.066648] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.596worker1 # [ 0.203967] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns597lb # [ 0.034695] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]598worker2 # [ 0.066649] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1599client # [ 0.041414] Unknown kernel command line parameters "regInfo=/nix/store/44ly9xw1plzr89mxxy7qmaqhksmlpcj4-closure-info/registration", will be passed to user space.600worker1 # [ 0.205477] APIC: Switch to symmetric I/O mode setup601client # [ 0.041427] random: crng init done602lb # [ 0.034697] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]603worker2 # [ 0.066712] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.604worker1 # [ 0.206439] x2apic enabled605client # [ 0.041428] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes606lb # [ 0.034698] [mem 0x60000000-0xafffffff] available for PCI devices607worker2 # [ 0.066714] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.608lb # [ 0.034700] Booting paravirtualized kernel on KVM609client # [ 0.043679] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)610worker1 # [ 0.207166] APIC: Switched APIC routing to: physical x2apic611worker2 # [ 0.066715] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.612client # [ 0.044798] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)613lb # [ 0.034704] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns614client # [ 0.044831] Fallback order for Node 0: 0615worker2 # [ 0.071376] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16616worker1 # [ 0.208952] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1617lb # [ 0.039226] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1618client # [ 0.044834] Built 1 zonelists, mobility grouping on. Total pages: 393078619worker2 # [ 0.071662] rcu: srcu_init: Setting srcu_struct sizes based on contention.620client # [ 0.044835] Policy zone: DMA32621lb # [ 0.041841] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152622lb # [ 0.041892] kvm-guest: PV spinlocks disabled, single CPU623client # [ 0.047765] mem auto-init: stack:all(zero), heap alloc:on, heap free:off624worker1 # [ 0.209916] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns625worker2 # [ 0.071669] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns626client # [ 0.051283] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1627client # [ 0.054784] allocated 3145728 bytes of page_ext628worker2 # [ 0.071785] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)629worker1 # [ 0.211566] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)630worker2 # [ 0.075321] Console: colour VGA+ 80x25631client # [ 0.065241] ftrace: allocating 48787 entries in 192 pages632worker2 # [ 0.075324] printk: legacy console [tty0] enabled633client # [ 0.065243] ftrace: allocated 192 pages with 2 groups634worker1 # [ 0.212652] x86/cpu: User Mode Instruction Prevention (UMIP) activated635client # [ 0.066141] Dynamic Preempt: lazy636worker2 # [ 0.104860] printk: legacy console [ttyS0] enabled637client # [ 0.066322] rcu: Preemptible hierarchical RCU implementation.638lb # [ 0.041894] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0639worker2 # [ 0.201935] ACPI: Core revision 20250807640worker1 # [ 0.213690] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127641client # [ 0.066323] rcu: RCU event tracing is enabled.642worker1 # [ 0.214563] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0643client # [ 0.066324] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.644worker2 # [ 0.202806] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns645lb # [ 0.042003] Unknown kernel command line parameters "regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration", will be passed to user space.646client # [ 0.066325] Trampoline variant of Tasks RCU enabled.647lb # [ 0.042016] random: crng init done648client # [ 0.066326] Rude variant of Tasks RCU enabled.649worker2 # [ 0.204311] APIC: Switch to symmetric I/O mode setup650worker1 # [ 0.215567] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto651client # [ 0.066326] Tracing variant of Tasks RCU enabled.652lb # [ 0.042017] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes653worker2 # [ 0.205295] x2apic enabled654client # [ 0.066327] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.655lb # [ 0.044282] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)656worker1 # [ 0.216563] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl657worker2 # [ 0.206076] APIC: Switched APIC routing to: physical x2apic658client # [ 0.066327] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1659lb # [ 0.045417] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)660worker1 # [ 0.217563] Transient Scheduler Attacks: Vulnerable: No microcode661lb # [ 0.045453] Fallback order for Node 0: 0662client # [ 0.066350] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.663worker1 # [ 0.218562] Spectre V2 : Mitigation: Enhanced / Automatic IBRS664lb # [ 0.045456] Built 1 zonelists, mobility grouping on. Total pages: 393078665worker2 # [ 0.207879] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1666lb # [ 0.045458] Policy zone: DMA32667client # [ 0.066352] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.668worker1 # [ 0.219562] Speculative Return Stack Overflow: Mitigation: Safe RET669lb # [ 0.048092] mem auto-init: stack:all(zero), heap alloc:on, heap free:off670client # [ 0.066353] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.671worker2 # [ 0.208851] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns672lb # [ 0.051671] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1673client # [ 0.070811] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16674lb # [ 0.055251] allocated 3145728 bytes of page_ext675lb # [ 0.065587] ftrace: allocating 48787 entries in 192 pages676client # [ 0.071086] rcu: srcu_init: Setting srcu_struct sizes based on contention.677worker2 # [ 0.210524] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)678lb # [ 0.065590] ftrace: allocated 192 pages with 2 groups679lb # [ 0.066484] Dynamic Preempt: lazy680client # [ 0.071091] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns681worker2 # [ 0.211610] x86/cpu: User Mode Instruction Prevention (UMIP) activated682lb # [ 0.066643] rcu: Preemptible hierarchical RCU implementation.683worker1 # [ 0.220562] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization684lb # [ 0.066643] rcu: RCU event tracing is enabled.685worker2 # [ 0.212659] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127686client # [ 0.071204] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)687worker1 # [ 0.222570] Spectre V2 : Enabling IBPB for BPF688client # [ 0.074728] Console: colour VGA+ 80x25689lb # [ 0.066644] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.690client # [ 0.074731] printk: legacy console [tty0] enabled691worker2 # [ 0.214201] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0692lb # [ 0.066645] Trampoline variant of Tasks RCU enabled.693worker1 # [ 0.223563] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier694client # [ 0.104122] printk: legacy console [ttyS0] enabled695lb # [ 0.066646] Rude variant of Tasks RCU enabled.696client # [ 0.199668] ACPI: Core revision 20250807697lb # [ 0.066646] Tracing variant of Tasks RCU enabled.698worker1 # [ 0.224563] active return thunk: srso_alias_return_thunk699worker2 # [ 0.214525] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto700lb # [ 0.066647] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.701worker1 # [ 0.225583] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'702worker2 # [ 0.216521] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl703lb # [ 0.066648] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1704client # [ 0.200781] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns705worker1 # [ 0.226563] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'706worker2 # [ 0.217521] Transient Scheduler Attacks: Vulnerable: No microcode707client # [ 0.202299] APIC: Switch to symmetric I/O mode setup708lb # [ 0.066671] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.709worker1 # [ 0.227563] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'710client # [ 0.203228] x2apic enabled711worker2 # [ 0.218520] Spectre V2 : Mitigation: Enhanced / Automatic IBRS712lb # [ 0.066673] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.713worker1 # [ 0.228562] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'714client # [ 0.203969] APIC: Switched APIC routing to: physical x2apic715worker2 # [ 0.219520] Speculative Return Stack Overflow: Mitigation: Safe RET716lb # [ 0.066674] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.717worker1 # [ 0.229562] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'718lb # [ 0.071136] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16719client # [ 0.205744] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1720worker1 # [ 0.230563] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'721lb # [ 0.071430] rcu: srcu_init: Setting srcu_struct sizes based on contention.722worker1 # [ 0.231562] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'723lb # [ 0.071439] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns724client # [ 0.206695] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns725worker2 # [ 0.220520] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization726worker1 # [ 0.233474] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256727lb # [ 0.071548] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)728client # [ 0.208271] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)729worker2 # [ 0.222527] Spectre V2 : Enabling IBPB for BPF730lb # [ 0.075117] Console: colour VGA+ 80x25731worker1 # [ 0.234280] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64732lb # [ 0.075120] printk: legacy console [tty0] enabled733client # [ 0.210353] x86/cpu: User Mode Instruction Prevention (UMIP) activated734worker1 # [ 0.235267] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512735worker2 # [ 0.223521] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier736lb # [ 0.104673] printk: legacy console [ttyS0] enabled737lb # [ 0.200943] ACPI: Core revision 20250807738client # [ 0.211437] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127739worker1 # [ 0.236251] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024740worker2 # [ 0.224521] active return thunk: srso_alias_return_thunk741worker1 # [ 0.237255] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8742client # [ 0.212268] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0743worker2 # [ 0.225542] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'744lb # [ 0.201787] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns745worker2 # [ 0.226520] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'746lb # [ 0.203312] APIC: Switch to symmetric I/O mode setup747worker1 # [ 0.237562] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.748client # [ 0.213271] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto749lb # [ 0.204280] x2apic enabled750worker2 # [ 0.228251] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'751client # [ 0.214267] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl752lb # [ 0.205092] APIC: Switched APIC routing to: physical x2apic753worker2 # [ 0.228521] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'754client # [ 0.215268] Transient Scheduler Attacks: Vulnerable: No microcode755worker2 # [ 0.230278] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'756client # [ 0.216267] Spectre V2 : Mitigation: Enhanced / Automatic IBRS757lb # [ 0.206954] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1758worker2 # [ 0.231275] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'759client # [ 0.217267] Speculative Return Stack Overflow: Mitigation: Safe RET760worker2 # [ 0.232311] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'761lb # [ 0.207973] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns762worker2 # [ 0.233521] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256763lb # [ 0.209750] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)764worker2 # [ 0.234520] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64765lb # [ 0.210837] x86/cpu: User Mode Instruction Prevention (UMIP) activated766worker2 # [ 0.235520] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512767client # [ 0.218267] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization768client # [ 0.220276] Spectre V2 : Enabling IBPB for BPF769worker2 # [ 0.236466] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024770lb # [ 0.212879] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127771worker2 # [ 0.237263] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8772lb # [ 0.213747] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0773client # [ 0.220982] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier774client # [ 0.222268] active return thunk: srso_alias_return_thunk775worker2 # [ 0.238520] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.776lb # [ 0.214745] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto777client # [ 0.223290] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'778client # [ 0.224267] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'779lb # [ 0.215747] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl780client # [ 0.225267] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'781lb # [ 0.217747] Transient Scheduler Attacks: Vulnerable: No microcode782client # [ 0.226267] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'783lb # [ 0.218746] Spectre V2 : Mitigation: Enhanced / Automatic IBRS784client # [ 0.227267] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'785lb # [ 0.219746] Speculative Return Stack Overflow: Mitigation: Safe RET786client # [ 0.228267] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'787client # [ 0.229267] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'788client # [ 0.230267] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256789client # [ 0.231267] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64790lb # [ 0.220746] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization791lb # [ 0.222754] Spectre V2 : Enabling IBPB for BPF792client # [ 0.232267] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512793client # [ 0.233267] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024794lb # [ 0.223747] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier795client # [ 0.234267] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8796lb # [ 0.224747] active return thunk: srso_alias_return_thunk797client # [ 0.235267] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.798lb # [ 0.225768] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'799lb # [ 0.226746] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'800lb # [ 0.227746] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'801lb # [ 0.229567] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'802lb # [ 0.230572] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'803lb # [ 0.231534] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'804lb # [ 0.232598] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'805lb # [ 0.233747] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256806lb # [ 0.234746] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807worker1 # [ 0.273146] Freeing SMP alternatives memory: 44K808lb # [ 0.235746] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512809worker1 # [ 0.273566] pid_max: default: 32768 minimum: 301810lb # [ 0.236746] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024811worker1 # [ 0.274661] LSM: initializing lsm=capability,landlock,yama,bpf,ima812lb # [ 0.237747] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8813worker1 # [ 0.275660] landlock: Up and running.814worker1 # [ 0.276563] Yama: becoming mindful.815lb # [ 0.238747] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.816worker1 # [ 0.277386] LSM support for eBPF active817worker1 # [ 0.278187] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818worker1 # [ 0.279601] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819worker1 # [ 0.282019] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820worker1 # [ 0.283151] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821worker1 # [ 0.283572] ... version: 2822worker2 # [ 0.274095] Freeing SMP alternatives memory: 44K823worker1 # [ 0.284311] ... bit width: 48824worker2 # [ 0.274524] pid_max: default: 32768 minimum: 301825worker1 # [ 0.284565] ... generic counters: 6826worker2 # [ 0.275620] LSM: initializing lsm=capability,landlock,yama,bpf,ima827worker1 # [ 0.285292] ... generic bitmap: 000000000000003f828worker2 # [ 0.276629] landlock: Up and running.829worker1 # [ 0.285565] ... fixed-purpose counters: 0830worker2 # [ 0.277522] Yama: becoming mindful.831worker1 # [ 0.286328] ... fixed-purpose bitmap: 0000000000000000832worker2 # [ 0.278329] LSM support for eBPF active833worker1 # [ 0.286565] ... value mask: 0000ffffffffffff834worker2 # [ 0.279112] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)835worker1 # [ 0.287497] ... max period: 00007fffffffffff836worker1 # [ 0.288283] ... global_ctrl mask: 000000000000003f837worker2 # [ 0.280436] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)838worker1 # [ 0.288677] signal: max sigframe size: 3376839worker1 # [ 0.289616] rcu: Hierarchical SRCU implementation.840worker2 # [ 0.282854] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)841worker1 # [ 0.290522] rcu: Max phase no-delay instances is 400.842worker2 # [ 0.284089] Performance Events: Fam17h+ core perfctr, AMD PMU driver.843worker2 # [ 0.284531] ... version: 2844worker2 # [ 0.285254] ... bit width: 48845client # [ 0.270808] Freeing SMP alternatives memory: 44K846worker2 # [ 0.285523] ... generic counters: 6847client # [ 0.271270] pid_max: default: 32768 minimum: 301848worker2 # [ 0.286213] ... generic bitmap: 000000000000003f849client # [ 0.272359] LSM: initializing lsm=capability,landlock,yama,bpf,ima850worker2 # [ 0.286523] ... fixed-purpose counters: 0851worker1 # [ 0.295920] smp: Bringing up secondary CPUs ...852client # [ 0.273370] landlock: Up and running.853worker1 # [ 0.296579] smp: Brought up 1 node, 1 CPU854worker2 # [ 0.287238] ... fixed-purpose bitmap: 0000000000000000855client # [ 0.273973] Yama: becoming mindful.856worker2 # [ 0.287523] ... value mask: 0000ffffffffffff857worker1 # [ 0.297278] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)858client # [ 0.274893] LSM support for eBPF active859worker2 # [ 0.288380] ... max period: 00007fffffffffff860client # [ 0.275388] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)861worker2 # [ 0.289184] ... global_ctrl mask: 000000000000003f862worker1 # [ 0.297731] Memory: 1454584K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)863worker2 # [ 0.289620] signal: max sigframe size: 3376864client # [ 0.276317] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)865worker1 # [ 0.298843] devtmpfs: initialized866worker2 # [ 0.290445] rcu: Hierarchical SRCU implementation.867worker1 # [ 0.299657] x86/mm: Memory block size: 128MB868worker2 # [ 0.291159] rcu: Max phase no-delay instances is 400.869client # [ 0.279134] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)870worker1 # [ 0.301424] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)871client # [ 0.279860] Performance Events: Fam17h+ core perfctr, AMD PMU driver.872client # [ 0.280272] ... version: 2873worker1 # [ 0.302506] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).874client # [ 0.280952] ... bit width: 48875worker1 # [ 0.303651] pinctrl core: initialized pinctrl subsystem876client # [ 0.281269] ... generic counters: 6877worker2 # [ 0.296149] smp: Bringing up secondary CPUs ...878client # [ 0.281968] ... generic bitmap: 000000000000003f879worker1 # [ 0.304792] PM: RTC time: 05:11:12, date: 2026-09-23880lb # [ 0.274085] Freeing SMP alternatives memory: 44K881worker2 # [ 0.296548] smp: Brought up 1 node, 1 CPU882client # [ 0.282281] ... fixed-purpose counters: 0883lb # [ 0.274751] pid_max: default: 32768 minimum: 301884client # [ 0.282943] ... fixed-purpose bitmap: 0000000000000000885worker2 # [ 0.297224] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)886lb # [ 0.275779] LSM: initializing lsm=capability,landlock,yama,bpf,ima887client # [ 0.283269] ... value mask: 0000ffffffffffff888lb # [ 0.276856] landlock: Up and running.889client # [ 0.284136] ... max period: 00007fffffffffff890lb # [ 0.277474] Yama: becoming mindful.891worker2 # [ 0.297725] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)892worker1 # [ 0.308404] NET: Registered PF_NETLINK/PF_ROUTE protocol family893client # [ 0.284920] ... global_ctrl mask: 000000000000003f894lb # [ 0.277973] LSM support for eBPF active895worker2 # [ 0.298806] devtmpfs: initialized896client # [ 0.285393] signal: max sigframe size: 3376897worker1 # [ 0.309801] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations898worker2 # [ 0.299592] x86/mm: Memory block size: 128MB899client # [ 0.286137] rcu: Hierarchical SRCU implementation.900lb # [ 0.278864] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)901client # [ 0.286814] rcu: Max phase no-delay instances is 400.902worker1 # [ 0.310589] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations903lb # [ 0.279787] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)904worker2 # [ 0.301343] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)905worker1 # [ 0.311858] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations906worker2 # [ 0.302442] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).907worker1 # [ 0.312576] audit: initializing netlink subsys (disabled)908lb # [ 0.282864] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)909worker2 # [ 0.303610] pinctrl core: initialized pinctrl subsystem910worker1 # [ 0.313740] thermal_sys: Registered thermal governor 'fair_share'911lb # [ 0.284312] Performance Events: Fam17h+ core perfctr, AMD PMU driver.912worker2 # [ 0.304708] PM: RTC time: 05:11:12, date: 2026-09-23913worker1 # [ 0.313742] thermal_sys: Registered thermal governor 'bang_bang'914client # [ 0.291995] smp: Bringing up secondary CPUs ...915lb # [ 0.284756] ... version: 2916worker1 # [ 0.314566] thermal_sys: Registered thermal governor 'step_wise'917client # [ 0.292284] smp: Brought up 1 node, 1 CPU918lb # [ 0.285438] ... bit width: 48919lb # [ 0.285748] ... generic counters: 6920worker1 # [ 0.315515] thermal_sys: Registered thermal governor 'user_space'921client # [ 0.292936] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)922lb # [ 0.286494] ... generic bitmap: 000000000000003f923worker1 # [ 0.316322] audit: type=2000 audit(1790140273.195:1): state=initialized audit_enabled=0 res=1924worker2 # [ 0.308256] NET: Registered PF_NETLINK/PF_ROUTE protocol family925lb # [ 0.286749] ... fixed-purpose counters: 0926client # [ 0.293504] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)927lb # [ 0.287466] ... fixed-purpose bitmap: 0000000000000000928worker1 # [ 0.317567] thermal_sys: Registered thermal governor 'power_allocator'929worker2 # [ 0.309017] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations930client # [ 0.294528] devtmpfs: initialized931lb # [ 0.287749] ... value mask: 0000ffffffffffff932worker1 # [ 0.317591] cpuidle: using governor menu933lb # [ 0.288598] ... max period: 00007fffffffffff934client # [ 0.295309] x86/mm: Memory block size: 128MB935worker2 # [ 0.309549] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations936worker1 # [ 0.319774] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5937lb # [ 0.289426] ... global_ctrl mask: 000000000000003f938worker2 # [ 0.310797] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations939lb # [ 0.289872] signal: max sigframe size: 3376940client # [ 0.297110] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)941worker1 # [ 0.320838] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]942worker2 # [ 0.311533] audit: initializing netlink subsys (disabled)943lb # [ 0.290724] rcu: Hierarchical SRCU implementation.944worker1 # [ 0.321568] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry945lb # [ 0.291338] rcu: Max phase no-delay instances is 400.946client # [ 0.298113] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).947worker2 # [ 0.312665] thermal_sys: Registered thermal governor 'fair_share'948worker1 # [ 0.322577] PCI: Using configuration type 1 for base access949client # [ 0.299360] pinctrl core: initialized pinctrl subsystem950worker2 # [ 0.312667] thermal_sys: Registered thermal governor 'bang_bang'951client # [ 0.300450] PM: RTC time: 05:11:12, date: 2026-09-23952worker1 # [ 0.323694] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.953worker2 # [ 0.313524] thermal_sys: Registered thermal governor 'step_wise'954worker2 # [ 0.314526] audit: type=2000 audit(1790140273.208:1): state=initialized audit_enabled=0 res=1955worker2 # [ 0.316526] thermal_sys: Registered thermal governor 'user_space'956lb # [ 0.296408] smp: Bringing up secondary CPUs ...957client # [ 0.304043] NET: Registered PF_NETLINK/PF_ROUTE protocol family958worker2 # [ 0.316528] thermal_sys: Registered thermal governor 'power_allocator'959lb # [ 0.296769] smp: Brought up 1 node, 1 CPU960worker2 # [ 0.317489] cpuidle: using governor menu961lb # [ 0.297461] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)962client # [ 0.305435] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations963worker2 # [ 0.319096] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5964worker1 # [ 0.329577] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages965client # [ 0.306295] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations966lb # [ 0.297996] Memory: 1454616K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)967worker1 # [ 0.330565] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page968worker2 # [ 0.319738] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]969client # [ 0.307550] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations970lb # [ 0.299038] devtmpfs: initialized971worker2 # [ 0.320526] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry972client # [ 0.308281] audit: initializing netlink subsys (disabled)973lb # [ 0.299879] x86/mm: Memory block size: 128MB974worker2 # [ 0.321535] PCI: Using configuration type 1 for base access975client # [ 0.309440] thermal_sys: Registered thermal governor 'fair_share'976worker1 # [ 0.333567] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages977client # [ 0.309442] thermal_sys: Registered thermal governor 'bang_bang'978worker2 # [ 0.322591] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.979worker1 # [ 0.334565] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page980lb # [ 0.301712] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)981client # [ 0.310271] thermal_sys: Registered thermal governor 'step_wise'982client # [ 0.311192] thermal_sys: Registered thermal governor 'user_space'983lb # [ 0.302663] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).984lb # [ 0.303850] pinctrl core: initialized pinctrl subsystem985client # [ 0.311995] audit: type=2000 audit(1790140273.222:1): state=initialized audit_enabled=0 res=1986lb # [ 0.304999] PM: RTC time: 05:11:12, date: 2026-09-23987client # [ 0.313272] thermal_sys: Registered thermal governor 'power_allocator'988client # [ 0.313286] cpuidle: using governor menu989worker2 # [ 0.327801] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages990worker2 # [ 0.328524] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page991client # [ 0.315485] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5992client # [ 0.316504] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]993lb # [ 0.308856] NET: Registered PF_NETLINK/PF_ROUTE protocol family994client # [ 0.317273] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry995lb # [ 0.310279] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations996client # [ 0.318281] PCI: Using configuration type 1 for base access997worker1 # [ 0.344995] ACPI: Added _OSI(Module Device)998lb # [ 0.310777] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations999worker1 # [ 0.345567] ACPI: Added _OSI(Processor Device)1000worker2 # [ 0.333524] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1001client # [ 0.319411] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1002worker1 # [ 0.346252] ACPI: Added _OSI(Processor Aggregator Device)1003worker2 # [ 0.334523] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1004lb # [ 0.312051] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1005lb # [ 0.312761] audit: initializing netlink subsys (disabled)1006lb # [ 0.313991] thermal_sys: Registered thermal governor 'fair_share'1007lb # [ 0.313993] thermal_sys: Registered thermal governor 'bang_bang'1008lb # [ 0.314750] thermal_sys: Registered thermal governor 'step_wise'1009lb # [ 0.315752] audit: type=2000 audit(1790140273.238:1): state=initialized audit_enabled=0 res=11010client # [ 0.326545] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1011lb # [ 0.317752] thermal_sys: Registered thermal governor 'user_space'1012client # [ 0.327271] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1013worker1 # [ 0.353927] ACPI: 1 ACPI AML tables successfully acquired and loaded1014lb # [ 0.317754] thermal_sys: Registered thermal governor 'power_allocator'1015lb # [ 0.318751] cpuidle: using governor menu1016worker2 # [ 0.344928] ACPI: Added _OSI(Module Device)1017lb # [ 0.321036] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51018worker2 # [ 0.345526] ACPI: Added _OSI(Processor Device)1019worker2 # [ 0.346236] ACPI: Added _OSI(Processor Aggregator Device)1020lb # [ 0.321984] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1021worker1 # [ 0.357987] ACPI: Interpreter enabled1022client # [ 0.332272] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1023worker1 # [ 0.358582] ACPI: PM: (supports S0 S3 S4 S5)1024lb # [ 0.322754] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1025client # [ 0.333270] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1026worker1 # [ 0.359261] ACPI: Using IOAPIC for interrupt routing1027lb # [ 0.323761] PCI: Using configuration type 1 for base access1028lb # [ 0.324862] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1029worker1 # [ 0.361629] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1030worker1 # [ 0.362566] PCI: Using E820 reservations for host bridge windows1031worker2 # [ 0.352468] ACPI: 1 ACPI AML tables successfully acquired and loaded1032lb # [ 0.330051] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1033worker1 # [ 0.365703] ACPI: Enabled 2 GPEs in block 00 to 3F1034client # [ 0.341886] ACPI: Added _OSI(Module Device)1035lb # [ 0.330750] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1036client # [ 0.342271] ACPI: Added _OSI(Processor Device)1037client # [ 0.343000] ACPI: Added _OSI(Processor Aggregator Device)1038worker2 # [ 0.358879] ACPI: Interpreter enabled1039worker2 # [ 0.359509] ACPI: PM: (supports S0 S3 S4 S5)1040worker2 # [ 0.360035] ACPI: Using IOAPIC for interrupt routing1041lb # [ 0.335750] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1042worker2 # [ 0.360600] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1043lb # [ 0.336749] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1044worker2 # [ 0.361524] PCI: Using E820 reservations for host bridge windows1045worker1 # [ 0.374472] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1046client # [ 0.349832] ACPI: 1 ACPI AML tables successfully acquired and loaded1047worker1 # [ 0.375375] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1048worker2 # [ 0.364613] ACPI: Enabled 2 GPEs in block 00 to 3F1049worker1 # [ 0.377377] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1050client # [ 0.353199] ACPI: Interpreter enabled1051worker1 # [ 0.378655] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1052client # [ 0.353720] ACPI: PM: (supports S0 S3 S4 S5)1053worker1 # [ 0.379902] PCI host bridge to bus 0000:001054client # [ 0.354270] ACPI: Using IOAPIC for interrupt routing1055worker1 # [ 0.380570] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1056client # [ 0.355126] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1057worker1 # [ 0.381565] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1058lb # [ 0.347104] ACPI: Added _OSI(Module Device)1059lb # [ 0.347750] ACPI: Added _OSI(Processor Device)1060worker1 # [ 0.382566] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1061lb # [ 0.348471] ACPI: Added _OSI(Processor Aggregator Device)1062worker2 # [ 0.373197] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1063worker1 # [ 0.383570] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1064worker2 # [ 0.373529] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1065worker1 # [ 0.384593] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1066client # [ 0.360270] PCI: Using E820 reservations for host bridge windows1067worker1 # [ 0.385565] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1068worker2 # [ 0.375101] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1069client # [ 0.361459] ACPI: Enabled 2 GPEs in block 00 to 3F1070worker1 # [ 0.386565] pci_bus 0000:00: root bus resource [bus 00-ff]1071worker2 # [ 0.375653] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1072worker1 # [ 0.387543] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1073worker2 # [ 0.376884] PCI host bridge to bus 0000:001074worker2 # [ 0.377527] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1075worker1 # [ 0.389028] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1076worker2 # [ 0.378523] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1077lb # [ 0.356763] ACPI: 1 ACPI AML tables successfully acquired and loaded1078worker2 # [ 0.379523] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1079worker2 # [ 0.380523] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1080worker2 # [ 0.381524] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1081client # [ 0.370153] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1082worker2 # [ 0.382523] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1083lb # [ 0.361152] ACPI: Interpreter enabled1084worker2 # [ 0.383523] pci_bus 0000:00: root bus resource [bus 00-ff]1085lb # [ 0.361764] ACPI: PM: (supports S0 S3 S4 S5)1086lb # [ 0.362492] ACPI: Using IOAPIC for interrupt routing1087worker2 # [ 0.384481] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1088client # [ 0.371936] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1089worker1 # [ 0.391650] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1090client # [ 0.373382] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1091worker2 # [ 0.385934] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1092worker1 # [ 0.392577] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1093client # [ 0.374431] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1094lb # [ 0.364829] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1095worker1 # [ 0.393576] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1096client # [ 0.375669] PCI host bridge to bus 0000:001097lb # [ 0.365750] PCI: Using E820 reservations for host bridge windows1098worker1 # [ 0.394835] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1099client # [ 0.376278] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1100client # [ 0.377273] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1101lb # [ 0.368848] ACPI: Enabled 2 GPEs in block 00 to 3F1102worker1 # [ 0.396322] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1103client # [ 0.378272] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1104client # [ 0.379272] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1105worker1 # [ 0.398598] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1106client # [ 0.380272] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1107worker2 # [ 0.388589] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1108worker1 # [ 0.399459] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1109worker2 # [ 0.389536] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1110client # [ 0.381272] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1111worker1 # [ 0.400399] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1112client # [ 0.382273] pci_bus 0000:00: root bus resource [bus 00-ff]1113worker2 # [ 0.390453] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1114worker1 # [ 0.401440] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1115client # [ 0.383390] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1116worker2 # [ 0.391877] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1117lb # [ 0.376386] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1118worker1 # [ 0.403065] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1119lb # [ 0.376756] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1120client # [ 0.384745] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1121worker2 # [ 0.393220] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1122lb # [ 0.377828] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1123worker1 # [ 0.404598] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1124worker1 # [ 0.405435] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1125lb # [ 0.378872] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1126worker2 # [ 0.395546] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1127worker1 # [ 0.406301] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1128lb # [ 0.380135] PCI host bridge to bus 0000:001129worker2 # [ 0.396369] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1130lb # [ 0.380754] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1131worker2 # [ 0.397242] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1132worker1 # [ 0.407139] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1133lb # [ 0.381749] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1134worker2 # [ 0.397529] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1135lb # [ 0.382749] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1136lb # [ 0.383749] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1137worker2 # [ 0.399370] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1138client # [ 0.387346] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1139lb # [ 0.384749] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1140worker1 # [ 0.409574] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1141client # [ 0.388372] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1142worker1 # [ 0.410572] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1143lb # [ 0.385749] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1144worker2 # [ 0.401552] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1145lb # [ 0.386749] pci_bus 0000:00: root bus resource [bus 00-ff]1146worker2 # [ 0.402368] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1147worker1 # [ 0.411539] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1148client # [ 0.389351] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1149worker2 # [ 0.403223] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1150lb # [ 0.387696] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1151client # [ 0.390625] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1152worker1 # [ 0.412955] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1153worker2 # [ 0.404100] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1154lb # [ 0.389170] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1155client # [ 0.391996] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1156worker1 # [ 0.414591] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1157worker1 # [ 0.415487] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158worker1 # [ 0.416383] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1159worker2 # [ 0.406782] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1160client # [ 0.395287] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1161worker1 # [ 0.417487] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1162worker2 # [ 0.407529] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1163client # [ 0.396231] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1164worker2 # [ 0.408444] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1165worker1 # [ 0.419154] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1166client # [ 0.397154] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1167worker2 # [ 0.409892] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1168client # [ 0.398572] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1169worker1 # [ 0.420415] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1170lb # [ 0.391890] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1171worker1 # [ 0.421379] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1172worker2 # [ 0.411563] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1173client # [ 0.399841] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1174lb # [ 0.392762] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1175worker2 # [ 0.412388] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1176worker1 # [ 0.422975] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1177lb # [ 0.393710] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1178worker2 # [ 0.413239] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1179worker2 # [ 0.413529] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1180lb # [ 0.395084] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1181client # [ 0.402286] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1182worker1 # [ 0.424578] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1183client # [ 0.403233] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1184worker1 # [ 0.425517] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1185lb # [ 0.396477] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1186worker2 # [ 0.415566] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1187client # [ 0.404127] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1188worker1 # [ 0.427132] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1189worker2 # [ 0.417398] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1190client # [ 0.405830] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1191lb # [ 0.399766] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1192worker1 # [ 0.428354] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1193worker2 # [ 0.418328] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1194lb # [ 0.400585] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1195worker1 # [ 0.429253] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1196worker2 # [ 0.419939] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1197lb # [ 0.401517] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1198client # [ 0.408305] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1199worker1 # [ 0.430115] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1200lb # [ 0.402622] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1201client # [ 0.409279] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1202worker2 # [ 0.421565] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1203worker1 # [ 0.431586] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1204client # [ 0.410298] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1205lb # [ 0.404300] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1206worker2 # [ 0.422545] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1207worker1 # [ 0.432509] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1208client # [ 0.411884] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1209worker2 # [ 0.424092] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1210worker1 # [ 0.434142] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1211lb # [ 0.406724] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1212lb # [ 0.407407] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1213worker2 # [ 0.425396] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1214worker1 # [ 0.435592] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1215lb # [ 0.407787] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1216worker1 # [ 0.436404] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1217worker2 # [ 0.426310] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1218client # [ 0.414302] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1219worker1 # [ 0.437264] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1220lb # [ 0.409320] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1221worker2 # [ 0.427931] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1222client # [ 0.415260] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1223worker1 # [ 0.438116] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1224client # [ 0.416121] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1225worker2 # [ 0.429536] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1226client # [ 0.417251] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1227worker1 # [ 0.439579] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1228worker2 # [ 0.430545] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1229lb # [ 0.412784] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1230client # [ 0.418691] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1231worker1 # [ 0.440411] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1232lb # [ 0.413678] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1233worker2 # [ 0.432061] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1234worker1 # [ 0.441311] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1235lb # [ 0.414523] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1236client # [ 0.420308] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1237worker1 # [ 0.442134] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1238lb # [ 0.416163] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1239worker2 # [ 0.433589] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1240client # [ 0.421298] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1241worker1 # [ 0.443102] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1242worker2 # [ 0.434475] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1243worker2 # [ 0.435292] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1244worker1 # [ 0.443765] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1245client # [ 0.422906] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1246lb # [ 0.418784] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1247worker1 # [ 0.445084] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1248lb # [ 0.419670] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1249worker2 # [ 0.436979] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1250worker1 # [ 0.445791] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1251client # [ 0.424284] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1252lb # [ 0.421231] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1253client # [ 0.425297] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1254lb # [ 0.421755] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1255worker2 # [ 0.439173] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1256worker1 # [ 0.447577] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1257worker2 # [ 0.439530] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1258client # [ 0.426881] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1259lb # [ 0.423290] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1260worker2 # [ 0.440529] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1261lb # [ 0.424762] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1262worker2 # [ 0.442091] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1263client # [ 0.428557] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1264lb # [ 0.425755] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1265worker2 # [ 0.443060] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1266client # [ 0.429299] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1267worker2 # [ 0.443724] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1268lb # [ 0.427337] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1269client # [ 0.430857] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1270worker2 # [ 0.445053] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1271lb # [ 0.429447] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1272client # [ 0.432306] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1273worker2 # [ 0.445768] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1274lb # [ 0.429772] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1275client # [ 0.433299] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1276worker2 # [ 0.447063] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1277worker1 # [ 0.448621] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1278lb # [ 0.431388] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1279client # [ 0.434870] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1280worker1 # [ 0.451537] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1281lb # [ 0.432792] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1282worker1 # [ 0.452544] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1283lb # [ 0.433771] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1284client # [ 0.437176] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1285worker1 # [ 0.453808] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1286client # [ 0.437993] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1287worker1 # [ 0.454817] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1288lb # [ 0.435316] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1289client # [ 0.438299] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1290lb # [ 0.436784] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1291client # [ 0.439839] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1292worker1 # [ 0.456601] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1293worker1 # [ 0.457417] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1294lb # [ 0.437760] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1295worker1 # [ 0.458591] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1296worker2 # [ 0.447769] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1297lb # [ 0.439282] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1298client # [ 0.442305] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1299worker1 # [ 0.460135] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1300worker2 # [ 0.450035] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1301client # [ 0.443280] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1302lb # [ 0.441788] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1303worker2 # [ 0.450759] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1304lb # [ 0.442676] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1305client # [ 0.444331] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1306worker2 # [ 0.451809] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1307worker1 # [ 0.463566] ACPI: PCI: Interrupt link LNKA configured for IRQ 101308lb # [ 0.443506] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1309client # [ 0.445863] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1310worker1 # [ 0.464610] ACPI: PCI: Interrupt link LNKB configured for IRQ 101311worker2 # [ 0.452787] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1312worker1 # [ 0.465631] ACPI: PCI: Interrupt link LNKC configured for IRQ 111313lb # [ 0.444327] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1314client # [ 0.447233] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1315worker2 # [ 0.454486] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1316worker1 # [ 0.466659] ACPI: PCI: Interrupt link LNKD configured for IRQ 111317worker2 # [ 0.455177] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1318worker1 # [ 0.467661] ACPI: PCI: Interrupt link LNKE configured for IRQ 101319client # [ 0.448252] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1320worker1 # [ 0.468661] ACPI: PCI: Interrupt link LNKF configured for IRQ 101321lb # [ 0.446490] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1322worker2 # [ 0.455867] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1323worker1 # [ 0.469663] ACPI: PCI: Interrupt link LNKG configured for IRQ 111324lb # [ 0.446757] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1325client # [ 0.450074] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1326worker2 # [ 0.457080] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1327worker1 # [ 0.470663] ACPI: PCI: Interrupt link LNKH configured for IRQ 111328lb # [ 0.447728] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1329worker1 # [ 0.471600] ACPI: PCI: Interrupt link GSIA configured for IRQ 161330client # [ 0.451236] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1331worker1 # [ 0.472572] ACPI: PCI: Interrupt link GSIB configured for IRQ 171332lb # [ 0.449184] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1333worker1 # [ 0.473522] ACPI: PCI: Interrupt link GSIC configured for IRQ 181334client # [ 0.453062] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1335worker1 # [ 0.474332] ACPI: PCI: Interrupt link GSID configured for IRQ 191336lb # [ 0.450538] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1337worker2 # [ 0.460625] ACPI: PCI: Interrupt link LNKA configured for IRQ 101338worker1 # [ 0.475351] ACPI: PCI: Interrupt link GSIE configured for IRQ 201339worker2 # [ 0.461637] ACPI: PCI: Interrupt link LNKB configured for IRQ 101340worker1 # [ 0.476283] ACPI: PCI: Interrupt link GSIF configured for IRQ 211341lb # [ 0.450950] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1342worker2 # [ 0.462624] ACPI: PCI: Interrupt link LNKC configured for IRQ 111343worker1 # [ 0.476577] ACPI: PCI: Interrupt link GSIG configured for IRQ 221344lb # [ 0.452594] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1345worker1 # [ 0.477582] ACPI: PCI: Interrupt link GSIH configured for IRQ 231346worker2 # [ 0.463620] ACPI: PCI: Interrupt link LNKD configured for IRQ 111347worker2 # [ 0.464623] ACPI: PCI: Interrupt link LNKE configured for IRQ 101348lb # [ 0.453584] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1349worker1 # [ 0.479593] iommu: Default domain type: Translated1350worker2 # [ 0.465628] ACPI: PCI: Interrupt link LNKF configured for IRQ 101351worker1 # [ 0.480373] iommu: DMA domain TLB invalidation policy: lazy mode1352lb # [ 0.455519] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1353worker2 # [ 0.466622] ACPI: PCI: Interrupt link LNKG configured for IRQ 111354worker1 # [ 0.481584] ACPI: bus type USB registered1355worker2 # [ 0.467620] ACPI: PCI: Interrupt link LNKH configured for IRQ 111356worker1 # [ 0.482361] usbcore: registered new interface driver usbfs1357worker2 # [ 0.468560] ACPI: PCI: Interrupt link GSIA configured for IRQ 161358worker1 # [ 0.483273] usbcore: registered new interface driver hub1359worker2 # [ 0.469530] ACPI: PCI: Interrupt link GSIB configured for IRQ 171360worker1 # [ 0.483574] usbcore: registered new device driver usb1361worker2 # [ 0.470539] ACPI: PCI: Interrupt link GSIC configured for IRQ 181362worker1 # [ 0.485299] NetLabel: Initializing1363client # [ 0.454257] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1364worker2 # [ 0.471538] ACPI: PCI: Interrupt link GSID configured for IRQ 191365worker1 # [ 0.485571] NetLabel: domain hash size = 1281366worker2 # [ 0.472536] ACPI: PCI: Interrupt link GSIE configured for IRQ 201367worker1 # [ 0.486312] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1368client # [ 0.456062] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1369worker1 # [ 0.486612] NetLabel: unlabeled traffic allowed by default1370worker2 # [ 0.473537] ACPI: PCI: Interrupt link GSIF configured for IRQ 211371client # [ 0.457302] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1372worker1 # [ 0.487538] PCI: Using ACPI for IRQ routing1373worker2 # [ 0.474540] ACPI: PCI: Interrupt link GSIG configured for IRQ 221374worker2 # [ 0.475482] ACPI: PCI: Interrupt link GSIH configured for IRQ 231375client # [ 0.458582] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1376worker2 # [ 0.477341] iommu: Default domain type: Translated1377client # [ 0.459650] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1378worker2 # [ 0.478137] iommu: DMA domain TLB invalidation policy: lazy mode1379lb # [ 0.455970] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1380worker2 # [ 0.478796] ACPI: bus type USB registered1381client # [ 0.461297] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1382worker2 # [ 0.479557] usbcore: registered new interface driver usbfs1383client # [ 0.462178] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1384lb # [ 0.458756] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1385worker2 # [ 0.480495] usbcore: registered new interface driver hub1386worker2 # [ 0.481202] usbcore: registered new device driver usb1387client # [ 0.463423] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1388lb # [ 0.459999] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1389worker2 # [ 0.482343] NetLabel: Initializing1390client # [ 0.464955] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1391worker2 # [ 0.483012] NetLabel: domain hash size = 1281392lb # [ 0.461050] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1393worker2 # [ 0.483523] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1394lb # [ 0.462005] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1395worker2 # [ 0.484533] NetLabel: unlabeled traffic allowed by default1396worker2 # [ 0.485458] PCI: Using ACPI for IRQ routing1397lb # [ 0.463807] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1398client # [ 0.468868] ACPI: PCI: Interrupt link LNKA configured for IRQ 101399lb # [ 0.464666] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1400client # [ 0.469376] ACPI: PCI: Interrupt link LNKB configured for IRQ 101401client # [ 0.470369] ACPI: PCI: Interrupt link LNKC configured for IRQ 111402lb # [ 0.465872] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1403client # [ 0.471369] ACPI: PCI: Interrupt link LNKD configured for IRQ 111404client # [ 0.472373] ACPI: PCI: Interrupt link LNKE configured for IRQ 101405lb # [ 0.467496] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1406client # [ 0.473365] ACPI: PCI: Interrupt link LNKF configured for IRQ 101407client # [ 0.474369] ACPI: PCI: Interrupt link LNKG configured for IRQ 111408client # [ 0.475379] ACPI: PCI: Interrupt link LNKH configured for IRQ 111409lb # [ 0.470882] ACPI: PCI: Interrupt link LNKA configured for IRQ 101410client # [ 0.476307] ACPI: PCI: Interrupt link GSIA configured for IRQ 161411lb # [ 0.471863] ACPI: PCI: Interrupt link LNKB configured for IRQ 101412client # [ 0.477288] ACPI: PCI: Interrupt link GSIB configured for IRQ 171413lb # [ 0.472878] ACPI: PCI: Interrupt link LNKC configured for IRQ 111414client # [ 0.478288] ACPI: PCI: Interrupt link GSIC configured for IRQ 181415lb # [ 0.473846] ACPI: PCI: Interrupt link LNKD configured for IRQ 111416client # [ 0.479286] ACPI: PCI: Interrupt link GSID configured for IRQ 191417client # [ 0.480281] ACPI: PCI: Interrupt link GSIE configured for IRQ 201418lb # [ 0.475093] ACPI: PCI: Interrupt link LNKE configured for IRQ 101419client # [ 0.481283] ACPI: PCI: Interrupt link GSIF configured for IRQ 211420lb # [ 0.475858] ACPI: PCI: Interrupt link LNKF configured for IRQ 101421client # [ 0.482281] ACPI: PCI: Interrupt link GSIG configured for IRQ 221422lb # [ 0.476851] ACPI: PCI: Interrupt link LNKG configured for IRQ 111423lb # [ 0.477852] ACPI: PCI: Interrupt link LNKH configured for IRQ 111424client # [ 0.483304] ACPI: PCI: Interrupt link GSIH configured for IRQ 231425lb # [ 0.478786] ACPI: PCI: Interrupt link GSIA configured for IRQ 161426lb # [ 0.479757] ACPI: PCI: Interrupt link GSIB configured for IRQ 171427client # [ 0.486186] iommu: Default domain type: Translated1428lb # [ 0.480755] ACPI: PCI: Interrupt link GSIC configured for IRQ 181429client # [ 0.486859] iommu: DMA domain TLB invalidation policy: lazy mode1430lb # [ 0.481732] ACPI: PCI: Interrupt link GSID configured for IRQ 191431client # [ 0.487556] ACPI: bus type USB registered1432lb # [ 0.482508] ACPI: PCI: Interrupt link GSIE configured for IRQ 201433client # [ 0.488298] usbcore: registered new interface driver usbfs1434lb # [ 0.482760] ACPI: PCI: Interrupt link GSIF configured for IRQ 211435client # [ 0.489240] usbcore: registered new interface driver hub1436lb # [ 0.483765] ACPI: PCI: Interrupt link GSIG configured for IRQ 221437client # [ 0.489992] usbcore: registered new device driver usb1438lb # [ 0.484730] ACPI: PCI: Interrupt link GSIH configured for IRQ 231439client # [ 0.491162] NetLabel: Initializing1440client # [ 0.491696] NetLabel: domain hash size = 1281441lb # [ 0.486589] iommu: Default domain type: Translated1442client # [ 0.492275] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1443lb # [ 0.487371] iommu: DMA domain TLB invalidation policy: lazy mode1444client # [ 0.493228] NetLabel: unlabeled traffic allowed by default1445lb # [ 0.488021] ACPI: bus type USB registered1446client # [ 0.494061] PCI: Using ACPI for IRQ routing1447lb # [ 0.488794] usbcore: registered new interface driver usbfs1448lb # [ 0.489695] usbcore: registered new interface driver hub1449worker1 # [ 0.532024] pci 0000:00:01.0: vgaarb: setting as boot VGA device1450lb # [ 0.490432] usbcore: registered new device driver usb1451worker1 # [ 0.532561] pci 0000:00:01.0: vgaarb: bridge control possible1452lb # [ 0.491954] NetLabel: Initializing1453worker1 # [ 0.532561] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1454worker1 # [ 0.532570] vgaarb: loaded1455worker1 # [ 0.533351] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01456worker1 # [ 0.534196] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1457lb # [ 0.492729] NetLabel: domain hash size = 1281458lb # [ 0.493360] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1459lb # [ 0.493843] NetLabel: unlabeled traffic allowed by default1460lb # [ 0.494762] PCI: Using ACPI for IRQ routing1461worker1 # [ 0.539671] clocksource: Switched to clocksource kvm-clock1462worker2 # [ 0.530009] pci 0000:00:01.0: vgaarb: setting as boot VGA device1463worker2 # [ 0.530519] pci 0000:00:01.0: vgaarb: bridge control possible1464worker1 # [ 0.543211] VFS: Disk quotas dquot_6.6.01465worker2 # [ 0.530519] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1466worker1 # [ 0.543941] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1467worker2 # [ 0.530529] vgaarb: loaded1468worker1 # [ 0.545270] pnp: PnP ACPI init1469worker2 # [ 0.531303] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01470worker1 # [ 0.546083] ACPI: IRQ 4 override to edge(!), high(!)1471worker2 # [ 0.531525] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1472worker1 # [ 0.547060] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1473worker1 # [ 0.548600] pnp: PnP ACPI: found 5 devices1474worker2 # [ 0.536615] clocksource: Switched to clocksource kvm-clock1475worker2 # [ 0.540434] VFS: Disk quotas dquot_6.6.01476worker2 # [ 0.541211] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1477worker1 # [ 0.556260] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1478worker2 # [ 0.542577] pnp: PnP ACPI init1479worker1 # [ 0.557706] clocksource: Switched to clocksource acpi_pm1480worker2 # [ 0.543414] ACPI: IRQ 4 override to edge(!), high(!)1481worker1 # [ 0.558687] NET: Registered PF_INET protocol family1482worker2 # [ 0.544415] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1483worker1 # [ 0.559859] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1484worker2 # [ 0.546018] pnp: PnP ACPI: found 5 devices1485worker2 # [ 0.553683] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1486client # [ 0.536995] pci 0000:00:01.0: vgaarb: setting as boot VGA device1487worker2 # [ 0.555197] clocksource: Switched to clocksource acpi_pm1488worker2 # [ 0.556230] NET: Registered PF_INET protocol family1489client # [ 0.537266] pci 0000:00:01.0: vgaarb: bridge control possible1490client # [ 0.537266] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1491worker2 # [ 0.557469] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1492client # [ 0.537274] vgaarb: loaded1493client # [ 0.538037] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01494client # [ 0.538271] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1495worker1 # [ 0.576534] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1496worker1 # [ 0.578142] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1497worker1 # [ 0.579507] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1498client # [ 0.543349] clocksource: Switched to clocksource kvm-clock1499lb # [ 0.538558] pci 0000:00:01.0: vgaarb: setting as boot VGA device1500worker1 # [ 0.580904] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1501lb # [ 0.538745] pci 0000:00:01.0: vgaarb: bridge control possible1502client # [ 0.545046] VFS: Disk quotas dquot_6.6.01503worker1 # [ 0.582171] TCP: Hash tables configured (established 16384 bind 16384)1504lb # [ 0.538745] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1505lb # [ 0.538754] vgaarb: loaded1506worker1 # [ 0.583392] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1507lb # [ 0.539540] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01508worker1 # [ 0.584689] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1509client # [ 0.547572] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1510lb # [ 0.539751] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1511client # [ 0.548874] pnp: PnP ACPI init1512worker1 # [ 0.585811] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1513client # [ 0.549661] ACPI: IRQ 4 override to edge(!), high(!)1514worker1 # [ 0.587070] NET: Registered PF_UNIX/PF_LOCAL protocol family1515worker1 # [ 0.588043] NET: Registered PF_XDP protocol family1516client # [ 0.550622] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1517worker1 # [ 0.588896] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1518client # [ 0.552162] pnp: PnP ACPI: found 5 devices1519worker2 # [ 0.574018] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1520worker1 # [ 0.589914] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1521lb # [ 0.543844] clocksource: Switched to clocksource kvm-clock1522worker2 # [ 0.575472] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1523worker1 # [ 0.591050] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1524worker1 # [ 0.592141] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1525worker2 # [ 0.576773] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1526worker1 # [ 0.593306] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1527lb # [ 0.547489] VFS: Disk quotas dquot_6.6.01528worker1 # [ 0.594420] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1529worker2 # [ 0.579100] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1530lb # [ 0.548274] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1531worker2 # [ 0.580346] TCP: Hash tables configured (established 16384 bind 16384)1532worker1 # [ 0.596225] ACPI: \_SB_.GSIA: Enabled at IRQ 161533lb # [ 0.549653] pnp: PnP ACPI init1534worker2 # [ 0.581706] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1535lb # [ 0.550557] ACPI: IRQ 4 override to edge(!), high(!)1536client # [ 0.559836] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1537worker1 # [ 0.598459] ACPI: \_SB_.GSIB: Enabled at IRQ 171538worker2 # [ 0.583069] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1539client # [ 0.561321] clocksource: Switched to clocksource acpi_pm1540lb # [ 0.551565] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1541client # [ 0.562286] NET: Registered PF_INET protocol family1542worker2 # [ 0.584190] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1543lb # [ 0.553078] pnp: PnP ACPI: found 5 devices1544worker1 # [ 0.600689] ACPI: \_SB_.GSIC: Enabled at IRQ 181545worker2 # [ 0.585448] NET: Registered PF_UNIX/PF_LOCAL protocol family1546client # [ 0.563492] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1547worker2 # [ 0.586430] NET: Registered PF_XDP protocol family1548worker1 # [ 0.602622] ACPI: \_SB_.GSID: Enabled at IRQ 191549worker2 # [ 0.587310] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1550worker2 # [ 0.588304] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1551worker1 # [ 0.604298] PCI: CLS 0 bytes, default 641552worker2 # [ 0.589311] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1553worker2 # [ 0.590388] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1554worker1 # [ 0.605197] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1555worker2 # [ 0.591471] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1556worker1 # [ 0.606909] Trying to unpack rootfs image as initramfs...1557worker2 # [ 0.592555] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1558lb # [ 0.560726] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1559lb # [ 0.562333] clocksource: Switched to clocksource acpi_pm1560worker2 # [ 0.594411] ACPI: \_SB_.GSIA: Enabled at IRQ 161561lb # [ 0.563345] NET: Registered PF_INET protocol family1562lb # [ 0.564587] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1563worker2 # [ 0.597210] ACPI: \_SB_.GSIB: Enabled at IRQ 171564worker2 # [ 0.599273] ACPI: \_SB_.GSIC: Enabled at IRQ 181565worker2 # [ 0.601221] ACPI: \_SB_.GSID: Enabled at IRQ 191566worker2 # [ 0.602926] PCI: CLS 0 bytes, default 641567client # [ 0.580392] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1568client # [ 0.581811] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1569worker2 # [ 0.603786] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1570worker2 # [ 0.605571] Trying to unpack rootfs image as initramfs...1571client # [ 0.583125] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1572client # [ 0.584595] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1573client # [ 0.585905] TCP: Hash tables configured (established 16384 bind 16384)1574client # [ 0.587231] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1575client # [ 0.588609] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1576client # [ 0.589827] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1577client # [ 0.591094] NET: Registered PF_UNIX/PF_LOCAL protocol family1578lb # [ 0.581289] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1579client # [ 0.592076] NET: Registered PF_XDP protocol family1580client # [ 0.592967] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1581lb # [ 0.583139] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1582client # [ 0.593996] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1583lb # [ 0.584502] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1584client # [ 0.595040] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1585lb # [ 0.585904] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1586client # [ 0.596160] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1587lb # [ 0.587185] TCP: Hash tables configured (established 16384 bind 16384)1588client # [ 0.597340] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1589lb # [ 0.588357] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1590client # [ 0.598486] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1591lb # [ 0.589625] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1592client # [ 0.600262] ACPI: \_SB_.GSIA: Enabled at IRQ 161593lb # [ 0.590801] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1594lb # [ 0.592094] NET: Registered PF_UNIX/PF_LOCAL protocol family1595lb # [ 0.593069] NET: Registered PF_XDP protocol family1596client # [ 0.602724] ACPI: \_SB_.GSIB: Enabled at IRQ 171597lb # [ 0.593944] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1598client # [ 0.604679] ACPI: \_SB_.GSIC: Enabled at IRQ 181599lb # [ 0.594966] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1600lb # [ 0.595968] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1601client # [ 0.606668] ACPI: \_SB_.GSID: Enabled at IRQ 191602lb # [ 0.597112] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1603lb # [ 0.598256] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1604client # [ 0.608360] PCI: CLS 0 bytes, default 641605lb # [ 0.599445] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1606client # [ 0.609307] Trying to unpack rootfs image as initramfs...1607lb # [ 0.601222] ACPI: \_SB_.GSIA: Enabled at IRQ 161608lb # [ 0.603740] ACPI: \_SB_.GSIB: Enabled at IRQ 171609worker1 # [ 0.651246] Initialise system trusted keyrings1610client # [ 0.613598] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1611lb # [ 0.605734] ACPI: \_SB_.GSIC: Enabled at IRQ 181612worker1 # [ 0.654116] workingset: timestamp_bits=40 max_order=19 bucket_order=01613lb # [ 0.607758] ACPI: \_SB_.GSID: Enabled at IRQ 191614lb # [ 0.609499] PCI: CLS 0 bytes, default 641615lb # [ 0.610492] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1616lb # [ 0.612268] Trying to unpack rootfs image as initramfs...1617worker2 # [ 0.649409] Initialise system trusted keyrings1618worker2 # [ 0.651935] workingset: timestamp_bits=40 max_order=19 bucket_order=01619worker1 # [ 0.677097] Key type asymmetric registered1620worker1 # [ 0.677873] Asymmetric key parser 'x509' registered1621worker1 # [ 0.678749] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1622worker1 # [ 0.684106] io scheduler mq-deadline registered1623worker1 # [ 0.684906] io scheduler kyber registered1624worker1 # [ 0.687563] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1625worker1 # [ 0.688861] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1626worker2 # [ 0.675042] Key type asymmetric registered1627worker2 # [ 0.675789] Asymmetric key parser 'x509' registered1628client # [ 0.654613] Initialise system trusted keyrings1629worker2 # [ 0.676669] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1630worker1 # [ 0.694599] Linux agpgart interface v0.1031631client # [ 0.657251] workingset: timestamp_bits=40 max_order=19 bucket_order=01632worker2 # [ 0.680039] io scheduler mq-deadline registered1633worker1 # [ 0.695423] ACPI: bus type drm_connector registered1634worker2 # [ 0.680767] io scheduler kyber registered1635worker1 # [ 0.698410] usbcore: registered new interface driver usbserial_generic1636worker1 # [ 0.699575] usbserial: USB Serial support registered for generic1637worker2 # [ 0.685062] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1638lb # [ 0.654751] Initialise system trusted keyrings1639worker2 # [ 0.686428] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1640worker1 # [ 0.702973] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1641worker1 # [ 0.702973] Please enable it if your BIOS has the CPPC option.1642lb # [ 0.658380] workingset: timestamp_bits=40 max_order=19 bucket_order=01643worker1 # [ 0.705235] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1644worker2 # [ 0.691744] Linux agpgart interface v0.1031645worker2 # [ 0.692493] ACPI: bus type drm_connector registered1646worker1 # [ 0.708101] drop_monitor: Initializing network drop monitor service1647worker1 # [ 0.709296] NET: Registered PF_INET6 protocol family1648worker2 # [ 0.695385] usbcore: registered new interface driver usbserial_generic1649worker2 # [ 0.696473] usbserial: USB Serial support registered for generic1650worker1 # [ 0.714107] Segment Routing with IPv61651worker1 # [ 0.714793] In-situ OAM (IOAM) with IPv61652worker2 # [ 0.698894] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1653worker2 # [ 0.698894] Please enable it if your BIOS has the CPPC option.1654worker2 # [ 0.701085] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1655worker1 # [ 0.717122] IPI shorthand broadcast: enabled1656client # [ 0.681294] Key type asymmetric registered1657client # [ 0.682064] Asymmetric key parser 'x509' registered1658worker2 # [ 0.705120] drop_monitor: Initializing network drop monitor service1659client # [ 0.682991] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1660worker2 # [ 0.706290] NET: Registered PF_INET6 protocol family1661client # [ 0.686315] io scheduler mq-deadline registered1662worker1 # [ 0.723840] sched_clock: Marking stable (587015045, 135933911)->(790683506, -67734550)1663client # [ 0.687121] io scheduler kyber registered1664lb # [ 0.679585] Key type asymmetric registered1665worker2 # [ 0.711407] Segment Routing with IPv61666worker2 # [ 0.712137] In-situ OAM (IOAM) with IPv61667lb # [ 0.680347] Asymmetric key parser 'x509' registered1668lb # [ 0.681234] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1669client # [ 0.691219] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1670worker1 # [ 0.729147] registered taskstats version 11671worker1 # [ 0.730109] Loading compiled-in X.509 certificates1672worker2 # [ 0.715178] IPI shorthand broadcast: enabled1673client # [ 0.692536] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1674lb # [ 0.686470] io scheduler mq-deadline registered1675lb # [ 0.687247] io scheduler kyber registered1676client # [ 0.698020] Linux agpgart interface v0.1031677client # [ 0.698855] ACPI: bus type drm_connector registered1678lb # [ 0.689939] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1679worker2 # [ 0.722441] sched_clock: Marking stable (587015913, 134850866)->(792362728, -70495949)1680lb # [ 0.691221] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1681client # [ 0.703340] usbcore: registered new interface driver usbserial_generic1682client # [ 0.704485] usbserial: USB Serial support registered for generic1683worker2 # [ 0.728002] registered taskstats version 11684lb # [ 0.697137] Linux agpgart interface v0.1031685worker2 # [ 0.728960] Loading compiled-in X.509 certificates1686lb # [ 0.697902] ACPI: bus type drm_connector registered1687client # [ 0.707212] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1688client # [ 0.707212] Please enable it if your BIOS has the CPPC option.1689client # [ 0.709501] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1690lb # [ 0.700767] usbcore: registered new interface driver usbserial_generic1691lb # [ 0.701923] usbserial: USB Serial support registered for generic1692worker1 # [ 0.748967] Demotion targets for Node 0: null1693worker1 # [ 0.750072] Key type .fscrypt registered1694worker1 # [ 0.750763] Key type fscrypt-provisioning registered1695client # [ 0.714359] drop_monitor: Initializing network drop monitor service1696client # [ 0.715616] NET: Registered PF_INET6 protocol family1697lb # [ 0.705354] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1698lb # [ 0.705354] Please enable it if your BIOS has the CPPC option.1699worker1 # [ 0.754058] ima: No TPM chip found, activating TPM-bypass!1700worker1 # [ 0.754958] ima: Allocated hash algorithm: sha11701lb # [ 0.707612] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1702worker1 # [ 0.755703] ima: No architecture policies found1703client # [ 0.718707] Segment Routing with IPv61704client # [ 0.719393] In-situ OAM (IOAM) with IPv61705lb # [ 0.711494] drop_monitor: Initializing network drop monitor service1706client # [ 0.722486] IPI shorthand broadcast: enabled1707lb # [ 0.712874] NET: Registered PF_INET6 protocol family1708worker1 # [ 0.760139] PM: Magic number: 10:890:1631709worker1 # [ 0.761784] RAS: Correctable Errors collector initialized.1710lb # [ 0.715765] Segment Routing with IPv61711worker2 # [ 0.747882] Demotion targets for Node 0: null1712worker2 # [ 0.748766] Key type .fscrypt registered1713worker2 # [ 0.749440] Key type fscrypt-provisioning registered1714lb # [ 0.718364] In-situ OAM (IOAM) with IPv61715worker2 # [ 0.750380] ima: No TPM chip found, activating TPM-bypass!1716lb # [ 0.719425] IPI shorthand broadcast: enabled1717client # [ 0.730891] sched_clock: Marking stable (597015247, 133174984)->(806815926, -76625695)1718worker1 # [ 0.769635] clk: Disabling unused clocks1719worker2 # [ 0.754885] ima: Allocated hash algorithm: sha11720worker2 # [ 0.755645] ima: No architecture policies found1721client # [ 0.734398] registered taskstats version 11722worker2 # [ 0.756641] PM: Magic number: 10:890:1631723worker1 # [ 0.771969] PM: genpd: Disabling unused power domains1724client # [ 0.735384] Loading compiled-in X.509 certificates1725worker2 # [ 0.760402] RAS: Correctable Errors collector initialized.1726lb # [ 0.729685] sched_clock: Marking stable (596015098, 133306779)->(805542582, -76220705)1727lb # [ 0.731247] registered taskstats version 11728lb # [ 0.732213] Loading compiled-in X.509 certificates1729worker2 # [ 0.769180] clk: Disabling unused clocks1730worker2 # [ 0.771889] PM: genpd: Disabling unused power domains1731client # [ 0.754204] Demotion targets for Node 0: null1732client # [ 0.755135] Key type .fscrypt registered1733client # [ 0.758195] Key type fscrypt-provisioning registered1734client # [ 0.759198] ima: No TPM chip found, activating TPM-bypass!1735lb # [ 0.750343] Demotion targets for Node 0: null1736client # [ 0.760143] ima: Allocated hash algorithm: sha11737client # [ 0.762223] ima: No architecture policies found1738lb # [ 0.753368] Key type .fscrypt registered1739client # [ 0.763236] PM: Magic number: 10:890:1631740lb # [ 0.754050] Key type fscrypt-provisioning registered1741lb # [ 0.755033] ima: No TPM chip found, activating TPM-bypass!1742client # [ 0.764947] RAS: Correctable Errors collector initialized.1743lb # [ 0.757340] ima: Allocated hash algorithm: sha11744lb # [ 0.758136] ima: No architecture policies found1745lb # [ 0.760333] PM: Magic number: 10:890:1631746lb # [ 0.761983] RAS: Correctable Errors collector initialized.1747client # [ 0.775931] clk: Disabling unused clocks1748client # [ 0.778219] PM: genpd: Disabling unused power domains1749lb # [ 0.771013] clk: Disabling unused clocks1750lb # [ 0.775350] PM: genpd: Disabling unused power domains1751worker1 # [ 0.939159] Freeing initrd memory: 29056K1752worker1 # [ 0.942451] Freeing unused decrypted memory: 2028K1753worker1 # [ 0.945235] Freeing unused kernel image (initmem) memory: 3652K1754worker1 # [ 0.946346] Write protecting the kernel read-only data: 32768k1755worker1 # [ 0.948281] Freeing unused kernel image (text/rodata gap) memory: 1184K1756worker1 # [ 0.949833] Freeing unused kernel image (rodata/data gap) memory: 720K1757worker2 # [ 0.943959] Freeing initrd memory: 29060K1758worker2 # [ 0.947409] Freeing unused decrypted memory: 2028K1759worker2 # [ 0.950366] Freeing unused kernel image (initmem) memory: 3652K1760worker2 # [ 0.951456] Write protecting the kernel read-only data: 32768k1761worker2 # [ 0.953416] Freeing unused kernel image (text/rodata gap) memory: 1184K1762worker2 # [ 0.954960] Freeing unused kernel image (rodata/data gap) memory: 720K1763client # [ 0.950140] Freeing initrd memory: 29056K1764lb # [ 0.940615] Freeing initrd memory: 29064K1765client # [ 0.953434] Freeing unused decrypted memory: 2028K1766lb # [ 0.943931] Freeing unused decrypted memory: 2028K1767client # [ 0.956155] Freeing unused kernel image (initmem) memory: 3652K1768lb # [ 0.946635] Freeing unused kernel image (initmem) memory: 3652K1769client # [ 0.957267] Write protecting the kernel read-only data: 32768k1770lb # [ 0.947738] Write protecting the kernel read-only data: 32768k1771client # [ 0.959130] Freeing unused kernel image (text/rodata gap) memory: 1184K1772lb # [ 0.949643] Freeing unused kernel image (text/rodata gap) memory: 1184K1773client # [ 0.960650] Freeing unused kernel image (rodata/data gap) memory: 720K1774lb # [ 0.951217] Freeing unused kernel image (rodata/data gap) memory: 720K1775worker1 # [ 1.000768] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776worker1 # [ 1.001870] Run /init as init process1777worker1 # [ 1.012745] systemd[1]: Inserted module 'autofs4'1778worker2 # [ 1.006693] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779worker2 # [ 1.007726] Run /init as init process1780worker1 # [ 1.032539] fuse: init (API version 7.45)1781worker2 # [ 1.019445] systemd[1]: Inserted module 'autofs4'1782worker1 # [ 1.040022] ACPI: \_SB_.GSIG: Enabled at IRQ 221783worker1 # [ 1.042338] ACPI: \_SB_.GSIH: Enabled at IRQ 231784worker1 # [ 1.045291] ACPI: \_SB_.GSIE: Enabled at IRQ 201785worker1 # [ 1.048063] ACPI: \_SB_.GSIF: Enabled at IRQ 211786client # [ 1.011601] x86/mm: Checked W+X mappings: passed, no W+X pages found.1787lb # [ 1.002248] x86/mm: Checked W+X mappings: passed, no W+X pages found.1788client # [ 1.012654] Run /init as init process1789lb # [ 1.003333] Run /init as init process1790worker1 # [ 1.053885] virtiofs virtio5: discovered new tag: nix-store1791worker2 # [ 1.039603] fuse: init (API version 7.45)1792worker1 # [ 1.055527] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1793client # [ 1.023427] systemd[1]: Inserted module 'autofs4'1794lb # [ 1.014104] systemd[1]: Inserted module 'autofs4'1795worker2 # [ 1.047223] ACPI: \_SB_.GSIG: Enabled at IRQ 221796worker1 # [ 1.063513] virtiofs virtio6: discovered new tag: shared1797worker2 # [ 1.049981] ACPI: \_SB_.GSIH: Enabled at IRQ 231798worker1 # [ 1.065153] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1799worker2 # [ 1.053513] ACPI: \_SB_.GSIE: Enabled at IRQ 201800worker1 # [ 1.069139] virtiofs virtio7: discovered new tag: xchg1801worker1 # [ 1.070610] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1802worker2 # [ 1.056274] ACPI: \_SB_.GSIF: Enabled at IRQ 211803worker2 # [ 1.061768] virtiofs virtio5: discovered new tag: nix-store1804worker2 # [ 1.063432] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1805client # [ 1.043375] fuse: init (API version 7.45)1806lb # [ 1.034073] fuse: init (API version 7.45)1807worker2 # [ 1.070310] virtiofs virtio6: discovered new tag: shared1808worker2 # [ 1.071816] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1809client # [ 1.050859] ACPI: \_SB_.GSIG: Enabled at IRQ 221810lb # [ 1.041553] ACPI: \_SB_.GSIG: Enabled at IRQ 221811client # [ 1.053296] ACPI: \_SB_.GSIH: Enabled at IRQ 231812lb # [ 1.043921] ACPI: \_SB_.GSIH: Enabled at IRQ 231813worker2 # [ 1.078164] virtiofs virtio7: discovered new tag: xchg1814lb # [ 1.046907] ACPI: \_SB_.GSIE: Enabled at IRQ 201815client # [ 1.056988] ACPI: \_SB_.GSIE: Enabled at IRQ 201816worker2 # [ 1.079639] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1817worker1 # [ 1.095888] systemd[1]: Successfully made /usr/ read-only.1818client # [ 1.059065] ACPI: \_SB_.GSIF: Enabled at IRQ 211819lb # [ 1.049642] ACPI: \_SB_.GSIF: Enabled at IRQ 211820client # [ 1.064166] virtiofs virtio5: discovered new tag: nix-store1821lb # [ 1.055659] virtiofs virtio5: discovered new tag: nix-store1822client # [ 1.065820] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1823lb # [ 1.057351] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1824client # [ 1.073321] virtiofs virtio6: discovered new tag: shared1825client # [ 1.074898] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1826lb # [ 1.065417] virtiofs virtio6: discovered new tag: shared1827lb # [ 1.067012] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1828client # [ 1.079301] virtiofs virtio7: discovered new tag: xchg1829lb # [ 1.070278] virtiofs virtio7: discovered new tag: xchg1830client # [ 1.080902] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1831lb # [ 1.071742] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1832worker2 # [ 1.103947] systemd[1]: Successfully made /usr/ read-only.1833client # [ 1.103047] systemd[1]: Successfully made /usr/ read-only.1834lb # [ 1.094234] systemd[1]: Successfully made /usr/ read-only.1835worker1 # [ 1.432506] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836worker1 # [ 1.444444] systemd[1]: Detected virtualization kvm.1837worker1 # [ 1.446567] systemd[1]: Detected architecture x86-64.1838worker1 # [ 1.448707] systemd[1]: Running in initrd.1839worker1 # [ 1.451208] systemd[1]: Initializing machine ID from random generator.1840worker1 # [ 1.454124] systemd[1]: Hostname set to <worker1>.1841worker2 # [ 1.440595] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842worker2 # [ 1.452715] systemd[1]: Detected virtualization kvm.1843worker2 # [ 1.454901] systemd[1]: Detected architecture x86-64.1844worker2 # [ 1.457046] systemd[1]: Running in initrd.1845worker2 # [ 1.459499] systemd[1]: Initializing machine ID from random generator.1846worker2 # [ 1.462494] systemd[1]: Hostname set to <worker2>.1847client # [ 1.438561] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848client # [ 1.443330] systemd[1]: Detected virtualization kvm.1849lb # [ 1.429707] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1850client # [ 1.444142] systemd[1]: Detected architecture x86-64.1851lb # [ 1.434661] systemd[1]: Detected virtualization kvm.1852client # [ 1.445003] systemd[1]: Running in initrd.1853lb # [ 1.435561] systemd[1]: Detected architecture x86-64.1854lb # [ 1.436493] systemd[1]: Running in initrd.1855client # [ 1.446050] systemd[1]: Initializing machine ID from random generator.1856client # [ 1.447232] systemd[1]: Hostname set to <client>.1857lb # [ 1.437650] systemd[1]: Initializing machine ID from random generator.1858lb # [ 1.438906] systemd[1]: Hostname set to <lb>.1859lb # [ 1.632128] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860worker1 # [ 1.680226] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861client # [ 1.648808] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862worker2 # [ 1.687222] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863lb # [ 1.670921] systemd[1]: Queued start job for default target Initrd Default Target.1864lb # [ 1.675742] systemd[1]: Created slice Slice /system/modprobe.1865lb # [ 1.676987] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866lb # [ 1.678479] systemd[1]: Expecting device /dev/disk/by-label/nixos...1867lb # [ 1.679645] systemd[1]: Reached target Path Units.1868lb # [ 1.680613] systemd[1]: Reached target Slice Units.1869lb # [ 1.681540] systemd[1]: Reached target Swaps.1870lb # [ 1.682393] systemd[1]: Reached target Timer Units.1871lb # [ 1.683422] systemd[1]: Listening on D-Bus System Message Bus Socket.1872lb # [ 1.684690] systemd[1]: Listening on Journal Socket (/dev/log).1873lb # [ 1.685915] systemd[1]: Listening on Journal Sockets.1874lb # [ 1.686996] systemd[1]: Listening on udev Control Socket.1875worker1 # [ 1.733709] systemd[1]: Queued start job for default target Initrd Default Target.1876client # [ 1.696803] systemd[1]: Queued start job for default target Initrd Default Target.1877lb # [ 1.688015] systemd[1]: Listening on udev Kernel Socket.1878lb # [ 1.689004] systemd[1]: Reached target Socket Units.1879lb # [ 1.690831] systemd[1]: Starting Create List of Static Device Nodes...1880worker1 # [ 1.738310] systemd[1]: Created slice Slice /system/modprobe.1881client # [ 1.701558] systemd[1]: Created slice Slice /system/modprobe.1882worker1 # [ 1.739501] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1883client # [ 1.702747] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1884worker1 # [ 1.740924] systemd[1]: Expecting device /dev/disk/by-label/nixos...1885client # [ 1.704146] systemd[1]: Expecting device /dev/disk/by-label/nixos...1886worker1 # [ 1.742080] systemd[1]: Reached target Path Units.1887lb # [ 1.695399] systemd[1]: Starting Load Kernel Module configfs...1888client # [ 1.705282] systemd[1]: Reached target Path Units.1889worker1 # [ 1.742978] systemd[1]: Reached target Slice Units.1890client # [ 1.706129] systemd[1]: Reached target Slice Units.1891worker1 # [ 1.743855] systemd[1]: Reached target Swaps.1892client # [ 1.707041] systemd[1]: Reached target Swaps.1893worker1 # [ 1.744636] systemd[1]: Reached target Timer Units.1894client # [ 1.707881] systemd[1]: Reached target Timer Units.1895worker1 # [ 1.745607] systemd[1]: Listening on D-Bus System Message Bus Socket.1896client # [ 1.708853] systemd[1]: Listening on D-Bus System Message Bus Socket.1897worker1 # [ 1.746839] systemd[1]: Listening on Journal Socket (/dev/log).1898client # [ 1.710094] systemd[1]: Listening on Journal Socket (/dev/log).1899worker1 # [ 1.748141] systemd[1]: Listening on Journal Sockets.1900client # [ 1.711304] systemd[1]: Listening on Journal Sockets.1901worker1 # [ 1.749170] systemd[1]: Listening on udev Control Socket.1902client # [ 1.712315] systemd[1]: Listening on udev Control Socket.1903worker1 # [ 1.750235] systemd[1]: Listening on udev Kernel Socket.1904client # [ 1.713424] systemd[1]: Listening on udev Kernel Socket.1905worker1 # [ 1.751235] systemd[1]: Reached target Socket Units.1906client # [ 1.714386] systemd[1]: Reached target Socket Units.1907worker1 # [ 1.753013] systemd[1]: Starting Create List of Static Device Nodes...1908client # [ 1.716147] systemd[1]: Starting Create List of Static Device Nodes...1909worker2 # [ 1.741057] systemd[1]: Queued start job for default target Initrd Default Target.1910worker1 # [ 1.756751] systemd[1]: Starting Load Kernel Module configfs...1911client # [ 1.720008] systemd[1]: Starting Load Kernel Module configfs...1912worker2 # [ 1.745220] systemd[1]: Created slice Slice /system/modprobe.1913worker2 # [ 1.746431] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1914worker2 # [ 1.747958] systemd[1]: Expecting device /dev/disk/by-label/nixos...1915worker2 # [ 1.749104] systemd[1]: Reached target Path Units.1916worker2 # [ 1.750052] systemd[1]: Reached target Slice Units.1917lb # [ 1.706398] systemd[1]: Starting Journal Service...1918worker2 # [ 1.750962] systemd[1]: Reached target Swaps.1919worker2 # [ 1.751751] systemd[1]: Reached target Timer Units.1920worker2 # [ 1.752729] systemd[1]: Listening on D-Bus System Message Bus Socket.1921worker2 # [ 1.754052] systemd[1]: Listening on Journal Socket (/dev/log).1922worker2 # [ 1.755273] systemd[1]: Listening on Journal Sockets.1923worker2 # [ 1.756312] systemd[1]: Listening on udev Control Socket.1924worker2 # [ 1.757373] systemd[1]: Listening on udev Kernel Socket.1925worker2 # [ 1.758392] systemd[1]: Reached target Socket Units.1926worker2 # [ 1.760208] systemd[1]: Starting Create List of Static Device Nodes...1927worker1 # [ 1.766425] systemd[1]: Starting Journal Service...1928client # [ 1.729263] systemd[1]: Starting Journal Service...1929lb # [ 1.731515] systemd[1]: Starting Load Kernel Modules...1930worker2 # [ 1.763935] systemd[1]: Starting Load Kernel Module configfs...1931lb # [ 1.735424] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1932lb # [ 1.742525] systemd[1]: Starting Coldplug All udev Devices...1933client # [ 1.754380] systemd[1]: Starting Load Kernel Modules...1934worker1 # [ 1.792127] systemd[1]: Starting Load Kernel Modules...1935client # [ 1.758290] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1936worker1 # [ 1.796043] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1937lb # [ 1.754029] systemd-journald[66]: Collecting audit messages is disabled.1938worker2 # [ 1.775093] systemd[1]: Starting Journal Service...1939client # [ 1.766268] systemd[1]: Starting Coldplug All udev Devices...1940worker1 # [ 1.804135] systemd[1]: Starting Coldplug All udev Devices...1941lb # [ 1.757467] systemd[1]: Finished Create List of Static Device Nodes.1942lb # [ 1.764967] systemd[1]: modprobe@configfs.service: Deactivated successfully.1943worker2 # [ 1.798942] systemd[1]: Starting Load Kernel Modules...1944client # [ 1.777036] systemd-journald[66]: Collecting audit messages is disabled.1945worker1 # [ 1.815484] systemd-journald[66]: Collecting audit messages is disabled.1946client # [ 1.780031] systemd[1]: Finished Create List of Static Device Nodes.1947worker1 # [ 1.817447] systemd[1]: Finished Create List of Static Device Nodes.1948worker2 # [ 1.802961] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1949lb # [ 1.771816] systemd[1]: Finished Load Kernel Module configfs.1950worker1 # [ 1.824536] systemd[1]: modprobe@configfs.service: Deactivated successfully.1951client # [ 1.787733] systemd[1]: modprobe@configfs.service: Deactivated successfully.1952lb # [ 1.777786] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1953worker2 # [ 1.812443] systemd[1]: Starting Coldplug All udev Devices...1954worker1 # [ 1.830333] systemd[1]: Finished Load Kernel Module configfs.1955client # [ 1.793626] systemd[1]: Finished Load Kernel Module configfs.1956worker2 # [ 1.819408] systemd-journald[66]: Collecting audit messages is disabled.1957lb # [ 1.790390] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1958worker1 # [ 1.837407] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1959client # [ 1.800619] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1960worker2 # [ 1.826955] systemd[1]: Finished Create List of Static Device Nodes.1961lb # [ 1.796178] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1962worker2 # [ 1.830692] systemd[1]: modprobe@configfs.service: Deactivated successfully.1963worker1 # [ 1.847034] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1964client # [ 1.812272] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1965worker2 # [ 1.837276] systemd[1]: Finished Load Kernel Module configfs.1966client # [ 1.815948] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1967lb # [ 1.807546] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1968worker2 # [ 1.842257] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1969worker1 # [ 1.860125] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1970client # [ 1.829413] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1971worker1 # [ 1.869162] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1972worker2 # [ 1.855951] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1973worker2 # [ 1.861352] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1974lb # [ 1.831900] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1975lb # [ 1.840592] systemd[1]: Starting Create Static Device Nodes in /dev...1976worker2 # [ 1.873203] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1977worker1 # [ 1.889510] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1978client # [ 1.852265] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1979worker1 # [ 1.897141] systemd[1]: Starting Create Static Device Nodes in /dev...1980lb # [ 1.852878] systemd[1]: Finished Load Kernel Modules.1981client # [ 1.863009] systemd[1]: Starting Create Static Device Nodes in /dev...1982lb # [ 1.861572] systemd[1]: Starting Apply Kernel Variables...1983worker1 # [ 1.910117] systemd[1]: Finished Load Kernel Modules.1984client # [ 1.873735] systemd[1]: Finished Load Kernel Modules.1985worker2 # [ 1.895988] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1986worker1 # [ 1.918023] systemd[1]: Starting Apply Kernel Variables...1987worker2 # [ 1.903069] systemd[1]: Starting Create Static Device Nodes in /dev...1988client # [ 1.882387] systemd[1]: Starting Apply Kernel Variables...1989lb # [ 1.747238] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.881824] systemd[1]: Finished Create Static Device Nodes in /dev.1990lb # 1991lb # [ 1.750318] systemd-modules-load[68]: Inserted module 'virtio_balloon'1992worker2 # [ 1.916420] systemd[1]: Finished Load Kernel Modules.1993lb # [ 1.752312] systemd-modules-load[68]: Inserted module 'virtio_gpu'1994lb # [ 1.890050] systemd[1]: Started Journal Service.1995worker1 # [ 1.938516] systemd[1]: Finished Create Static Device Nodes in /dev.1996client # [ 1.766220] systemd-modules-load[68]: Inserted module 'dm_mod'1997client # [ 1.770526] systemd-modules-load[68]: Inserted module 'virtio_balloon'1998worker2 # [ 1.926079] systemd[1]: Starting Apply Kernel Variables...1999client # [ 1.904567] systemd[1]: Finished Create Static Device Nodes in /dev.2000client # [ 1.772501] systemd-modules-load[68]: Inserted module 'virtio_gpu'
loading 2,916 hidden lines…
4917worker1 # [ 45.562344] nix-grpc-daemon[780]: ts=2026-09-23T05:11:58Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04918worker1 # [ 45.564965] nix-grpc-daemon[780]: ts=2026-09-23T05:11:58Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=14919worker2 # [ 45.559272] nix-daemon[852]: reaped child process 898, status = succeeded4920worker2 # [ 45.593772] nix-grpc-daemon[783]: ts=2026-09-23T05:11:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474804921worker2 # [ 45.606211] nix-grpc-daemon[783]: ts=2026-09-23T05:11:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:474804922worker2 # [ 45.620357] nix-grpc-daemon[880]: time=2026-09-23T05:11:58.486Z level=INFO msg="All 1 paths already cached"4923client # copying 1 paths to 'lb:50051'...4924client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4925worker2 # [ 45.628723] nix-grpc-daemon[783]: ts=2026-09-23T05:11:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:474804926worker2 # [ 45.636804] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4927worker2 # [ 45.737208] nix-daemon[852]: accepted connection from pid 923, user nix-grpc-daemon (trusted)4928worker2 # [ 45.752217] nix-daemon[852]: reaped child process 926, status = succeeded4929lb # [ 45.729185] niks3-server[976]: 2026/09/23 05:11:58 INFO Received uploads request method=POST path=/api/pending_closures4930worker2 # [ 45.768380] nix-grpc-daemon[880]: time=2026-09-23T05:11:58.634Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4931worker2 # [ 45.769822] nix-grpc-daemon[880]: time=2026-09-23T05:11:58.635Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4932lb # [ 45.764333] niks3-server[976]: 2026/09/23 05:11:58 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4933lb # [ 45.770376] niks3-server[976]: 2026/09/23 05:11:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4934lb # [ 45.775494] niks3-server[976]: 2026/09/23 05:11:58 INFO Signed narinfos id=11 count=14935worker2 # [ 45.808783] nix-grpc-daemon[880]: time=2026-09-23T05:11:58.674Z level=INFO msg="Uploading 1 narinfos"4936lb # [ 45.784928] niks3-server[976]: 2026/09/23 05:11:58 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4937lb # [ 45.795350] niks3-server[976]: 2026/09/23 05:11:58 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4938lb # [ 45.804182] niks3-server[976]: 2026/09/23 05:11:58 INFO Completed upload id=114939worker2 # [ 45.834493] nix-grpc-daemon[880]: time=2026-09-23T05:11:58.700Z level=INFO msg="Upload complete. (184ms)"4940lb # [ 45.806579] niks3-server[976]: 2026/09/23 05:11:58 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4941worker2 # [ 45.838645] nix-grpc-daemon[783]: ts=2026-09-23T05:11:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=1 nar_bytes_in=6244942worker2 # [ 45.843980] nix-grpc-daemon[783]: ts=2026-09-23T05:11:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474804943worker2 # [ 45.855089] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4944worker2 # [ 45.864125] nix-daemon[852]: reaped child process 921, status = succeeded4945client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4946worker2 # [ 45.867126] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4947client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4948client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4949client # farm-b> LOG-farm-b-sig4950worker2 # [ 46.705386] nix-daemon[852]: reaped child process 930, status = succeeded4951worker2 # [ 46.784692] nix-daemon[852]: accepted connection from pid 946, user nix-grpc-daemon (trusted)4952worker2 # [ 46.798941] nix-daemon[852]: reaped child process 949, status = succeeded4953lb # [ 46.775491] niks3-server[976]: 2026/09/23 05:11:59 INFO Received uploads request method=POST path=/api/pending_closures4954worker2 # [ 46.813826] nix-grpc-daemon[880]: time=2026-09-23T05:11:59.679Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4955worker2 # [ 46.815458] nix-grpc-daemon[880]: time=2026-09-23T05:11:59.681Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4956lb # [ 46.815950] niks3-server[976]: 2026/09/23 05:12:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4957lb # [ 46.823515] niks3-server[976]: 2026/09/23 05:12:00 INFO Signed narinfos id=12 count=14958worker2 # [ 46.857701] nix-grpc-daemon[880]: time=2026-09-23T05:11:59.723Z level=INFO msg="Uploading 1 narinfos"4959lb # [ 46.833938] niks3-server[976]: 2026/09/23 05:12:00 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4960lb # [ 46.840402] niks3-server[976]: 2026/09/23 05:12:00 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4961lb # [ 46.849665] niks3-server[976]: 2026/09/23 05:12:00 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4962lb # [ 46.857887] niks3-server[976]: 2026/09/23 05:12:00 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4963lb # [ 46.861063] niks3-server[976]: 2026/09/23 05:12:00 INFO Completed upload id=124964worker2 # [ 46.891680] nix-grpc-daemon[880]: time=2026-09-23T05:11:59.757Z level=INFO msg="Upload complete. (217ms)"4965lb # [ 46.864152] niks3-server[976]: 2026/09/23 05:12:00 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4966worker2 # [ 46.897753] nix-grpc-daemon[783]: ts=2026-09-23T05:11:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14967worker1 # [ 46.918899] nix-grpc-daemon[780]: ts=2026-09-23T05:12:00Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04968worker1 # [ 46.923455] nix-grpc-daemon[780]: ts=2026-09-23T05:12:00Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=14969worker2 # [ 46.913561] nix-daemon[852]: reaped child process 928, status = succeeded4970worker2 # [ 46.947795] nix-grpc-daemon[783]: ts=2026-09-23T05:11:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474804971lb # [ 46.918972] envoy[912]: conn peer=[2001:db8:1::1]:53964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9490 tx=4350 ms=57264972lb # [ 46.922606] envoy[912]: conn peer=[2001:db8:1::1]:53970 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=644973lb # [ 46.926680] envoy[912]: conn peer=[2001:db8:1::1]:53980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4122 tx=1859 ms=17784974worker2 # [ 46.966425] nix-grpc-daemon[783]: ts=2026-09-23T05:11:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:474804975worker2 # [ 46.979142] nix-grpc-daemon[880]: time=2026-09-23T05:11:59.845Z level=INFO msg="All 2 paths already cached"4976client # copying 1 paths to 'lb:50051'...4977client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4978worker2 # [ 46.988841] nix-grpc-daemon[783]: ts=2026-09-23T05:11:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:474804979worker2 # [ 46.997249] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4980worker2 # [ 47.098461] nix-daemon[852]: accepted connection from pid 953, user nix-grpc-daemon (trusted)4981worker2 # [ 47.112300] nix-daemon[852]: reaped child process 956, status = succeeded4982lb # [ 47.089193] niks3-server[976]: 2026/09/23 05:12:00 INFO Received uploads request method=POST path=/api/pending_closures4983worker2 # [ 47.128315] nix-grpc-daemon[880]: time=2026-09-23T05:11:59.994Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4984worker2 # [ 47.129863] nix-grpc-daemon[880]: time=2026-09-23T05:11:59.995Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4985lb # [ 47.122816] niks3-server[976]: 2026/09/23 05:12:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4986lb # [ 47.129290] niks3-server[976]: 2026/09/23 05:12:00 INFO Signed narinfos id=13 count=14987worker2 # [ 47.164028] nix-grpc-daemon[880]: time=2026-09-23T05:12:00.029Z level=INFO msg="Uploading 1 narinfos"4988lb # [ 47.136243] niks3-server[976]: 2026/09/23 05:12:00 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4989lb # [ 47.148729] niks3-server[976]: 2026/09/23 05:12:00 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4990lb # [ 47.156512] niks3-server[976]: 2026/09/23 05:12:00 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4991lb # [ 47.164607] niks3-server[976]: 2026/09/23 05:12:00 INFO Completed upload id=134992worker2 # [ 47.194267] nix-grpc-daemon[880]: time=2026-09-23T05:12:00.060Z level=INFO msg="Upload complete. (183ms)"4993lb # [ 47.167274] niks3-server[976]: 2026/09/23 05:12:00 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4994worker2 # [ 47.198707] nix-grpc-daemon[783]: ts=2026-09-23T05:12:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=1 nar_bytes_in=7524995worker2 # [ 47.204289] nix-grpc-daemon[783]: ts=2026-09-23T05:12:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474804996worker2 # [ 47.215865] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4997worker2 # [ 47.223580] nix-daemon[852]: reaped child process 951, status = succeeded4998worker2 # [ 47.228551] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4999client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...5000client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5001client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5002client # farm-top> LOG-farm-top-sig5003worker2 # [ 48.066174] nix-daemon[852]: reaped child process 960, status = succeeded5004worker2 # [ 48.151544] nix-daemon[852]: accepted connection from pid 976, user nix-grpc-daemon (trusted)5005worker2 # [ 48.164577] nix-daemon[852]: reaped child process 979, status = succeeded5006lb # [ 48.140357] niks3-server[976]: 2026/09/23 05:12:01 INFO Received uploads request method=POST path=/api/pending_closures5007worker2 # [ 48.179439] nix-grpc-daemon[880]: time=2026-09-23T05:12:01.045Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5008worker2 # [ 48.180908] nix-grpc-daemon[880]: time=2026-09-23T05:12:01.046Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5009lb # [ 48.176277] niks3-server[976]: 2026/09/23 05:12:01 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5010lb # [ 48.194215] niks3-server[976]: 2026/09/23 05:12:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5011lb # [ 48.203095] niks3-server[976]: 2026/09/23 05:12:01 INFO Signed narinfos id=14 count=15012worker2 # [ 48.236313] nix-grpc-daemon[880]: time=2026-09-23T05:12:01.101Z level=INFO msg="Uploading 1 narinfos"5013lb # [ 48.206528] niks3-server[976]: 2026/09/23 05:12:01 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5014lb # [ 48.213159] niks3-server[976]: 2026/09/23 05:12:01 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5015lb # [ 48.221656] niks3-server[976]: 2026/09/23 05:12:01 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5016lb # [ 48.224955] niks3-server[976]: 2026/09/23 05:12:01 INFO Completed upload id=145017worker2 # [ 48.255444] nix-grpc-daemon[880]: time=2026-09-23T05:12:01.121Z level=INFO msg="Upload complete. (218ms)"5018lb # [ 48.227944] niks3-server[976]: 2026/09/23 05:12:01 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5019worker2 # [ 48.261789] nix-grpc-daemon[783]: ts=2026-09-23T05:12:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15020worker2 # [ 48.276305] nix-daemon[852]: reaped child process 958, status = succeeded5021client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.72 seconds)5022client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5023client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.11 seconds)5024client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5025worker1 # [ 48.524895] nix-grpc-daemon[780]: ts=2026-09-23T05:12:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:356645026client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5027client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5028client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5029client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5030worker1 # [ 48.550065] nix-grpc-daemon[780]: ts=2026-09-23T05:12:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:356645031worker1 # [ 48.556707] nix-grpc-daemon[780]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5032worker1 # [ 48.571135] nix-grpc-daemon[780]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5033worker1 # [ 48.583357] nix-grpc-daemon[780]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5034client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.20 seconds)5035client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5036worker2 # [ 48.719560] nix-grpc-daemon[783]: ts=2026-09-23T05:12:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:503225037worker2 # [ 48.726070] nix-grpc-daemon[783]: ts=2026-09-23T05:12:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:503225038client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.18 seconds)5039client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5040client # finding garbage collector roots...5041client # removing stale temporary roots file "/nix/var/nix/temproots/802"5042client # removing stale temporary roots file "/nix/var/nix/temproots/897"5043client # removing stale temporary roots file "/nix/var/nix/temproots/852"5044client # removing stale temporary roots file "/nix/var/nix/temproots/882"5045client # removing stale temporary roots file "/nix/var/nix/temproots/860"5046client # removing stale temporary roots file "/nix/var/nix/temproots/819"5047client # removing stale temporary roots file "/nix/var/nix/temproots/840"5048client # removing stale temporary roots file "/nix/var/nix/temproots/874"5049client # deleting specified paths...5050client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5051client # deleting unused links...5052client # note: hard linking is currently saving 0.0 KiB5053worker1 # [ 49.027212] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406245054worker1 # [ 49.035115] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245055worker1 # [ 49.045961] nix-grpc-daemon[878]: time=2026-09-23T05:12:02.149Z level=INFO msg="All 1 paths already cached"5056worker2 # [ 49.039563] nix-grpc-daemon[783]: ts=2026-09-23T05:12:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:474805057worker1 # [ 49.061787] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406245058client # copying 3 paths...5059worker2 # [ 49.057834] nix-grpc-daemon[783]: ts=2026-09-23T05:12:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:474805060client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5061worker2 # [ 49.074081] nix-grpc-daemon[783]: ts=2026-09-23T05:12:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:474805062worker2 # [ 49.080364] nix-grpc-daemon[783]: ts=2026-09-23T05:12:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=1 nar_bytes_out=2325063worker1 # [ 49.095207] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406245064worker2 # [ 49.088278] nix-grpc-daemon[783]: ts=2026-09-23T05:12:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:474805065worker1 # [ 49.101267] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_out=1765066worker2 # [ 49.092987] nix-grpc-daemon[783]: ts=2026-09-23T05:12:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=1 nar_bytes_out=1285067client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5068client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5069client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.44 seconds)5070(finished: subtest: outputs read back through any worker carry the cache signature, in 5.64 seconds)5071subtest: two clients wanting the same drv build it once5072client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5073client # [ 49.230285] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5074client # Running as unit: dedup1.service; invocation ID: d533823de151480da8d93c2e442fd6075075client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.04 seconds)5076client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5077client # [ 49.362785] nix[950]: don't know how to build these paths:5078client # [ 49.363783] nix[950]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5079worker2 # [ 49.382677] nix-grpc-daemon[783]: ts=2026-09-23T05:12:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:474805080worker1 # [ 49.405388] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05081worker1 # [ 49.409542] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=15082worker1 # [ 49.459410] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245083worker1 # [ 49.471827] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245084client # [ 49.448336] nix[950]: copying 1 paths to 'lb:50051'...5085worker1 # [ 49.483845] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406245086client # [ 49.450463] nix[950]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5087worker1 # [ 49.496423] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5088worker1 # [ 49.608089] nix-daemon[863]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)5089worker1 # [ 49.624518] nix-daemon[863]: reaped child process 1061, status = succeeded5090lb # [ 49.588713] niks3-server[976]: 2026/09/23 05:12:02 INFO Received uploads request method=POST path=/api/pending_closures5091worker1 # [ 49.641900] nix-grpc-daemon[878]: time=2026-09-23T05:12:02.745Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5092worker1 # [ 49.643931] nix-grpc-daemon[878]: time=2026-09-23T05:12:02.747Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5093lb # [ 49.618357] niks3-server[976]: 2026/09/23 05:12:02 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5094lb # [ 49.626592] niks3-server[976]: 2026/09/23 05:12:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5095lb # [ 49.632505] niks3-server[976]: 2026/09/23 05:12:02 INFO Signed narinfos id=15 count=15096worker1 # [ 49.679585] nix-grpc-daemon[878]: time=2026-09-23T05:12:02.783Z level=INFO msg="Uploading 1 narinfos"5097lb # [ 49.647257] niks3-server[976]: 2026/09/23 05:12:02 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5098lb # [ 49.649904] niks3-server[976]: 2026/09/23 05:12:02 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5099lb # [ 49.652427] niks3-server[976]: 2026/09/23 05:12:02 INFO Completed upload id=155100worker1 # [ 49.696934] nix-grpc-daemon[878]: time=2026-09-23T05:12:02.800Z level=INFO msg="Upload complete. (188ms)"5101lb # [ 49.654422] niks3-server[976]: 2026/09/23 05:12:02 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5102worker1 # [ 49.701310] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_in=4325103worker1 # [ 49.706758] nix-grpc-daemon[780]: ts=2026-09-23T05:12:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245104worker1 # [ 49.715478] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5105worker1 # [ 49.720730] nix-daemon[863]: reaped child process 1056, status = succeeded5106worker1 # [ 49.725149] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5107client # [ 49.690485] nix[950]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5108client # [ 50.282962] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5109client # Running as unit: dedup2.service; invocation ID: 2688bdd8ef854ee2a53cf891a249a7845110client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)5111worker1 # [ 50.480921] nix-grpc-daemon[780]: ts=2026-09-23T05:12:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406245112client # [ 50.457648] nix[966]: this derivation will be built:5113worker2 # [ 50.481427] nix-grpc-daemon[783]: ts=2026-09-23T05:12:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:474805114worker2 # [ 50.492173] nix-grpc-daemon[783]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...5115client # [ 50.488305] nix[966]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5116worker1 # [ 50.530306] nix-grpc-daemon[780]: ts=2026-09-23T05:12:03Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05117worker1 # [ 50.541320] nix-grpc-daemon[780]: ts=2026-09-23T05:12:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245118client # [ 50.512577] nix[966]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5119client: must succeed: sleep 25120client: (finished: must succeed: sleep 2, in 2.02 seconds)5121client: waiting for success: ! systemctl is-active dedup1 dedup25122worker1 # [ 53.767964] nix-daemon[863]: reaped child process 1065, status = succeeded5123worker1 # [ 53.850026] nix-daemon[863]: accepted connection from pid 1097, user nix-grpc-daemon (trusted)5124worker1 # [ 53.861427] nix-daemon[863]: reaped child process 1100, status = succeeded5125lb # [ 53.826511] niks3-server[976]: 2026/09/23 05:12:07 INFO Received uploads request method=POST path=/api/pending_closures5126worker1 # [ 53.879427] nix-grpc-daemon[878]: time=2026-09-23T05:12:06.982Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5127worker1 # [ 53.881706] nix-grpc-daemon[878]: time=2026-09-23T05:12:06.984Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5128lb # [ 53.873235] niks3-server[976]: 2026/09/23 05:12:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5129lb # [ 53.882731] niks3-server[976]: 2026/09/23 05:12:07 INFO Signed narinfos id=16 count=15130worker1 # [ 53.930452] nix-grpc-daemon[878]: time=2026-09-23T05:12:07.034Z level=INFO msg="Uploading 1 narinfos"5131lb # [ 53.894374] niks3-server[976]: 2026/09/23 05:12:07 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5132lb # [ 54.038452] niks3-server[976]: 2026/09/23 05:12:07 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5133lb # [ 54.052203] niks3-server[976]: 2026/09/23 05:12:07 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5134lb # [ 54.062175] niks3-server[976]: 2026/09/23 05:12:07 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5135lb # [ 54.069666] niks3-server[976]: 2026/09/23 05:12:07 INFO Completed upload id=165136worker1 # [ 54.113911] nix-grpc-daemon[878]: time=2026-09-23T05:12:07.217Z level=INFO msg="Upload complete. (373ms)"5137lb # [ 54.077342] niks3-server[976]: 2026/09/23 05:12:07 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5138worker1 # [ 54.123289] nix-grpc-daemon[780]: ts=2026-09-23T05:12:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15139worker1 # [ 54.130801] nix-grpc-daemon[780]: ts=2026-09-23T05:12:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15140worker1 # [ 54.144589] nix-daemon[863]: reaped child process 1063, status = succeeded5141client # [ 54.126520] systemd[1]: dedup2.service: Deactivated successfully.5142client # [ 54.130563] systemd[1]: dedup2.service: Consumed 146ms CPU time over 3.846s wall clock time, 35.4M memory peak, 6.5K incoming IP traffic, 6K outgoing IP traffic.5143client # [ 54.164966] systemd[1]: dedup1.service: Deactivated successfully.5144client # [ 54.168169] systemd[1]: dedup1.service: Consumed 118ms CPU time over 4.936s wall clock time, 43.5M memory peak, 7.4K incoming IP traffic, 8.6K outgoing IP traffic.5145client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)5146client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5147client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5148worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5149worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5150worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5151worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5152worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5153worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5154worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5155worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5156(finished: subtest: two clients wanting the same drv build it once, in 5.74 seconds)5157subtest: --timeout and --max-silent-time reach the builder5158worker2 # [ 55.099508] nix-grpc-daemon[783]: ts=2026-09-23T05:12:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:474805159worker1 # [ 55.128889] nix-grpc-daemon[780]: ts=2026-09-23T05:12:08Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05160worker1 # [ 55.133899] nix-grpc-daemon[780]: ts=2026-09-23T05:12:08Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=15161worker2 # [ 55.122407] nix-grpc-daemon[783]: ts=2026-09-23T05:12:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474805162worker2 # [ 55.133697] nix-grpc-daemon[783]: ts=2026-09-23T05:12:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:474805163worker2 # [ 55.144972] nix-grpc-daemon[783]: ts=2026-09-23T05:12:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:474805164worker2 # [ 55.156091] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5165worker2 # [ 55.260443] nix-daemon[852]: accepted connection from pid 1005, user nix-grpc-daemon (trusted)5166worker2 # [ 55.276140] nix-daemon[852]: reaped child process 1008, status = succeeded5167lb # [ 55.251603] niks3-server[976]: 2026/09/23 05:12:08 INFO Received uploads request method=POST path=/api/pending_closures5168worker2 # [ 55.589769] nix-grpc-daemon[880]: time=2026-09-23T05:12:08.454Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5169worker2 # [ 55.593997] nix-grpc-daemon[880]: time=2026-09-23T05:12:08.454Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5170lb # [ 55.580646] niks3-server[976]: 2026/09/23 05:12:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5171lb # [ 55.586493] niks3-server[976]: 2026/09/23 05:12:08 INFO Signed narinfos id=17 count=15172worker2 # [ 55.620519] nix-grpc-daemon[880]: time=2026-09-23T05:12:08.486Z level=INFO msg="Uploading 1 narinfos"5173lb # [ 55.601566] niks3-server[976]: 2026/09/23 05:12:08 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5174lb # [ 55.618671] niks3-server[976]: 2026/09/23 05:12:08 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5175lb # [ 55.633097] niks3-server[976]: 2026/09/23 05:12:08 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5176lb # [ 55.691333] niks3-server[976]: 2026/09/23 05:12:08 INFO Completed upload id=175177worker2 # [ 55.721552] nix-grpc-daemon[880]: time=2026-09-23T05:12:08.587Z level=INFO msg="Upload complete. (552ms)"5178worker2 # [ 55.726777] nix-grpc-daemon[783]: ts=2026-09-23T05:12:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=1 nar_bytes_in=4245179lb # [ 55.701669] niks3-server[976]: 2026/09/23 05:12:08 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5180worker2 # [ 55.733338] nix-grpc-daemon[783]: ts=2026-09-23T05:12:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474805181worker2 # [ 55.747170] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5182worker2 # [ 55.756891] nix-daemon[852]: reaped child process 1003, status = succeeded5183worker2 # [ 55.768191] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5184lb # [ 56.919184] envoy[912]: conn peer=[2001:db8:1::1]:47908 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9494 tx=4384 ms=45535185lb # [ 56.923375] envoy[912]: conn peer=[2001:db8:1::1]:47914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2887 ms=1855186lb # [ 56.927089] envoy[912]: conn peer=[2001:db8:1::1]:47922 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=1095187lb # [ 56.930642] envoy[912]: conn peer=[2001:db8:1::1]:47916 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=427 tx=427 ms=1245188lb # [ 56.934253] envoy[912]: conn peer=[2001:db8:1::1]:47920 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=425 tx=393 ms=1175189lb # [ 56.938448] envoy[912]: conn peer=[2001:db8:1::1]:47942 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1589 tx=1599 ms=36665190lb # [ 56.942114] envoy[912]: conn peer=[2001:db8:1::1]:47938 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3024 tx=1536 ms=47875191worker2 # [ 61.586305] nix-grpc-daemon[783]: ts=2026-09-23T05:12:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=5 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05192worker2 # [ 61.598382] nix-daemon[852]: reaped child process 1010, status = succeeded5193worker2 # [ 61.599544] nix-daemon[852]: reaped child process 1012, status = succeeded5194worker1 # [ 61.803854] nix-grpc-daemon[780]: ts=2026-09-23T05:12:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406245195worker1 # [ 61.817114] nix-grpc-daemon[780]: ts=2026-09-23T05:12:14Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05196worker1 # [ 61.822397] nix-grpc-daemon[780]: ts=2026-09-23T05:12:14Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=15197worker1 # [ 61.828476] nix-grpc-daemon[780]: ts=2026-09-23T05:12:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245198worker1 # [ 61.839643] nix-grpc-daemon[780]: ts=2026-09-23T05:12:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245199worker1 # [ 61.850401] nix-grpc-daemon[780]: ts=2026-09-23T05:12:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406245200worker1 # [ 61.858947] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5201worker1 # [ 61.957400] nix-daemon[863]: accepted connection from pid 1117, user nix-grpc-daemon (trusted)5202worker1 # [ 61.971745] nix-daemon[863]: reaped child process 1120, status = succeeded5203lb # [ 61.934209] niks3-server[976]: 2026/09/23 05:12:15 INFO Received uploads request method=POST path=/api/pending_closures5204worker1 # [ 61.987538] nix-grpc-daemon[878]: time=2026-09-23T05:12:15.091Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5205worker1 # [ 61.989456] nix-grpc-daemon[878]: time=2026-09-23T05:12:15.092Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5206lb # [ 61.965167] niks3-server[976]: 2026/09/23 05:12:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5207lb # [ 61.972671] niks3-server[976]: 2026/09/23 05:12:15 INFO Signed narinfos id=18 count=15208worker1 # [ 62.020313] nix-grpc-daemon[878]: time=2026-09-23T05:12:15.123Z level=INFO msg="Uploading 1 narinfos"5209lb # [ 61.981139] niks3-server[976]: 2026/09/23 05:12:15 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5210lb # [ 61.992366] niks3-server[976]: 2026/09/23 05:12:15 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5211lb # [ 61.999246] niks3-server[976]: 2026/09/23 05:12:15 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5212lb # [ 62.006626] niks3-server[976]: 2026/09/23 05:12:15 INFO Completed upload id=185213worker1 # [ 62.050898] nix-grpc-daemon[878]: time=2026-09-23T05:12:15.154Z level=INFO msg="Upload complete. (180ms)"5214worker1 # [ 62.054928] nix-grpc-daemon[780]: ts=2026-09-23T05:12:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_in=4245215worker1 # [ 62.060270] nix-grpc-daemon[780]: ts=2026-09-23T05:12:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245216lb # [ 62.016364] niks3-server[976]: 2026/09/23 05:12:15 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5217worker1 # [ 62.069483] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5218worker1 # [ 62.077822] nix-daemon[863]: reaped child process 1115, status = succeeded5219worker1 # [ 62.082386] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5220lb # [ 66.919760] envoy[912]: conn peer=[2001:db8:1::1]:59924 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2983 tx=953 ms=65415221worker1 # [ 67.696555] nix-grpc-daemon[780]: ts=2026-09-23T05:12:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=5 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05222worker1 # [ 67.721427] nix-daemon[863]: reaped child process 1122, status = succeeded5223worker1 # [ 67.724145] nix-daemon[863]: reaped child process 1124, status = succeeded5224(finished: subtest: --timeout and --max-silent-time reach the builder, in 12.85 seconds)5225subtest: the first client leaving does not fail the second5226client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5227client # [ 67.824916] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5228client # Running as unit: share1.service; invocation ID: 396fe7a7cd0b4eedbb1b119c964843d45229client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5230client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5231worker2 # [ 67.990909] nix-grpc-daemon[783]: ts=2026-09-23T05:12:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:474805232client # [ 67.977114] nix[1040]: don't know how to build these paths:5233client # [ 67.978056] nix[1040]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5234worker1 # [ 68.019714] nix-grpc-daemon[780]: ts=2026-09-23T05:12:21Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05235worker2 # [ 68.007872] nix-grpc-daemon[783]: ts=2026-09-23T05:12:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474805236worker1 # [ 68.024406] nix-grpc-daemon[780]: ts=2026-09-23T05:12:21Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15237worker2 # [ 68.020227] nix-grpc-daemon[783]: ts=2026-09-23T05:12:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:474805238client # [ 68.009737] nix[1040]: copying 1 paths to 'lb:50051'...5239client # [ 68.010656] nix[1040]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5240worker2 # [ 68.030789] nix-grpc-daemon[783]: ts=2026-09-23T05:12:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:474805241worker2 # [ 68.039703] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5242worker2 # [ 68.142202] nix-daemon[852]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)5243worker2 # [ 68.156882] nix-daemon[852]: reaped child process 1038, status = succeeded5244lb # [ 68.132607] niks3-server[976]: 2026/09/23 05:12:21 INFO Received uploads request method=POST path=/api/pending_closures5245worker2 # [ 68.170983] nix-grpc-daemon[880]: time=2026-09-23T05:12:21.036Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5246worker2 # [ 68.172914] nix-grpc-daemon[880]: time=2026-09-23T05:12:21.038Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5247lb # [ 68.163924] niks3-server[976]: 2026/09/23 05:12:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5248lb # [ 68.169123] niks3-server[976]: 2026/09/23 05:12:21 INFO Signed narinfos id=19 count=15249worker2 # [ 68.203034] nix-grpc-daemon[880]: time=2026-09-23T05:12:21.068Z level=INFO msg="Uploading 1 narinfos"5250lb # [ 68.178467] niks3-server[976]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5251lb # [ 68.189832] niks3-server[976]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5252lb # [ 68.197327] niks3-server[976]: 2026/09/23 05:12:21 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5253lb # [ 68.204538] niks3-server[976]: 2026/09/23 05:12:21 INFO Completed upload id=195254worker2 # [ 68.235328] nix-grpc-daemon[880]: time=2026-09-23T05:12:21.101Z level=INFO msg="Upload complete. (181ms)"5255lb # [ 68.208447] niks3-server[976]: 2026/09/23 05:12:21 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5256worker2 # [ 68.239960] nix-grpc-daemon[783]: ts=2026-09-23T05:12:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=1 nar_bytes_in=4325257worker2 # [ 68.247572] nix-grpc-daemon[783]: ts=2026-09-23T05:12:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474805258worker2 # [ 68.257196] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5259worker2 # [ 68.265564] nix-daemon[852]: reaped child process 1033, status = succeeded5260worker2 # [ 68.270698] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5261client # [ 68.250201] nix[1040]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5262client # [ 68.892907] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5263client # Running as unit: share2.service; invocation ID: 0513d02602724c8987f85491c490c7815264client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5265worker1 # [ 69.071419] nix-grpc-daemon[780]: ts=2026-09-23T05:12:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406245266client # [ 69.040456] nix[1056]: don't know how to build these paths:5267client # [ 69.041373] nix[1056]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5268worker1 # [ 69.079409] nix-grpc-daemon[780]: ts=2026-09-23T05:12:22Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05269client # [ 69.048215] nix[1056]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5270worker2 # [ 69.068386] nix-grpc-daemon[783]: ts=2026-09-23T05:12:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474805271client: must succeed: sleep 2; systemctl kill -s INT share15272client # [ 72.143353] systemd[1]: share1.service: Sent signal SIGINT to main process 1040 (nix) on client request.5273client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5274client # [ 72.154866] nix[1040]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5275client: waiting for success: ! systemctl is-active share15276client # [ 72.160266] nix[1040]: error: interrupted by the user5277client # [ 72.201398] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5278client # [ 72.203713] systemd[1]: share1.service: Failed with result 'exit-code'.5279client # [ 72.206084] systemd[1]: share1.service: Consumed 118ms CPU time over 4.379s wall clock time, 41.5M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.5280client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5281client: waiting for success: ! systemctl is-active share25282worker2 # [ 73.491859] nix-daemon[852]: reaped child process 1042, status = succeeded5283worker2 # [ 73.565898] nix-daemon[852]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)5284worker2 # [ 73.578352] nix-daemon[852]: reaped child process 1079, status = succeeded5285lb # [ 73.555071] niks3-server[976]: 2026/09/23 05:12:26 INFO Received uploads request method=POST path=/api/pending_closures5286worker2 # [ 73.594154] nix-grpc-daemon[880]: time=2026-09-23T05:12:26.459Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5287worker2 # [ 73.595651] nix-grpc-daemon[880]: time=2026-09-23T05:12:26.461Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5288lb # [ 73.608384] niks3-server[976]: 2026/09/23 05:12:26 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5289lb # [ 73.614620] niks3-server[976]: 2026/09/23 05:12:26 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5290lb # [ 73.621184] niks3-server[976]: 2026/09/23 05:12:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5291worker2 # [ 73.655760] nix-grpc-daemon[880]: time=2026-09-23T05:12:26.521Z level=INFO msg="Uploading 1 narinfos"5292lb # [ 73.625833] niks3-server[976]: 2026/09/23 05:12:26 INFO Signed narinfos id=20 count=15293lb # [ 73.635634] niks3-server[976]: 2026/09/23 05:12:26 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5294lb # [ 73.638253] niks3-server[976]: 2026/09/23 05:12:26 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5295lb # [ 73.641222] niks3-server[976]: 2026/09/23 05:12:26 INFO Completed upload id=205296worker2 # [ 73.671734] nix-grpc-daemon[880]: time=2026-09-23T05:12:26.537Z level=INFO msg="Upload complete. (190ms)"5297worker2 # [ 73.677763] nix-grpc-daemon[783]: ts=2026-09-23T05:12:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15298worker2 # [ 73.684250] nix-grpc-daemon[783]: ts=2026-09-23T05:12:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15299worker2 # [ 73.697274] nix-daemon[852]: reaped child process 1040, status = succeeded5300client # [ 73.731410] systemd[1]: share2.service: Deactivated successfully.5301client # [ 73.735101] systemd[1]: share2.service: Consumed 119ms CPU time over 4.841s wall clock time, 35.5M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5302client: (finished: waiting for success: ! systemctl is-active share2, in 1.09 seconds)5303client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5304client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds)5305(finished: subtest: the first client leaving does not fail the second, in 6.76 seconds)5306subtest: build hook: nix-daemon with builders = grpc://lb5307client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5308client # [ 74.635827] systemd[1]: Started Nix Daemon.5309client # [ 74.724910] nix-daemon[1108]: accepted connection from pid 1105, user root (trusted)5310worker2 # [ 74.861777] nix-grpc-daemon[783]: ts=2026-09-23T05:12:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:474805311worker2 # [ 74.867112] nix-grpc-daemon[783]: ts=2026-09-23T05:12:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:474805312worker1 # [ 74.894154] nix-grpc-daemon[780]: ts=2026-09-23T05:12:27Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05313worker1 # [ 74.900431] nix-grpc-daemon[780]: ts=2026-09-23T05:12:27Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15314worker1 # [ 74.905506] nix-grpc-daemon[780]: ts=2026-09-23T05:12:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245315worker1 # [ 74.913704] nix-grpc-daemon[780]: ts=2026-09-23T05:12:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245316worker1 # [ 74.925975] nix-grpc-daemon[780]: ts=2026-09-23T05:12:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406245317worker1 # [ 74.936981] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5318worker1 # [ 75.038571] nix-daemon[863]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5319worker1 # [ 75.051112] nix-daemon[863]: reaped child process 1172, status = succeeded5320lb # [ 75.013612] niks3-server[976]: 2026/09/23 05:12:28 INFO Received uploads request method=POST path=/api/pending_closures5321worker1 # [ 75.066347] nix-grpc-daemon[878]: time=2026-09-23T05:12:28.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5322worker1 # [ 75.067810] nix-grpc-daemon[878]: time=2026-09-23T05:12:28.171Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5323lb # [ 75.045123] niks3-server[976]: 2026/09/23 05:12:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5324lb # [ 75.051193] niks3-server[976]: 2026/09/23 05:12:28 INFO Signed narinfos id=21 count=15325worker1 # [ 75.098764] nix-grpc-daemon[878]: time=2026-09-23T05:12:28.202Z level=INFO msg="Uploading 1 narinfos"5326lb # [ 75.064728] niks3-server[976]: 2026/09/23 05:12:28 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5327lb # [ 75.069739] niks3-server[976]: 2026/09/23 05:12:28 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5328lb # [ 75.078120] niks3-server[976]: 2026/09/23 05:12:28 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5329lb # [ 75.082908] niks3-server[976]: 2026/09/23 05:12:28 INFO Completed upload id=215330worker1 # [ 75.129757] nix-grpc-daemon[878]: time=2026-09-23T05:12:28.233Z level=INFO msg="Upload complete. (181ms)"5331lb # [ 75.088883] niks3-server[976]: 2026/09/23 05:12:28 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5332worker1 # [ 75.133776] nix-grpc-daemon[780]: ts=2026-09-23T05:12:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_in=5125333worker1 # [ 75.138989] nix-grpc-daemon[780]: ts=2026-09-23T05:12:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245334worker1 # [ 75.149180] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5335worker1 # [ 75.156389] nix-daemon[863]: reaped child process 1167, status = succeeded5336worker1 # [ 75.160935] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5337worker1 # [ 75.995961] nix-daemon[863]: reaped child process 1176, status = succeeded5338worker1 # [ 76.082755] nix-daemon[863]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)5339worker1 # [ 76.095906] nix-daemon[863]: reaped child process 1195, status = succeeded5340lb # [ 76.055850] niks3-server[976]: 2026/09/23 05:12:29 INFO Received uploads request method=POST path=/api/pending_closures5341worker1 # [ 76.109839] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.213Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5342worker1 # [ 76.111992] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.215Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5343lb # [ 76.093969] niks3-server[976]: 2026/09/23 05:12:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5344worker1 # [ 76.147466] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.251Z level=INFO msg="Uploading 1 narinfos"5345lb # [ 76.103108] niks3-server[976]: 2026/09/23 05:12:29 INFO Signed narinfos id=22 count=15346lb # [ 76.110646] niks3-server[976]: 2026/09/23 05:12:29 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5347lb # [ 76.115627] niks3-server[976]: 2026/09/23 05:12:29 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5348lb # [ 76.123804] niks3-server[976]: 2026/09/23 05:12:29 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5349lb # [ 76.129640] niks3-server[976]: 2026/09/23 05:12:29 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5350lb # [ 76.137233] niks3-server[976]: 2026/09/23 05:12:29 INFO Completed upload id=225351worker1 # [ 76.181911] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.285Z level=INFO msg="Upload complete. (214ms)"5352worker1 # [ 76.189533] nix-grpc-daemon[780]: ts=2026-09-23T05:12:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15353worker2 # [ 76.178425] nix-grpc-daemon[783]: ts=2026-09-23T05:12:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:474805354worker2 # [ 76.187693] nix-grpc-daemon[783]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5355worker1 # [ 76.203235] nix-daemon[863]: reaped child process 1174, status = succeeded5356worker2 # [ 76.208123] nix-grpc-daemon[783]: ts=2026-09-23T05:12:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:474805357worker2 # [ 76.213273] nix-grpc-daemon[783]: ts=2026-09-23T05:12:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=1 nar_bytes_out=1285358worker2 # [ 76.414190] nix-grpc-daemon[783]: ts=2026-09-23T05:12:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:474805359worker1 # [ 76.431712] nix-grpc-daemon[780]: ts=2026-09-23T05:12:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245360worker1 # [ 76.440888] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.544Z level=INFO msg="All 1 paths already cached"5361worker1 # [ 76.455225] nix-grpc-daemon[780]: ts=2026-09-23T05:12:29Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05362worker1 # [ 76.460690] nix-grpc-daemon[780]: ts=2026-09-23T05:12:29Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15363worker1 # [ 76.466431] nix-grpc-daemon[780]: ts=2026-09-23T05:12:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245364worker1 # [ 76.478193] nix-grpc-daemon[780]: ts=2026-09-23T05:12:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245365worker1 # [ 76.492688] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.596Z level=INFO msg="All 1 paths already cached"5366worker1 # [ 76.500788] nix-grpc-daemon[780]: ts=2026-09-23T05:12:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406245367worker1 # [ 76.510174] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5368worker1 # [ 76.610931] nix-daemon[863]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5369worker1 # [ 76.623911] nix-daemon[863]: reaped child process 1204, status = succeeded5370lb # [ 76.586901] niks3-server[976]: 2026/09/23 05:12:29 INFO Received uploads request method=POST path=/api/pending_closures5371worker1 # [ 76.644772] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.748Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5372worker1 # [ 76.646173] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.748Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5373lb # [ 76.619861] niks3-server[976]: 2026/09/23 05:12:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5374lb # [ 76.624101] niks3-server[976]: 2026/09/23 05:12:29 INFO Signed narinfos id=23 count=15375worker1 # [ 76.671629] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.775Z level=INFO msg="Uploading 1 narinfos"5376lb # [ 76.638801] niks3-server[976]: 2026/09/23 05:12:29 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5377lb # [ 76.644700] niks3-server[976]: 2026/09/23 05:12:29 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5378lb # [ 76.651358] niks3-server[976]: 2026/09/23 05:12:29 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5379lb # [ 76.657329] niks3-server[976]: 2026/09/23 05:12:29 INFO Completed upload id=235380worker1 # [ 76.701445] nix-grpc-daemon[878]: time=2026-09-23T05:12:29.805Z level=INFO msg="Upload complete. (179ms)"5381worker1 # [ 76.705643] nix-grpc-daemon[780]: ts=2026-09-23T05:12:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_in=6245382worker1 # [ 76.711276] nix-grpc-daemon[780]: ts=2026-09-23T05:12:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245383worker1 # [ 76.722896] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5384worker1 # [ 76.728203] nix-daemon[863]: reaped child process 1199, status = succeeded5385worker1 # [ 76.732626] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5386lb # [ 76.700994] niks3-server[976]: 2026/09/23 05:12:29 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5387lb # [ 76.920567] envoy[912]: conn peer=[2001:db8:1::1]:59926 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=957 ms=59085388lb # [ 76.924303] envoy[912]: conn peer=[2001:db8:1::1]:58552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=772 ms=42005389lb # [ 76.927982] envoy[912]: conn peer=[2001:db8:1::1]:58564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1197 ms=46755390lb # [ 76.931577] envoy[912]: conn peer=[2001:db8:1::1]:37850 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3406 tx=1946 ms=14035391lb # [ 76.935128] envoy[912]: conn peer=[2001:db8:1::1]:37864 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=333 ms=555392worker1 # [ 77.974845] nix-daemon[863]: reaped child process 1208, status = succeeded5393worker1 # [ 78.053308] nix-daemon[863]: accepted connection from pid 1224, user nix-grpc-daemon (trusted)5394worker1 # [ 78.067160] nix-daemon[863]: reaped child process 1227, status = succeeded5395lb # [ 78.030560] niks3-server[976]: 2026/09/23 05:12:31 INFO Received uploads request method=POST path=/api/pending_closures5396worker1 # [ 78.083800] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.187Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5397worker1 # [ 78.086243] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.189Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5398lb # [ 78.068157] niks3-server[976]: 2026/09/23 05:12:31 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5399lb # [ 78.084286] niks3-server[976]: 2026/09/23 05:12:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5400worker1 # [ 78.136437] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.240Z level=INFO msg="Uploading 1 narinfos"5401lb # [ 78.092589] niks3-server[976]: 2026/09/23 05:12:31 INFO Signed narinfos id=24 count=15402lb # [ 78.097994] niks3-server[976]: 2026/09/23 05:12:31 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5403lb # [ 78.102444] niks3-server[976]: 2026/09/23 05:12:31 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5404lb # [ 78.112942] niks3-server[976]: 2026/09/23 05:12:31 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5405lb # [ 78.115524] niks3-server[976]: 2026/09/23 05:12:31 INFO Completed upload id=245406worker1 # [ 78.159962] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.263Z level=INFO msg="Upload complete. (219ms)"5407lb # [ 78.117646] niks3-server[976]: 2026/09/23 05:12:31 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5408worker1 # [ 78.165664] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15409worker1 # [ 78.177594] nix-daemon[863]: reaped child process 1206, status = succeeded5410worker2 # [ 78.197877] nix-grpc-daemon[783]: ts=2026-09-23T05:12:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:474805411worker2 # [ 78.209804] nix-grpc-daemon[783]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5412worker1 # [ 78.251689] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406245413worker1 # [ 78.257421] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_out=1765414worker2 # [ 78.412940] nix-grpc-daemon[783]: ts=2026-09-23T05:12:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:474805415worker1 # [ 78.430154] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245416worker1 # [ 78.449436] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.553Z level=INFO msg="All 2 paths already cached"5417worker1 # [ 78.464188] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05418worker1 # [ 78.469626] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15419worker1 # [ 78.475320] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245420worker1 # [ 78.486872] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245421worker1 # [ 78.634972] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.738Z level=INFO msg="All 2 paths already cached"5422worker1 # [ 78.645178] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406245423worker1 # [ 78.655169] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5424worker1 # [ 78.766180] nix-daemon[863]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)5425worker1 # [ 78.780072] nix-daemon[863]: reaped child process 1236, status = succeeded5426lb # [ 78.743344] niks3-server[976]: 2026/09/23 05:12:31 INFO Received uploads request method=POST path=/api/pending_closures5427worker1 # [ 78.795594] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.899Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5428worker1 # [ 78.797457] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.900Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5429lb # [ 78.774159] niks3-server[976]: 2026/09/23 05:12:31 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5430lb # [ 78.780939] niks3-server[976]: 2026/09/23 05:12:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5431lb # [ 78.786585] niks3-server[976]: 2026/09/23 05:12:31 INFO Signed narinfos id=25 count=15432worker1 # [ 78.835970] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.939Z level=INFO msg="Uploading 1 narinfos"5433lb # [ 78.795845] niks3-server[976]: 2026/09/23 05:12:31 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5434lb # [ 78.811241] niks3-server[976]: 2026/09/23 05:12:31 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5435lb # [ 78.817105] niks3-server[976]: 2026/09/23 05:12:32 INFO Completed upload id=255436worker1 # [ 78.861615] nix-grpc-daemon[878]: time=2026-09-23T05:12:31.965Z level=INFO msg="Upload complete. (193ms)"5437lb # [ 78.823237] niks3-server[976]: 2026/09/23 05:12:32 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5438worker1 # [ 78.866363] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_in=7525439worker1 # [ 78.873280] nix-grpc-daemon[780]: ts=2026-09-23T05:12:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245440worker1 # [ 78.884064] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5441worker1 # [ 78.897210] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5442worker1 # [ 78.899503] nix-daemon[863]: reaped child process 1231, status = succeeded5443worker1 # [ 79.912161] nix-daemon[863]: reaped child process 1240, status = succeeded5444worker1 # [ 79.987739] nix-daemon[863]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5445worker1 # [ 80.000393] nix-daemon[863]: reaped child process 1259, status = succeeded5446lb # [ 79.965145] niks3-server[976]: 2026/09/23 05:12:33 INFO Received uploads request method=POST path=/api/pending_closures5447worker1 # [ 80.016284] nix-grpc-daemon[878]: time=2026-09-23T05:12:33.119Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5448worker1 # [ 80.017949] nix-grpc-daemon[878]: time=2026-09-23T05:12:33.121Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5449lb # [ 80.005277] niks3-server[976]: 2026/09/23 05:12:33 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5450lb # [ 80.015327] niks3-server[976]: 2026/09/23 05:12:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5451worker1 # [ 80.065336] nix-grpc-daemon[878]: time=2026-09-23T05:12:33.168Z level=INFO msg="Uploading 1 narinfos"5452lb # [ 80.022141] niks3-server[976]: 2026/09/23 05:12:33 INFO Signed narinfos id=26 count=15453lb # [ 80.027825] niks3-server[976]: 2026/09/23 05:12:33 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5454lb # [ 80.033651] niks3-server[976]: 2026/09/23 05:12:33 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5455lb # [ 80.037613] niks3-server[976]: 2026/09/23 05:12:33 INFO Completed upload id=265456worker1 # [ 80.082414] nix-grpc-daemon[878]: time=2026-09-23T05:12:33.186Z level=INFO msg="Upload complete. (176ms)"5457lb # [ 80.041719] niks3-server[976]: 2026/09/23 05:12:33 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5458lb # [ 80.044930] niks3-server[976]: 2026/09/23 05:12:33 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5459worker1 # [ 80.090070] nix-grpc-daemon[780]: ts=2026-09-23T05:12:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15460worker1 # [ 80.099372] nix-grpc-daemon[780]: ts=2026-09-23T05:12:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406245461worker1 # [ 80.107058] nix-daemon[863]: reaped child process 1238, status = succeeded5462worker1 # [ 80.120230] nix-grpc-daemon[780]: ts=2026-09-23T05:12:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406245463worker1 # [ 80.122496] nix-grpc-daemon[780]: ts=2026-09-23T05:12:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_out=2325464client # [ 80.198310] nix-daemon[1108]: reaped child process 1110, status = succeeded5465client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.66 seconds)5466client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5467client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5468client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5469client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5470client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5471client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5472(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.73 seconds)5473subtest: an input only one worker has still reaches the builder5474client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5475client # this derivation will be built:5476client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5477client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5478client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds)5479client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5480client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5481worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5482worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5483worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5484worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5485client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25486client # [ 80.865694] nix-daemon[1108]: accepted connection from pid 1191, user root (trusted)5487client # this derivation will be built:5488client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5489worker1 # [ 81.017682] nix-grpc-daemon[780]: ts=2026-09-23T05:12:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:406245490client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5491worker2 # [ 81.007532] nix-grpc-daemon[783]: ts=2026-09-23T05:12:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:474805492client # copying 1 paths...5493client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5494worker1 # [ 81.033825] nix-grpc-daemon[780]: ts=2026-09-23T05:12:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406245495worker1 # [ 81.044991] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5496worker1 # [ 81.152206] nix-daemon[863]: accepted connection from pid 1268, user nix-grpc-daemon (trusted)5497worker1 # [ 81.165657] nix-daemon[863]: reaped child process 1271, status = succeeded5498lb # [ 81.127929] niks3-server[976]: 2026/09/23 05:12:34 INFO Received uploads request method=POST path=/api/pending_closures5499worker1 # [ 81.180750] nix-grpc-daemon[878]: time=2026-09-23T05:12:34.283Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5500worker1 # [ 81.182229] nix-grpc-daemon[878]: time=2026-09-23T05:12:34.283Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5501lb # [ 81.158798] niks3-server[976]: 2026/09/23 05:12:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5502lb # [ 81.164658] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5503lb # [ 81.170592] niks3-server[976]: 2026/09/23 05:12:34 INFO Signed narinfos id=27 count=15504worker1 # [ 81.218065] nix-grpc-daemon[878]: time=2026-09-23T05:12:34.321Z level=INFO msg="Uploading 1 narinfos"5505lb # [ 81.181152] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5506lb # [ 81.192649] niks3-server[976]: 2026/09/23 05:12:34 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5507lb # [ 81.198197] niks3-server[976]: 2026/09/23 05:12:34 INFO Completed upload id=275508worker1 # [ 81.242202] nix-grpc-daemon[878]: time=2026-09-23T05:12:34.345Z level=INFO msg="Upload complete. (187ms)"5509lb # [ 81.202212] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5510worker1 # [ 81.247603] nix-grpc-daemon[780]: ts=2026-09-23T05:12:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_in=1205511worker1 # [ 81.261322] nix-daemon[863]: reaped child process 1266, status = succeeded5512worker1 # [ 81.264201] nix-grpc-daemon[780]: ts=2026-09-23T05:12:34Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05513worker1 # [ 81.266261] nix-grpc-daemon[780]: ts=2026-09-23T05:12:34Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15514worker2 # [ 81.254522] nix-grpc-daemon[783]: ts=2026-09-23T05:12:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474805515worker2 # [ 81.265384] nix-grpc-daemon[783]: ts=2026-09-23T05:12:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:474805516client # copying 2 paths to 'lb:50051'...5517client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5518worker2 # [ 81.281277] nix-grpc-daemon[783]: ts=2026-09-23T05:12:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:474805519client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5520worker2 # [ 81.295556] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5521worker2 # [ 81.406046] nix-daemon[852]: accepted connection from pid 1088, user nix-grpc-daemon (trusted)5522worker2 # [ 81.422874] nix-daemon[852]: reaped child process 1091, status = succeeded5523lb # [ 81.397901] niks3-server[976]: 2026/09/23 05:12:34 INFO Received uploads request method=POST path=/api/pending_closures5524lb # [ 81.406380] niks3-server[976]: 2026/09/23 05:12:34 INFO Received uploads request method=POST path=/api/pending_closures5525worker2 # [ 81.446717] nix-grpc-daemon[880]: time=2026-09-23T05:12:34.312Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5526worker2 # [ 81.451160] nix-grpc-daemon[880]: time=2026-09-23T05:12:34.314Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5527worker2 # [ 81.453227] nix-grpc-daemon[880]: time=2026-09-23T05:12:34.314Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5528lb # [ 81.450606] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5529lb # [ 81.455084] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5530lb # [ 81.463281] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5531lb # [ 81.472379] niks3-server[976]: 2026/09/23 05:12:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5532lb # [ 81.474496] niks3-server[976]: 2026/09/23 05:12:34 INFO Signed narinfos id=28 count=15533lb # [ 81.476637] niks3-server[976]: 2026/09/23 05:12:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5534lb # [ 81.479058] niks3-server[976]: 2026/09/23 05:12:34 INFO Signed narinfos id=29 count=25535worker2 # [ 81.510100] nix-grpc-daemon[880]: time=2026-09-23T05:12:34.375Z level=INFO msg="Uploading 3 narinfos"5536lb # [ 81.482491] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5537lb # [ 81.493707] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5538lb # [ 81.497205] niks3-server[976]: 2026/09/23 05:12:34 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5539lb # [ 81.504087] niks3-server[976]: 2026/09/23 05:12:34 INFO Completed upload id=295540lb # [ 81.505454] niks3-server[976]: 2026/09/23 05:12:34 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5541lb # [ 81.507862] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5542lb # [ 81.510492] niks3-server[976]: 2026/09/23 05:12:34 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5543lb # [ 81.512551] niks3-server[976]: 2026/09/23 05:12:34 INFO Completed upload id=285544worker2 # [ 81.544497] nix-grpc-daemon[880]: time=2026-09-23T05:12:34.410Z level=INFO msg="Upload complete. (235ms)"5545worker2 # [ 81.549931] nix-grpc-daemon[783]: ts=2026-09-23T05:12:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=2 nar_bytes_in=9765546worker2 # [ 81.555694] nix-grpc-daemon[783]: ts=2026-09-23T05:12:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474805547worker2 # [ 81.567366] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5548worker2 # [ 81.580720] nix-daemon[852]: reaped child process 1086, status = succeeded5549worker2 # [ 81.589958] nix-grpc-daemon[783]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5550client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5551worker2 # [ 81.604147] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5552client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5553client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5554worker2 # [ 82.636641] nix-daemon[852]: reaped child process 1096, status = succeeded5555worker2 # [ 82.731038] nix-daemon[852]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)5556worker2 # [ 82.744053] nix-daemon[852]: reaped child process 1115, status = succeeded5557lb # [ 82.720727] niks3-server[976]: 2026/09/23 05:12:35 INFO Received uploads request method=POST path=/api/pending_closures5558worker2 # [ 82.758925] nix-grpc-daemon[880]: time=2026-09-23T05:12:35.624Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5559lb # [ 82.750396] niks3-server[976]: 2026/09/23 05:12:35 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5560lb # [ 82.758302] niks3-server[976]: 2026/09/23 05:12:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5561worker2 # [ 82.797868] nix-grpc-daemon[880]: time=2026-09-23T05:12:35.663Z level=INFO msg="Uploading 1 narinfos"5562lb # [ 82.767698] niks3-server[976]: 2026/09/23 05:12:35 INFO Signed narinfos id=30 count=15563lb # [ 82.773843] niks3-server[976]: 2026/09/23 05:12:35 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5564lb # [ 82.786510] niks3-server[976]: 2026/09/23 05:12:35 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5565lb # [ 82.792146] niks3-server[976]: 2026/09/23 05:12:35 INFO Completed upload id=305566worker2 # [ 82.824814] nix-grpc-daemon[880]: time=2026-09-23T05:12:35.690Z level=INFO msg="Upload complete. (211ms)"5567lb # [ 82.798105] niks3-server[976]: 2026/09/23 05:12:35 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5568worker2 # [ 82.829710] nix-grpc-daemon[783]: ts=2026-09-23T05:12:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15569client # copying 1 paths...5570worker2 # [ 82.842589] nix-daemon[852]: reaped child process 1093, status = succeeded5571worker1 # [ 82.854544] nix-grpc-daemon[780]: ts=2026-09-23T05:12:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406245572worker1 # [ 82.863582] nix-grpc-daemon[780]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5573client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5574worker1 # [ 82.888885] nix-grpc-daemon[780]: ts=2026-09-23T05:12:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406245575worker1 # [ 82.894254] nix-grpc-daemon[780]: ts=2026-09-23T05:12:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_out=1205576client # [ 82.971834] nix-daemon[1108]: reaped child process 1194, status = succeeded5577client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.23 seconds)5578client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25579client # [ 83.073093] nix-daemon[1108]: accepted connection from pid 1212, user root (trusted)5580client # this derivation will be built:5581client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5582worker2 # [ 83.210190] nix-grpc-daemon[783]: ts=2026-09-23T05:12:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:474805583client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5584worker2 # [ 83.215967] nix-grpc-daemon[783]: ts=2026-09-23T05:12:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:474805585worker2 # [ 83.220893] nix-grpc-daemon[880]: time=2026-09-23T05:12:36.086Z level=INFO msg="All 1 paths already cached"5586worker1 # [ 83.245136] nix-grpc-daemon[780]: ts=2026-09-23T05:12:36Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05587worker1 # [ 83.252118] nix-grpc-daemon[780]: ts=2026-09-23T05:12:36Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15588worker1 # [ 83.258157] nix-grpc-daemon[780]: ts=2026-09-23T05:12:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245589worker1 # [ 83.269740] nix-grpc-daemon[780]: ts=2026-09-23T05:12:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245590client # copying 1 paths to 'lb:50051'...5591client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5592worker1 # [ 83.287835] nix-grpc-daemon[780]: ts=2026-09-23T05:12:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406245593worker1 # [ 83.299091] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5594worker1 # [ 83.315459] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)5595worker1 # [ 83.383194] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 174 ms (attempt 2/5)5596worker1 # [ 83.565065] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 3/5)5597worker1 # [ 83.714119] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 294 ms (attempt 4/5)5598worker1 # [ 84.011868] nix-grpc-daemon[780]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5599worker1 # [ 84.015509] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5600worker1 # [ 84.020686] nix-grpc-daemon[780]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5601worker1 # [ 84.134159] nix-daemon[863]: accepted connection from pid 1285, user nix-grpc-daemon (trusted)5602worker1 # [ 84.151051] nix-daemon[863]: reaped child process 1288, status = succeeded5603lb # [ 84.114211] niks3-server[976]: 2026/09/23 05:12:37 INFO Received uploads request method=POST path=/api/pending_closures5604worker1 # [ 84.166713] nix-grpc-daemon[878]: time=2026-09-23T05:12:37.269Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5605worker1 # [ 84.168182] nix-grpc-daemon[878]: time=2026-09-23T05:12:37.269Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5606lb # [ 84.141734] niks3-server[976]: 2026/09/23 05:12:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5607lb # [ 84.149439] niks3-server[976]: 2026/09/23 05:12:37 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5608lb # [ 84.155121] niks3-server[976]: 2026/09/23 05:12:37 INFO Signed narinfos id=31 count=15609worker1 # [ 84.203334] nix-grpc-daemon[878]: time=2026-09-23T05:12:37.306Z level=INFO msg="Uploading 1 narinfos"5610lb # [ 84.163858] niks3-server[976]: 2026/09/23 05:12:37 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5611lb # [ 84.177328] niks3-server[976]: 2026/09/23 05:12:37 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5612lb # [ 84.183292] niks3-server[976]: 2026/09/23 05:12:37 INFO Completed upload id=315613worker1 # [ 84.230288] nix-grpc-daemon[878]: time=2026-09-23T05:12:37.333Z level=INFO msg="Upload complete. (191ms)"5614lb # [ 84.186923] niks3-server[976]: 2026/09/23 05:12:37 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5615worker1 # [ 84.234807] nix-grpc-daemon[780]: ts=2026-09-23T05:12:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_in=5605616worker1 # [ 84.241815] nix-grpc-daemon[780]: ts=2026-09-23T05:12:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406245617worker1 # [ 84.253194] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5618worker1 # [ 84.264172] nix-daemon[863]: reaped child process 1278, status = succeeded5619worker1 # [ 84.267157] nix-daemon[863]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5620client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5621client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5622client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5623worker1 # [ 85.101794] nix-daemon[863]: reaped child process 1292, status = succeeded5624worker1 # [ 85.190380] nix-daemon[863]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5625worker1 # [ 85.202284] nix-daemon[863]: reaped child process 1311, status = succeeded5626lb # [ 85.166459] niks3-server[976]: 2026/09/23 05:12:38 INFO Received uploads request method=POST path=/api/pending_closures5627worker1 # [ 85.218924] nix-grpc-daemon[878]: time=2026-09-23T05:12:38.322Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5628lb # [ 85.195362] niks3-server[976]: 2026/09/23 05:12:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5629lb # [ 85.203312] niks3-server[976]: 2026/09/23 05:12:38 INFO Signed narinfos id=32 count=15630worker1 # [ 85.251194] nix-grpc-daemon[878]: time=2026-09-23T05:12:38.354Z level=INFO msg="Uploading 1 narinfos"5631lb # [ 85.211279] niks3-server[976]: 2026/09/23 05:12:38 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5632lb # [ 85.225143] niks3-server[976]: 2026/09/23 05:12:38 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5633lb # [ 85.228530] niks3-server[976]: 2026/09/23 05:12:38 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5634lb # [ 85.236263] niks3-server[976]: 2026/09/23 05:12:38 INFO Completed upload id=325635worker1 # [ 85.280540] nix-grpc-daemon[878]: time=2026-09-23T05:12:38.384Z level=INFO msg="Upload complete. (203ms)"5636lb # [ 85.239927] niks3-server[976]: 2026/09/23 05:12:38 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5637worker1 # [ 85.285202] nix-grpc-daemon[780]: ts=2026-09-23T05:12:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15638client # copying 1 paths...5639worker1 # [ 85.296507] nix-grpc-daemon[780]: ts=2026-09-23T05:12:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406245640client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5641worker1 # [ 85.303970] nix-daemon[863]: reaped child process 1290, status = succeeded5642worker1 # [ 85.317048] nix-grpc-daemon[780]: ts=2026-09-23T05:12:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406245643worker1 # [ 85.319146] nix-grpc-daemon[780]: ts=2026-09-23T05:12:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40624 duration_s=0 paths=1 nar_bytes_out=1205644client # [ 85.393266] nix-daemon[1108]: reaped child process 1215, status = succeeded5645client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.42 seconds)5646client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25647client # [ 85.494564] nix-daemon[1108]: accepted connection from pid 1234, user root (trusted)5648client # this derivation will be built:5649client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5650worker1 # [ 85.644411] nix-grpc-daemon[780]: ts=2026-09-23T05:12:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:406245651client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5652worker1 # [ 85.650906] nix-grpc-daemon[780]: ts=2026-09-23T05:12:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406245653worker1 # [ 85.655531] nix-grpc-daemon[878]: time=2026-09-23T05:12:38.759Z level=INFO msg="All 1 paths already cached"5654worker1 # [ 85.669482] nix-grpc-daemon[780]: ts=2026-09-23T05:12:38Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05655worker2 # [ 85.659819] nix-grpc-daemon[783]: ts=2026-09-23T05:12:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474805656worker1 # [ 85.674440] nix-grpc-daemon[780]: ts=2026-09-23T05:12:38Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15657worker2 # [ 85.670732] nix-grpc-daemon[783]: ts=2026-09-23T05:12:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:474805658worker2 # [ 85.683846] nix-grpc-daemon[880]: time=2026-09-23T05:12:38.549Z level=INFO msg="All 1 paths already cached"5659client # copying 1 paths to 'lb:50051'...5660client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5661worker2 # [ 85.691755] nix-grpc-daemon[783]: ts=2026-09-23T05:12:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:474805662worker2 # [ 85.702814] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5663worker2 # [ 85.799854] nix-daemon[852]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5664worker2 # [ 85.813831] nix-daemon[852]: reaped child process 1122, status = succeeded5665lb # [ 85.789086] niks3-server[976]: 2026/09/23 05:12:38 INFO Received uploads request method=POST path=/api/pending_closures5666worker2 # [ 85.843474] nix-grpc-daemon[880]: time=2026-09-23T05:12:38.709Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5667worker2 # [ 85.844897] nix-grpc-daemon[880]: time=2026-09-23T05:12:38.709Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5668lb # [ 85.833504] niks3-server[976]: 2026/09/23 05:12:39 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5669lb # [ 85.840513] niks3-server[976]: 2026/09/23 05:12:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5670lb # [ 85.850434] niks3-server[976]: 2026/09/23 05:12:39 INFO Signed narinfos id=33 count=15671worker2 # [ 85.880594] nix-grpc-daemon[880]: time=2026-09-23T05:12:38.746Z level=INFO msg="Uploading 1 narinfos"5672lb # [ 85.854960] niks3-server[976]: 2026/09/23 05:12:39 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5673lb # [ 85.866588] niks3-server[976]: 2026/09/23 05:12:39 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5674lb # [ 85.870054] niks3-server[976]: 2026/09/23 05:12:39 INFO Completed upload id=335675lb # [ 85.871994] niks3-server[976]: 2026/09/23 05:12:39 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5676worker2 # [ 85.901903] nix-grpc-daemon[880]: time=2026-09-23T05:12:38.767Z level=INFO msg="Upload complete. (187ms)"5677worker2 # [ 85.906835] nix-grpc-daemon[783]: ts=2026-09-23T05:12:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=1 nar_bytes_in=5605678worker2 # [ 85.914163] nix-grpc-daemon[783]: ts=2026-09-23T05:12:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:474805679worker2 # [ 85.926424] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5680worker2 # [ 85.932975] nix-daemon[852]: reaped child process 1117, status = succeeded5681worker2 # [ 85.941458] nix-daemon[852]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5682client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5683client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5684lb # [ 86.921590] envoy[912]: conn peer=[2001:db8:1::1]:37876 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3863 tx=2140 ms=18405685lb # [ 86.925551] envoy[912]: conn peer=[2001:db8:1::1]:37888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=195686lb # [ 86.929068] envoy[912]: conn peer=[2001:db8:1::1]:37894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=585687lb # [ 86.932639] envoy[912]: conn peer=[2001:db8:1::1]:37890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4315 tx=2385 ms=17485688lb # [ 86.936204] envoy[912]: conn peer=[2001:db8:1::1]:37986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=555689lb # [ 86.939653] envoy[912]: conn peer=[2001:db8:1::1]:37896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4228 tx=2032 ms=19295690lb # [ 86.943195] envoy[912]: conn peer=[2001:db8:1::1]:37998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545691lb # [ 86.947216] envoy[912]: conn peer=[2001:db8:1::1]:37996 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3754 tx=2120 ms=21475692client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5693worker2 # [ 87.386667] nix-daemon[852]: reaped child process 1126, status = succeeded5694worker2 # [ 87.463935] nix-daemon[852]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)5695worker2 # [ 87.478936] nix-daemon[852]: reaped child process 1145, status = succeeded5696lb # [ 87.453937] niks3-server[976]: 2026/09/23 05:12:40 INFO Received uploads request method=POST path=/api/pending_closures5697worker2 # [ 87.493206] nix-grpc-daemon[880]: time=2026-09-23T05:12:40.358Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5698lb # [ 87.484521] niks3-server[976]: 2026/09/23 05:12:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5699lb # [ 87.493499] niks3-server[976]: 2026/09/23 05:12:40 INFO Signed narinfos id=34 count=15700worker2 # [ 87.526966] nix-grpc-daemon[880]: time=2026-09-23T05:12:40.392Z level=INFO msg="Uploading 1 narinfos"5701lb # [ 87.499393] niks3-server[976]: 2026/09/23 05:12:40 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5702lb # [ 87.504636] niks3-server[976]: 2026/09/23 05:12:40 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5703lb # [ 87.517846] niks3-server[976]: 2026/09/23 05:12:40 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5704lb # [ 87.524429] niks3-server[976]: 2026/09/23 05:12:40 INFO Completed upload id=345705worker2 # [ 87.555083] nix-grpc-daemon[880]: time=2026-09-23T05:12:40.420Z level=INFO msg="Upload complete. (205ms)"5706lb # [ 87.528215] niks3-server[976]: 2026/09/23 05:12:40 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5707worker2 # [ 87.560620] nix-grpc-daemon[783]: ts=2026-09-23T05:12:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15708client # copying 1 paths...5709worker2 # [ 87.573613] nix-grpc-daemon[783]: ts=2026-09-23T05:12:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:474805710client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5711worker2 # [ 87.579985] nix-daemon[852]: reaped child process 1124, status = succeeded5712worker2 # [ 87.593342] nix-grpc-daemon[783]: ts=2026-09-23T05:12:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:474805713worker2 # [ 87.597321] nix-grpc-daemon[783]: ts=2026-09-23T05:12:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47480 duration_s=0 paths=1 nar_bytes_out=1205714client # [ 87.685820] nix-daemon[1108]: reaped child process 1237, status = succeeded5715client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.29 seconds)5716(finished: subtest: an input only one worker has still reaches the builder, in 7.42 seconds)5717subtest: upload whose reference the worker lacks is completed from the cache5718client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5719client # this derivation will be built:5720client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5721client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5722client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.37 seconds)5723client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5724client # this derivation will be built:5725client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5726client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5727client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.25 seconds)5728client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5729client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5730worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5731worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5732worker2 # [ 88.604477] systemd[1]: Stopping Nix worker-protocol over gRPC...5733worker2 # [ 89.164104] nix-grpc-daemon[783]: ts=2026-09-23T05:12:42Z level=info event=signal_exit5734worker1 # [ 90.700383] nix-grpc-daemon[780]: ts=2026-09-23T05:12:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:402305735worker1 # [ 90.704462] nix-grpc-daemon[780]: ts=2026-09-23T05:12:43Z level=info event=worker_gone addr=192.168.1.4:500515736worker2 # [ 90.693332] nix-grpc-daemon[783]: ts=2026-09-23T05:12:43Z level=info event=scheduler_restarting5737worker2 # [ 90.702772] nix-grpc-daemon[783]: ts=2026-09-23T05:12:43Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5738worker2 # [ 90.724476] nix-daemon[852]: reaped child process 854, status = succeeded5739worker2 # [ 91.013165] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5740worker2 # [ 91.016242] systemd[1]: Stopped Nix worker-protocol over gRPC.5741worker2 # [ 91.020465] systemd[1]: nix-grpc-daemon.service: Consumed 1.828s CPU time over 1min 14.004s wall clock time, 60.1M memory peak, 79.3K incoming IP traffic, 190.1K outgoing IP traffic.5742worker2 # [ 91.031957] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5743worker2 # [ 91.036705] systemd[1]: Closed Nix worker-protocol over gRPC.5744worker2 # [ 91.040313] systemd[1]: nix-grpc-daemon.socket: Consumed 335us CPU time over 1min 22.121s wall clock time, 256K memory peak, 83.8K incoming IP traffic, 74.7K outgoing IP traffic.5745lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5746lb: must succeed: curl -sf localhost:9901/clusters5747lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5748lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5749client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25750worker1 # [ 91.260249] nix-grpc-daemon[780]: ts=2026-09-23T05:12:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406245751worker1 # [ 91.265159] nix-grpc-daemon[780]: ts=2026-09-23T05:12:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406245752client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5753client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5754worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5755worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5756worker2 # [ 91.341253] systemd[1]: Listening on Nix worker-protocol over gRPC.5757worker1 # [ 91.398972] systemd[1]: Stopping Nix worker-protocol over gRPC...5758worker1 # [ 92.124704] nix-grpc-daemon[780]: ts=2026-09-23T05:12:45Z level=info event=signal_exit5759worker2 # [ 92.297261] systemd[1]: Starting scheduler-token.service...5760worker2 # [ 92.357245] systemd[1]: scheduler-token.service: Deactivated successfully.5761worker2 # [ 92.360408] systemd[1]: Finished scheduler-token.service.5762worker2 # [ 92.362769] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 63ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5763worker2 # [ 92.371660] systemd[1]: Starting Nix worker-protocol over gRPC...5764worker2 # [ 92.452233] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5765worker2 # [ 92.454280] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5766worker2 # [ 92.460935] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15767worker2 # [ 92.462887] nix-grpc-daemon[1169]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5768worker2 # [ 92.464255] nix-grpc-daemon[1169]: E0000 00:00:1790140365.326061 1169 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5769worker2 # [ 92.472172] nix-grpc-daemon[1169]: E0000 00:00:1790140365.333108 1169 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5770worker2 # [ 92.475992] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5771worker2 # [ 92.480147] systemd[1]: Started Nix worker-protocol over gRPC.5772worker2 # [ 92.482620] nix-grpc-daemon[1169]: E0000 00:00:1790140365.345429 1211 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5773worker2 # [ 92.494535] nix-grpc-daemon[1169]: E0000 00:00:1790140365.360480 1211 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5774worker2 # [ 92.498205] nix-grpc-daemon[1169]: E0000 00:00:1790140365.362618 1211 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5775worker2 # [ 92.508882] nix-grpc-daemon[1169]: E0000 00:00:1790140365.374848 1211 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5776worker2 # [ 92.518093] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=scheduler_connected addr=lb:500515777worker2 # [ 92.519691] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5778lb # [ 92.647846] niks3-server[976]: 2026/09/23 05:12:45 INFO lead: released remote=192.168.1.3:538305779worker2 # [ 92.721280] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=scheduler_connected addr=lb:500515780worker2 # [ 92.725280] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5781worker2 # [ 92.926143] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=scheduler_connected addr=lb:500515782worker2 # [ 92.930215] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5783worker2 # [ 93.131297] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:45Z level=info event=scheduler_connected addr=lb:500515784worker2 # [ 93.135263] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5785worker2 # [ 93.336259] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:46Z level=info event=scheduler_connected addr=lb:500515786worker2 # [ 93.340183] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5787worker2 # [ 93.542579] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:46Z level=info event=scheduler_connected addr=lb:500515788worker2 # [ 93.545827] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5789worker1 # [ 93.696651] nix-grpc-daemon[780]: ts=2026-09-23T05:12:46Z level=info event=scheduler_restarting5790worker1 # [ 93.700366] nix-grpc-daemon[780]: ts=2026-09-23T05:12:46Z level=info event=scheduler_restarting addr=lb:500515791worker1 # [ 93.703986] nix-grpc-daemon[780]: ts=2026-09-23T05:12:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:402305792worker1 # [ 93.707681] nix-grpc-daemon[780]: ts=2026-09-23T05:12:46Z level=info event=worker_gone addr=192.168.1.3:500515793worker1 # [ 93.729811] nix-daemon[863]: reaped child process 865, status = succeeded5794worker2 # [ 93.743789] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:46Z level=info event=scheduler_connected addr=lb:500515795worker2 # [ 93.747743] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5796worker1 # [ 93.909488] nix-grpc-daemon[780]: ts=2026-09-23T05:12:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5797worker2 # [ 93.948758] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:46Z level=info event=scheduler_connected addr=lb:500515798worker2 # [ 93.952810] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5799worker2 # [ 94.153837] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_connected addr=lb:500515800worker2 # [ 94.157882] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5801worker1 # [ 94.276606] nix-grpc-daemon[780]: E0000 00:00:1790140367.380059 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5802worker1 # [ 94.281549] nix-grpc-daemon[780]: E0000 00:00:1790140367.380266 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5803worker2 # [ 94.359076] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_connected addr=lb:500515804worker2 # [ 94.363302] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5805worker2 # [ 94.565520] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_connected addr=lb:500515806worker2 # [ 94.568843] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5807worker2 # [ 94.766911] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_connected addr=lb:500515808worker2 # [ 94.770909] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5809worker2 # [ 94.971971] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_connected addr=lb:500515810worker1 # [ 94.990402] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5811worker2 # [ 94.975957] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5812worker1 # [ 94.994298] systemd[1]: Stopped Nix worker-protocol over gRPC.5813worker1 # [ 94.998830] systemd[1]: nix-grpc-daemon.service: Consumed 2.596s CPU time over 1min 18.033s wall clock time, 69M memory peak, 116K incoming IP traffic, 249.8K outgoing IP traffic.5814worker1 # [ 95.010781] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5815worker1 # [ 95.015668] systemd[1]: Closed Nix worker-protocol over gRPC.5816worker1 # [ 95.019208] systemd[1]: nix-grpc-daemon.socket: Consumed 312us CPU time over 1min 27.342s wall clock time, 256K memory peak, 122.9K incoming IP traffic, 104.9K outgoing IP traffic.5817lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5818lb: must succeed: curl -sf localhost:9901/clusters5819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5820lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5821client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25822worker2 # [ 95.177202] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_connected addr=lb:500515823worker2 # [ 95.181140] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5824worker2 # [ 95.207132] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368645825worker2 # [ 95.219386] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5826worker2 # [ 95.235575] nix-grpc-daemon[1169]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 1/5)5827worker2 # [ 95.291195] nix-grpc-daemon[1169]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 2/5)5828worker2 # [ 95.382293] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_connected addr=lb:500515829worker2 # [ 95.386323] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5830worker2 # [ 95.418187] nix-grpc-daemon[1169]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 3/5)5831worker2 # [ 95.470338] nix-grpc-daemon[1169]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 685 ms (attempt 4/5)5832worker2 # [ 95.588093] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_connected addr=lb:500515833worker2 # [ 95.591839] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5834worker2 # [ 95.793227] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_connected addr=lb:500515835worker2 # [ 95.797126] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5836worker2 # [ 95.998158] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_connected addr=lb:500515837worker2 # [ 96.001986] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5838worker2 # [ 96.163727] nix-grpc-daemon[1169]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5839worker2 # [ 96.167461] nix-grpc-daemon[1169]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5840client # copying 2 paths...5841client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5842worker2 # [ 96.190160] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368645843client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5844worker2 # [ 96.200265] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5845worker2 # [ 96.205652] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=scheduler_connected addr=lb:500515846worker2 # [ 96.209086] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5847worker2 # [ 96.235787] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=niks3_push_started pid=12305848worker2 # [ 96.332727] nix-grpc-daemon[1230]: time=2026-09-23T05:12:49.198Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5849worker2 # [ 96.408296] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=scheduler_connected addr=lb:500515850worker2 # [ 96.412173] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5851worker2 # [ 96.425943] nix-daemon[852]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5852worker2 # [ 96.438531] nix-daemon[852]: reaped child process 1246, status = succeeded5853lb # [ 96.419927] niks3-server[976]: 2026/09/23 05:12:49 INFO Received uploads request method=POST path=/api/pending_closures5854lb # [ 96.429517] niks3-server[976]: 2026/09/23 05:12:49 INFO Received uploads request method=POST path=/api/pending_closures5855worker2 # [ 96.468054] nix-grpc-daemon[1230]: time=2026-09-23T05:12:49.334Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5856worker2 # [ 96.469660] nix-grpc-daemon[1230]: time=2026-09-23T05:12:49.335Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5857worker2 # [ 96.473177] nix-grpc-daemon[1230]: time=2026-09-23T05:12:49.337Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5858lb # [ 96.477263] niks3-server[976]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5859lb # [ 96.484652] niks3-server[976]: 2026/09/23 05:12:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5860lb # [ 96.487979] niks3-server[976]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5861lb # [ 96.490691] niks3-server[976]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5862lb # [ 96.492727] niks3-server[976]: 2026/09/23 05:12:49 INFO Signed narinfos id=36 count=25863lb # [ 96.495575] niks3-server[976]: 2026/09/23 05:12:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5864worker2 # [ 96.526920] nix-grpc-daemon[1230]: time=2026-09-23T05:12:49.391Z level=INFO msg="Uploading 3 narinfos"5865lb # [ 96.497204] niks3-server[976]: 2026/09/23 05:12:49 INFO Signed narinfos id=35 count=15866lb # [ 96.506170] niks3-server[976]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5867lb # [ 96.513809] niks3-server[976]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5868lb # [ 96.515428] niks3-server[976]: 2026/09/23 05:12:49 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5869lb # [ 96.519342] niks3-server[976]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5870lb # [ 96.521752] niks3-server[976]: 2026/09/23 05:12:49 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5871lb # [ 96.525093] niks3-server[976]: 2026/09/23 05:12:49 INFO Completed upload id=355872lb # [ 96.526820] niks3-server[976]: 2026/09/23 05:12:49 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5873lb # [ 96.550622] niks3-server[976]: 2026/09/23 05:12:49 INFO Completed upload id=365874worker2 # [ 96.581347] nix-grpc-daemon[1230]: time=2026-09-23T05:12:49.447Z level=INFO msg="Upload complete. (245ms)"5875worker2 # [ 96.587831] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36864 duration_s=0 paths=2 nar_bytes_in=3125876worker2 # [ 96.602084] nix-daemon[852]: reaped child process 1228, status = succeeded5877worker2 # [ 96.611104] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=scheduler_connected addr=lb:500515878client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.55 seconds)5879worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5880worker2 # [ 96.615190] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5881worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5882worker1 # [ 96.678372] systemd[1]: Listening on Nix worker-protocol over gRPC.5883worker1 # [ 96.682082] systemd[1]: Starting Nix worker-protocol over gRPC...5884worker1 # [ 96.766243] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5885worker1 # [ 96.768249] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5886worker1 # [ 96.771582] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15887worker1 # [ 96.775899] nix-grpc-daemon[1336]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5888worker1 # [ 96.777264] nix-grpc-daemon[1336]: E0000 00:00:1790140369.879287 1336 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5889worker1 # [ 96.782737] nix-grpc-daemon[1336]: E0000 00:00:1790140369.884853 1336 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5890worker1 # [ 96.787287] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5891worker1 # [ 96.790942] systemd[1]: Started Nix worker-protocol over gRPC.5892lb # [ 96.748878] niks3-server[976]: 2026/09/23 05:12:49 INFO lead: acquired remote=192.168.1.3:393625893worker1 # [ 96.795100] nix-grpc-daemon[1336]: E0000 00:00:1790140369.896285 1378 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5894worker1 # [ 96.800065] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:49Z level=info event=scheduler_take_over5895worker1 # [ 96.807429] nix-grpc-daemon[1336]: E0000 00:00:1790140369.911179 1378 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5896worker1 # [ 96.812132] nix-grpc-daemon[1336]: E0000 00:00:1790140369.913462 1378 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5897lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5898lb: must succeed: curl -sf localhost:9901/clusters5899worker1 # [ 96.822059] nix-grpc-daemon[1336]: E0000 00:00:1790140369.925780 1378 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5900worker2 # [ 96.816112] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=scheduler_connected addr=lb:500515901worker1 # [ 96.836417] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:49Z level=info event=scheduler_connected addr=lb:500515902worker1 # [ 96.837869] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5903worker2 # [ 96.821692] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5904lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5905lb # [ 96.922248] envoy[912]: conn peer=[2001:db8:1::1]:38018 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=565906lb # [ 96.925982] envoy[912]: conn peer=[2001:db8:1::1]:38012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=2084 ms=20205907lb # [ 96.929759] envoy[912]: conn peer=[2001:db8:1::4]:34802 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=59549 tx=3182 ms=735025908lb # [ 96.933383] envoy[912]: conn peer=[2001:db8:1::1]:38024 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215909lb # [ 96.937601] envoy[912]: conn peer=[2001:db8:1::3]:60106 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=16303 tx=3455 ms=767435910lb # [ 96.941818] envoy[912]: conn peer=[2001:db8:1::1]:51238 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=282 ms=13915911worker2 # [ 97.023120] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:49Z level=info event=scheduler_connected addr=lb:500515912worker1 # [ 97.039272] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:50Z level=info event=scheduler_connected addr=lb:500515913worker1 # [ 97.046709] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:50Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:529485914worker1 # [ 97.051250] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:50Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05915worker1 # [ 97.060983] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:50Z level=info event=oidc_jwks provider=mock keys=15916worker1 # [ 97.063973] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:50Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:529485917worker1 # [ 97.068278] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:50Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05918lb: must succeed: curl -sf localhost:9901/clusters5919lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5920lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.09 seconds)5921(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.18 seconds)5922subtest: a niks3 restart does not cost the leader its role5923lb: must succeed: curl -sf localhost:9901/clusters5924lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5925lb: must succeed: curl -sf localhost:9901/clusters5926lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5927lb # [ 97.928373] niks3-server[976]: 2026/09/23 05:12:51 INFO Shutdown signal received, draining in-flight requests timeout=10s5928lb # [ 97.931363] systemd[1]: Stopping niks3 server...5929lb # [ 97.933798] niks3-server[976]: 2026/09/23 05:12:51 INFO lead: released remote=192.168.1.3:393625930worker2 # [ 97.965161] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:50Z level=info event=scheduler_lock_lost status=2005931worker1 # [ 97.979655] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:51Z level=info event=scheduler_lock_lost status=2005932lb # [ 97.944483] systemd[1]: niks3.service: Deactivated successfully.5933lb # [ 97.946069] systemd[1]: Stopped niks3 server.5934lb # [ 97.947150] systemd[1]: niks3.service: Consumed 774ms CPU time over 1min 17.520s wall clock time, 28.6M memory peak, 95.6K incoming IP traffic, 137.8K outgoing IP traffic.5935lb # [ 97.953892] systemd[1]: Starting niks3 server...5936lb # [ 98.002953] niks3-server[1116]: 2026/09/23 05:12:51 goose: no migrations to run. current version: 202609200000005937lb # [ 98.009033] niks3-server[1116]: 2026/09/23 05:12:51 OK 1_commit_pending_closure.sql (4.2ms)5938lb # [ 98.012988] niks3-server[1116]: 2026/09/23 05:12:51 OK 2_object_stats_trigger.sql (2.16ms)5939lb # [ 98.014135] niks3-server[1116]: 2026/09/23 05:12:51 goose: up to current file version: 25940lb # [ 98.015473] niks3-server[1116]: 2026/09/23 05:12:51 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5941lb # [ 98.026634] niks3-server[1116]: 2026/09/23 05:12:51 INFO Read proxy enabled — serving cache objects from S35942lb # [ 98.027923] niks3-server[1116]: 2026/09/23 05:12:51 INFO Using socket-activated listener address=0.0.0.0:57515943lb # [ 98.029865] systemd[1]: Started niks3 server.5944lb # [ 98.030772] niks3-server[1116]: 2026/09/23 05:12:51 INFO systemd watchdog enabled interval=15s5945lb # [ 98.032463] niks3-server[1116]: 2026/09/23 05:12:51 INFO Starting HTTP server address=0.0.0.0:57515946lb: waiting for unit niks3.service5947lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5948lb: must succeed: curl -sf localhost:9901/clusters5949lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5950worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5951worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5952lb: must succeed: curl -sf localhost:9901/clusters5953lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5954lb: must succeed: curl -sf localhost:9901/clusters5955lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5956worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5957worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5958client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25959worker2 # [ 98.394892] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:368645960client # don't know how to build these paths:5961client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5962worker1 # [ 98.423059] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529605963worker1 # [ 98.434300] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)5964worker1 # [ 98.452610] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)5965worker1 # [ 98.488940] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 2/5)5966worker1 # [ 98.577107] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 238 ms (attempt 3/5)5967worker1 # [ 98.817496] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 4/5)5968worker1 # [ 98.865369] nix-grpc-daemon[1336]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5969worker1 # [ 98.868664] nix-grpc-daemon[1336]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5970worker1 # [ 98.898574] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:52Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05971worker1 # [ 98.904086] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:52Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.3:50051 clients=15972worker1 # [ 98.943666] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529605973worker1 # [ 98.956811] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529605974client # copying 1 paths to 'lb:50051'...5975client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5976worker1 # [ 98.969378] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529605977worker1 # [ 98.977821] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)5978lb # [ 98.942848] niks3-server[1116]: 2026/09/23 05:12:52 INFO lead: acquired remote=192.168.1.3:393845979worker1 # [ 99.001422] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:52Z level=info event=niks3_push_started pid=14095980worker1 # [ 99.104799] nix-grpc-daemon[1409]: time=2026-09-23T05:12:52.208Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5981worker1 # [ 99.202100] nix-daemon[863]: accepted connection from pid 1422, user nix-grpc-daemon (trusted)5982worker1 # [ 99.214167] nix-daemon[863]: reaped child process 1425, status = succeeded5983lb # [ 99.177308] niks3-server[1116]: 2026/09/23 05:12:52 INFO Received uploads request method=POST path=/api/pending_closures5984worker1 # [ 99.234083] nix-grpc-daemon[1409]: time=2026-09-23T05:12:52.337Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5985worker1 # [ 99.235600] nix-grpc-daemon[1409]: time=2026-09-23T05:12:52.339Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5986lb # [ 99.228668] niks3-server[1116]: 2026/09/23 05:12:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5987lb # [ 99.236980] niks3-server[1116]: 2026/09/23 05:12:52 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5988lb # [ 99.254133] niks3-server[1116]: 2026/09/23 05:12:52 INFO Signed narinfos id=37 count=15989worker1 # [ 99.299973] nix-grpc-daemon[1409]: time=2026-09-23T05:12:52.403Z level=INFO msg="Uploading 1 narinfos"5990lb # [ 99.260196] niks3-server[1116]: 2026/09/23 05:12:52 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5991lb # [ 99.264583] niks3-server[1116]: 2026/09/23 05:12:52 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5992lb # [ 99.270780] niks3-server[1116]: 2026/09/23 05:12:52 INFO Completed upload id=375993worker1 # [ 99.315520] nix-grpc-daemon[1409]: time=2026-09-23T05:12:52.419Z level=INFO msg="Upload complete. (207ms)"5994lb # [ 99.274487] niks3-server[1116]: 2026/09/23 05:12:52 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5995worker1 # [ 99.320523] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52960 duration_s=0 paths=1 nar_bytes_in=5365996worker1 # [ 99.327245] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529605997worker1 # [ 99.338105] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)5998worker1 # [ 99.349538] nix-daemon[863]: reaped child process 1407, status = succeeded5999worker1 # [ 99.352681] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6000client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...6001client # farm-a> node-a: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6002client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6003client # farm-a> LOG-farm-a-after-niks3-restart6004worker1 # [ 99.966778] nix-daemon[863]: reaped child process 1429, status = succeeded6005worker1 # [ 100.044848] nix-daemon[863]: accepted connection from pid 1445, user nix-grpc-daemon (trusted)6006worker1 # [ 100.057960] nix-daemon[863]: reaped child process 1448, status = succeeded6007lb # [ 100.021227] niks3-server[1116]: 2026/09/23 05:12:53 INFO Received uploads request method=POST path=/api/pending_closures6008worker1 # [ 100.077512] nix-grpc-daemon[1409]: time=2026-09-23T05:12:53.181Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6009worker1 # [ 100.079041] nix-grpc-daemon[1409]: time=2026-09-23T05:12:53.182Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"6010lb # [ 100.071471] niks3-server[1116]: 2026/09/23 05:12:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6011worker1 # [ 100.121912] nix-grpc-daemon[1409]: time=2026-09-23T05:12:53.225Z level=INFO msg="Uploading 1 narinfos"6012lb # [ 100.077105] niks3-server[1116]: 2026/09/23 05:12:53 INFO Signed narinfos id=38 count=16013lb # [ 100.084797] niks3-server[1116]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6014lb # [ 100.088918] niks3-server[1116]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst6015lb # [ 100.097875] niks3-server[1116]: 2026/09/23 05:12:53 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6016lb # [ 100.101600] niks3-server[1116]: 2026/09/23 05:12:53 INFO Completed upload id=386017worker1 # [ 100.146296] nix-grpc-daemon[1409]: time=2026-09-23T05:12:53.249Z level=INFO msg="Upload complete. (187ms)"6018lb # [ 100.104290] niks3-server[1116]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls6019lb # [ 100.107320] niks3-server[1116]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6020worker1 # [ 100.152453] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52960 duration_s=0 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16021worker1 # [ 100.163628] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:53Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06022worker1 # [ 100.168277] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:53Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.3:50051 clients=16023worker1 # [ 100.173791] nix-daemon[863]: reaped child process 1427, status = succeeded6024worker1 # [ 100.208509] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529606025worker1 # [ 100.220989] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529606026worker1 # [ 100.235501] nix-grpc-daemon[1409]: time=2026-09-23T05:12:53.339Z level=INFO msg="All 1 paths already cached"6027client # copying 1 paths to 'lb:50051'...6028client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6029worker1 # [ 100.244763] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529606030worker1 # [ 100.255130] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6031worker1 # [ 100.350848] nix-daemon[863]: accepted connection from pid 1454, user nix-grpc-daemon (trusted)6032worker1 # [ 100.364173] nix-daemon[863]: reaped child process 1457, status = succeeded6033lb # [ 100.328465] niks3-server[1116]: 2026/09/23 05:12:53 INFO Received uploads request method=POST path=/api/pending_closures6034worker1 # [ 100.383446] nix-grpc-daemon[1409]: time=2026-09-23T05:12:53.487Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6035worker1 # [ 100.385079] nix-grpc-daemon[1409]: time=2026-09-23T05:12:53.488Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6036lb # [ 100.362865] niks3-server[1116]: 2026/09/23 05:12:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6037lb # [ 100.368901] niks3-server[1116]: 2026/09/23 05:12:53 INFO Signed narinfos id=39 count=16038worker1 # [ 100.416547] nix-grpc-daemon[1409]: time=2026-09-23T05:12:53.520Z level=INFO msg="Uploading 1 narinfos"6039lb # [ 100.379138] niks3-server[1116]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6040lb # [ 100.384045] niks3-server[1116]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6041lb # [ 100.398304] niks3-server[1116]: 2026/09/23 05:12:53 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6042lb # [ 100.402679] niks3-server[1116]: 2026/09/23 05:12:53 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6043lb # [ 100.405650] niks3-server[1116]: 2026/09/23 05:12:53 INFO Completed upload id=396044worker1 # [ 100.450266] nix-grpc-daemon[1409]: time=2026-09-23T05:12:53.553Z level=INFO msg="Upload complete. (185ms)"6045worker1 # [ 100.455092] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52960 duration_s=0 paths=1 nar_bytes_in=6566046worker1 # [ 100.460868] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529606047worker1 # [ 100.469619] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6048worker1 # [ 100.478559] nix-daemon[863]: reaped child process 1452, status = succeeded6049client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6050worker1 # [ 100.485100] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6051client # farm-b> node-a: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6052client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6053client # farm-b> LOG-farm-b-after-niks3-restart6054worker1 # [ 101.517875] nix-daemon[863]: reaped child process 1461, status = succeeded6055worker1 # [ 101.607695] nix-daemon[863]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6056worker1 # [ 101.619775] nix-daemon[863]: reaped child process 1480, status = succeeded6057lb # [ 101.583534] niks3-server[1116]: 2026/09/23 05:12:54 INFO Received uploads request method=POST path=/api/pending_closures6058worker1 # [ 101.636182] nix-grpc-daemon[1409]: time=2026-09-23T05:12:54.739Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6059worker1 # [ 101.638541] nix-grpc-daemon[1409]: time=2026-09-23T05:12:54.741Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6060lb # [ 101.623071] niks3-server[1116]: 2026/09/23 05:12:54 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6061lb # [ 101.628468] niks3-server[1116]: 2026/09/23 05:12:54 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6062lb # [ 101.634884] niks3-server[1116]: 2026/09/23 05:12:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6063lb # [ 101.641820] niks3-server[1116]: 2026/09/23 05:12:54 INFO Signed narinfos id=40 count=16064worker1 # [ 101.689810] nix-grpc-daemon[1409]: time=2026-09-23T05:12:54.793Z level=INFO msg="Uploading 1 narinfos"6065lb # [ 101.649818] niks3-server[1116]: 2026/09/23 05:12:54 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6066lb # [ 101.659498] niks3-server[1116]: 2026/09/23 05:12:54 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6067lb # [ 101.662269] niks3-server[1116]: 2026/09/23 05:12:54 INFO Completed upload id=406068worker1 # [ 101.707120] nix-grpc-daemon[1409]: time=2026-09-23T05:12:54.810Z level=INFO msg="Upload complete. (215ms)"6069lb # [ 101.667163] niks3-server[1116]: 2026/09/23 05:12:54 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6070worker1 # [ 101.716879] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52960 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16071worker1 # [ 101.726557] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:54Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06072worker1 # [ 101.730270] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:54Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.3:50051 clients=16073worker1 # [ 101.738760] nix-daemon[863]: reaped child process 1459, status = succeeded6074worker1 # [ 101.771185] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529606075worker1 # [ 101.784596] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529606076worker1 # [ 101.797760] nix-grpc-daemon[1409]: time=2026-09-23T05:12:54.901Z level=INFO msg="All 2 paths already cached"6077client # copying 1 paths to 'lb:50051'...6078client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6079worker1 # [ 101.806873] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529606080worker1 # [ 101.816331] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6081worker1 # [ 101.913542] nix-daemon[863]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6082worker1 # [ 101.926495] nix-daemon[863]: reaped child process 1489, status = succeeded6083lb # [ 101.890744] niks3-server[1116]: 2026/09/23 05:12:55 INFO Received uploads request method=POST path=/api/pending_closures6084worker1 # [ 101.943123] nix-grpc-daemon[1409]: time=2026-09-23T05:12:55.046Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6085worker1 # [ 101.945063] nix-grpc-daemon[1409]: time=2026-09-23T05:12:55.048Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6086lb # [ 101.922341] niks3-server[1116]: 2026/09/23 05:12:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6087lb # [ 101.927647] niks3-server[1116]: 2026/09/23 05:12:55 INFO Signed narinfos id=41 count=16088worker1 # [ 101.975917] nix-grpc-daemon[1409]: time=2026-09-23T05:12:55.079Z level=INFO msg="Uploading 1 narinfos"6089lb # [ 101.937207] niks3-server[1116]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6090lb # [ 101.949541] niks3-server[1116]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6091lb # [ 101.957469] niks3-server[1116]: 2026/09/23 05:12:55 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6092lb # [ 101.965954] niks3-server[1116]: 2026/09/23 05:12:55 INFO Completed upload id=416093lb # [ 101.968140] niks3-server[1116]: 2026/09/23 05:12:55 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6094worker1 # [ 102.012066] nix-grpc-daemon[1409]: time=2026-09-23T05:12:55.115Z level=INFO msg="Upload complete. (185ms)"6095worker1 # [ 102.017946] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52960 duration_s=0 paths=1 nar_bytes_in=7846096worker1 # [ 102.022826] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529606097worker1 # [ 102.032102] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6098worker1 # [ 102.040789] nix-daemon[863]: reaped child process 1484, status = succeeded6099worker1 # [ 102.044939] nix-daemon[863]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)6100client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6101client # farm-top> node-a: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6102client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6103client # farm-top> LOG-farm-top-after-niks3-restart6104worker1 # [ 102.879772] nix-daemon[863]: reaped child process 1493, status = succeeded6105worker1 # [ 102.967285] nix-daemon[863]: accepted connection from pid 1509, user nix-grpc-daemon (trusted)6106worker1 # [ 102.978708] nix-daemon[863]: reaped child process 1512, status = succeeded6107lb # [ 102.943407] niks3-server[1116]: 2026/09/23 05:12:56 INFO Received uploads request method=POST path=/api/pending_closures6108worker1 # [ 102.996941] nix-grpc-daemon[1409]: time=2026-09-23T05:12:56.100Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6109worker1 # [ 102.998435] nix-grpc-daemon[1409]: time=2026-09-23T05:12:56.102Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6110lb # [ 102.988749] niks3-server[1116]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6111lb # [ 102.993557] niks3-server[1116]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6112lb # [ 103.001368] niks3-server[1116]: 2026/09/23 05:12:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6113lb # [ 103.006887] niks3-server[1116]: 2026/09/23 05:12:56 INFO Signed narinfos id=42 count=16114worker1 # [ 103.053859] nix-grpc-daemon[1409]: time=2026-09-23T05:12:56.157Z level=INFO msg="Uploading 1 narinfos"6115lb # [ 103.009805] niks3-server[1116]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6116lb # [ 103.021545] niks3-server[1116]: 2026/09/23 05:12:56 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6117lb # [ 103.024676] niks3-server[1116]: 2026/09/23 05:12:56 INFO Completed upload id=426118lb # [ 103.027417] niks3-server[1116]: 2026/09/23 05:12:56 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6119worker1 # [ 103.070780] nix-grpc-daemon[1409]: time=2026-09-23T05:12:56.174Z level=INFO msg="Upload complete. (218ms)"6120worker1 # [ 103.079321] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52960 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16121worker1 # [ 103.093080] nix-daemon[863]: reaped child process 1491, status = succeeded6122client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 4.85 seconds)6123client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6124client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)6125(finished: subtest: a niks3 restart does not cost the leader its role, in 5.32 seconds)6126subtest: scheduler down: the other node takes the lock and keeps it6127lb: must succeed: curl -sf localhost:9901/clusters6128lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6129lb: must succeed: curl -sf localhost:9901/clusters6130lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6131lb: must succeed: curl -sf localhost:9901/clusters6132lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6133lb: must succeed: curl -sf localhost:9901/clusters6134lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6135worker1 # [ 103.384640] systemd[1]: Stopping Nix worker-protocol over gRPC...6136worker1 # [ 103.785455] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:56Z level=info event=signal_exit6137lb # [ 103.946734] niks3-server[1116]: 2026/09/23 05:12:57 INFO lead: released remote=192.168.1.3:393846138worker2 # [ 104.979178] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:57Z level=info event=scheduler_restarting addr=lb:500516139worker1 # [ 104.996565] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:58Z level=info event=scheduler_restarting6140worker1 # [ 105.001809] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:529486141worker1 # [ 105.006264] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:58Z level=info event=scheduler_restarting addr=lb:500516142worker1 # [ 105.009688] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:58Z level=info event=worker_gone addr=192.168.1.4:500516143worker1 # [ 105.013320] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:529486144worker1 # [ 105.017524] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:58Z level=info event=worker_gone addr=192.168.1.3:500516145worker1 # [ 105.021995] nix-grpc-daemon[1336]: E0000 00:00:1790140378.125129 1382 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true6146worker1 # [ 105.027185] nix-grpc-daemon[1336]: E0000 00:00:1790140378.125295 1382 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6147worker1 # [ 105.046899] nix-daemon[863]: reaped child process 1397, status = succeeded6148worker2 # [ 105.179286] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:500516149worker2 # [ 105.183561] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6150worker1 # [ 105.209093] nix-grpc-daemon[1336]: ts=2026-09-23T05:12:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6151worker2 # [ 105.384955] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:500516152worker2 # [ 105.388952] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6153worker2 # [ 105.589986] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:500516154worker2 # [ 105.594146] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6155worker2 # [ 105.795244] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:500516156worker2 # [ 105.799269] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6157worker2 # [ 106.001288] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:500516158worker2 # [ 106.005211] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6159worker1 # [ 106.199127] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6160worker1 # [ 106.202498] systemd[1]: Stopped Nix worker-protocol over gRPC.6161worker1 # [ 106.206760] systemd[1]: nix-grpc-daemon.service: Consumed 793ms CPU time over 9.515s wall clock time, 36.5M memory peak, 36K incoming IP traffic, 59.6K outgoing IP traffic.6162worker1 # [ 106.217220] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6163worker1 # [ 106.221705] systemd[1]: Closed Nix worker-protocol over gRPC.6164worker2 # [ 106.206704] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:500516165worker2 # [ 106.211198] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6166worker1 # [ 106.225456] systemd[1]: nix-grpc-daemon.socket: Consumed 392us CPU time over 9.546s wall clock time, 256K memory peak, 32.5K incoming IP traffic, 23.4K outgoing IP traffic.6167worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6168worker2 # [ 106.412255] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:500516169worker2 # [ 106.416310] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6170worker2 # [ 106.617358] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:500516171worker2 # [ 106.621424] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6172worker2 # [ 106.822676] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:500516173worker2 # [ 106.826567] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6174lb # [ 106.922662] envoy[912]: conn peer=[2001:db8:1::1]:51240 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9666 tx=4418 ms=46816175lb # [ 106.926613] envoy[912]: conn peer=[2001:db8:1::3]:44832 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1328 tx=513 ms=83686176worker2 # [ 107.028892] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:500516177worker2 # [ 107.032749] nix-grpc-daemon[1169]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6178worker2 # [ 107.233880] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:500516179worker2 # [ 107.237895] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6180worker2 # [ 107.439144] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:500516181worker2 # [ 107.443164] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6182worker2 # [ 107.644268] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:500516183worker2 # [ 107.648435] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6184worker2 # [ 107.849480] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:500516185worker2 # [ 107.853429] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6186worker2 # [ 108.055415] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:500516187worker2 # [ 108.058972] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6188worker2 # [ 108.260459] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:500516189worker2 # [ 108.264488] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6190worker2 # [ 108.465703] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:500516191worker2 # [ 108.469919] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6192worker2 # [ 108.671162] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:500516193worker2 # [ 108.675158] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6194worker2 # [ 108.876402] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:500516195worker2 # [ 108.880402] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6196lb # [ 108.941182] niks3-server[1116]: 2026/09/23 05:13:02 INFO lead: acquired remote=192.168.1.4:397266197worker2 # [ 108.971503] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_take_over6198worker2 # [ 109.082428] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:500516199worker2 # [ 109.086364] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6200worker2 # [ 109.287710] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:500516201worker2 # [ 109.291756] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6202worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 3.22 seconds)6203worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6204worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6205worker2 # [ 109.492465] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:500516206worker2 # [ 109.495683] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6207worker2 # [ 109.696925] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:500516208worker2 # [ 109.716606] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:02Z level=info event=oidc_jwks provider=mock keys=16209worker2 # [ 109.719658] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:592246210worker2 # [ 109.723988] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06211worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6212worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6213client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26214client # don't know how to build these paths:6215client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6216worker2 # [ 110.652252] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:368646217worker2 # [ 110.658710] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368646218worker2 # [ 110.681117] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:03Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06219worker2 # [ 110.687416] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:03Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=16220worker2 # [ 110.731146] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368646221worker2 # [ 110.744412] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368646222client # copying 1 paths to 'lb:50051'...6223client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6224worker2 # [ 110.757906] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368646225worker2 # [ 110.766857] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6226worker2 # [ 110.871664] nix-daemon[852]: accepted connection from pid 1287, user nix-grpc-daemon (trusted)6227worker2 # [ 110.884663] nix-daemon[852]: reaped child process 1290, status = succeeded6228lb # [ 110.863066] niks3-server[1116]: 2026/09/23 05:13:04 INFO Received uploads request method=POST path=/api/pending_closures6229worker2 # [ 110.905563] nix-grpc-daemon[1230]: time=2026-09-23T05:13:03.771Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6230worker2 # [ 110.907766] nix-grpc-daemon[1230]: time=2026-09-23T05:13:03.773Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6231lb # [ 110.902542] niks3-server[1116]: 2026/09/23 05:13:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6232lb # [ 110.907319] niks3-server[1116]: 2026/09/23 05:13:04 INFO Signed narinfos id=43 count=16233worker2 # [ 110.941417] nix-grpc-daemon[1230]: time=2026-09-23T05:13:03.806Z level=INFO msg="Uploading 1 narinfos"6234lb # [ 110.916851] niks3-server[1116]: 2026/09/23 05:13:04 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6235lb # [ 110.929334] niks3-server[1116]: 2026/09/23 05:13:04 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6236lb # [ 110.933833] niks3-server[1116]: 2026/09/23 05:13:04 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6237lb # [ 110.941335] niks3-server[1116]: 2026/09/23 05:13:04 INFO Completed upload id=436238worker2 # [ 110.971556] nix-grpc-daemon[1230]: time=2026-09-23T05:13:03.837Z level=INFO msg="Upload complete. (190ms)"6239lb # [ 110.943841] niks3-server[1116]: 2026/09/23 05:13:04 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6240worker2 # [ 110.975483] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36864 duration_s=0 paths=1 nar_bytes_in=5206241worker2 # [ 110.980994] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368646242worker2 # [ 110.991487] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6243worker2 # [ 111.000349] nix-daemon[852]: reaped child process 1285, status = succeeded6244worker2 # [ 111.004418] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6245client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6246client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6247client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6248client # farm-a> LOG-farm-a-on-standby6249worker2 # [ 112.446720] nix-daemon[852]: reaped child process 1294, status = succeeded6250worker2 # [ 112.530761] nix-daemon[852]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)6251worker2 # [ 112.545344] nix-daemon[852]: reaped child process 1313, status = succeeded6252lb # [ 112.522072] niks3-server[1116]: 2026/09/23 05:13:05 INFO Received uploads request method=POST path=/api/pending_closures6253worker2 # [ 112.562213] nix-grpc-daemon[1230]: time=2026-09-23T05:13:05.427Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6254worker2 # [ 112.565060] nix-grpc-daemon[1230]: time=2026-09-23T05:13:05.429Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6255lb # [ 112.557261] niks3-server[1116]: 2026/09/23 05:13:05 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6256lb # [ 112.570898] niks3-server[1116]: 2026/09/23 05:13:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6257worker2 # [ 112.608791] nix-grpc-daemon[1230]: time=2026-09-23T05:13:05.474Z level=INFO msg="Uploading 1 narinfos"6258lb # [ 112.579183] niks3-server[1116]: 2026/09/23 05:13:05 INFO Signed narinfos id=44 count=16259lb # [ 112.583957] niks3-server[1116]: 2026/09/23 05:13:05 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6260lb # [ 112.590767] niks3-server[1116]: 2026/09/23 05:13:05 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6261lb # [ 112.599054] niks3-server[1116]: 2026/09/23 05:13:05 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6262lb # [ 112.601803] niks3-server[1116]: 2026/09/23 05:13:05 INFO Completed upload id=446263worker2 # [ 112.632468] nix-grpc-daemon[1230]: time=2026-09-23T05:13:05.498Z level=INFO msg="Upload complete. (219ms)"6264lb # [ 112.604824] niks3-server[1116]: 2026/09/23 05:13:05 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6265worker2 # [ 112.639496] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36864 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16266worker2 # [ 112.652565] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:05Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06267worker2 # [ 112.657218] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:05Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16268worker2 # [ 112.667815] nix-daemon[852]: reaped child process 1292, status = succeeded6269worker2 # [ 112.697126] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368646270worker2 # [ 112.710359] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368646271worker2 # [ 112.723156] nix-grpc-daemon[1230]: time=2026-09-23T05:13:05.589Z level=INFO msg="All 1 paths already cached"6272client # copying 1 paths to 'lb:50051'...6273client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6274worker2 # [ 112.731814] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368646275worker2 # [ 112.740174] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6276worker2 # [ 112.838074] nix-daemon[852]: accepted connection from pid 1319, user nix-grpc-daemon (trusted)6277worker2 # [ 112.851482] nix-daemon[852]: reaped child process 1322, status = succeeded6278lb # [ 112.829842] niks3-server[1116]: 2026/09/23 05:13:06 INFO Received uploads request method=POST path=/api/pending_closures6279worker2 # [ 112.868418] nix-grpc-daemon[1230]: time=2026-09-23T05:13:05.734Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6280worker2 # [ 112.869920] nix-grpc-daemon[1230]: time=2026-09-23T05:13:05.735Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6281lb # [ 112.863684] niks3-server[1116]: 2026/09/23 05:13:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6282worker2 # [ 112.902502] nix-grpc-daemon[1230]: time=2026-09-23T05:13:05.768Z level=INFO msg="Uploading 1 narinfos"6283lb # [ 112.871947] niks3-server[1116]: 2026/09/23 05:13:06 INFO Signed narinfos id=45 count=16284lb # [ 112.876837] niks3-server[1116]: 2026/09/23 05:13:06 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6285lb # [ 112.895196] niks3-server[1116]: 2026/09/23 05:13:06 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6286worker2 # [ 112.932813] nix-grpc-daemon[1230]: time=2026-09-23T05:13:05.798Z level=INFO msg="Upload complete. (180ms)"6287lb # [ 112.903966] niks3-server[1116]: 2026/09/23 05:13:06 INFO Completed upload id=456288lb # [ 112.905252] niks3-server[1116]: 2026/09/23 05:13:06 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6289lb # [ 112.911158] niks3-server[1116]: 2026/09/23 05:13:06 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6290worker2 # [ 112.937520] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36864 duration_s=0 paths=1 nar_bytes_in=6406291worker2 # [ 112.943898] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368646292worker2 # [ 112.956183] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6293worker2 # [ 112.960568] nix-daemon[852]: reaped child process 1317, status = succeeded6294worker2 # [ 112.965891] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6295client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6296client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6297client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6298client # farm-b> LOG-farm-b-on-standby6299worker2 # [ 114.003713] nix-daemon[852]: reaped child process 1326, status = succeeded6300worker2 # [ 114.089205] nix-daemon[852]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)6301worker2 # [ 114.102787] nix-daemon[852]: reaped child process 1345, status = succeeded6302lb # [ 114.081577] niks3-server[1116]: 2026/09/23 05:13:07 INFO Received uploads request method=POST path=/api/pending_closures6303worker2 # [ 114.128703] nix-grpc-daemon[1230]: time=2026-09-23T05:13:06.994Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6304worker2 # [ 114.130603] nix-grpc-daemon[1230]: time=2026-09-23T05:13:06.996Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6305lb # [ 114.134273] niks3-server[1116]: 2026/09/23 05:13:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6306worker2 # [ 114.177462] nix-grpc-daemon[1230]: time=2026-09-23T05:13:07.043Z level=INFO msg="Uploading 1 narinfos"6307lb # [ 114.146564] niks3-server[1116]: 2026/09/23 05:13:07 INFO Signed narinfos id=46 count=16308lb # [ 114.154346] niks3-server[1116]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6309lb # [ 114.158483] niks3-server[1116]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6310lb # [ 114.167187] niks3-server[1116]: 2026/09/23 05:13:07 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6311lb # [ 114.168864] niks3-server[1116]: 2026/09/23 05:13:07 INFO Completed upload id=466312worker2 # [ 114.200655] nix-grpc-daemon[1230]: time=2026-09-23T05:13:07.066Z level=INFO msg="Upload complete. (227ms)"6313lb # [ 114.172624] niks3-server[1116]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6314lb # [ 114.174666] niks3-server[1116]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6315worker2 # [ 114.212684] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36864 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16316worker2 # [ 114.220405] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:07Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06317worker2 # [ 114.223677] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:07Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16318worker2 # [ 114.233422] nix-daemon[852]: reaped child process 1324, status = succeeded6319worker2 # [ 114.262947] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368646320worker2 # [ 114.275338] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368646321worker2 # [ 114.289160] nix-grpc-daemon[1230]: time=2026-09-23T05:13:07.155Z level=INFO msg="All 2 paths already cached"6322client # copying 1 paths to 'lb:50051'...6323client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6324worker2 # [ 114.298779] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368646325worker2 # [ 114.306960] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6326worker2 # [ 114.408285] nix-daemon[852]: accepted connection from pid 1351, user nix-grpc-daemon (trusted)6327worker2 # [ 114.423168] nix-daemon[852]: reaped child process 1354, status = succeeded6328lb # [ 114.399998] niks3-server[1116]: 2026/09/23 05:13:07 INFO Received uploads request method=POST path=/api/pending_closures6329worker2 # [ 114.438944] nix-grpc-daemon[1230]: time=2026-09-23T05:13:07.304Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6330worker2 # [ 114.440421] nix-grpc-daemon[1230]: time=2026-09-23T05:13:07.306Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6331lb # [ 114.433132] niks3-server[1116]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6332lb # [ 114.439511] niks3-server[1116]: 2026/09/23 05:13:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6333worker2 # [ 114.477193] nix-grpc-daemon[1230]: time=2026-09-23T05:13:07.343Z level=INFO msg="Uploading 1 narinfos"6334lb # [ 114.447133] niks3-server[1116]: 2026/09/23 05:13:07 INFO Signed narinfos id=47 count=16335lb # [ 114.453698] niks3-server[1116]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6336lb # [ 114.467195] niks3-server[1116]: 2026/09/23 05:13:07 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6337lb # [ 114.476637] niks3-server[1116]: 2026/09/23 05:13:07 INFO Completed upload id=476338worker2 # [ 114.507273] nix-grpc-daemon[1230]: time=2026-09-23T05:13:07.373Z level=INFO msg="Upload complete. (187ms)"6339lb # [ 114.478442] niks3-server[1116]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6340worker2 # [ 114.511338] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36864 duration_s=0 paths=1 nar_bytes_in=7686341worker2 # [ 114.516286] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368646342worker2 # [ 114.526480] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6343worker2 # [ 114.536355] nix-daemon[852]: reaped child process 1349, status = succeeded6344worker2 # [ 114.540831] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6345client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6346client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6347client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6348client # farm-top> LOG-farm-top-on-standby6349worker2 # [ 115.578506] nix-daemon[852]: reaped child process 1358, status = succeeded6350worker2 # [ 115.659093] nix-daemon[852]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)6351worker2 # [ 115.674616] nix-daemon[852]: reaped child process 1377, status = succeeded6352lb # [ 115.651577] niks3-server[1116]: 2026/09/23 05:13:08 INFO Received uploads request method=POST path=/api/pending_closures6353worker2 # [ 115.690914] nix-grpc-daemon[1230]: time=2026-09-23T05:13:08.556Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6354worker2 # [ 115.692477] nix-grpc-daemon[1230]: time=2026-09-23T05:13:08.558Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6355lb # [ 115.695842] niks3-server[1116]: 2026/09/23 05:13:08 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6356lb # [ 115.703981] niks3-server[1116]: 2026/09/23 05:13:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6357lb # [ 115.709866] niks3-server[1116]: 2026/09/23 05:13:08 INFO Signed narinfos id=48 count=16358worker2 # [ 115.743904] nix-grpc-daemon[1230]: time=2026-09-23T05:13:08.609Z level=INFO msg="Uploading 1 narinfos"6359lb # [ 115.719439] niks3-server[1116]: 2026/09/23 05:13:08 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6360lb # [ 115.723480] niks3-server[1116]: 2026/09/23 05:13:08 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6361lb # [ 115.732937] niks3-server[1116]: 2026/09/23 05:13:08 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6362lb # [ 115.737588] niks3-server[1116]: 2026/09/23 05:13:08 INFO Completed upload id=486363worker2 # [ 115.768270] nix-grpc-daemon[1230]: time=2026-09-23T05:13:08.634Z level=INFO msg="Upload complete. (221ms)"6364lb # [ 115.740188] niks3-server[1116]: 2026/09/23 05:13:08 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6365worker2 # [ 115.776696] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36864 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16366worker2 # [ 115.791114] nix-daemon[852]: reaped child process 1356, status = succeeded6367client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.30 seconds)6368client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6369client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6370worker1 # [ 115.982902] systemd[1]: Listening on Nix worker-protocol over gRPC.6371worker1 # [ 115.990381] systemd[1]: Starting Nix worker-protocol over gRPC...6372worker1 # [ 116.077198] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6373worker1 # [ 116.079224] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6374worker1 # [ 116.082810] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16375worker1 # [ 116.087079] nix-grpc-daemon[1530]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6376worker1 # [ 116.088429] nix-grpc-daemon[1530]: E0000 00:00:1790140389.190201 1530 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6377worker1 # [ 116.093404] nix-grpc-daemon[1530]: E0000 00:00:1790140389.195768 1530 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6378worker1 # [ 116.097404] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6379worker1 # [ 116.101953] systemd[1]: Started Nix worker-protocol over gRPC.6380worker1 # [ 116.104266] nix-grpc-daemon[1530]: E0000 00:00:1790140389.206977 1571 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6381worker1 # [ 116.117065] nix-grpc-daemon[1530]: E0000 00:00:1790140389.220493 1571 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6382worker1 # [ 116.121105] nix-grpc-daemon[1530]: E0000 00:00:1790140389.220619 1571 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6383lb: must succeed: curl -sf localhost:9901/clusters6384worker1 # [ 116.131490] nix-grpc-daemon[1530]: E0000 00:00:1790140389.235241 1571 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6385worker1 # [ 116.144206] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:500516386worker2 # [ 116.132365] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:592246387worker2 # [ 116.136149] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06388lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6389worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6390worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6391lb: must succeed: curl -sf localhost:9901/clusters6392lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6393lb: must succeed: curl -sf localhost:9901/clusters6394lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6395worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6396worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6397(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.09 seconds)6398subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6399client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6400client # [ 116.330122] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6401client # Running as unit: rs.service; invocation ID: d2426e314954477eb24473b7ca6bfc776402client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6403worker1 # [ 116.503994] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:598266404worker1 # [ 116.515628] nix-daemon[863]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6405client # [ 116.493614] nix[1361]: don't know how to build these paths:6406client # [ 116.494528] nix[1361]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6407worker2 # [ 116.521907] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:09Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06408worker2 # [ 116.528397] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:09Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16409worker2 # [ 116.566823] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368646410worker2 # [ 116.580339] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368646411client # [ 116.569504] nix[1361]: copying 1 paths to 'lb:50051'...6412worker2 # [ 116.591135] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368646413client # [ 116.572395] nix[1361]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6414worker2 # [ 116.601801] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6415worker2 # [ 116.705218] nix-daemon[852]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6416worker2 # [ 116.718728] nix-daemon[852]: reaped child process 1393, status = succeeded6417lb # [ 116.695569] niks3-server[1116]: 2026/09/23 05:13:09 INFO Received uploads request method=POST path=/api/pending_closures6418worker2 # [ 116.736564] nix-grpc-daemon[1230]: time=2026-09-23T05:13:09.602Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6419worker2 # [ 116.738462] nix-grpc-daemon[1230]: time=2026-09-23T05:13:09.604Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6420lb # [ 116.732348] niks3-server[1116]: 2026/09/23 05:13:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6421worker2 # [ 116.768684] nix-grpc-daemon[1230]: time=2026-09-23T05:13:09.634Z level=INFO msg="Uploading 1 narinfos"6422lb # [ 116.738640] niks3-server[1116]: 2026/09/23 05:13:09 INFO Signed narinfos id=49 count=16423lb # [ 116.744231] niks3-server[1116]: 2026/09/23 05:13:09 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6424lb # [ 116.750507] niks3-server[1116]: 2026/09/23 05:13:09 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6425lb # [ 116.759245] niks3-server[1116]: 2026/09/23 05:13:09 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6426lb # [ 116.762076] niks3-server[1116]: 2026/09/23 05:13:09 INFO Completed upload id=496427worker2 # [ 116.792623] nix-grpc-daemon[1230]: time=2026-09-23T05:13:09.658Z level=INFO msg="Upload complete. (176ms)"6428lb # [ 116.767235] niks3-server[1116]: 2026/09/23 05:13:09 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6429worker2 # [ 116.797658] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36864 duration_s=0 paths=1 nar_bytes_in=4166430worker2 # [ 116.803266] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368646431worker2 # [ 116.811030] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6432worker2 # [ 116.816898] nix-daemon[852]: reaped child process 1388, status = succeeded6433worker2 # [ 116.821149] nix-daemon[852]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)6434client # [ 116.800317] nix[1361]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6435lb # [ 116.923117] envoy[912]: conn peer=[2001:db8:1::1]:52660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9586 tx=4345 ms=51376436lb: must succeed: curl -sf localhost:9901/clusters6437lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6438lb: must succeed: curl -sf localhost:9901/clusters6439lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6440lb: must succeed: curl -sf localhost:9901/clusters6441lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6442lb: must succeed: curl -sf localhost:9901/clusters6443lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6444worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6445worker1 # [ 117.938679] systemd-journald[284]: Received client request to rotate journal, rotating.6446worker1 # [ 117.958872] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/6c9b3e48d2e647a4ab5c38ce309a1480.6447worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6448worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6449worker2 # [ 117.853349] systemd[1]: Stopping Nix worker-protocol over gRPC...6450worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6451client: must succeed: sleep 5.06452worker2 # [ 118.483754] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:11Z level=info event=draining6453client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6454worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6455worker2 # [ 123.271475] nix-daemon[852]: reaped child process 1397, status = succeeded6456worker2 # [ 123.346851] nix-daemon[852]: accepted connection from pid 1434, user nix-grpc-daemon (trusted)6457worker2 # [ 123.359927] nix-daemon[852]: reaped child process 1437, status = succeeded6458lb # [ 123.339264] niks3-server[1116]: 2026/09/23 05:13:16 INFO Received uploads request method=POST path=/api/pending_closures6459worker2 # [ 123.378809] nix-grpc-daemon[1230]: time=2026-09-23T05:13:16.244Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6460worker2 # [ 123.380965] nix-grpc-daemon[1230]: time=2026-09-23T05:13:16.246Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6461lb # [ 123.383071] niks3-server[1116]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6462lb # [ 123.390380] niks3-server[1116]: 2026/09/23 05:13:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6463worker2 # [ 123.426905] nix-grpc-daemon[1230]: time=2026-09-23T05:13:16.292Z level=INFO msg="Uploading 1 narinfos"6464lb # [ 123.396233] niks3-server[1116]: 2026/09/23 05:13:16 INFO Signed narinfos id=50 count=16465lb # [ 123.400930] niks3-server[1116]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6466lb # [ 123.414995] niks3-server[1116]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6467lb # [ 123.418972] niks3-server[1116]: 2026/09/23 05:13:16 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6468lb # [ 123.422991] niks3-server[1116]: 2026/09/23 05:13:16 INFO Completed upload id=506469worker2 # [ 123.453604] nix-grpc-daemon[1230]: time=2026-09-23T05:13:16.319Z level=INFO msg="Upload complete. (218ms)"6470lb # [ 123.425878] niks3-server[1116]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6471worker2 # [ 123.462603] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36864 duration_s=6 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16472worker2 # [ 123.477451] nix-daemon[852]: reaped child process 1395, status = succeeded6473worker2 # [ 123.484128] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:16Z level=info event=signal_exit6474client # [ 123.469531] systemd[1]: rs.service: Deactivated successfully.6475client # [ 123.473221] systemd[1]: rs.service: Consumed 127ms CPU time over 7.142s wall clock time, 39.4M memory peak, 7.3K incoming IP traffic, 8.6K outgoing IP traffic.6476lb # [ 123.942266] niks3-server[1116]: 2026/09/23 05:13:17 INFO lead: released remote=192.168.1.4:397266477worker1 # [ 124.989623] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:18Z level=info event=scheduler_restarting addr=lb:500516478worker2 # [ 124.978432] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:17Z level=info event=scheduler_restarting6479worker2 # [ 124.982068] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:17Z level=info event=scheduler_restarting addr=lb:500516480worker2 # [ 124.985876] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:592246481worker2 # [ 124.989324] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:17Z level=info event=worker_gone addr=192.168.1.4:500516482worker2 # [ 124.992485] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:592246483worker2 # [ 124.995972] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:17Z level=info event=worker_gone addr=192.168.1.3:500516484worker2 # [ 125.011951] nix-daemon[852]: reaped child process 1222, status = succeeded6485worker1 # [ 125.189638] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:18Z level=info event=scheduler_connected addr=lb:500516486worker1 # [ 125.194969] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6487worker2 # [ 125.188476] nix-grpc-daemon[1169]: ts=2026-09-23T05:13:18Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6488worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.14 seconds)6489client: waiting for success: ! systemctl is-active rs6490client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6491client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6492client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6493client: must fail: journalctl -u rs | grep -q 'reconnecting'6494client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6495lb: must succeed: curl -sf localhost:9901/clusters6496lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6497worker1 # [ 125.395151] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:18Z level=info event=scheduler_connected addr=lb:500516498worker1 # [ 125.399168] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6499worker2 # [ 125.539196] nix-grpc-daemon[1169]: E0000 00:00:1790140398.404818 1215 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6500worker2 # [ 125.544219] nix-grpc-daemon[1169]: E0000 00:00:1790140398.405292 1215 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6501worker1 # [ 125.599938] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:18Z level=info event=scheduler_connected addr=lb:500516502worker1 # [ 125.604376] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6503worker1 # [ 125.804965] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:18Z level=info event=scheduler_connected addr=lb:500516504worker1 # [ 125.809215] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6505worker1 # [ 126.010831] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=scheduler_connected addr=lb:500516506worker1 # [ 126.014992] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6507worker1 # [ 126.105453] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=scheduler_take_over6508lb # [ 126.061211] niks3-server[1116]: 2026/09/23 05:13:19 INFO lead: acquired remote=192.168.1.3:591226509worker1 # [ 126.215922] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=scheduler_connected addr=lb:500516510worker1 # [ 126.220301] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6511worker2 # [ 126.237193] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6512worker2 # [ 126.240139] systemd[1]: Stopped Nix worker-protocol over gRPC.6513worker2 # [ 126.243457] systemd[1]: nix-grpc-daemon.service: Consumed 1.266s CPU time over 33.865s wall clock time, 41.8M memory peak, 64.1K incoming IP traffic, 163.1K outgoing IP traffic.6514worker2 # [ 126.257610] systemd[1]: Starting scheduler-token.service...6515worker2 # [ 126.302085] systemd[1]: scheduler-token.service: Deactivated successfully.6516worker2 # [ 126.304070] systemd[1]: Finished scheduler-token.service.6517worker2 # [ 126.304801] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 45ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6518worker2 # [ 126.309269] systemd[1]: Starting Nix worker-protocol over gRPC...6519lb: must succeed: curl -sf localhost:9901/clusters6520worker2 # [ 126.390193] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6521worker2 # [ 126.392315] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6522worker2 # [ 126.395594] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16523worker2 # [ 126.399911] nix-grpc-daemon[1447]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6524worker2 # [ 126.401200] nix-grpc-daemon[1447]: E0000 00:00:1790140399.265761 1447 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6525lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6526worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6527worker2 # [ 126.407351] nix-grpc-daemon[1447]: E0000 00:00:1790140399.271005 1447 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6528worker1 # [ 126.421480] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=scheduler_connected addr=lb:500516529worker2 # [ 126.410771] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6530worker2 # [ 126.412679] systemd[1]: Started Nix worker-protocol over gRPC.6531worker2 # [ 126.415058] nix-grpc-daemon[1447]: E0000 00:00:1790140399.280962 1489 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6532worker1 # [ 126.430129] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:351026533worker1 # [ 126.433948] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06534worker2 # [ 126.427225] nix-grpc-daemon[1447]: E0000 00:00:1790140399.293179 1489 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6535worker2 # [ 126.431053] nix-grpc-daemon[1447]: E0000 00:00:1790140399.295368 1489 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6536worker2 # [ 126.441233] nix-grpc-daemon[1447]: E0000 00:00:1790140399.307203 1489 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6537worker2 # [ 126.450168] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:19Z level=info event=scheduler_connected addr=lb:500516538worker1 # [ 126.470751] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=oidc_jwks provider=mock keys=16539worker1 # [ 126.474104] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:351026540worker1 # [ 126.477342] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06541worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6542(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 10.17 seconds)6543subtest: scheduler restart mid-build: build finishes, no double build6544client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6545client # [ 126.500687] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6546client # Running as unit: mid.service; invocation ID: bdf5dc5c6c994eb6879bc91926d66bf46547client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6548worker1 # [ 126.672150] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:598266549client # [ 126.644244] nix[1397]: don't know how to build these paths:6550client # [ 126.645128] nix[1397]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6551worker1 # [ 126.688205] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06552worker1 # [ 126.693498] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16553worker1 # [ 126.736827] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:598266554worker1 # [ 126.748543] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)6555worker1 # [ 126.806590] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 164 ms (attempt 2/5)6556lb # [ 126.924588] envoy[912]: conn peer=[2001:db8:1::1]:35636 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1518 ms=69876557lb # [ 126.928439] envoy[912]: conn peer=[2001:db8:1::4]:57168 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=39225 tx=2130 ms=326636558worker1 # [ 126.973197] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 287 ms (attempt 3/5)6559worker1 # [ 127.262771] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 196 ms (attempt 4/5)6560worker1 # [ 127.460993] nix-grpc-daemon[1530]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6561worker1 # [ 127.464273] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6562worker1 # [ 127.481532] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:598266563client # [ 127.457760] nix[1397]: copying 1 paths to 'lb:50051'...6564client # [ 127.461673] nix[1397]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6565worker1 # [ 127.497354] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:598266566worker1 # [ 127.509125] nix-daemon[863]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6567worker1 # [ 127.531503] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:20Z level=info event=niks3_push_started pid=16466568worker1 # [ 127.637911] nix-grpc-daemon[1646]: time=2026-09-23T05:13:20.740Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6569worker1 # [ 127.739823] nix-daemon[863]: accepted connection from pid 1661, user nix-grpc-daemon (trusted)6570worker1 # [ 127.753355] nix-daemon[863]: reaped child process 1666, status = succeeded6571lb # [ 127.714775] niks3-server[1116]: 2026/09/23 05:13:20 INFO Received uploads request method=POST path=/api/pending_closures6572worker1 # [ 127.767461] nix-grpc-daemon[1646]: time=2026-09-23T05:13:20.871Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6573worker1 # [ 127.769171] nix-grpc-daemon[1646]: time=2026-09-23T05:13:20.872Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6574lb # [ 127.752817] niks3-server[1116]: 2026/09/23 05:13:20 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6575lb # [ 127.758223] niks3-server[1116]: 2026/09/23 05:13:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6576lb # [ 127.764519] niks3-server[1116]: 2026/09/23 05:13:20 INFO Signed narinfos id=51 count=16577worker1 # [ 127.811997] nix-grpc-daemon[1646]: time=2026-09-23T05:13:20.915Z level=INFO msg="Uploading 1 narinfos"6578lb # [ 127.778256] niks3-server[1116]: 2026/09/23 05:13:20 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6579lb # [ 127.786306] niks3-server[1116]: 2026/09/23 05:13:20 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6580lb # [ 127.792790] niks3-server[1116]: 2026/09/23 05:13:20 INFO Completed upload id=516581worker1 # [ 127.836974] nix-grpc-daemon[1646]: time=2026-09-23T05:13:20.940Z level=INFO msg="Upload complete. (196ms)"6582lb # [ 127.795719] niks3-server[1116]: 2026/09/23 05:13:20 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6583worker1 # [ 127.840957] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59826 duration_s=0 paths=1 nar_bytes_in=4246584worker1 # [ 127.846500] nix-grpc-daemon[1530]: ts=2026-09-23T05:13:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:598266585worker1 # [ 127.858060] nix-daemon[863]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6586worker1 # [ 127.861557] nix-daemon[863]: reaped child process 1644, status = succeeded6587worker1 # [ 127.867268] nix-daemon[863]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6588client # [ 127.832912] nix[1397]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6589lb: must succeed: curl -sf localhost:9901/clusters6590lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6591lb: must succeed: curl -sf localhost:9901/clusters6592lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6593worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6594worker1 # [ 129.079347] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1530 (nix-grpc-daemon) on client request.6595worker1 # [ 129.081092] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6596lb # [ 129.043420] niks3-server[1116]: 2026/09/23 05:13:22 INFO lead: released remote=192.168.1.3:591226597worker1 # [ 129.095814] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6598client # [ 129.059609] nix[1397]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6599worker2 # [ 129.079697] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:21Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6600worker1 # [ 129.098164] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6601worker1 # [ 129.100171] systemd[1]: nix-grpc-daemon.service: Consumed 378ms CPU time over 13.105s wall clock time, 35.6M memory peak, 15.5K incoming IP traffic, 24.9K outgoing IP traffic.6602client # [ 129.064429] nix[1397]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6603worker1 # [ 129.107967] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6604worker1 # [ 129.110313] systemd[1]: Starting Nix worker-protocol over gRPC...6605worker1 # [ 129.146505] nix-daemon[863]: reaped child process 1668, status = succeeded6606worker1 # [ 129.148596] nix-daemon[863]: reaped child process 1590, status = succeeded6607worker1 # [ 129.160770] nix-daemon[1670]: unexpected Nix daemon error: error: interrupted by the user6608worker1 # [ 129.173850] nix-daemon[863]: reaped child process 1670, status = failed with exit code 16609worker1 # [ 129.208687] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6610worker1 # [ 129.210662] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6611worker1 # [ 129.213954] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16612worker1 # [ 129.218354] nix-grpc-daemon[1702]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6613worker1 # [ 129.219707] nix-grpc-daemon[1702]: E0000 00:00:1790140402.322003 1702 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6614worker1 # [ 129.227900] nix-grpc-daemon[1702]: E0000 00:00:1790140402.327515 1702 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6615worker1 # [ 129.231112] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6616worker1 # [ 129.233980] systemd[1]: Started Nix worker-protocol over gRPC.6617lb # [ 129.190211] niks3-server[1116]: 2026/09/23 05:13:22 INFO lead: acquired remote=192.168.1.3:570706618worker1 # [ 129.237143] nix-grpc-daemon[1702]: E0000 00:00:1790140402.339427 1745 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6619worker1 # [ 129.241205] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=scheduler_take_over6620worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.20 seconds)6621client: must succeed: sleep 5.06622worker1 # [ 129.256298] nix-grpc-daemon[1702]: E0000 00:00:1790140402.359993 1745 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6623worker1 # [ 129.261058] nix-grpc-daemon[1702]: E0000 00:00:1790140402.362335 1745 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6624worker1 # [ 129.271350] nix-grpc-daemon[1702]: E0000 00:00:1790140402.375052 1745 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6625worker1 # [ 129.280227] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=scheduler_connected addr=lb:500516626worker1 # [ 129.283470] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:351186627worker1 # [ 129.285069] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06628worker2 # [ 129.280509] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:22Z level=info event=scheduler_connected addr=lb:500516629worker1 # [ 129.299073] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=oidc_jwks provider=mock keys=16630worker1 # [ 129.300523] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:351186631worker1 # [ 129.302235] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06632worker1 # [ 129.609621] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06633worker1 # [ 129.615292] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16634worker1 # [ 129.658675] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351206635worker1 # [ 129.669173] nix-daemon[863]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6636worker1 # [ 129.684354] nix-daemon[863]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6637client # [ 129.663408] nix[1397]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6638worker1 # [ 129.699624] nix-daemon[863]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6639client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6640client: waiting for success: ! systemctl is-active mid6641worker1 # [ 134.522995] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:27Z level=info event=niks3_push_started pid=17876642worker1 # [ 134.551920] nix-daemon[863]: reaped child process 1762, status = succeeded6643worker1 # [ 134.641285] nix-grpc-daemon[1787]: time=2026-09-23T05:13:27.742Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6644worker1 # [ 134.733896] nix-daemon[863]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6645worker1 # [ 134.748549] nix-daemon[863]: reaped child process 1803, status = succeeded6646lb # [ 134.714357] niks3-server[1116]: 2026/09/23 05:13:27 INFO Received uploads request method=POST path=/api/pending_closures6647worker1 # [ 134.768562] nix-grpc-daemon[1787]: time=2026-09-23T05:13:27.872Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6648worker1 # [ 134.770205] nix-grpc-daemon[1787]: time=2026-09-23T05:13:27.873Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6649lb # [ 134.765160] niks3-server[1116]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6650lb # [ 134.770089] niks3-server[1116]: 2026/09/23 05:13:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6651worker1 # [ 134.822829] nix-grpc-daemon[1787]: time=2026-09-23T05:13:27.926Z level=INFO msg="Uploading 1 narinfos"6652lb # [ 134.778393] niks3-server[1116]: 2026/09/23 05:13:27 INFO Signed narinfos id=52 count=16653lb # [ 134.798935] niks3-server[1116]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6654lb # [ 134.802091] niks3-server[1116]: 2026/09/23 05:13:27 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6655lb # [ 134.807265] niks3-server[1116]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6656lb # [ 134.811117] niks3-server[1116]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6657lb # [ 134.813915] niks3-server[1116]: 2026/09/23 05:13:28 INFO Completed upload id=526658worker1 # [ 134.858755] nix-grpc-daemon[1787]: time=2026-09-23T05:13:27.962Z level=INFO msg="Upload complete. (218ms)"6659worker1 # [ 134.866543] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35120 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16660worker1 # [ 134.883504] nix-daemon[863]: reaped child process 1760, status = succeeded6661client # [ 134.869555] systemd[1]: mid.service: Deactivated successfully.6662client # [ 134.873120] systemd[1]: mid.service: Consumed 141ms CPU time over 8.368s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.4K outgoing IP traffic.6663client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6664client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6665client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6666(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.09 seconds)6667subtest: interrupting the client stops the build on the worker6668client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6669client # [ 135.564472] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6670client # Running as unit: intr.service; invocation ID: d066dad26254488a8f7d6afb5e67740b6671client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6672worker1 # [ 135.717490] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:351206673client # [ 135.689771] nix[1433]: don't know how to build these paths:6674client # [ 135.690695] nix[1433]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6675worker1 # [ 135.733158] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:28Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06676worker1 # [ 135.737393] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:28Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16677worker2 # [ 135.771847] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530486678worker2 # [ 135.783782] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6679worker2 # [ 135.800772] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)6680worker2 # [ 135.889763] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)6681worker2 # [ 136.006900] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 3/5)6682worker2 # [ 136.041749] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 251 ms (attempt 4/5)6683worker2 # [ 136.295489] nix-grpc-daemon[1447]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6684worker2 # [ 136.299157] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6685worker2 # [ 136.315353] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530486686client # [ 136.306268] nix[1433]: copying 1 paths to 'lb:50051'...6687worker2 # [ 136.327749] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530486688client # [ 136.308918] nix[1433]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6689worker2 # [ 136.338307] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6690worker2 # [ 136.363367] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:29Z level=info event=niks3_push_started pid=15366691worker2 # [ 136.472084] nix-grpc-daemon[1536]: time=2026-09-23T05:13:29.336Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6692worker2 # [ 136.561312] nix-daemon[852]: accepted connection from pid 1549, user nix-grpc-daemon (trusted)6693worker2 # [ 136.574329] nix-daemon[852]: reaped child process 1552, status = succeeded6694lb # [ 136.552248] niks3-server[1116]: 2026/09/23 05:13:29 INFO Received uploads request method=POST path=/api/pending_closures6695worker2 # [ 136.592975] nix-grpc-daemon[1536]: time=2026-09-23T05:13:29.458Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6696worker2 # [ 136.594511] nix-grpc-daemon[1536]: time=2026-09-23T05:13:29.460Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6697lb # [ 136.591580] niks3-server[1116]: 2026/09/23 05:13:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6698worker2 # [ 136.634242] nix-grpc-daemon[1536]: time=2026-09-23T05:13:29.500Z level=INFO msg="Uploading 1 narinfos"6699lb # [ 136.604211] niks3-server[1116]: 2026/09/23 05:13:29 INFO Signed narinfos id=53 count=16700lb # [ 136.608582] niks3-server[1116]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6701lb # [ 136.614154] niks3-server[1116]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6702lb # [ 136.625130] niks3-server[1116]: 2026/09/23 05:13:29 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6703lb # [ 136.630190] niks3-server[1116]: 2026/09/23 05:13:29 INFO Completed upload id=536704worker2 # [ 136.660558] nix-grpc-daemon[1536]: time=2026-09-23T05:13:29.526Z level=INFO msg="Upload complete. (186ms)"6705worker2 # [ 136.663857] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 paths=1 nar_bytes_in=4246706lb # [ 136.635985] niks3-server[1116]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6707worker2 # [ 136.669140] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530486708worker2 # [ 136.678829] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6709worker2 # [ 136.683744] nix-daemon[852]: reaped child process 1534, status = succeeded6710worker2 # [ 136.687898] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6711client # [ 136.667487] nix[1433]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6712lb # [ 136.925449] envoy[912]: conn peer=[2001:db8:1::3]:44302 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2031 tx=556 ms=129566713lb # [ 136.929350] envoy[912]: conn peer=[2001:db8:1::1]:38296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3921 tx=1939 ms=82156714client: must succeed: systemctl kill -s INT intr6715client # [ 137.880729] systemd[1]: intr.service: Sent signal SIGINT to main process 1433 (nix) on client request.6716worker2 # [ 137.904219] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:30Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6717client # [ 137.886868] nix[1433]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6718client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6719client # [ 137.893820] nix[1433]: error: interrupted by the user6720client # [ 137.920140] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6721client # [ 137.923168] systemd[1]: intr.service: Failed with result 'exit-code'.6722client # [ 137.926402] systemd[1]: intr.service: Consumed 115ms CPU time over 2.358s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.6723worker2 # [ 138.106456] nix-daemon[852]: reaped child process 1554, status = succeeded6724worker2 # [ 138.108408] nix-daemon[1556]: unexpected Nix daemon error: error: interrupted by the user6725worker2 # [ 138.112665] nix-daemon[852]: reaped child process 1556, status = failed with exit code 16726(finished: subtest: interrupting the client stops the build on the worker, in 3.54 seconds)6727subtest: a deploy mid-build drains: the build finishes, the new generation takes over6728client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6729client # [ 139.127920] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6730client # Running as unit: sw.service; invocation ID: 6d40d62dec5d4435ab6b0f943990bcba6731client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6732client # [ 139.265222] nix[1454]: don't know how to build these paths:6733client # [ 139.266095] nix[1454]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6734worker1 # [ 139.299510] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:351206735worker1 # [ 139.310204] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:32Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06736worker1 # [ 139.315960] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:32Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16737worker1 # [ 139.359272] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351206738worker1 # [ 139.370990] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)6739worker1 # [ 139.432837] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 2/5)6740worker1 # [ 139.456346] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 3/5)6741worker1 # [ 139.466903] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 478 ms (attempt 4/5)6742worker1 # [ 139.953267] nix-grpc-daemon[1702]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6743worker1 # [ 139.957363] nix-grpc-daemon[1702]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6744worker1 # [ 139.975371] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351206745client # [ 139.950961] nix[1454]: copying 1 paths to 'lb:50051'...6746client # [ 139.955178] nix[1454]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6747worker1 # [ 139.989834] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351206748worker1 # [ 140.001928] nix-daemon[863]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6749worker1 # [ 140.112648] nix-daemon[863]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)6750worker1 # [ 140.128054] nix-daemon[863]: reaped child process 1842, status = succeeded6751lb # [ 140.091769] niks3-server[1116]: 2026/09/23 05:13:33 INFO Received uploads request method=POST path=/api/pending_closures6752worker1 # [ 140.146612] nix-grpc-daemon[1787]: time=2026-09-23T05:13:33.250Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6753worker1 # [ 140.149062] nix-grpc-daemon[1787]: time=2026-09-23T05:13:33.251Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6754lb # [ 140.125756] niks3-server[1116]: 2026/09/23 05:13:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6755worker1 # [ 140.177968] nix-grpc-daemon[1787]: time=2026-09-23T05:13:33.281Z level=INFO msg="Uploading 1 narinfos"6756lb # [ 140.133236] niks3-server[1116]: 2026/09/23 05:13:33 INFO Signed narinfos id=54 count=16757lb # [ 140.140156] niks3-server[1116]: 2026/09/23 05:13:33 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6758lb # [ 140.147147] niks3-server[1116]: 2026/09/23 05:13:33 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6759lb # [ 140.157122] niks3-server[1116]: 2026/09/23 05:13:33 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6760lb # [ 140.169924] niks3-server[1116]: 2026/09/23 05:13:33 INFO Completed upload id=546761worker1 # [ 140.214639] nix-grpc-daemon[1787]: time=2026-09-23T05:13:33.318Z level=INFO msg="Upload complete. (200ms)"6762lb # [ 140.172366] niks3-server[1116]: 2026/09/23 05:13:33 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6763worker1 # [ 140.219553] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35120 duration_s=0 paths=1 nar_bytes_in=4166764worker1 # [ 140.226340] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351206765worker1 # [ 140.240124] nix-daemon[863]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6766worker1 # [ 140.249449] nix-daemon[863]: reaped child process 1837, status = succeeded6767worker1 # [ 140.254173] nix-daemon[863]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6768client # [ 140.219456] nix[1454]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6769worker1: must succeed: systemctl reload nix-grpc-daemon.service6770worker1 # [ 141.582205] systemd[1]: Reloading Nix worker-protocol over gRPC...6771worker1 # [ 141.609639] systemd[1]: Reloaded Nix worker-protocol over gRPC.6772worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6773worker1: must succeed: pgrep -f 'read -t [9]0 x'6774worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6775client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26776worker2 # [ 141.754848] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:530486777client # don't know how to build these paths:6778client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6779worker1 # [ 141.778496] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351206780worker1 # [ 141.797262] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:34Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06781worker1 # [ 141.799100] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:34Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16782worker2 # [ 141.829080] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530486783worker2 # [ 141.840973] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530486784client # copying 1 paths to 'lb:50051'...6785client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6786worker2 # [ 141.852465] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530486787worker2 # [ 141.865475] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6788worker2 # [ 141.968120] nix-daemon[852]: accepted connection from pid 1607, user nix-grpc-daemon (trusted)6789worker2 # [ 141.981218] nix-daemon[852]: reaped child process 1610, status = succeeded6790lb # [ 141.961832] niks3-server[1116]: 2026/09/23 05:13:35 INFO Received uploads request method=POST path=/api/pending_closures6791worker2 # [ 141.999326] nix-grpc-daemon[1536]: time=2026-09-23T05:13:34.865Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6792worker2 # [ 142.001570] nix-grpc-daemon[1536]: time=2026-09-23T05:13:34.866Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6793lb # [ 141.993130] niks3-server[1116]: 2026/09/23 05:13:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6794worker2 # [ 142.032231] nix-grpc-daemon[1536]: time=2026-09-23T05:13:34.898Z level=INFO msg="Uploading 1 narinfos"6795lb # [ 142.002161] niks3-server[1116]: 2026/09/23 05:13:35 INFO Signed narinfos id=55 count=16796lb # [ 142.007505] niks3-server[1116]: 2026/09/23 05:13:35 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6797lb # [ 142.013362] niks3-server[1116]: 2026/09/23 05:13:35 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6798lb # [ 142.022727] niks3-server[1116]: 2026/09/23 05:13:35 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6799lb # [ 142.030888] niks3-server[1116]: 2026/09/23 05:13:35 INFO Completed upload id=556800worker2 # [ 142.060888] nix-grpc-daemon[1536]: time=2026-09-23T05:13:34.926Z level=INFO msg="Upload complete. (183ms)"6801lb # [ 142.034165] niks3-server[1116]: 2026/09/23 05:13:35 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6802worker2 # [ 142.065145] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 paths=1 nar_bytes_in=5286803worker2 # [ 142.069894] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530486804worker2 # [ 142.081115] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6805worker2 # [ 142.086772] nix-daemon[852]: reaped child process 1605, status = succeeded6806worker2 # [ 142.090972] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6807client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6808client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6809worker1 # [ 142.232135] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:35Z level=info event=draining6810client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6811client # farm-a> LOG-farm-a-during-drain6812worker2 # [ 143.326772] nix-daemon[852]: reaped child process 1614, status = succeeded6813worker2 # [ 143.415241] nix-daemon[852]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6814worker2 # [ 143.429780] nix-daemon[852]: reaped child process 1633, status = succeeded6815lb # [ 143.406225] niks3-server[1116]: 2026/09/23 05:13:36 INFO Received uploads request method=POST path=/api/pending_closures6816worker2 # [ 143.449872] nix-grpc-daemon[1536]: time=2026-09-23T05:13:36.315Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6817worker2 # [ 143.451346] nix-grpc-daemon[1536]: time=2026-09-23T05:13:36.317Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6818lb # [ 143.456259] niks3-server[1116]: 2026/09/23 05:13:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6819worker2 # [ 143.493093] nix-grpc-daemon[1536]: time=2026-09-23T05:13:36.359Z level=INFO msg="Uploading 1 narinfos"6820lb # [ 143.462676] niks3-server[1116]: 2026/09/23 05:13:36 INFO Signed narinfos id=56 count=16821lb # [ 143.476851] niks3-server[1116]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6822lb # [ 143.480985] niks3-server[1116]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6823lb # [ 143.487653] niks3-server[1116]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6824lb # [ 143.495631] niks3-server[1116]: 2026/09/23 05:13:36 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6825lb # [ 143.499602] niks3-server[1116]: 2026/09/23 05:13:36 INFO Completed upload id=566826worker2 # [ 143.530312] nix-grpc-daemon[1536]: time=2026-09-23T05:13:36.396Z level=INFO msg="Upload complete. (231ms)"6827lb # [ 143.501669] niks3-server[1116]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6828worker2 # [ 143.536548] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16829worker1 # [ 143.555844] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:36Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06830worker1 # [ 143.558233] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:36Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16831worker2 # [ 143.553134] nix-daemon[852]: reaped child process 1612, status = succeeded6832worker2 # [ 143.586523] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530486833worker2 # [ 143.599327] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530486834worker2 # [ 143.612949] nix-grpc-daemon[1536]: time=2026-09-23T05:13:36.478Z level=INFO msg="All 1 paths already cached"6835client # copying 1 paths to 'lb:50051'...6836worker2 # [ 143.621720] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530486837client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6838worker2 # [ 143.647231] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6839worker2 # [ 143.751562] nix-daemon[852]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)6840worker2 # [ 143.764682] nix-daemon[852]: reaped child process 1640, status = succeeded6841lb # [ 143.743740] niks3-server[1116]: 2026/09/23 05:13:36 INFO Received uploads request method=POST path=/api/pending_closures6842worker2 # [ 143.782559] nix-grpc-daemon[1536]: time=2026-09-23T05:13:36.648Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6843worker2 # [ 143.784140] nix-grpc-daemon[1536]: time=2026-09-23T05:13:36.650Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6844lb # [ 143.776258] niks3-server[1116]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6845lb # [ 143.783472] niks3-server[1116]: 2026/09/23 05:13:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6846lb # [ 143.788497] niks3-server[1116]: 2026/09/23 05:13:36 INFO Signed narinfos id=57 count=16847worker2 # [ 143.821920] nix-grpc-daemon[1536]: time=2026-09-23T05:13:36.687Z level=INFO msg="Uploading 1 narinfos"6848lb # [ 143.802239] niks3-server[1116]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6849lb # [ 143.809788] niks3-server[1116]: 2026/09/23 05:13:36 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6850lb # [ 143.816441] niks3-server[1116]: 2026/09/23 05:13:37 INFO Completed upload id=576851worker2 # [ 143.847050] nix-grpc-daemon[1536]: time=2026-09-23T05:13:36.712Z level=INFO msg="Upload complete. (186ms)"6852lb # [ 143.820277] niks3-server[1116]: 2026/09/23 05:13:37 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6853worker2 # [ 143.851681] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 paths=1 nar_bytes_in=6406854worker2 # [ 143.857545] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530486855worker2 # [ 143.869111] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6856worker2 # [ 143.876248] nix-daemon[852]: reaped child process 1635, status = succeeded6857worker2 # [ 143.881405] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6858client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6859client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6860client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6861client # farm-b> LOG-farm-b-during-drain6862worker2 # [ 144.920309] nix-daemon[852]: reaped child process 1644, status = succeeded6863worker2 # [ 144.997843] nix-daemon[852]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6864worker2 # [ 145.010849] nix-daemon[852]: reaped child process 1663, status = succeeded6865lb # [ 144.991711] niks3-server[1116]: 2026/09/23 05:13:38 INFO Received uploads request method=POST path=/api/pending_closures6866worker2 # [ 145.029398] nix-grpc-daemon[1536]: time=2026-09-23T05:13:37.895Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6867worker2 # [ 145.031770] nix-grpc-daemon[1536]: time=2026-09-23T05:13:37.896Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6868lb # [ 145.028712] niks3-server[1116]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6869lb # [ 145.037925] niks3-server[1116]: 2026/09/23 05:13:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6870lb # [ 145.048782] niks3-server[1116]: 2026/09/23 05:13:38 INFO Signed narinfos id=58 count=16871worker2 # [ 145.082160] nix-grpc-daemon[1536]: time=2026-09-23T05:13:37.947Z level=INFO msg="Uploading 1 narinfos"6872lb # [ 145.051615] niks3-server[1116]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6873lb # [ 145.058403] niks3-server[1116]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6874lb # [ 145.070107] niks3-server[1116]: 2026/09/23 05:13:38 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6875lb # [ 145.074674] niks3-server[1116]: 2026/09/23 05:13:38 INFO Completed upload id=586876lb # [ 145.077239] niks3-server[1116]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6877worker2 # [ 145.106373] nix-grpc-daemon[1536]: time=2026-09-23T05:13:37.972Z level=INFO msg="Upload complete. (217ms)"6878worker2 # [ 145.112888] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16879worker1 # [ 145.131595] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:38Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06880worker1 # [ 145.134262] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:38Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16881worker2 # [ 145.129454] nix-daemon[852]: reaped child process 1642, status = succeeded6882worker2 # [ 145.160953] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530486883worker2 # [ 145.173417] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530486884worker2 # [ 145.186324] nix-grpc-daemon[1536]: time=2026-09-23T05:13:38.052Z level=INFO msg="All 2 paths already cached"6885client # copying 1 paths to 'lb:50051'...6886worker2 # [ 145.195411] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530486887client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6888worker2 # [ 145.207668] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6889worker2 # [ 145.313065] nix-daemon[852]: accepted connection from pid 1667, user nix-grpc-daemon (trusted)6890worker2 # [ 145.328042] nix-daemon[852]: reaped child process 1670, status = succeeded6891lb # [ 145.303838] niks3-server[1116]: 2026/09/23 05:13:38 INFO Received uploads request method=POST path=/api/pending_closures6892worker2 # [ 145.343058] nix-grpc-daemon[1536]: time=2026-09-23T05:13:38.209Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6893worker2 # [ 145.344657] nix-grpc-daemon[1536]: time=2026-09-23T05:13:38.210Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6894lb # [ 145.337996] niks3-server[1116]: 2026/09/23 05:13:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6895worker2 # [ 145.373876] nix-grpc-daemon[1536]: time=2026-09-23T05:13:38.239Z level=INFO msg="Uploading 1 narinfos"6896lb # [ 145.343263] niks3-server[1116]: 2026/09/23 05:13:38 INFO Signed narinfos id=59 count=16897lb # [ 145.349942] niks3-server[1116]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6898lb # [ 145.359668] niks3-server[1116]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6899lb # [ 145.368156] niks3-server[1116]: 2026/09/23 05:13:38 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6900lb # [ 145.373703] niks3-server[1116]: 2026/09/23 05:13:38 INFO Completed upload id=596901lb # [ 145.377813] niks3-server[1116]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6902worker2 # [ 145.406672] nix-grpc-daemon[1536]: time=2026-09-23T05:13:38.272Z level=INFO msg="Upload complete. (184ms)"6903worker2 # [ 145.411572] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 paths=1 nar_bytes_in=7686904worker2 # [ 145.419267] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530486905worker2 # [ 145.428538] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6906worker2 # [ 145.438173] nix-daemon[852]: reaped child process 1665, status = succeeded6907worker2 # [ 145.442838] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6908client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6909client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6910client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6911client # farm-top> LOG-farm-top-during-drain6912worker2 # [ 146.688406] nix-daemon[852]: reaped child process 1674, status = succeeded6913worker2 # [ 146.776375] nix-daemon[852]: accepted connection from pid 1690, user nix-grpc-daemon (trusted)6914worker2 # [ 146.792859] nix-daemon[852]: reaped child process 1693, status = succeeded6915lb # [ 146.768456] niks3-server[1116]: 2026/09/23 05:13:39 INFO Received uploads request method=POST path=/api/pending_closures6916worker2 # [ 146.807281] nix-grpc-daemon[1536]: time=2026-09-23T05:13:39.673Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6917worker2 # [ 146.808824] nix-grpc-daemon[1536]: time=2026-09-23T05:13:39.674Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6918lb # [ 146.808950] niks3-server[1116]: 2026/09/23 05:13:39 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6919lb # [ 146.817216] niks3-server[1116]: 2026/09/23 05:13:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6920lb # [ 146.822346] niks3-server[1116]: 2026/09/23 05:13:40 INFO Signed narinfos id=60 count=16921worker2 # [ 146.856134] nix-grpc-daemon[1536]: time=2026-09-23T05:13:39.721Z level=INFO msg="Uploading 1 narinfos"6922lb # [ 146.829797] niks3-server[1116]: 2026/09/23 05:13:40 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6923lb # [ 146.839123] niks3-server[1116]: 2026/09/23 05:13:40 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6924lb # [ 146.848589] niks3-server[1116]: 2026/09/23 05:13:40 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6925lb # [ 146.853809] niks3-server[1116]: 2026/09/23 05:13:40 INFO Completed upload id=606926worker2 # [ 146.884481] nix-grpc-daemon[1536]: time=2026-09-23T05:13:39.750Z level=INFO msg="Upload complete. (233ms)"6927lb # [ 146.855850] niks3-server[1116]: 2026/09/23 05:13:40 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6928worker2 # [ 146.890674] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16929worker2 # [ 146.906164] nix-daemon[852]: reaped child process 1672, status = succeeded6930client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.29 seconds)6931client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6932lb # [ 146.925320] envoy[912]: conn peer=[2001:db8:1::1]:57018 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2967 tx=829 ms=22236933lb # [ 146.929129] envoy[912]: conn peer=[2001:db8:1::1]:57028 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9594 tx=4379 ms=51566934client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)6935worker1: must succeed: pkill -f 'read -t [9]0 x'6936worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6937client: waiting for success: ! systemctl is-active sw6938worker1 # [ 147.316226] nix-daemon[863]: reaped child process 1846, status = succeeded6939worker1 # [ 147.372980] nix-daemon[863]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6940worker1 # [ 147.386212] nix-daemon[863]: reaped child process 1897, status = succeeded6941lb # [ 147.348157] niks3-server[1116]: 2026/09/23 05:13:40 INFO Received uploads request method=POST path=/api/pending_closures6942worker1 # [ 147.402258] nix-grpc-daemon[1787]: time=2026-09-23T05:13:40.505Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6943worker1 # [ 147.403767] nix-grpc-daemon[1787]: time=2026-09-23T05:13:40.507Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6944lb # [ 147.387121] niks3-server[1116]: 2026/09/23 05:13:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6945worker1 # [ 147.439772] nix-grpc-daemon[1787]: time=2026-09-23T05:13:40.543Z level=INFO msg="Uploading 1 narinfos"6946lb # [ 147.395132] niks3-server[1116]: 2026/09/23 05:13:40 INFO Signed narinfos id=61 count=16947lb # [ 147.406490] niks3-server[1116]: 2026/09/23 05:13:40 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6948lb # [ 147.412111] niks3-server[1116]: 2026/09/23 05:13:40 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6949lb # [ 147.418260] niks3-server[1116]: 2026/09/23 05:13:40 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6950lb # [ 147.425887] niks3-server[1116]: 2026/09/23 05:13:40 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6951lb # [ 147.430427] niks3-server[1116]: 2026/09/23 05:13:40 INFO Completed upload id=616952lb # [ 147.432843] niks3-server[1116]: 2026/09/23 05:13:40 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6953worker1 # [ 147.476638] nix-grpc-daemon[1787]: time=2026-09-23T05:13:40.580Z level=INFO msg="Upload complete. (208ms)"6954worker1 # [ 147.485361] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35120 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16955worker1 # [ 147.499221] nix-daemon[863]: reaped child process 1844, status = succeeded6956client # [ 147.478458] systemd[1]: sw.service: Deactivated successfully.6957client # [ 147.482631] systemd[1]: sw.service: Consumed 129ms CPU time over 8.353s wall clock time, 41.4M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic.6958client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6959client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6960client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6961worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6962worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.04 seconds)6963lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6964lb: must succeed: curl -sf localhost:9901/clusters6965worker1 # [ 148.234120] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:41Z level=info event=signal_exit6966lb # [ 148.197857] niks3-server[1116]: 2026/09/23 05:13:41 INFO lead: released remote=192.168.1.3:570706967lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6968worker2 # [ 149.229343] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:42Z level=info event=scheduler_restarting addr=lb:500516969worker1 # [ 149.246400] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:42Z level=info event=scheduler_restarting6970worker1 # [ 149.250530] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:351186971worker1 # [ 149.254261] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:42Z level=info event=worker_gone addr=192.168.1.4:500516972worker1 # [ 149.257838] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:42Z level=info event=scheduler_restarting addr=lb:500516973worker1 # [ 149.261358] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:351186974worker1 # [ 149.264848] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:42Z level=info event=worker_gone addr=192.168.1.3:500516975lb: must succeed: curl -sf localhost:9901/clusters6976worker1 # [ 149.280554] nix-daemon[863]: reaped child process 1758, status = succeeded6977worker1 # [ 149.289664] nix-grpc-daemon[1702]: E0000 00:00:1790140422.393247 1749 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6978worker1 # [ 149.294906] nix-grpc-daemon[1702]: E0000 00:00:1790140422.393345 1749 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6979lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6980worker2 # [ 149.430186] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500516981worker2 # [ 149.434416] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6982worker1 # [ 149.457282] nix-grpc-daemon[1702]: ts=2026-09-23T05:13:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6983worker2 # [ 149.636133] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500516984worker2 # [ 149.640238] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6985worker2 # [ 149.841404] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500516986worker2 # [ 149.845469] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6987worker2 # [ 150.047137] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:42Z level=info event=scheduler_connected addr=lb:500516988worker2 # [ 150.051338] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6989worker2 # [ 150.253391] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500516990worker2 # [ 150.257439] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6991lb: must succeed: curl -sf localhost:9901/clusters6992lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6993worker2 # [ 150.458616] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500516994worker2 # [ 150.462769] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6995worker2 # [ 150.664156] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500516996worker2 # [ 150.668283] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6997worker2 # [ 150.869665] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500516998worker2 # [ 150.873795] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6999worker2 # [ 151.075141] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_connected addr=lb:500517000worker2 # [ 151.079195] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7001worker1 # [ 151.262892] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.7002worker1 # [ 151.266588] systemd[1]: nix-grpc-daemon.service: Consumed 632ms CPU time over 22.166s wall clock time, 36M memory peak, 24.9K incoming IP traffic, 32.6K outgoing IP traffic.7003worker2 # [ 151.282222] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_connected addr=lb:500517004worker2 # [ 151.285549] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7005lb: must succeed: curl -sf localhost:9901/clusters7006lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7007lb # [ 151.386126] niks3-server[1116]: 2026/09/23 05:13:44 INFO lead: acquired remote=192.168.1.4:509487008worker2 # [ 151.416491] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_take_over7009worker2 # [ 151.483339] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_connected addr=lb:500517010worker2 # [ 151.487466] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7011worker2 # [ 151.688904] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_connected addr=lb:500517012worker2 # [ 151.693271] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7013worker2 # [ 151.894435] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_connected addr=lb:500517014worker2 # [ 151.898574] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7015worker2 # [ 152.099967] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_connected addr=lb:500517016worker2 # [ 152.104198] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7017worker1 # [ 152.259980] systemd[1]: Starting Nix worker-protocol over gRPC...7018worker2 # [ 152.306352] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=info event=scheduler_connected addr=lb:500517019worker2 # [ 152.325126] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=info event=oidc_jwks provider=mock keys=17020worker2 # [ 152.328964] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:334427021worker2 # [ 152.332934] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07022worker1 # [ 152.362376] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"7023worker1 # [ 152.364637] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc7024worker1 # [ 152.368055] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=17025worker1 # [ 152.372367] nix-grpc-daemon[1909]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR7026worker1 # [ 152.373743] nix-grpc-daemon[1909]: E0000 00:00:1790140425.476020 1909 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7027worker1 # [ 152.379415] nix-grpc-daemon[1909]: E0000 00:00:1790140425.481642 1909 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7028worker1 # [ 152.383164] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket7029worker1 # [ 152.384908] systemd[1]: Started Nix worker-protocol over gRPC.7030worker1 # [ 152.391192] nix-grpc-daemon[1909]: E0000 00:00:1790140425.494955 1951 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7031worker1 # [ 152.405120] nix-grpc-daemon[1909]: E0000 00:00:1790140425.508798 1951 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7032worker1 # [ 152.410057] nix-grpc-daemon[1909]: E0000 00:00:1790140425.511114 1951 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7033lb: must succeed: curl -sf localhost:9901/clusters7034worker1 # [ 152.419346] nix-grpc-daemon[1909]: E0000 00:00:1790140425.523047 1951 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7035worker1 # [ 152.431893] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:45Z level=info event=scheduler_connected addr=lb:500517036worker2 # [ 152.419880] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:334427037worker2 # [ 152.425972] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07038lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7039lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 4.25 seconds)7040lb: must succeed: curl -sf localhost:9901/clusters7041lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7042worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7043worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7044(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.42 seconds)7045subtest: queue gauges show a burst while every build blocks7046lb: must succeed: curl -sf localhost:9901/clusters7047lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7048lb: must succeed: curl -sf localhost:9901/clusters7049lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7050worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7051worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7052client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'7053client # [ 152.648684] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".7054client # Running as unit: burst.service; invocation ID: f473a846af9247df82fd196fb4c67c417055client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)7056lb: must succeed: curl -sf localhost:9901/clusters7057lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7058lb: must succeed: curl -sf localhost:9901/clusters7059lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7060worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7061worker1 # [ 152.819866] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:419267062worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7063worker1 # [ 152.831966] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7064client # [ 152.809776] nix[1510]: don't know how to build these paths:7065client # [ 152.810811] nix[1510]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7066client # [ 152.812351] nix[1510]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7067client # [ 152.813427] nix[1510]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7068client # [ 152.814464] nix[1510]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7069worker2 # [ 152.841674] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=debug event=want client=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=07070worker2 # [ 152.848191] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=17071worker2 # [ 152.852603] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=debug event=want client=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=07072worker2 # [ 152.856469] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=17073worker2 # [ 152.860762] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=debug event=want client=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=07074worker2 # [ 152.864765] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=debug event=want client=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=07075worker2 # [ 152.892137] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487076worker1 # [ 152.905956] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267077worker1 # [ 152.917960] nix-grpc-daemon[1909]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)7078worker2 # [ 152.905675] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530487079worker1 # [ 152.926544] nix-grpc-daemon[1909]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)7080client # [ 152.896224] nix[1510]: copying 1 paths to 'lb:50051'...7081worker2 # [ 152.917716] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530487082client # [ 152.899252] nix[1510]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...7083worker2 # [ 152.929236] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7084worker2 # [ 153.031425] nix-daemon[852]: accepted connection from pid 1709, user nix-grpc-daemon (trusted)7085worker2 # [ 153.046602] nix-daemon[852]: reaped child process 1712, status = succeeded7086lb # [ 153.023371] niks3-server[1116]: 2026/09/23 05:13:46 INFO Received uploads request method=POST path=/api/pending_closures7087worker2 # [ 153.063502] nix-grpc-daemon[1536]: time=2026-09-23T05:13:45.929Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7088worker2 # [ 153.065496] nix-grpc-daemon[1536]: time=2026-09-23T05:13:45.931Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"7089worker1 # [ 153.078699] nix-grpc-daemon[1909]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 310 ms (attempt 3/5)7090lb # [ 153.053706] niks3-server[1116]: 2026/09/23 05:13:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7091lb # [ 153.061378] niks3-server[1116]: 2026/09/23 05:13:46 INFO Signed narinfos id=62 count=17092worker2 # [ 153.094795] nix-grpc-daemon[1536]: time=2026-09-23T05:13:45.960Z level=INFO msg="Uploading 1 narinfos"7093lb # [ 153.071232] niks3-server[1116]: 2026/09/23 05:13:46 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls7094lb # [ 153.079219] niks3-server[1116]: 2026/09/23 05:13:46 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst7095lb # [ 153.088719] niks3-server[1116]: 2026/09/23 05:13:46 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7096lb # [ 153.096159] niks3-server[1116]: 2026/09/23 05:13:46 INFO Completed upload id=627097worker2 # [ 153.126808] nix-grpc-daemon[1536]: time=2026-09-23T05:13:45.992Z level=INFO msg="Upload complete. (185ms)"7098lb # [ 153.098656] niks3-server[1116]: 2026/09/23 05:13:46 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo7099worker2 # [ 153.131185] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 paths=1 nar_bytes_in=4167100worker2 # [ 153.136277] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487101worker2 # [ 153.147161] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7102worker2 # [ 153.155156] nix-daemon[852]: reaped child process 1707, status = succeeded7103worker2 # [ 153.160959] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7104client # [ 153.139305] nix[1510]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...7105worker1 # [ 153.391427] nix-grpc-daemon[1909]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 276 ms (attempt 4/5)7106worker1 # [ 153.669194] nix-grpc-daemon[1909]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7107worker1 # [ 153.672739] nix-grpc-daemon[1909]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7108worker1 # [ 153.689499] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267109client # [ 153.666211] nix[1510]: copying 1 paths to 'lb:50051'...7110client # [ 153.668110] nix[1510]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...7111worker1 # [ 153.701689] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267112worker1 # [ 153.711173] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7113worker1 # [ 153.734441] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:46Z level=info event=niks3_push_started pid=19707114lb: must succeed: curl -sf localhost:9901/clusters7115worker1 # [ 153.839538] nix-grpc-daemon[1970]: time=2026-09-23T05:13:46.942Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""7116lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7117lb: must succeed: curl -sf localhost:9901/clusters7118lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7119worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7120worker1 # [ 153.928640] nix-daemon[863]: accepted connection from pid 1983, user nix-grpc-daemon (trusted)7121worker1 # [ 153.940262] nix-daemon[863]: reaped child process 1986, status = succeeded7122lb # [ 153.903780] niks3-server[1116]: 2026/09/23 05:13:47 INFO Received uploads request method=POST path=/api/pending_closures7123worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7124worker1 # [ 153.956353] nix-grpc-daemon[1970]: time=2026-09-23T05:13:47.060Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7125worker1 # [ 153.959864] nix-grpc-daemon[1970]: time=2026-09-23T05:13:47.061Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"7126lb # [ 153.940050] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst7127lb # [ 153.945140] niks3-server[1116]: 2026/09/23 05:13:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7128worker1 # [ 153.994095] nix-grpc-daemon[1970]: time=2026-09-23T05:13:47.097Z level=INFO msg="Uploading 1 narinfos"7129lb # [ 153.949212] niks3-server[1116]: 2026/09/23 05:13:47 INFO Signed narinfos id=63 count=17130lb # [ 153.957079] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls7131lb # [ 153.962584] niks3-server[1116]: 2026/09/23 05:13:47 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7132lb # [ 153.965975] niks3-server[1116]: 2026/09/23 05:13:47 INFO Completed upload id=637133worker1 # [ 154.011610] nix-grpc-daemon[1970]: time=2026-09-23T05:13:47.115Z level=INFO msg="Upload complete. (168ms)"7134lb # [ 153.968080] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo7135worker1 # [ 154.013875] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=4167136worker1 # [ 154.017126] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267137worker1 # [ 154.024107] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7138worker1 # [ 154.027255] nix-daemon[863]: reaped child process 1968, status = succeeded7139worker1 # [ 154.032092] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7140client # [ 153.999096] nix[1510]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...7141worker2 # [ 154.202889] nix-daemon[852]: reaped child process 1716, status = succeeded7142worker2 # [ 154.284541] nix-daemon[852]: accepted connection from pid 1740, user nix-grpc-daemon (trusted)7143worker2 # [ 154.300243] nix-daemon[852]: reaped child process 1743, status = succeeded7144lb # [ 154.277942] niks3-server[1116]: 2026/09/23 05:13:47 INFO Received uploads request method=POST path=/api/pending_closures7145worker2 # [ 154.316821] nix-grpc-daemon[1536]: time=2026-09-23T05:13:47.182Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7146worker2 # [ 154.318723] nix-grpc-daemon[1536]: time=2026-09-23T05:13:47.184Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"7147lb # [ 154.322535] niks3-server[1116]: 2026/09/23 05:13:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7148worker2 # [ 154.359477] nix-grpc-daemon[1536]: time=2026-09-23T05:13:47.225Z level=INFO msg="Uploading 1 narinfos"7149lb # [ 154.328858] niks3-server[1116]: 2026/09/23 05:13:47 INFO Signed narinfos id=64 count=17150lb # [ 154.335493] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7151lb # [ 154.340172] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls7152lb # [ 154.346240] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst7153lb # [ 154.353933] niks3-server[1116]: 2026/09/23 05:13:47 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7154lb # [ 154.358695] niks3-server[1116]: 2026/09/23 05:13:47 INFO Completed upload id=647155lb # [ 154.360222] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo7156worker2 # [ 154.389242] nix-grpc-daemon[1536]: time=2026-09-23T05:13:47.255Z level=INFO msg="Upload complete. (222ms)"7157worker2 # [ 154.400321] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=1 outputs=17158worker2 # [ 154.405283] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:47Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=17159worker2 # [ 154.416421] nix-daemon[852]: reaped child process 1714, status = succeeded7160worker2 # [ 154.447469] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487161worker2 # [ 154.460435] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530487162client # [ 154.450331] nix[1510]: copying 1 paths to 'lb:50051'...7163worker2 # [ 154.471863] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530487164client # [ 154.453150] nix[1510]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...7165worker2 # [ 154.484142] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7166worker2 # [ 154.586895] nix-daemon[852]: accepted connection from pid 1747, user nix-grpc-daemon (trusted)7167worker2 # [ 154.601761] nix-daemon[852]: reaped child process 1750, status = succeeded7168lb # [ 154.578721] niks3-server[1116]: 2026/09/23 05:13:47 INFO Received uploads request method=POST path=/api/pending_closures7169worker2 # [ 154.617330] nix-grpc-daemon[1536]: time=2026-09-23T05:13:47.483Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7170worker2 # [ 154.618858] nix-grpc-daemon[1536]: time=2026-09-23T05:13:47.484Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"7171lb # [ 154.610806] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst7172lb # [ 154.618170] niks3-server[1116]: 2026/09/23 05:13:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7173lb # [ 154.622742] niks3-server[1116]: 2026/09/23 05:13:47 INFO Signed narinfos id=65 count=17174worker2 # [ 154.657316] nix-grpc-daemon[1536]: time=2026-09-23T05:13:47.523Z level=INFO msg="Uploading 1 narinfos"7175lb # [ 154.631935] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls7176lb # [ 154.643766] niks3-server[1116]: 2026/09/23 05:13:47 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7177lb # [ 154.650334] niks3-server[1116]: 2026/09/23 05:13:47 INFO Completed upload id=657178worker2 # [ 154.682526] nix-grpc-daemon[1536]: time=2026-09-23T05:13:47.548Z level=INFO msg="Upload complete. (187ms)"7179lb # [ 154.653428] niks3-server[1116]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo7180worker2 # [ 154.686689] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 paths=1 nar_bytes_in=4167181worker2 # [ 154.691554] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487182worker2 # [ 154.704069] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7183worker2 # [ 154.707936] nix-daemon[852]: reaped child process 1745, status = succeeded7184worker2 # [ 154.713258] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7185client # [ 154.692249] nix[1510]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...7186worker1 # [ 155.273078] nix-daemon[863]: reaped child process 1994, status = succeeded7187worker1 # [ 155.356932] nix-daemon[863]: accepted connection from pid 2015, user nix-grpc-daemon (trusted)7188worker1 # [ 155.371834] nix-daemon[863]: reaped child process 2018, status = succeeded7189lb # [ 155.335945] niks3-server[1116]: 2026/09/23 05:13:48 INFO Received uploads request method=POST path=/api/pending_closures7190worker1 # [ 155.389431] nix-grpc-daemon[1970]: time=2026-09-23T05:13:48.492Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7191worker1 # [ 155.390946] nix-grpc-daemon[1970]: time=2026-09-23T05:13:48.494Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"7192lb # [ 155.376652] niks3-server[1116]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst7193lb # [ 155.387145] niks3-server[1116]: 2026/09/23 05:13:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7194worker1 # [ 155.440109] nix-grpc-daemon[1970]: time=2026-09-23T05:13:48.543Z level=INFO msg="Uploading 1 narinfos"7195lb # [ 155.397043] niks3-server[1116]: 2026/09/23 05:13:48 INFO Signed narinfos id=66 count=17196lb # [ 155.409076] niks3-server[1116]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls7197lb # [ 155.418077] niks3-server[1116]: 2026/09/23 05:13:48 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7198lb # [ 155.420421] niks3-server[1116]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7199lb # [ 155.424680] niks3-server[1116]: 2026/09/23 05:13:48 INFO Completed upload id=667200worker1 # [ 155.469360] nix-grpc-daemon[1970]: time=2026-09-23T05:13:48.572Z level=INFO msg="Upload complete. (227ms)"7201lb # [ 155.427899] niks3-server[1116]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo7202worker2 # [ 155.462448] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:48Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=17203worker1 # [ 155.475981] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=2 outputs=17204worker1 # [ 155.492335] nix-daemon[863]: reaped child process 1992, status = succeeded7205worker1 # [ 155.520548] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267206worker1 # [ 155.533512] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267207client # [ 155.510836] nix[1510]: copying 1 paths to 'lb:50051'...7208worker1 # [ 155.545612] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267209client # [ 155.512891] nix[1510]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...7210worker1 # [ 155.558140] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7211worker1 # [ 155.664665] nix-daemon[863]: accepted connection from pid 2022, user nix-grpc-daemon (trusted)7212worker1 # [ 155.681743] nix-daemon[863]: reaped child process 2025, status = succeeded7213lb # [ 155.642335] niks3-server[1116]: 2026/09/23 05:13:48 INFO Received uploads request method=POST path=/api/pending_closures7214worker1 # [ 155.697487] nix-grpc-daemon[1970]: time=2026-09-23T05:13:48.801Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7215worker1 # [ 155.699022] nix-grpc-daemon[1970]: time=2026-09-23T05:13:48.802Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"7216lb # [ 155.675771] niks3-server[1116]: 2026/09/23 05:13:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7217lb # [ 155.683549] niks3-server[1116]: 2026/09/23 05:13:48 INFO Signed narinfos id=67 count=17218worker1 # [ 155.731379] nix-grpc-daemon[1970]: time=2026-09-23T05:13:48.834Z level=INFO msg="Uploading 1 narinfos"7219lb # [ 155.691734] niks3-server[1116]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls7220lb # [ 155.697134] niks3-server[1116]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst7221lb # [ 155.709360] niks3-server[1116]: 2026/09/23 05:13:48 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7222lb # [ 155.713614] niks3-server[1116]: 2026/09/23 05:13:48 INFO Completed upload id=677223worker1 # [ 155.760195] nix-grpc-daemon[1970]: time=2026-09-23T05:13:48.863Z level=INFO msg="Upload complete. (190ms)"7224lb # [ 155.718987] niks3-server[1116]: 2026/09/23 05:13:48 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo7225worker1 # [ 155.763116] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=4167226worker1 # [ 155.768509] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267227worker1 # [ 155.781064] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7228worker1 # [ 155.785032] nix-daemon[863]: reaped child process 2020, status = succeeded7229worker1 # [ 155.790164] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7230client # [ 155.755460] nix[1510]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...7231worker2 # [ 156.332225] nix-daemon[852]: reaped child process 1754, status = succeeded7232worker2 # [ 156.410245] nix-daemon[852]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)7233worker2 # [ 156.425053] nix-daemon[852]: reaped child process 1782, status = succeeded7234lb # [ 156.403189] niks3-server[1116]: 2026/09/23 05:13:49 INFO Received uploads request method=POST path=/api/pending_closures7235worker2 # [ 156.441536] nix-grpc-daemon[1536]: time=2026-09-23T05:13:49.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7236worker2 # [ 156.443303] nix-grpc-daemon[1536]: time=2026-09-23T05:13:49.309Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"7237lb # [ 156.448874] niks3-server[1116]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7238lb # [ 156.453486] niks3-server[1116]: 2026/09/23 05:13:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7239lb # [ 156.459656] niks3-server[1116]: 2026/09/23 05:13:49 INFO Signed narinfos id=68 count=17240worker2 # [ 156.493181] nix-grpc-daemon[1536]: time=2026-09-23T05:13:49.358Z level=INFO msg="Uploading 1 narinfos"7241lb # [ 156.467133] niks3-server[1116]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst7242lb # [ 156.480175] niks3-server[1116]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls7243lb # [ 156.483449] niks3-server[1116]: 2026/09/23 05:13:49 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7244lb # [ 156.489494] niks3-server[1116]: 2026/09/23 05:13:49 INFO Completed upload id=687245worker2 # [ 156.520155] nix-grpc-daemon[1536]: time=2026-09-23T05:13:49.385Z level=INFO msg="Upload complete. (200ms)"7246lb # [ 156.492969] niks3-server[1116]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo7247worker2 # [ 156.527367] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=3 outputs=17248worker2 # [ 156.541972] nix-daemon[852]: reaped child process 1752, status = succeeded7249lb # [ 156.926463] envoy[912]: conn peer=[2001:db8:1::1]:57022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1557 ms=81957250lb # [ 156.930147] envoy[912]: conn peer=[2001:db8:1::3]:38638 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=975 tx=361 ms=201717251worker1 # [ 157.410438] nix-daemon[863]: reaped child process 2029, status = succeeded7252worker1 # [ 157.493778] nix-daemon[863]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)7253worker1 # [ 157.506688] nix-daemon[863]: reaped child process 2057, status = succeeded7254lb # [ 157.471387] niks3-server[1116]: 2026/09/23 05:13:50 INFO Received uploads request method=POST path=/api/pending_closures7255worker1 # [ 157.523843] nix-grpc-daemon[1970]: time=2026-09-23T05:13:50.627Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7256worker1 # [ 157.525430] nix-grpc-daemon[1970]: time=2026-09-23T05:13:50.629Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"7257lb # [ 157.510973] niks3-server[1116]: 2026/09/23 05:13:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7258lb # [ 157.516568] niks3-server[1116]: 2026/09/23 05:13:50 INFO Signed narinfos id=69 count=17259worker1 # [ 157.564777] nix-grpc-daemon[1970]: time=2026-09-23T05:13:50.668Z level=INFO msg="Uploading 1 narinfos"7260lb # [ 157.528694] niks3-server[1116]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls7261lb # [ 157.533538] niks3-server[1116]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7262lb # [ 157.541231] niks3-server[1116]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst7263lb # [ 157.551581] niks3-server[1116]: 2026/09/23 05:13:50 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7264lb # [ 157.558385] niks3-server[1116]: 2026/09/23 05:13:50 INFO Completed upload id=697265worker1 # [ 157.603130] nix-grpc-daemon[1970]: time=2026-09-23T05:13:50.706Z level=INFO msg="Upload complete. (205ms)"7266lb # [ 157.561854] niks3-server[1116]: 2026/09/23 05:13:50 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo7267worker1 # [ 157.608506] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=4 outputs=17268worker1 # [ 157.622201] nix-daemon[863]: reaped child process 2027, status = succeeded7269client # [ 157.606755] systemd[1]: burst.service: Deactivated successfully.7270client # [ 157.610933] systemd[1]: burst.service: Consumed 159ms CPU time over 4.960s wall clock time, 54.1M memory peak, 14.1K incoming IP traffic, 18.6K outgoing IP traffic.7271client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }7272client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)7273lb: must succeed: curl -sf localhost:9901/clusters7274lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7275lb: must succeed: curl -sf localhost:9901/clusters7276lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7277worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7278worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7279(finished: subtest: queue gauges show a burst while every build blocks, in 6.21 seconds)7280subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7281client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17282worker1 # [ 158.893680] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:419267283worker2 # [ 158.888884] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530487284worker2 # [ 158.916216] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:51Z level=debug event=want client=2 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07285worker2 # [ 158.920726] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:51Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17286worker2 # [ 158.925310] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:51Z level=debug event=want client=2 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07287worker2 # [ 158.929231] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:51Z level=debug event=want client=2 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07288worker1 # [ 158.980350] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267289worker1 # [ 158.993912] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267290worker1 # [ 159.005813] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267291worker1 # [ 159.017264] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7292worker1 # [ 159.130805] nix-daemon[863]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)7293worker1 # [ 159.145427] nix-daemon[863]: reaped child process 2068, status = succeeded7294lb # [ 159.110383] niks3-server[1116]: 2026/09/23 05:13:52 INFO Received uploads request method=POST path=/api/pending_closures7295worker1 # [ 159.162703] nix-grpc-daemon[1970]: time=2026-09-23T05:13:52.266Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7296worker1 # [ 159.164980] nix-grpc-daemon[1970]: time=2026-09-23T05:13:52.268Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7297lb # [ 159.138455] niks3-server[1116]: 2026/09/23 05:13:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7298lb # [ 159.142353] niks3-server[1116]: 2026/09/23 05:13:52 INFO Signed narinfos id=70 count=17299worker1 # [ 159.189949] nix-grpc-daemon[1970]: time=2026-09-23T05:13:52.293Z level=INFO msg="Uploading 1 narinfos"7300lb # [ 159.149709] niks3-server[1116]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7301lb # [ 159.155809] niks3-server[1116]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7302lb # [ 159.160493] niks3-server[1116]: 2026/09/23 05:13:52 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7303lb # [ 159.165091] niks3-server[1116]: 2026/09/23 05:13:52 INFO Completed upload id=707304worker1 # [ 159.209542] nix-grpc-daemon[1970]: time=2026-09-23T05:13:52.313Z level=INFO msg="Upload complete. (179ms)"7305lb # [ 159.167104] niks3-server[1116]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7306worker1 # [ 159.213247] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=5047307worker1 # [ 159.216122] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267308worker1 # [ 159.225377] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7309worker1 # [ 159.227341] nix-daemon[863]: reaped child process 2063, status = succeeded7310worker1 # [ 159.232898] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7311worker1 # [ 160.077495] nix-daemon[863]: reaped child process 2072, status = succeeded7312worker1 # [ 160.160516] nix-daemon[863]: accepted connection from pid 2088, user nix-grpc-daemon (trusted)7313worker1 # [ 160.175697] nix-daemon[863]: reaped child process 2091, status = succeeded7314lb # [ 160.139324] niks3-server[1116]: 2026/09/23 05:13:53 INFO Received uploads request method=POST path=/api/pending_closures7315worker1 # [ 160.192572] nix-grpc-daemon[1970]: time=2026-09-23T05:13:53.296Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7316worker1 # [ 160.194238] nix-grpc-daemon[1970]: time=2026-09-23T05:13:53.297Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7317lb # [ 160.181776] niks3-server[1116]: 2026/09/23 05:13:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7318worker1 # [ 160.232079] nix-grpc-daemon[1970]: time=2026-09-23T05:13:53.335Z level=INFO msg="Uploading 1 narinfos"7319lb # [ 160.188185] niks3-server[1116]: 2026/09/23 05:13:53 INFO Signed narinfos id=71 count=17320lb # [ 160.197175] niks3-server[1116]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7321lb # [ 160.202562] niks3-server[1116]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7322lb # [ 160.212480] niks3-server[1116]: 2026/09/23 05:13:53 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7323lb # [ 160.217585] niks3-server[1116]: 2026/09/23 05:13:53 INFO Completed upload id=717324worker1 # [ 160.265153] nix-grpc-daemon[1970]: time=2026-09-23T05:13:53.368Z level=INFO msg="Upload complete. (224ms)"7325lb # [ 160.223375] niks3-server[1116]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7326lb # [ 160.226571] niks3-server[1116]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7327worker1 # [ 160.271985] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=5 outputs=17328worker2 # [ 160.259116] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:53Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17329worker2 # [ 160.269352] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:53Z level=debug event=want client=2 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07330worker1 # [ 160.288754] nix-daemon[863]: reaped child process 2070, status = succeeded7331worker1 # [ 160.318273] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267332worker1 # [ 160.331748] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267333worker1 # [ 160.343966] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267334worker1 # [ 160.354396] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7335worker1 # [ 160.465139] nix-daemon[863]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)7336worker1 # [ 160.481086] nix-daemon[863]: reaped child process 2098, status = succeeded7337lb # [ 160.442564] niks3-server[1116]: 2026/09/23 05:13:53 INFO Received uploads request method=POST path=/api/pending_closures7338worker1 # [ 160.496145] nix-grpc-daemon[1970]: time=2026-09-23T05:13:53.599Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7339worker1 # [ 160.497845] nix-grpc-daemon[1970]: time=2026-09-23T05:13:53.601Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7340lb # [ 160.476480] niks3-server[1116]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7341lb # [ 160.483328] niks3-server[1116]: 2026/09/23 05:13:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7342worker1 # [ 160.535860] nix-grpc-daemon[1970]: time=2026-09-23T05:13:53.639Z level=INFO msg="Uploading 1 narinfos"7343lb # [ 160.491150] niks3-server[1116]: 2026/09/23 05:13:53 INFO Signed narinfos id=72 count=17344lb # [ 160.499729] niks3-server[1116]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7345lb # [ 160.507950] niks3-server[1116]: 2026/09/23 05:13:53 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7346lb # [ 160.517635] niks3-server[1116]: 2026/09/23 05:13:53 INFO Completed upload id=727347lb # [ 160.519508] niks3-server[1116]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7348worker1 # [ 160.563500] nix-grpc-daemon[1970]: time=2026-09-23T05:13:53.667Z level=INFO msg="Upload complete. (195ms)"7349worker1 # [ 160.568257] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=5047350worker1 # [ 160.574330] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267351worker1 # [ 160.587187] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7352worker1 # [ 160.595257] nix-daemon[863]: reaped child process 2093, status = succeeded7353worker1 # [ 160.600529] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7354worker1 # [ 161.638476] nix-daemon[863]: reaped child process 2102, status = succeeded7355worker1 # [ 161.718453] nix-daemon[863]: accepted connection from pid 2118, user nix-grpc-daemon (trusted)7356worker1 # [ 161.731162] nix-daemon[863]: reaped child process 2121, status = succeeded7357lb # [ 161.695208] niks3-server[1116]: 2026/09/23 05:13:54 INFO Received uploads request method=POST path=/api/pending_closures7358worker1 # [ 161.747965] nix-grpc-daemon[1970]: time=2026-09-23T05:13:54.851Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7359worker1 # [ 161.749420] nix-grpc-daemon[1970]: time=2026-09-23T05:13:54.853Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7360lb # [ 161.739143] niks3-server[1116]: 2026/09/23 05:13:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7361lb # [ 161.745949] niks3-server[1116]: 2026/09/23 05:13:54 INFO Signed narinfos id=73 count=17362worker1 # [ 161.793840] nix-grpc-daemon[1970]: time=2026-09-23T05:13:54.897Z level=INFO msg="Uploading 1 narinfos"7363lb # [ 161.754757] niks3-server[1116]: 2026/09/23 05:13:54 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7364lb # [ 161.757828] niks3-server[1116]: 2026/09/23 05:13:54 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7365lb # [ 161.768105] niks3-server[1116]: 2026/09/23 05:13:54 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7366lb # [ 161.773758] niks3-server[1116]: 2026/09/23 05:13:54 INFO Completed upload id=737367worker1 # [ 161.818236] nix-grpc-daemon[1970]: time=2026-09-23T05:13:54.921Z level=INFO msg="Upload complete. (210ms)"7368lb # [ 161.779593] niks3-server[1116]: 2026/09/23 05:13:54 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7369lb # [ 161.782077] niks3-server[1116]: 2026/09/23 05:13:54 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7370worker1 # [ 161.824210] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=6 outputs=17371worker2 # [ 161.811544] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:54Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17372worker1 # [ 161.833310] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267373worker2 # [ 161.821733] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:54Z level=debug event=want client=2 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07374worker1 # [ 161.843528] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267375worker1 # [ 161.849933] nix-daemon[863]: reaped child process 2100, status = succeeded7376worker1 # [ 161.854922] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267377worker1 # [ 161.863072] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7378worker1 # [ 161.947693] nix-daemon[863]: accepted connection from pid 2125, user nix-grpc-daemon (trusted)7379worker1 # [ 161.960567] nix-daemon[863]: reaped child process 2128, status = succeeded7380lb # [ 161.923822] niks3-server[1116]: 2026/09/23 05:13:55 INFO Received uploads request method=POST path=/api/pending_closures7381worker1 # [ 161.975996] nix-grpc-daemon[1970]: time=2026-09-23T05:13:55.079Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7382worker1 # [ 161.977736] nix-grpc-daemon[1970]: time=2026-09-23T05:13:55.081Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7383lb # [ 161.954204] niks3-server[1116]: 2026/09/23 05:13:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7384worker1 # [ 162.004477] nix-grpc-daemon[1970]: time=2026-09-23T05:13:55.108Z level=INFO msg="Uploading 1 narinfos"7385lb # [ 161.961144] niks3-server[1116]: 2026/09/23 05:13:55 INFO Signed narinfos id=74 count=17386lb # [ 161.966989] niks3-server[1116]: 2026/09/23 05:13:55 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7387lb # [ 161.975244] niks3-server[1116]: 2026/09/23 05:13:55 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7388lb # [ 161.985518] niks3-server[1116]: 2026/09/23 05:13:55 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7389worker1 # [ 162.035916] nix-grpc-daemon[1970]: time=2026-09-23T05:13:55.139Z level=INFO msg="Upload complete. (167ms)"7390lb # [ 161.991471] niks3-server[1116]: 2026/09/23 05:13:55 INFO Completed upload id=747391worker1 # [ 162.037987] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=5047392lb # [ 161.996619] niks3-server[1116]: 2026/09/23 05:13:55 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7393worker1 # [ 162.041105] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267394worker1 # [ 162.047833] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7395worker1 # [ 162.053074] nix-daemon[863]: reaped child process 2123, status = succeeded7396worker1 # [ 162.055849] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7397worker1 # [ 163.495863] nix-daemon[863]: reaped child process 2132, status = succeeded7398worker1 # [ 163.575855] nix-daemon[863]: accepted connection from pid 2148, user nix-grpc-daemon (trusted)7399worker1 # [ 163.587808] nix-daemon[863]: reaped child process 2151, status = succeeded7400lb # [ 163.550904] niks3-server[1116]: 2026/09/23 05:13:56 INFO Received uploads request method=POST path=/api/pending_closures7401worker1 # [ 163.604221] nix-grpc-daemon[1970]: time=2026-09-23T05:13:56.707Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7402worker1 # [ 163.605683] nix-grpc-daemon[1970]: time=2026-09-23T05:13:56.709Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7403lb # [ 163.594932] niks3-server[1116]: 2026/09/23 05:13:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7404worker1 # [ 163.645390] nix-grpc-daemon[1970]: time=2026-09-23T05:13:56.748Z level=INFO msg="Uploading 1 narinfos"7405lb # [ 163.600699] niks3-server[1116]: 2026/09/23 05:13:56 INFO Signed narinfos id=75 count=17406lb # [ 163.608175] niks3-server[1116]: 2026/09/23 05:13:56 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7407lb # [ 163.616742] niks3-server[1116]: 2026/09/23 05:13:56 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7408lb # [ 163.624947] niks3-server[1116]: 2026/09/23 05:13:56 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7409lb # [ 163.630096] niks3-server[1116]: 2026/09/23 05:13:56 INFO Completed upload id=757410worker1 # [ 163.674701] nix-grpc-daemon[1970]: time=2026-09-23T05:13:56.778Z level=INFO msg="Upload complete. (210ms)"7411lb # [ 163.637336] niks3-server[1116]: 2026/09/23 05:13:56 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7412lb # [ 163.639491] niks3-server[1116]: 2026/09/23 05:13:56 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7413worker1 # [ 163.680605] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=7 outputs=17414worker2 # [ 163.667868] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:56Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17415worker2 # [ 163.677934] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:56Z level=debug event=want client=2 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07416worker1 # [ 163.696205] nix-daemon[863]: reaped child process 2130, status = succeeded7417worker1 # [ 163.733766] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267418worker1 # [ 163.745680] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267419worker1 # [ 163.758272] nix-grpc-daemon[1970]: time=2026-09-23T05:13:56.861Z level=INFO msg="All 1 paths already cached"7420worker1 # [ 163.765579] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267421worker1 # [ 163.773675] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7422worker1 # [ 163.871674] nix-daemon[863]: accepted connection from pid 2155, user nix-grpc-daemon (trusted)7423worker1 # [ 163.885561] nix-daemon[863]: reaped child process 2158, status = succeeded7424lb # [ 163.847545] niks3-server[1116]: 2026/09/23 05:13:57 INFO Received uploads request method=POST path=/api/pending_closures7425worker1 # [ 163.899659] nix-grpc-daemon[1970]: time=2026-09-23T05:13:57.003Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7426worker1 # [ 163.901227] nix-grpc-daemon[1970]: time=2026-09-23T05:13:57.004Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7427lb # [ 163.879601] niks3-server[1116]: 2026/09/23 05:13:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7428lb # [ 163.884951] niks3-server[1116]: 2026/09/23 05:13:57 INFO Signed narinfos id=76 count=17429worker1 # [ 163.932368] nix-grpc-daemon[1970]: time=2026-09-23T05:13:57.036Z level=INFO msg="Uploading 1 narinfos"7430lb # [ 163.892204] niks3-server[1116]: 2026/09/23 05:13:57 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7431lb # [ 163.904264] niks3-server[1116]: 2026/09/23 05:13:57 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7432lb # [ 163.911769] niks3-server[1116]: 2026/09/23 05:13:57 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7433worker1 # [ 163.961887] nix-grpc-daemon[1970]: time=2026-09-23T05:13:57.065Z level=INFO msg="Upload complete. (174ms)"7434lb # [ 163.917449] niks3-server[1116]: 2026/09/23 05:13:57 INFO Completed upload id=767435worker1 # [ 163.964211] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=6247436lb # [ 163.923054] niks3-server[1116]: 2026/09/23 05:13:57 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7437worker1 # [ 163.970372] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267438worker1 # [ 163.981973] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7439worker1 # [ 163.985938] nix-daemon[863]: reaped child process 2153, status = succeeded7440worker1 # [ 163.990898] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7441worker1 # [ 164.805798] nix-daemon[863]: reaped child process 2162, status = succeeded7442worker1 # [ 164.881634] nix-daemon[863]: accepted connection from pid 2178, user nix-grpc-daemon (trusted)7443worker1 # [ 164.894081] nix-daemon[863]: reaped child process 2181, status = succeeded7444lb # [ 164.858618] niks3-server[1116]: 2026/09/23 05:13:58 INFO Received uploads request method=POST path=/api/pending_closures7445worker1 # [ 164.912292] nix-grpc-daemon[1970]: time=2026-09-23T05:13:58.014Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7446worker1 # [ 164.913655] nix-grpc-daemon[1970]: time=2026-09-23T05:13:58.014Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7447lb # [ 164.903220] niks3-server[1116]: 2026/09/23 05:13:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7448lb # [ 164.910206] niks3-server[1116]: 2026/09/23 05:13:58 INFO Signed narinfos id=77 count=17449worker1 # [ 164.959246] nix-grpc-daemon[1970]: time=2026-09-23T05:13:58.062Z level=INFO msg="Uploading 1 narinfos"7450lb # [ 164.918526] niks3-server[1116]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7451lb # [ 164.926749] niks3-server[1116]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7452lb # [ 164.930224] niks3-server[1116]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7453lb # [ 164.941715] niks3-server[1116]: 2026/09/23 05:13:58 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7454lb # [ 164.945869] niks3-server[1116]: 2026/09/23 05:13:58 INFO Completed upload id=777455worker1 # [ 164.991691] nix-grpc-daemon[1970]: time=2026-09-23T05:13:58.095Z level=INFO msg="Upload complete. (194ms)"7456lb # [ 164.949179] niks3-server[1116]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7457worker1 # [ 164.997614] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=8 outputs=17458worker2 # [ 164.985697] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:57Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17459worker1 # [ 165.013433] nix-daemon[863]: reaped child process 2160, status = succeeded7460worker1 # [ 165.042586] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267461worker2 # [ 165.032463] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:57Z level=debug event=want client=2 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07462worker1 # [ 165.055527] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267463worker1 # [ 165.067799] nix-grpc-daemon[1970]: time=2026-09-23T05:13:58.171Z level=INFO msg="All 1 paths already cached"7464worker1 # [ 165.075088] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267465worker1 # [ 165.085131] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7466worker1 # [ 165.181056] nix-daemon[863]: accepted connection from pid 2185, user nix-grpc-daemon (trusted)7467worker1 # [ 165.194261] nix-daemon[863]: reaped child process 2188, status = succeeded7468lb # [ 165.156961] niks3-server[1116]: 2026/09/23 05:13:58 INFO Received uploads request method=POST path=/api/pending_closures7469worker1 # [ 165.209200] nix-grpc-daemon[1970]: time=2026-09-23T05:13:58.312Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7470worker1 # [ 165.210723] nix-grpc-daemon[1970]: time=2026-09-23T05:13:58.314Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7471lb # [ 165.190091] niks3-server[1116]: 2026/09/23 05:13:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7472lb # [ 165.195952] niks3-server[1116]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7473lb # [ 165.202662] niks3-server[1116]: 2026/09/23 05:13:58 INFO Signed narinfos id=78 count=17474worker1 # [ 165.249969] nix-grpc-daemon[1970]: time=2026-09-23T05:13:58.353Z level=INFO msg="Uploading 1 narinfos"7475lb # [ 165.210515] niks3-server[1116]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7476lb # [ 165.222676] niks3-server[1116]: 2026/09/23 05:13:58 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7477worker1 # [ 165.272976] nix-grpc-daemon[1970]: time=2026-09-23T05:13:58.376Z level=INFO msg="Upload complete. (177ms)"7478lb # [ 165.228840] niks3-server[1116]: 2026/09/23 05:13:58 INFO Completed upload id=787479lb # [ 165.232639] niks3-server[1116]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7480worker1 # [ 165.275446] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=6247481worker1 # [ 165.280491] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267482worker1 # [ 165.291988] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7483worker1 # [ 165.297972] nix-daemon[863]: reaped child process 2183, status = succeeded7484worker1 # [ 165.300822] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7485worker1 # [ 166.515779] nix-daemon[863]: reaped child process 2192, status = succeeded7486worker1 # [ 166.593045] nix-daemon[863]: accepted connection from pid 2208, user nix-grpc-daemon (trusted)7487worker1 # [ 166.604700] nix-daemon[863]: reaped child process 2211, status = succeeded7488lb # [ 166.568358] niks3-server[1116]: 2026/09/23 05:13:59 INFO Received uploads request method=POST path=/api/pending_closures7489worker1 # [ 166.621576] nix-grpc-daemon[1970]: time=2026-09-23T05:13:59.725Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7490worker1 # [ 166.623224] nix-grpc-daemon[1970]: time=2026-09-23T05:13:59.726Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7491lb # [ 166.608185] niks3-server[1116]: 2026/09/23 05:13:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7492lb # [ 166.615062] niks3-server[1116]: 2026/09/23 05:13:59 INFO Signed narinfos id=79 count=17493worker1 # [ 166.663459] nix-grpc-daemon[1970]: time=2026-09-23T05:13:59.767Z level=INFO msg="Uploading 1 narinfos"7494lb # [ 166.625688] niks3-server[1116]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7495lb # [ 166.631641] niks3-server[1116]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7496lb # [ 166.635451] niks3-server[1116]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7497lb # [ 166.644170] niks3-server[1116]: 2026/09/23 05:13:59 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7498lb # [ 166.650034] niks3-server[1116]: 2026/09/23 05:13:59 INFO Completed upload id=797499worker1 # [ 166.694520] nix-grpc-daemon[1970]: time=2026-09-23T05:13:59.798Z level=INFO msg="Upload complete. (186ms)"7500lb # [ 166.651726] niks3-server[1116]: 2026/09/23 05:13:59 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7501worker1 # [ 166.699523] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=9 outputs=17502worker2 # [ 166.689780] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:59Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17503worker1 # [ 166.707276] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267504worker2 # [ 166.696480] nix-grpc-daemon[1447]: ts=2026-09-23T05:13:59Z level=debug event=want client=2 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07505worker1 # [ 166.716576] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267506worker1 # [ 166.724728] nix-daemon[863]: reaped child process 2190, status = succeeded7507worker1 # [ 166.726776] nix-grpc-daemon[1970]: time=2026-09-23T05:13:59.830Z level=INFO msg="All 1 paths already cached"7508worker1 # [ 166.731092] nix-grpc-daemon[1909]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267509worker1 # [ 166.735828] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7510worker1 # [ 166.821350] nix-daemon[863]: accepted connection from pid 2215, user nix-grpc-daemon (trusted)7511worker1 # [ 166.835611] nix-daemon[863]: reaped child process 2218, status = succeeded7512lb # [ 166.796484] niks3-server[1116]: 2026/09/23 05:13:59 INFO Received uploads request method=POST path=/api/pending_closures7513worker1 # [ 166.850220] nix-grpc-daemon[1970]: time=2026-09-23T05:13:59.953Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7514worker1 # [ 166.852243] nix-grpc-daemon[1970]: time=2026-09-23T05:13:59.955Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7515lb # [ 166.831547] niks3-server[1116]: 2026/09/23 05:14:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7516lb # [ 166.837513] niks3-server[1116]: 2026/09/23 05:14:00 INFO Signed narinfos id=80 count=17517worker1 # [ 166.885538] nix-grpc-daemon[1970]: time=2026-09-23T05:13:59.989Z level=INFO msg="Uploading 1 narinfos"7518lb # [ 166.844381] niks3-server[1116]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7519lb # [ 166.855298] niks3-server[1116]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7520lb # [ 166.867180] niks3-server[1116]: 2026/09/23 05:14:00 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7521lb # [ 166.873783] niks3-server[1116]: 2026/09/23 05:14:00 INFO Completed upload id=807522worker1 # [ 166.918497] nix-grpc-daemon[1970]: time=2026-09-23T05:14:00.022Z level=INFO msg="Upload complete. (177ms)"7523lb # [ 166.876037] niks3-server[1116]: 2026/09/23 05:14:00 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7524worker1 # [ 166.923471] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=6247525worker1 # [ 166.929323] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267526worker1 # [ 166.939877] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7527worker1 # [ 166.950149] nix-daemon[863]: reaped child process 2213, status = succeeded7528worker1 # [ 166.954820] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7529lb # [ 166.926635] envoy[912]: conn peer=[2001:db8:1::1]:42606 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9604 tx=5161 ms=48047530worker1 # [ 168.370350] nix-daemon[863]: reaped child process 2222, status = succeeded7531worker1 # [ 168.449113] nix-daemon[863]: accepted connection from pid 2238, user nix-grpc-daemon (trusted)7532worker1 # [ 168.463133] nix-daemon[863]: reaped child process 2241, status = succeeded7533lb # [ 168.424446] niks3-server[1116]: 2026/09/23 05:14:01 INFO Received uploads request method=POST path=/api/pending_closures7534worker1 # [ 168.478471] nix-grpc-daemon[1970]: time=2026-09-23T05:14:01.581Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7535worker1 # [ 168.483178] nix-grpc-daemon[1970]: time=2026-09-23T05:14:01.583Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7536lb # [ 168.466774] niks3-server[1116]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7537lb # [ 168.479387] niks3-server[1116]: 2026/09/23 05:14:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7538worker1 # [ 168.530917] nix-grpc-daemon[1970]: time=2026-09-23T05:14:01.634Z level=INFO msg="Uploading 1 narinfos"7539lb # [ 168.486394] niks3-server[1116]: 2026/09/23 05:14:01 INFO Signed narinfos id=81 count=17540lb # [ 168.489814] niks3-server[1116]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7541lb # [ 168.493974] niks3-server[1116]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7542lb # [ 168.500060] niks3-server[1116]: 2026/09/23 05:14:01 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7543lb # [ 168.504079] niks3-server[1116]: 2026/09/23 05:14:01 INFO Completed upload id=817544worker1 # [ 168.548705] nix-grpc-daemon[1970]: time=2026-09-23T05:14:01.652Z level=INFO msg="Upload complete. (186ms)"7545lb # [ 168.505929] niks3-server[1116]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7546worker1 # [ 168.554624] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=10 outputs=17547worker2 # [ 168.541962] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:01Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17548worker2 # [ 168.552619] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:01Z level=debug event=want client=2 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07549worker1 # [ 168.569241] nix-daemon[863]: reaped child process 2220, status = succeeded7550worker1 # [ 168.602361] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267551worker1 # [ 168.614612] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267552worker1 # [ 168.627062] nix-grpc-daemon[1970]: time=2026-09-23T05:14:01.730Z level=INFO msg="All 2 paths already cached"7553worker1 # [ 168.635065] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267554worker1 # [ 168.645193] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7555worker1 # [ 168.745076] nix-daemon[863]: accepted connection from pid 2245, user nix-grpc-daemon (trusted)7556worker1 # [ 168.759329] nix-daemon[863]: reaped child process 2248, status = succeeded7557lb # [ 168.721888] niks3-server[1116]: 2026/09/23 05:14:01 INFO Received uploads request method=POST path=/api/pending_closures7558worker1 # [ 168.775174] nix-grpc-daemon[1970]: time=2026-09-23T05:14:01.878Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7559worker1 # [ 168.776921] nix-grpc-daemon[1970]: time=2026-09-23T05:14:01.880Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7560lb # [ 168.753894] niks3-server[1116]: 2026/09/23 05:14:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7561worker1 # [ 168.804512] nix-grpc-daemon[1970]: time=2026-09-23T05:14:01.908Z level=INFO msg="Uploading 1 narinfos"7562lb # [ 168.760078] niks3-server[1116]: 2026/09/23 05:14:01 INFO Signed narinfos id=82 count=17563lb # [ 168.768873] niks3-server[1116]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7564lb # [ 168.780465] niks3-server[1116]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7565lb # [ 168.787102] niks3-server[1116]: 2026/09/23 05:14:01 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7566lb # [ 168.795896] niks3-server[1116]: 2026/09/23 05:14:01 INFO Completed upload id=827567worker1 # [ 168.840396] nix-grpc-daemon[1970]: time=2026-09-23T05:14:01.944Z level=INFO msg="Upload complete. (183ms)"7568lb # [ 168.798523] niks3-server[1116]: 2026/09/23 05:14:01 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7569worker1 # [ 168.844581] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=7527570worker1 # [ 168.850944] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267571worker1 # [ 168.860083] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7572worker1 # [ 168.870151] nix-daemon[863]: reaped child process 2243, status = succeeded7573worker1 # [ 168.875335] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7574worker1 # [ 169.891244] nix-daemon[863]: reaped child process 2252, status = succeeded7575worker1 # [ 169.967812] nix-daemon[863]: accepted connection from pid 2268, user nix-grpc-daemon (trusted)7576worker1 # [ 169.979843] nix-daemon[863]: reaped child process 2271, status = succeeded7577lb # [ 169.943279] niks3-server[1116]: 2026/09/23 05:14:03 INFO Received uploads request method=POST path=/api/pending_closures7578worker1 # [ 169.996464] nix-grpc-daemon[1970]: time=2026-09-23T05:14:03.099Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7579worker1 # [ 169.998145] nix-grpc-daemon[1970]: time=2026-09-23T05:14:03.101Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7580lb # [ 169.984402] niks3-server[1116]: 2026/09/23 05:14:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7581worker1 # [ 170.035441] nix-grpc-daemon[1970]: time=2026-09-23T05:14:03.139Z level=INFO msg="Uploading 1 narinfos"7582lb # [ 169.991128] niks3-server[1116]: 2026/09/23 05:14:03 INFO Signed narinfos id=83 count=17583lb # [ 169.998777] niks3-server[1116]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7584lb # [ 170.004641] niks3-server[1116]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7585lb # [ 170.009891] niks3-server[1116]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7586lb # [ 170.020625] niks3-server[1116]: 2026/09/23 05:14:03 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7587lb # [ 170.025862] niks3-server[1116]: 2026/09/23 05:14:03 INFO Completed upload id=837588worker1 # [ 170.070531] nix-grpc-daemon[1970]: time=2026-09-23T05:14:03.174Z level=INFO msg="Upload complete. (188ms)"7589lb # [ 170.029251] niks3-server[1116]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7590worker2 # [ 170.063146] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:02Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17591worker1 # [ 170.076952] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=11 outputs=17592worker1 # [ 170.092802] nix-daemon[863]: reaped child process 2250, status = succeeded7593worker1 # [ 170.124828] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267594worker1 # [ 170.137632] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267595worker1 # [ 170.150703] nix-grpc-daemon[1970]: time=2026-09-23T05:14:03.254Z level=INFO msg="All 2 paths already cached"7596worker1 # [ 170.158771] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267597worker1 # [ 170.168170] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7598worker1 # [ 170.275885] nix-daemon[863]: accepted connection from pid 2275, user nix-grpc-daemon (trusted)7599worker1 # [ 170.292788] nix-daemon[863]: reaped child process 2278, status = succeeded7600lb # [ 170.254164] niks3-server[1116]: 2026/09/23 05:14:03 INFO Received uploads request method=POST path=/api/pending_closures7601worker1 # [ 170.308066] nix-grpc-daemon[1970]: time=2026-09-23T05:14:03.411Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7602worker1 # [ 170.309429] nix-grpc-daemon[1970]: time=2026-09-23T05:14:03.411Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7603lb # [ 170.282167] niks3-server[1116]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7604lb # [ 170.289593] niks3-server[1116]: 2026/09/23 05:14:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7605lb # [ 170.294546] niks3-server[1116]: 2026/09/23 05:14:03 INFO Signed narinfos id=84 count=17606worker1 # [ 170.342371] nix-grpc-daemon[1970]: time=2026-09-23T05:14:03.445Z level=INFO msg="Uploading 1 narinfos"7607lb # [ 170.309399] niks3-server[1116]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7608lb # [ 170.318104] niks3-server[1116]: 2026/09/23 05:14:03 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7609lb # [ 170.324520] niks3-server[1116]: 2026/09/23 05:14:03 INFO Completed upload id=847610worker1 # [ 170.368948] nix-grpc-daemon[1970]: time=2026-09-23T05:14:03.472Z level=INFO msg="Upload complete. (188ms)"7611lb # [ 170.326836] niks3-server[1116]: 2026/09/23 05:14:03 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7612worker1 # [ 170.373296] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=7527613worker1 # [ 170.378322] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267614worker1 # [ 170.389353] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7615worker1 # [ 170.394940] nix-daemon[863]: reaped child process 2273, status = succeeded7616worker1 # [ 170.400302] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7617worker1 # [ 171.443438] nix-daemon[863]: reaped child process 2282, status = succeeded7618worker1 # [ 171.528103] nix-daemon[863]: accepted connection from pid 2298, user nix-grpc-daemon (trusted)7619worker1 # [ 171.541670] nix-daemon[863]: reaped child process 2301, status = succeeded7620lb # [ 171.506651] niks3-server[1116]: 2026/09/23 05:14:04 INFO Received uploads request method=POST path=/api/pending_closures7621worker1 # [ 171.561511] nix-grpc-daemon[1970]: time=2026-09-23T05:14:04.664Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7622worker1 # [ 171.563191] nix-grpc-daemon[1970]: time=2026-09-23T05:14:04.666Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7623lb # [ 171.550349] niks3-server[1116]: 2026/09/23 05:14:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7624lb # [ 171.557167] niks3-server[1116]: 2026/09/23 05:14:04 INFO Signed narinfos id=85 count=17625worker1 # [ 171.608680] nix-grpc-daemon[1970]: time=2026-09-23T05:14:04.712Z level=INFO msg="Uploading 1 narinfos"7626lb # [ 171.568200] niks3-server[1116]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7627lb # [ 171.573639] niks3-server[1116]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7628lb # [ 171.578109] niks3-server[1116]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7629lb # [ 171.586526] niks3-server[1116]: 2026/09/23 05:14:04 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7630lb # [ 171.590352] niks3-server[1116]: 2026/09/23 05:14:04 INFO Completed upload id=857631worker1 # [ 171.635212] nix-grpc-daemon[1970]: time=2026-09-23T05:14:04.738Z level=INFO msg="Upload complete. (226ms)"7632lb # [ 171.593148] niks3-server[1116]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7633worker1 # [ 171.641594] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=12 outputs=17634worker2 # [ 171.628743] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:04Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17635worker1 # [ 171.657908] nix-daemon[863]: reaped child process 2280, status = succeeded7636worker1 # [ 171.692249] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267637worker1 # [ 171.705402] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419267638worker1 # [ 171.719826] nix-grpc-daemon[1970]: time=2026-09-23T05:14:04.823Z level=INFO msg="All 2 paths already cached"7639worker1 # [ 171.727999] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419267640worker1 # [ 171.736888] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7641worker1 # [ 171.842124] nix-daemon[863]: accepted connection from pid 2305, user nix-grpc-daemon (trusted)7642worker1 # [ 171.858815] nix-daemon[863]: reaped child process 2308, status = succeeded7643lb # [ 171.819904] niks3-server[1116]: 2026/09/23 05:14:05 INFO Received uploads request method=POST path=/api/pending_closures7644worker1 # [ 171.872297] nix-grpc-daemon[1970]: time=2026-09-23T05:14:04.976Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7645worker1 # [ 171.873813] nix-grpc-daemon[1970]: time=2026-09-23T05:14:04.977Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7646lb # [ 171.851812] niks3-server[1116]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7647lb # [ 171.861973] niks3-server[1116]: 2026/09/23 05:14:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7648worker1 # [ 171.912647] nix-grpc-daemon[1970]: time=2026-09-23T05:14:05.016Z level=INFO msg="Uploading 1 narinfos"7649lb # [ 171.867916] niks3-server[1116]: 2026/09/23 05:14:05 INFO Signed narinfos id=86 count=17650lb # [ 171.872615] niks3-server[1116]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7651lb # [ 171.886842] niks3-server[1116]: 2026/09/23 05:14:05 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7652lb # [ 171.892661] niks3-server[1116]: 2026/09/23 05:14:05 INFO Completed upload id=867653worker1 # [ 171.939541] nix-grpc-daemon[1970]: time=2026-09-23T05:14:05.043Z level=INFO msg="Upload complete. (189ms)"7654lb # [ 171.897903] niks3-server[1116]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7655worker1 # [ 171.944552] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=0 paths=1 nar_bytes_in=7527656worker1 # [ 171.950750] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419267657worker1 # [ 171.964117] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7658worker1 # [ 171.973132] nix-daemon[863]: reaped child process 2303, status = succeeded7659worker1 # [ 171.977220] nix-daemon[863]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7660worker1 # [ 173.214804] nix-daemon[863]: reaped child process 2312, status = succeeded7661worker1 # [ 173.304107] nix-daemon[863]: accepted connection from pid 2328, user nix-grpc-daemon (trusted)7662worker1 # [ 173.318425] nix-daemon[863]: reaped child process 2331, status = succeeded7663lb # [ 173.282342] niks3-server[1116]: 2026/09/23 05:14:06 INFO Received uploads request method=POST path=/api/pending_closures7664worker1 # [ 173.335530] nix-grpc-daemon[1970]: time=2026-09-23T05:14:06.438Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7665worker1 # [ 173.337330] nix-grpc-daemon[1970]: time=2026-09-23T05:14:06.440Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7666lb # [ 173.331170] niks3-server[1116]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7667lb # [ 173.334858] niks3-server[1116]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7668lb # [ 173.341118] niks3-server[1116]: 2026/09/23 05:14:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7669lb # [ 173.348041] niks3-server[1116]: 2026/09/23 05:14:06 INFO Signed narinfos id=87 count=17670worker1 # [ 173.392997] nix-grpc-daemon[1970]: time=2026-09-23T05:14:06.496Z level=INFO msg="Uploading 1 narinfos"7671lb # [ 173.356603] niks3-server[1116]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7672lb # [ 173.363583] niks3-server[1116]: 2026/09/23 05:14:06 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7673lb # [ 173.368464] niks3-server[1116]: 2026/09/23 05:14:06 INFO Completed upload id=877674worker1 # [ 173.413313] nix-grpc-daemon[1970]: time=2026-09-23T05:14:06.516Z level=INFO msg="Upload complete. (226ms)"7675lb # [ 173.371196] niks3-server[1116]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7676worker1 # [ 173.419858] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41926 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=13 outputs=17677worker1 # [ 173.435511] nix-daemon[863]: reaped child process 2310, status = succeeded7678client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.76 seconds)7679worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7680worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7681client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7682client # [ 173.561540] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7683client # Running as unit: gpu.service; invocation ID: 66a148ea0dea490d9c7c451545400aa47684client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7685lb: must succeed: curl -sf localhost:9901/clusters7686lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7687lb: must succeed: curl -sf localhost:9901/clusters7688lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7689worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7690worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7691worker2 # [ 173.716561] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:530487692client # [ 173.702092] nix[1580]: don't know how to build these paths:7693client # [ 173.702973] nix[1580]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7694worker2 # [ 173.726157] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530487695client # [ 173.726662] nix[1580]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7696worker2 # [ 173.746504] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:06Z level=debug event=want client=3 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07697lb: must succeed: curl -sf localhost:9901/clusters7698lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7699lb: must succeed: curl -sf localhost:9901/clusters7700lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7701worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7702worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7703client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7704lb # [ 176.927224] envoy[912]: conn peer=[2001:db8:1::1]:47490 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27150 tx=12301 ms=145837705client # [ 203.727828] nix[1580]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7706client # [ 203.737727] nix[1580]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7707client # [ 203.769674] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7708client # [ 203.773074] systemd[1]: gpu.service: Failed with result 'exit-code'.7709client # [ 203.777899] systemd[1]: gpu.service: Consumed 134ms CPU time over 30.208s wall clock time, 35.6M memory peak, 5.5K incoming IP traffic, 5.2K outgoing IP traffic.7710client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.43 seconds)7711client: must succeed: journalctl -u gpu -o cat7712client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7713lb: must succeed: curl -sf localhost:9901/clusters7714lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7715lb: must succeed: curl -sf localhost:9901/clusters7716lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7717worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7718worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7719(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.66 seconds)7720subtest: low disk drains a worker and builds go to the other7721worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7722worker1 # [ 205.409232] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:38Z level=info event=unhealthy reason=min_free7723worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)7724worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7725worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.02 seconds)7726client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27727client # don't know how to build these paths:7728client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7729worker2 # [ 205.567572] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:530487730worker2 # [ 205.573581] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530487731worker2 # [ 205.592773] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:38Z level=debug event=want client=4 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07732worker2 # [ 205.597904] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:38Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17733worker2 # [ 205.636956] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487734worker2 # [ 205.649868] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530487735client # copying 1 paths to 'lb:50051'...7736client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7737worker2 # [ 205.662570] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530487738worker2 # [ 205.674495] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7739worker2 # [ 205.776337] nix-daemon[852]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)7740worker2 # [ 205.790588] nix-daemon[852]: reaped child process 1827, status = succeeded7741lb # [ 205.769318] niks3-server[1116]: 2026/09/23 05:14:38 INFO Received uploads request method=POST path=/api/pending_closures7742worker2 # [ 205.808470] nix-grpc-daemon[1536]: time=2026-09-23T05:14:38.674Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7743worker2 # [ 205.810513] nix-grpc-daemon[1536]: time=2026-09-23T05:14:38.676Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7744lb # [ 205.799118] niks3-server[1116]: 2026/09/23 05:14:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7745lb # [ 205.807608] niks3-server[1116]: 2026/09/23 05:14:38 INFO Signed narinfos id=88 count=17746worker2 # [ 205.841718] nix-grpc-daemon[1536]: time=2026-09-23T05:14:38.707Z level=INFO msg="Uploading 1 narinfos"7747lb # [ 205.816797] niks3-server[1116]: 2026/09/23 05:14:39 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7748lb # [ 205.823508] niks3-server[1116]: 2026/09/23 05:14:39 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7749lb # [ 205.832069] niks3-server[1116]: 2026/09/23 05:14:39 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7750lb # [ 205.839321] niks3-server[1116]: 2026/09/23 05:14:39 INFO Completed upload id=887751worker2 # [ 205.869971] nix-grpc-daemon[1536]: time=2026-09-23T05:14:38.735Z level=INFO msg="Upload complete. (183ms)"7752worker2 # [ 205.874129] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 paths=1 nar_bytes_in=5127753lb # [ 205.847642] niks3-server[1116]: 2026/09/23 05:14:39 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7754worker2 # [ 205.878874] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487755worker2 # [ 205.890106] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7756worker2 # [ 205.895148] nix-daemon[852]: reaped child process 1822, status = succeeded7757worker2 # [ 205.899868] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7758client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7759client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7760client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7761client # farm-a> LOG-farm-a-drain7762worker2 # [ 206.715570] nix-daemon[852]: reaped child process 1831, status = succeeded7763worker2 # [ 206.791183] nix-daemon[852]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)7764worker2 # [ 206.804699] nix-daemon[852]: reaped child process 1850, status = succeeded7765lb # [ 206.782882] niks3-server[1116]: 2026/09/23 05:14:39 INFO Received uploads request method=POST path=/api/pending_closures7766worker2 # [ 206.821517] nix-grpc-daemon[1536]: time=2026-09-23T05:14:39.687Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7767worker2 # [ 206.823035] nix-grpc-daemon[1536]: time=2026-09-23T05:14:39.689Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7768lb # [ 206.822705] niks3-server[1116]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7769lb # [ 206.833946] niks3-server[1116]: 2026/09/23 05:14:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7770lb # [ 206.842643] niks3-server[1116]: 2026/09/23 05:14:40 INFO Signed narinfos id=89 count=17771worker2 # [ 206.875457] nix-grpc-daemon[1536]: time=2026-09-23T05:14:39.741Z level=INFO msg="Uploading 1 narinfos"7772lb # [ 206.846740] niks3-server[1116]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7773lb # [ 206.852273] niks3-server[1116]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7774lb # [ 206.863077] niks3-server[1116]: 2026/09/23 05:14:40 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7775lb # [ 206.866279] niks3-server[1116]: 2026/09/23 05:14:40 INFO Completed upload id=897776worker2 # [ 206.896754] nix-grpc-daemon[1536]: time=2026-09-23T05:14:39.762Z level=INFO msg="Upload complete. (190ms)"7777lb # [ 206.868730] niks3-server[1116]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7778worker2 # [ 206.902468] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=14 outputs=17779worker2 # [ 206.914696] nix-daemon[852]: reaped child process 1829, status = succeeded7780worker2 # [ 206.949261] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:39Z level=debug event=want client=4 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07781worker2 # [ 206.954674] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:39Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17782lb # [ 206.928288] envoy[912]: conn peer=[2001:db8:1::1]:53846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300527783worker2 # [ 206.960204] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487784worker2 # [ 206.972247] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530487785worker2 # [ 206.985114] nix-grpc-daemon[1536]: time=2026-09-23T05:14:39.850Z level=INFO msg="All 1 paths already cached"7786client # copying 1 paths to 'lb:50051'...7787client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7788worker2 # [ 206.993596] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530487789worker2 # [ 207.003272] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7790worker2 # [ 207.098411] nix-daemon[852]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7791worker2 # [ 207.112324] nix-daemon[852]: reaped child process 1859, status = succeeded7792lb # [ 207.090201] niks3-server[1116]: 2026/09/23 05:14:40 INFO Received uploads request method=POST path=/api/pending_closures7793worker2 # [ 207.128316] nix-grpc-daemon[1536]: time=2026-09-23T05:14:39.994Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7794worker2 # [ 207.130028] nix-grpc-daemon[1536]: time=2026-09-23T05:14:39.995Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7795lb # [ 207.122546] niks3-server[1116]: 2026/09/23 05:14:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7796worker2 # [ 207.159033] nix-grpc-daemon[1536]: time=2026-09-23T05:14:40.024Z level=INFO msg="Uploading 1 narinfos"7797lb # [ 207.128776] niks3-server[1116]: 2026/09/23 05:14:40 INFO Signed narinfos id=90 count=17798lb # [ 207.134508] niks3-server[1116]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7799lb # [ 207.145611] niks3-server[1116]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7800lb # [ 207.155242] niks3-server[1116]: 2026/09/23 05:14:40 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7801lb # [ 207.161864] niks3-server[1116]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7802lb # [ 207.165897] niks3-server[1116]: 2026/09/23 05:14:40 INFO Completed upload id=907803worker2 # [ 207.196481] nix-grpc-daemon[1536]: time=2026-09-23T05:14:40.062Z level=INFO msg="Upload complete. (182ms)"7804worker2 # [ 207.200919] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 paths=1 nar_bytes_in=6247805worker2 # [ 207.208113] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487806worker2 # [ 207.217818] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7807worker2 # [ 207.229154] nix-daemon[852]: reaped child process 1854, status = succeeded7808worker2 # [ 207.231964] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7809client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7810client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7811client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7812client # farm-b> LOG-farm-b-drain7813worker2 # [ 207.847940] nix-daemon[852]: reaped child process 1863, status = succeeded7814worker2 # [ 207.923378] nix-daemon[852]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)7815worker2 # [ 207.937920] nix-daemon[852]: reaped child process 1882, status = succeeded7816lb # [ 207.913679] niks3-server[1116]: 2026/09/23 05:14:41 INFO Received uploads request method=POST path=/api/pending_closures7817worker2 # [ 207.952473] nix-grpc-daemon[1536]: time=2026-09-23T05:14:40.818Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7818worker2 # [ 207.954054] nix-grpc-daemon[1536]: time=2026-09-23T05:14:40.820Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7819lb # [ 207.956536] niks3-server[1116]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7820lb # [ 207.964369] niks3-server[1116]: 2026/09/23 05:14:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7821lb # [ 207.969888] niks3-server[1116]: 2026/09/23 05:14:41 INFO Signed narinfos id=91 count=17822worker2 # [ 208.004320] nix-grpc-daemon[1536]: time=2026-09-23T05:14:40.870Z level=INFO msg="Uploading 1 narinfos"7823lb # [ 207.978822] niks3-server[1116]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7824lb # [ 207.988563] niks3-server[1116]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7825lb # [ 207.995553] niks3-server[1116]: 2026/09/23 05:14:41 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7826lb # [ 208.001108] niks3-server[1116]: 2026/09/23 05:14:41 INFO Completed upload id=917827worker2 # [ 208.031527] nix-grpc-daemon[1536]: time=2026-09-23T05:14:40.897Z level=INFO msg="Upload complete. (193ms)"7828lb # [ 208.003458] niks3-server[1116]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7829worker2 # [ 208.042203] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=15 outputs=17830worker2 # [ 208.050780] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:40Z level=debug event=want client=4 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07831worker2 # [ 208.055207] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:40Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17832worker2 # [ 208.065982] nix-daemon[852]: reaped child process 1861, status = succeeded7833worker2 # [ 208.095872] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487834worker2 # [ 208.108774] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:530487835worker2 # [ 208.122123] nix-grpc-daemon[1536]: time=2026-09-23T05:14:40.987Z level=INFO msg="All 2 paths already cached"7836client # copying 1 paths to 'lb:50051'...7837client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7838worker2 # [ 208.131716] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:530487839worker2 # [ 208.142183] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7840worker2 # [ 208.243757] nix-daemon[852]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7841worker2 # [ 208.259569] nix-daemon[852]: reaped child process 1891, status = succeeded7842lb # [ 208.235614] niks3-server[1116]: 2026/09/23 05:14:41 INFO Received uploads request method=POST path=/api/pending_closures7843worker2 # [ 208.274071] nix-grpc-daemon[1536]: time=2026-09-23T05:14:41.140Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7844worker2 # [ 208.276085] nix-grpc-daemon[1536]: time=2026-09-23T05:14:41.141Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7845lb # [ 208.265908] niks3-server[1116]: 2026/09/23 05:14:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7846lb # [ 208.272146] niks3-server[1116]: 2026/09/23 05:14:41 INFO Signed narinfos id=92 count=17847worker2 # [ 208.305610] nix-grpc-daemon[1536]: time=2026-09-23T05:14:41.171Z level=INFO msg="Uploading 1 narinfos"7848lb # [ 208.280069] niks3-server[1116]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7849lb # [ 208.290948] niks3-server[1116]: 2026/09/23 05:14:41 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7850lb # [ 208.296692] niks3-server[1116]: 2026/09/23 05:14:41 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7851worker2 # [ 208.332506] nix-grpc-daemon[1536]: time=2026-09-23T05:14:41.198Z level=INFO msg="Upload complete. (178ms)"7852lb # [ 208.302309] niks3-server[1116]: 2026/09/23 05:14:41 INFO Completed upload id=927853worker2 # [ 208.336355] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=0 paths=1 nar_bytes_in=7527854worker2 # [ 208.341400] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:530487855worker2 # [ 208.350584] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7856worker2 # [ 208.357520] nix-daemon[852]: reaped child process 1886, status = succeeded7857worker2 # [ 208.362131] nix-daemon[852]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)7858client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7859client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7860client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7861client # farm-top> LOG-farm-top-drain7862worker2 # [ 209.378569] nix-daemon[852]: reaped child process 1895, status = succeeded7863worker2 # [ 209.454185] nix-daemon[852]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)7864worker2 # [ 209.468078] nix-daemon[852]: reaped child process 1914, status = succeeded7865lb # [ 209.444622] niks3-server[1116]: 2026/09/23 05:14:42 INFO Received uploads request method=POST path=/api/pending_closures7866worker2 # [ 209.485363] nix-grpc-daemon[1536]: time=2026-09-23T05:14:42.351Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7867worker2 # [ 209.486851] nix-grpc-daemon[1536]: time=2026-09-23T05:14:42.352Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7868lb # [ 209.483932] niks3-server[1116]: 2026/09/23 05:14:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7869lb # [ 209.490427] niks3-server[1116]: 2026/09/23 05:14:42 INFO Signed narinfos id=93 count=17870worker2 # [ 209.524218] nix-grpc-daemon[1536]: time=2026-09-23T05:14:42.389Z level=INFO msg="Uploading 1 narinfos"7871lb # [ 209.500581] niks3-server[1116]: 2026/09/23 05:14:42 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7872lb # [ 209.507856] niks3-server[1116]: 2026/09/23 05:14:42 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7873lb # [ 209.512654] niks3-server[1116]: 2026/09/23 05:14:42 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7874lb # [ 209.522847] niks3-server[1116]: 2026/09/23 05:14:42 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7875lb # [ 209.528527] niks3-server[1116]: 2026/09/23 05:14:42 INFO Completed upload id=937876worker2 # [ 209.559197] nix-grpc-daemon[1536]: time=2026-09-23T05:14:42.425Z level=INFO msg="Upload complete. (189ms)"7877lb # [ 209.531551] niks3-server[1116]: 2026/09/23 05:14:42 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7878worker2 # [ 209.568131] nix-grpc-daemon[1447]: ts=2026-09-23T05:14:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53048 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=16 outputs=17879worker2 # [ 209.581494] nix-daemon[852]: reaped child process 1893, status = succeeded7880client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.18 seconds)7881client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7882client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7883worker1: must succeed: rm /nix/.rw-store/fill7884worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7885worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7886worker1 # [ 210.411222] nix-grpc-daemon[1909]: ts=2026-09-23T05:14:43Z level=info event=healthy reason=min_free7887worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.04 seconds)7888(finished: subtest: low disk drains a worker and builds go to the other, in 6.54 seconds)7889(finished: run the VM test script, in 211.88 seconds)7890test script finished in 212.06s7891cleanup7892kill QemuMachine (pid 50)7893client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7894client # [2026-09-23T05:14:44Z INFO virtiofsd] Client disconnected, shutting down7895client # [2026-09-23T05:14:44Z INFO virtiofsd] Client disconnected, shutting down7896client # [2026-09-23T05:14:44Z INFO virtiofsd] Client disconnected, shutting down7897kill QemuMachine (pid 49)7898lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7899lb # [2026-09-23T05:14:44Z INFO virtiofsd] Client disconnected, shutting down7900lb # [2026-09-23T05:14:44Z INFO virtiofsd] Client disconnected, shutting down7901lb # [2026-09-23T05:14:44Z INFO virtiofsd] Client disconnected, shutting down7902kill QemuMachine (pid 51)7903worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7904worker1 # [2026-09-23T05:14:44Z INFO virtiofsd] Client disconnected, shutting down7905worker1 # [2026-09-23T05:14:44Z INFO virtiofsd] Client disconnected, shutting down7906worker1 # [2026-09-23T05:14:44Z INFO virtiofsd] Client disconnected, shutting down7907kill QemuMachine (pid 52)7908worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7909worker2 # [2026-09-23T05:14:45Z INFO virtiofsd] Client disconnected, shutting down7910worker2 # [2026-09-23T05:14:45Z INFO virtiofsd] Client disconnected, shutting down7911worker2 # [2026-09-23T05:14:45Z INFO virtiofsd] Client disconnected, shutting down7912(finished: cleanup, in 0.99 seconds)7913additionally exposed symbols:7914 client, lb, worker1, worker2,7915 vlan1,7916 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh