vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #559
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.RQhQzuTa25', fmt=raw size=107374182414client: QEMU running (pid 49)15client # mke2fs 1.47.4 (6-Mar-2025)16lb # Disk image does not exist, creating the virtualisation disk image...17client # Discarding device blocks: 0/262144 done18lb # Formatting '/build/vm-state-lb/tmp.6CF368vp42', fmt=raw size=107374182419client # Creating filesystem with 262144 4k blocks and 65536 inodes20lb # mke2fs 1.47.4 (6-Mar-2025)21lb: QEMU running (pid 50)22worker1 # Disk image does not exist, creating the virtualisation disk image...23lb # Discarding device blocks: 0/262144 done24worker1: QEMU running (pid 51)25client # Filesystem UUID: d1837d7b-7cad-41be-9369-6315479279f826worker1 # Formatting '/build/vm-state-worker1/tmp.CvNYMxuPFd', fmt=raw size=107374182427client # Superblock backups stored on blocks:28worker1 # mke2fs 1.47.4 (6-Mar-2025)29client # 32768, 98304, 163840, 22937630worker1 # Discarding device blocks: 0/262144 done31client # 32worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes33client # Allocating group tables: 0/8 done34worker1 # Filesystem UUID: ce910eb5-ac87-49db-9b0e-64075be9814735client # Writing inode tables: 0/8 done36worker1 # Superblock backups stored on blocks:37client # Creating journal (8192 blocks): done38worker1 # 32768, 98304, 163840, 22937639client # Writing superblocks and filesystem accounting information: 0/8 done40worker1 # 41client # 42worker1 # Allocating group tables: 0/8 done43client # Virtualisation disk image created.44worker1 # Writing inode tables: 0/8 done45client # Starting virtiofs daemons...46worker1 # Creating journal (8192 blocks): done47client # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)48worker1 # Writing superblocks and filesystem accounting information: 0/8 done49client # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether50worker1 # 51client # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...52lb # Creating filesystem with 262144 4k blocks and 65536 inodes53client # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)54client # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether55worker1 # Virtualisation disk image created.56client # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)57worker1 # Starting virtiofs daemons...58client # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether59worker1 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)60client # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...61worker1 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether62client # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...63worker1 # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...64client # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests65worker2 # Disk image does not exist, creating the virtualisation disk image...66worker1 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67client # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests68worker2 # Formatting '/build/vm-state-worker2/tmp.670QnHUrxS', fmt=raw size=107374182469worker1 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70client # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests71worker2 # mke2fs 1.47.4 (6-Mar-2025)72worker1 # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...73worker2: QEMU running (pid 52)74lb # Filesystem UUID: 3538cfd2-bd36-4280-86ae-357e2a48f39275worker2 # Discarding device blocks: 0/262144 done76lb # Superblock backups stored on blocks:77worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes78lb # 32768, 98304, 163840, 22937679worker1 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80(finished: start all VMs, in 0.22 seconds)81worker2 # Filesystem UUID: d9fcfd20-3f20-4f1d-b028-6a9b6653a44a82lb # 83worker1 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84client: waiting for unit network-addresses-eth1.service85worker2 # Superblock backups stored on blocks:86lb # Allocating group tables: 0/8 done87worker1 # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...88client: waiting for the VM to finish booting89worker2 # 32768, 98304, 163840, 22937690lb # Writing inode tables: 0/8 done91worker2 # 92lb # Creating journal (8192 blocks): done93worker2 # Allocating group tables: 0/8 done94lb # Writing superblocks and filesystem accounting information: 0/8 done95worker2 # Writing inode tables: 0/8 done96lb # 97worker2 # Creating journal (8192 blocks): done98lb # Virtualisation disk image created.99worker2 # Writing superblocks and filesystem accounting information: 0/8 done100lb # Starting virtiofs daemons...101worker2 # 102lb # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)103worker2 # Virtualisation disk image created.104lb # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether105worker2 # Starting virtiofs daemons...106lb # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...107worker2 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)108lb # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)109worker2 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether110lb # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether111worker2 # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...112lb # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...113worker2 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)114lb # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)115worker2 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether116lb # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether117worker2 # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...118lb # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...119worker2 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests121worker2 # [2026-09-25T16:31:22Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether122lb # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests123worker2 # [2026-09-25T16:31:22Z INFO virtiofsd] Waiting for vhost-user socket connection...124lb # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests125worker2 # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests126worker1 # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests127worker2 # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests128worker1 # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests129worker2 # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-25T16:31:22Z INFO virtiofsd] Client connected, servicing requests131worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker1 # 136worker1 # 137client # 138client # 139lb # 140worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141lb # 142client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00143lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00144worker2 # 145worker2 # 146worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...147worker1 # 148worker1 # 149worker1 # 150worker1 # 151worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00152client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...153client # 154client # 155client # 156client # 157lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...158lb # 159lb # 160lb # 161lb # 162worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00163client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00164lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00165worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...166worker2 # 167worker2 # 168worker2 # 169worker2 # 170worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00171worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...172worker1 # 173worker1 # 174client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...175lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...176client # 177lb # 178client # 179lb # 180worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker2 # 182worker2 # 183lb # Booting from ROM...184worker1 # Booting from ROM...185client # Booting from ROM...186worker2 # Booting from ROM...187lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0189lb # [ 0.000000] BIOS-provided physical RAM map:190lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026192lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved193lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved194lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable195lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved196worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/f94wjck2a7wq46iir6a6zbavnfk7qfqj-nixos-system-worker1-test/init regInfo=/nix/store/bmgjg3l7l2l8lkg7xd7sfzlrxaklnwwz-closure-info/registration console=ttyS0,115200n8 console=tty0197lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved198worker1 # [ 0.000000] BIOS-provided physical RAM map:199lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved200worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable201lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved202worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved203lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved204worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026205worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved206lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved207worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable208lb # [ 0.000000] NX (Execute Disable) protection: active209lb # [ 0.000000] APIC: Static calls initialized210worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved211lb # [ 0.000000] SMBIOS 2.8 present.212worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved213lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014214worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved215worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m68sx34dr7yy1drfgmzf0cyxm52cz9hp-nixos-system-worker2-test/init regInfo=/nix/store/zijq974g613q511fgj3vy2szbqsd8kv2-closure-info/registration console=ttyS0,115200n8 console=tty0216lb # [ 0.000000] DMI: Memory slots populated: 1/1217lb # [ 0.000000] Hypervisor detected: KVM218worker2 # [ 0.000000] BIOS-provided physical RAM map:219worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved220lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000221worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable222worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved223lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00224worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved225worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved226lb # [ 0.000000] kvm-clock: using sched offset of 475972540 cycles227worker1 # [ 0.000000] NX (Execute Disable) protection: active228worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved229worker1 # [ 0.000000] APIC: Static calls initialized230lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns231worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable232worker1 # [ 0.000000] SMBIOS 2.8 present.233lb # [ 0.000005] tsc: Detected 2400.012 MHz processor234client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026235worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved236lb # [ 0.000823] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000237worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014238worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved239worker1 # [ 0.000000] DMI: Memory slots populated: 1/1240lb # [ 0.000850] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs241worker1 # [ 0.000000] Hypervisor detected: KVM242worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved243lb # [ 0.000853] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT244worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000245worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved246lb # [ 0.002767] found SMP MP-table at [mem 0x000f5450-0x000f545f]247worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00248lb # [ 0.002779] Using GB pages for direct mapping249client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6rlhi5dcdiiclafqnpzgp1y7qxb2ara2-nixos-system-client-test/init regInfo=/nix/store/h3j26zp3yqdh88ddln89hmpjd3kw928v-closure-info/registration console=ttyS0,115200n8 console=tty0250worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved251worker1 # [ 0.000000] kvm-clock: using sched offset of 478269960 cycles252lb # [ 0.002923] RAMDISK: [mem 0x5e36e000-0x5ffcffff]253client # [ 0.000000] BIOS-provided physical RAM map:254worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved255lb # [ 0.002932] ACPI: Early table checksum verification disabled256client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable257worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns258worker2 # [ 0.000000] NX (Execute Disable) protection: active259lb # [ 0.002935] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )260worker1 # [ 0.000004] tsc: Detected 2400.012 MHz processor261worker2 # [ 0.000000] APIC: Static calls initialized262client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved263worker2 # [ 0.000000] SMBIOS 2.8 present.264worker1 # [ 0.000803] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000265lb # [ 0.002939] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)266client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved267worker1 # [ 0.000835] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs268worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014269lb # [ 0.002943] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)270client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable271worker2 # [ 0.000000] DMI: Memory slots populated: 1/1272worker1 # [ 0.000837] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT273client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved274worker2 # [ 0.000000] Hypervisor detected: KVM275lb # [ 0.002950] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)276worker1 # [ 0.002731] found SMP MP-table at [mem 0x000f5450-0x000f545f]277worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000278client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved279lb # [ 0.002952] ACPI: FACS 0x000000005FFE0000 000040280worker1 # [ 0.002741] Using GB pages for direct mapping281worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00282worker1 # [ 0.002811] RAMDISK: [mem 0x5e370000-0x5ffcffff]283client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved284lb # [ 0.002954] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)285worker2 # [ 0.000000] kvm-clock: using sched offset of 460816586 cycles286worker1 # [ 0.002825] ACPI: Early table checksum verification disabled287client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved288lb # [ 0.002955] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)289worker1 # [ 0.002827] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )290client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved291worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns292lb # [ 0.002957] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)293worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor294client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved295worker1 # [ 0.002831] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)296worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000297client # [ 0.000000] NX (Execute Disable) protection: active298lb # [ 0.002958] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)299worker1 # [ 0.002835] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)300client # [ 0.000000] APIC: Static calls initialized301client # [ 0.000000] SMBIOS 2.8 present.302lb # [ 0.002960] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]303worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs304worker1 # [ 0.002842] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)305worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT306lb # [ 0.002960] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]307client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014308worker1 # [ 0.002844] ACPI: FACS 0x000000005FFE0000 000040309client # [ 0.000000] DMI: Memory slots populated: 1/1310worker2 # [ 0.002728] found SMP MP-table at [mem 0x000f5450-0x000f545f]311lb # [ 0.002961] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]312client # [ 0.000000] Hypervisor detected: KVM313worker1 # [ 0.002845] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)314worker2 # [ 0.002738] Using GB pages for direct mapping315lb # [ 0.002961] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]316client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000317worker2 # [ 0.002808] RAMDISK: [mem 0x5e36f000-0x5ffcffff]318worker1 # [ 0.002847] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)319client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00320lb # [ 0.002962] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]321worker2 # [ 0.002815] ACPI: Early table checksum verification disabled322client # [ 0.000000] kvm-clock: using sched offset of 474737876 cycles323worker2 # [ 0.002818] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )324worker1 # [ 0.002849] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)325lb # [ 0.002962] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]326lb # [ 0.002963] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]327worker2 # [ 0.002822] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)328client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns329worker1 # [ 0.002850] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)330lb # [ 0.003194] No NUMA configuration found331client # [ 0.000005] tsc: Detected 2400.012 MHz processor332worker1 # [ 0.002851] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]333worker2 # [ 0.002825] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)334lb # [ 0.003195] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]335client # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000336lb # [ 0.003197] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]337worker1 # [ 0.002852] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]338worker2 # [ 0.002832] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)339lb # [ 0.005629] Zone ranges:340client # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs341worker2 # [ 0.002834] ACPI: FACS 0x000000005FFE0000 000040342worker1 # [ 0.002853] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]343lb # [ 0.005630] DMA [mem 0x0000000000001000-0x0000000000ffffff]344client # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT345worker1 # [ 0.002853] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]346worker2 # [ 0.002835] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)347lb # [ 0.005631] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]348client # [ 0.002721] found SMP MP-table at [mem 0x000f5450-0x000f545f]349lb # [ 0.005633] Normal empty350client # [ 0.002732] Using GB pages for direct mapping351lb # [ 0.005633] Device empty352worker1 # [ 0.002854] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]353worker2 # [ 0.002837] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)354client # [ 0.002903] RAMDISK: [mem 0x5e370000-0x5ffcffff]355lb # [ 0.005634] Movable zone start for each node356worker1 # [ 0.002854] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]357lb # [ 0.005635] Early memory node ranges358client # [ 0.002913] ACPI: Early table checksum verification disabled359worker2 # [ 0.002838] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)360lb # [ 0.005635] node 0: [mem 0x0000000000001000-0x000000000009efff]361client # [ 0.002915] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )362worker1 # [ 0.002855] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]363worker1 # [ 0.003075] No NUMA configuration found364lb # [ 0.005636] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]365worker2 # [ 0.002840] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)366client # [ 0.002919] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)367worker1 # [ 0.003076] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]368lb # [ 0.005637] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]369worker2 # [ 0.002841] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]370worker1 # [ 0.003078] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]371client # [ 0.002923] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)372lb # [ 0.005659] On node 0, zone DMA: 1 pages in unavailable ranges373worker1 # [ 0.005304] Zone ranges:374worker2 # [ 0.002842] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]375lb # [ 0.005911] On node 0, zone DMA: 97 pages in unavailable ranges376client # [ 0.002930] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)377worker1 # [ 0.005305] DMA [mem 0x0000000000001000-0x0000000000ffffff]378worker2 # [ 0.002843] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]379lb # [ 0.031528] On node 0, zone DMA32: 40 pages in unavailable ranges380client # [ 0.002933] ACPI: FACS 0x000000005FFE0000 000040381worker1 # [ 0.005306] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]382lb # [ 0.031985] ACPI: PM-Timer IO Port: 0x608383worker2 # [ 0.002843] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]384worker1 # [ 0.005307] Normal empty385worker1 # [ 0.005308] Device empty386client # [ 0.002934] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)387lb # [ 0.031996] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])388worker2 # [ 0.002844] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]389worker1 # [ 0.005309] Movable zone start for each node390worker1 # [ 0.005310] Early memory node ranges391lb # [ 0.032021] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23392client # [ 0.002936] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)393worker2 # [ 0.002844] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]394lb # [ 0.032024] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)395worker1 # [ 0.005310] node 0: [mem 0x0000000000001000-0x000000000009efff]396client # [ 0.002937] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)397worker2 # [ 0.002845] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]398worker1 # [ 0.005311] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]399lb # [ 0.032025] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)400worker2 # [ 0.003075] No NUMA configuration found401client # [ 0.002939] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)402lb # [ 0.032026] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)403worker1 # [ 0.005312] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]404worker2 # [ 0.003076] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]405client # [ 0.002940] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]406worker1 # [ 0.005330] On node 0, zone DMA: 1 pages in unavailable ranges407worker2 # [ 0.003078] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]408lb # [ 0.032027] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)409worker2 # [ 0.005335] Zone ranges:410client # [ 0.002941] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]411worker1 # [ 0.005586] On node 0, zone DMA: 97 pages in unavailable ranges412lb # [ 0.032028] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)413worker2 # [ 0.005336] DMA [mem 0x0000000000001000-0x0000000000ffffff]414worker1 # [ 0.031199] On node 0, zone DMA32: 40 pages in unavailable ranges415client # [ 0.002942] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]416lb # [ 0.032031] ACPI: Using ACPI (MADT) for SMP configuration information417worker1 # [ 0.031657] ACPI: PM-Timer IO Port: 0x608418worker2 # [ 0.005337] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]419worker2 # [ 0.005338] Normal empty420lb # [ 0.032031] ACPI: HPET id: 0x8086a201 base: 0xfed00000421client # [ 0.002942] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]422worker2 # [ 0.005339] Device empty423lb # [ 0.032035] TSC deadline timer available424worker1 # [ 0.031668] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])425worker2 # [ 0.005340] Movable zone start for each node426client # [ 0.002943] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]427lb # [ 0.032038] CPU topo: Max. logical packages: 1428worker1 # [ 0.031694] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23429worker2 # [ 0.005340] Early memory node ranges430lb # [ 0.032039] CPU topo: Max. logical dies: 1431client # [ 0.002943] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]432worker1 # [ 0.031697] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)433worker2 # [ 0.005341] node 0: [mem 0x0000000000001000-0x000000000009efff]434lb # [ 0.032039] CPU topo: Max. dies per package: 1435lb # [ 0.032043] CPU topo: Max. threads per core: 1436client # [ 0.002944] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]437worker2 # [ 0.005342] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]438worker1 # [ 0.031698] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)439client # [ 0.003191] No NUMA configuration found440lb # [ 0.032043] CPU topo: Num. cores per package: 1441worker1 # [ 0.031699] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)442worker2 # [ 0.005343] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]443lb # [ 0.032044] CPU topo: Num. threads per package: 1444client # [ 0.003192] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]445worker1 # [ 0.031700] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)446worker2 # [ 0.005362] On node 0, zone DMA: 1 pages in unavailable ranges447lb # [ 0.032044] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs448client # [ 0.003194] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]449client # [ 0.005619] Zone ranges:450worker2 # [ 0.005624] On node 0, zone DMA: 97 pages in unavailable ranges451worker1 # [ 0.031701] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)452lb # [ 0.032061] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()453client # [ 0.005620] DMA [mem 0x0000000000001000-0x0000000000ffffff]454worker2 # [ 0.031809] On node 0, zone DMA32: 40 pages in unavailable ranges455worker1 # [ 0.031703] ACPI: Using ACPI (MADT) for SMP configuration information456lb # [ 0.032096] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]457client # [ 0.005621] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]458worker2 # [ 0.032290] ACPI: PM-Timer IO Port: 0x608459worker1 # [ 0.031704] ACPI: HPET id: 0x8086a201 base: 0xfed00000460client # [ 0.005622] Normal empty461client # [ 0.005623] Device empty462worker1 # [ 0.031708] TSC deadline timer available463lb # [ 0.032097] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]464worker2 # [ 0.032300] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])465client # [ 0.005624] Movable zone start for each node466worker1 # [ 0.031712] CPU topo: Max. logical packages: 1467lb # [ 0.032099] [mem 0x60000000-0xafffffff] available for PCI devices468worker2 # [ 0.032326] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23469client # [ 0.005624] Early memory node ranges470worker1 # [ 0.031713] CPU topo: Max. logical dies: 1471lb # [ 0.032100] Booting paravirtualized kernel on KVM472worker2 # [ 0.032328] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)473client # [ 0.005625] node 0: [mem 0x0000000000001000-0x000000000009efff]474worker1 # [ 0.031713] CPU topo: Max. dies per package: 1475worker1 # [ 0.031716] CPU topo: Max. threads per core: 1476client # [ 0.005626] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]477worker2 # [ 0.032330] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)478lb # [ 0.032103] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns479worker1 # [ 0.031717] CPU topo: Num. cores per package: 1480client # [ 0.005627] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]481worker2 # [ 0.032331] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)482lb # [ 0.036659] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1483worker1 # [ 0.031717] CPU topo: Num. threads per package: 1484client # [ 0.005646] On node 0, zone DMA: 1 pages in unavailable ranges485worker2 # [ 0.032332] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)486worker1 # [ 0.031717] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs487lb # [ 0.038763] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152488client # [ 0.005929] On node 0, zone DMA: 97 pages in unavailable ranges489worker2 # [ 0.032333] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)490lb # [ 0.038805] kvm-guest: PV spinlocks disabled, single CPU491worker1 # [ 0.031735] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()492client # [ 0.034060] On node 0, zone DMA32: 40 pages in unavailable ranges493client # [ 0.034529] ACPI: PM-Timer IO Port: 0x608494worker2 # [ 0.032335] ACPI: Using ACPI (MADT) for SMP configuration information495worker1 # [ 0.031764] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]496worker2 # [ 0.032336] ACPI: HPET id: 0x8086a201 base: 0xfed00000497client # [ 0.034540] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])498worker2 # [ 0.032339] TSC deadline timer available499worker1 # [ 0.031766] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]500client # [ 0.034568] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23501worker2 # [ 0.032343] CPU topo: Max. logical packages: 1502worker1 # [ 0.031767] [mem 0x60000000-0xafffffff] available for PCI devices503worker2 # [ 0.032344] CPU topo: Max. logical dies: 1504client # [ 0.034571] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)505worker1 # [ 0.031768] Booting paravirtualized kernel on KVM506worker2 # [ 0.032344] CPU topo: Max. dies per package: 1507lb # [ 0.038807] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0508client # [ 0.034572] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)509worker2 # [ 0.032348] CPU topo: Max. threads per core: 1510worker1 # [ 0.031771] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns511worker2 # [ 0.032348] CPU topo: Num. cores per package: 1512client # [ 0.034574] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)513worker2 # [ 0.032349] CPU topo: Num. threads per package: 1514lb # [ 0.038898] Unknown kernel command line parameters "regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration", will be passed to user space.515client # [ 0.034575] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)516worker1 # [ 0.036250] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1517lb # [ 0.038911] random: crng init done518worker2 # [ 0.032349] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs519worker1 # [ 0.038347] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152520client # [ 0.034575] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)521lb # [ 0.038912] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes522worker1 # [ 0.038389] kvm-guest: PV spinlocks disabled, single CPU523worker2 # [ 0.032366] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()524client # [ 0.034578] ACPI: Using ACPI (MADT) for SMP configuration information525lb # [ 0.040994] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)526client # [ 0.034579] ACPI: HPET id: 0x8086a201 base: 0xfed00000527worker2 # [ 0.032395] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]528client # [ 0.034583] TSC deadline timer available529lb # [ 0.042032] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)530client # [ 0.034587] CPU topo: Max. logical packages: 1531worker2 # [ 0.032396] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]532lb # [ 0.042066] Fallback order for Node 0: 0533client # [ 0.034588] CPU topo: Max. logical dies: 1534worker2 # [ 0.032398] [mem 0x60000000-0xafffffff] available for PCI devices535lb # [ 0.042070] Built 1 zonelists, mobility grouping on. Total pages: 393078536client # [ 0.034588] CPU topo: Max. dies per package: 1537worker2 # [ 0.032399] Booting paravirtualized kernel on KVM538lb # [ 0.042070] Policy zone: DMA32539client # [ 0.034592] CPU topo: Max. threads per core: 1540worker1 # [ 0.038391] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/f94wjck2a7wq46iir6a6zbavnfk7qfqj-nixos-system-worker1-test/init regInfo=/nix/store/bmgjg3l7l2l8lkg7xd7sfzlrxaklnwwz-closure-info/registration console=ttyS0,115200n8 console=tty0541client # [ 0.034592] CPU topo: Num. cores per package: 1542lb # [ 0.044691] mem auto-init: stack:all(zero), heap alloc:on, heap free:off543worker2 # [ 0.032402] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns544client # [ 0.034593] CPU topo: Num. threads per package: 1545lb # [ 0.047667] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1546client # [ 0.034593] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs547worker2 # [ 0.036860] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1548lb # [ 0.050891] allocated 3145728 bytes of page_ext549worker1 # [ 0.038480] Unknown kernel command line parameters "regInfo=/nix/store/bmgjg3l7l2l8lkg7xd7sfzlrxaklnwwz-closure-info/registration", will be passed to user space.550lb # [ 0.060749] ftrace: allocating 48787 entries in 192 pages551worker1 # [ 0.038493] random: crng init done552worker2 # [ 0.038983] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152553client # [ 0.034611] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()554lb # [ 0.060750] ftrace: allocated 192 pages with 2 groups555worker2 # [ 0.039024] kvm-guest: PV spinlocks disabled, single CPU556worker1 # [ 0.038494] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes557client # [ 0.034642] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]558lb # [ 0.061577] Dynamic Preempt: lazy559lb # [ 0.061738] rcu: Preemptible hierarchical RCU implementation.560worker1 # [ 0.040576] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)561client # [ 0.034644] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]562lb # [ 0.061739] rcu: RCU event tracing is enabled.563client # [ 0.034645] [mem 0x60000000-0xafffffff] available for PCI devices564worker1 # [ 0.041796] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)565lb # [ 0.061740] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.566client # [ 0.034647] Booting paravirtualized kernel on KVM567worker1 # [ 0.041834] Fallback order for Node 0: 0568lb # [ 0.061741] Trampoline variant of Tasks RCU enabled.569worker1 # [ 0.041837] Built 1 zonelists, mobility grouping on. Total pages: 393078570lb # [ 0.061742] Rude variant of Tasks RCU enabled.571client # [ 0.034650] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns572worker1 # [ 0.041838] Policy zone: DMA32573worker2 # [ 0.039026] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/m68sx34dr7yy1drfgmzf0cyxm52cz9hp-nixos-system-worker2-test/init regInfo=/nix/store/zijq974g613q511fgj3vy2szbqsd8kv2-closure-info/registration console=ttyS0,115200n8 console=tty0574lb # [ 0.061742] Tracing variant of Tasks RCU enabled.575worker1 # [ 0.044224] mem auto-init: stack:all(zero), heap alloc:on, heap free:off576client # [ 0.039149] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1577lb # [ 0.061743] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.578worker1 # [ 0.047166] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1579client # [ 0.041428] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152580worker2 # [ 0.039117] Unknown kernel command line parameters "regInfo=/nix/store/zijq974g613q511fgj3vy2szbqsd8kv2-closure-info/registration", will be passed to user space.581lb # [ 0.061743] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1582worker1 # [ 0.050378] allocated 3145728 bytes of page_ext583client # [ 0.041476] kvm-guest: PV spinlocks disabled, single CPU584worker2 # [ 0.039129] random: crng init done585worker1 # [ 0.060229] ftrace: allocating 48787 entries in 192 pages586lb # [ 0.061759] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.587worker2 # [ 0.039130] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes588worker1 # [ 0.060231] ftrace: allocated 192 pages with 2 groups589worker1 # [ 0.061045] Dynamic Preempt: lazy590lb # [ 0.061760] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.591worker2 # [ 0.041220] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)592worker1 # [ 0.061172] rcu: Preemptible hierarchical RCU implementation.593worker1 # [ 0.061172] rcu: RCU event tracing is enabled.594worker2 # [ 0.042269] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)595lb # [ 0.061761] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.596worker2 # [ 0.042304] Fallback order for Node 0: 0597lb # [ 0.066183] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16598worker1 # [ 0.061173] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.599client # [ 0.041478] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6rlhi5dcdiiclafqnpzgp1y7qxb2ara2-nixos-system-client-test/init regInfo=/nix/store/h3j26zp3yqdh88ddln89hmpjd3kw928v-closure-info/registration console=ttyS0,115200n8 console=tty0600worker1 # [ 0.061174] Trampoline variant of Tasks RCU enabled.601worker2 # [ 0.042307] Built 1 zonelists, mobility grouping on. Total pages: 393078602lb # [ 0.066450] rcu: srcu_init: Setting srcu_struct sizes based on contention.603worker2 # [ 0.042308] Policy zone: DMA32604worker1 # [ 0.061175] Rude variant of Tasks RCU enabled.605worker1 # [ 0.061175] Tracing variant of Tasks RCU enabled.606client # [ 0.041573] Unknown kernel command line parameters "regInfo=/nix/store/h3j26zp3yqdh88ddln89hmpjd3kw928v-closure-info/registration", will be passed to user space.607worker2 # [ 0.044910] mem auto-init: stack:all(zero), heap alloc:on, heap free:off608lb # [ 0.066456] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns609client # [ 0.041586] random: crng init done610worker2 # [ 0.047912] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1611worker1 # [ 0.061176] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.612client # [ 0.041586] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes613worker2 # [ 0.051131] allocated 3145728 bytes of page_ext614lb # [ 0.066552] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)615worker1 # [ 0.061177] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1616lb # [ 0.070103] Console: colour VGA+ 80x25617worker2 # [ 0.060872] ftrace: allocating 48787 entries in 192 pages618client # [ 0.043845] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)619lb # [ 0.070106] printk: legacy console [tty0] enabled620worker2 # [ 0.060874] ftrace: allocated 192 pages with 2 groups621worker1 # [ 0.061233] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.622lb # [ 0.099567] printk: legacy console [ttyS0] enabled623worker2 # [ 0.061686] Dynamic Preempt: lazy624client # [ 0.045245] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)625lb # [ 0.196238] ACPI: Core revision 20250807626client # [ 0.045276] Fallback order for Node 0: 0627worker2 # [ 0.061845] rcu: Preemptible hierarchical RCU implementation.628worker1 # [ 0.061234] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.629worker2 # [ 0.061845] rcu: RCU event tracing is enabled.630client # [ 0.045279] Built 1 zonelists, mobility grouping on. Total pages: 393078631client # [ 0.045280] Policy zone: DMA32632worker1 # [ 0.061235] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.633worker2 # [ 0.061846] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.634lb # [ 0.197096] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns635client # [ 0.047924] mem auto-init: stack:all(zero), heap alloc:on, heap free:off636worker1 # [ 0.065782] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16637worker2 # [ 0.061847] Trampoline variant of Tasks RCU enabled.638lb # [ 0.198575] APIC: Switch to symmetric I/O mode setup639worker2 # [ 0.061848] Rude variant of Tasks RCU enabled.640client # [ 0.051225] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1641worker1 # [ 0.066051] rcu: srcu_init: Setting srcu_struct sizes based on contention.642worker2 # [ 0.061848] Tracing variant of Tasks RCU enabled.643lb # [ 0.199538] x2apic enabled644client # [ 0.054701] allocated 3145728 bytes of page_ext645client # [ 0.064629] ftrace: allocating 48787 entries in 192 pages646worker1 # [ 0.066057] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns647worker2 # [ 0.061849] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.648lb # [ 0.200269] APIC: Switched APIC routing to: physical x2apic649client # [ 0.064630] ftrace: allocated 192 pages with 2 groups650client # [ 0.065500] Dynamic Preempt: lazy651worker2 # [ 0.061849] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1652worker1 # [ 0.066156] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)653client # [ 0.065688] rcu: Preemptible hierarchical RCU implementation.654worker1 # [ 0.069680] Console: colour VGA+ 80x25655client # [ 0.065689] rcu: RCU event tracing is enabled.656worker2 # [ 0.061869] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.657lb # [ 0.202058] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1658worker1 # [ 0.069683] printk: legacy console [tty0] enabled659worker1 # [ 0.099718] printk: legacy console [ttyS0] enabled660client # [ 0.065690] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.661worker2 # [ 0.061871] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.662client # [ 0.065691] Trampoline variant of Tasks RCU enabled.663worker1 # [ 0.198450] ACPI: Core revision 20250807664lb # [ 0.203059] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns665client # [ 0.065692] Rude variant of Tasks RCU enabled.666worker2 # [ 0.061872] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.667client # [ 0.065692] Tracing variant of Tasks RCU enabled.668worker2 # [ 0.066212] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16669worker1 # [ 0.199286] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns670lb # [ 0.204715] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)671client # [ 0.065693] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.672worker2 # [ 0.066481] rcu: srcu_init: Setting srcu_struct sizes based on contention.673worker1 # [ 0.200815] APIC: Switch to symmetric I/O mode setup674client # [ 0.065693] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1675lb # [ 0.205805] x86/cpu: User Mode Instruction Prevention (UMIP) activated676worker1 # [ 0.201812] x2apic enabled677worker2 # [ 0.066488] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns678client # [ 0.065710] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.679lb # [ 0.206855] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127680worker1 # [ 0.202594] APIC: Switched APIC routing to: physical x2apic681worker2 # [ 0.066585] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)682client # [ 0.065712] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.683lb # [ 0.207713] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0684worker2 # [ 0.070213] Console: colour VGA+ 80x25685worker2 # [ 0.070215] printk: legacy console [tty0] enabled686client # [ 0.065713] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.687worker2 # [ 0.100195] printk: legacy console [ttyS0] enabled688worker1 # [ 0.204499] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1689lb # [ 0.208716] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto690client # [ 0.070500] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16691worker2 # [ 0.198781] ACPI: Core revision 20250807692client # [ 0.070761] rcu: srcu_init: Setting srcu_struct sizes based on contention.693lb # [ 0.209713] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl694worker1 # [ 0.205484] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns695client # [ 0.070770] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns696worker2 # [ 0.199645] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns697lb # [ 0.210713] Transient Scheduler Attacks: Vulnerable: No microcode698worker1 # [ 0.207149] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)699worker2 # [ 0.201149] APIC: Switch to symmetric I/O mode setup700client # [ 0.070887] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)701lb # [ 0.211712] Spectre V2 : Mitigation: Enhanced / Automatic IBRS702client # [ 0.074406] Console: colour VGA+ 80x25703worker2 # [ 0.202136] x2apic enabled704worker1 # [ 0.208235] x86/cpu: User Mode Instruction Prevention (UMIP) activated705client # [ 0.074409] printk: legacy console [tty0] enabled706lb # [ 0.212712] Speculative Return Stack Overflow: Mitigation: Safe RET707client # [ 0.103942] printk: legacy console [ttyS0] enabled708worker2 # [ 0.202914] APIC: Switched APIC routing to: physical x2apic709worker1 # [ 0.210031] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127710client # [ 0.199302] ACPI: Core revision 20250807711worker1 # [ 0.210843] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0712worker2 # [ 0.204741] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1713client # [ 0.200227] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns714worker1 # [ 0.211943] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto715client # [ 0.201737] APIC: Switch to symmetric I/O mode setup716lb # [ 0.213713] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization717worker2 # [ 0.205762] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns718client # [ 0.202717] x2apic enabled719lb # [ 0.215719] Spectre V2 : Enabling IBPB for BPF720worker1 # [ 0.213146] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl721worker2 # [ 0.207457] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)722client # [ 0.203499] APIC: Switched APIC routing to: physical x2apic723worker1 # [ 0.214146] Transient Scheduler Attacks: Vulnerable: No microcode724lb # [ 0.216459] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier725worker2 # [ 0.208540] x86/cpu: User Mode Instruction Prevention (UMIP) activated726lb # [ 0.217713] active return thunk: srso_alias_return_thunk727worker1 # [ 0.215146] Spectre V2 : Mitigation: Enhanced / Automatic IBRS728client # [ 0.205306] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1729worker2 # [ 0.209612] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127730worker1 # [ 0.216050] Speculative Return Stack Overflow: Mitigation: Safe RET731lb # [ 0.218733] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'732worker2 # [ 0.211140] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0733client # [ 0.206296] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns734lb # [ 0.219712] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'735lb # [ 0.220713] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'736client # [ 0.207961] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)737worker2 # [ 0.212243] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto738lb # [ 0.221712] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'739client # [ 0.209047] x86/cpu: User Mode Instruction Prevention (UMIP) activated740worker1 # [ 0.216876] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization741worker2 # [ 0.213454] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl742lb # [ 0.222712] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'743worker1 # [ 0.218153] Spectre V2 : Enabling IBPB for BPF744client # [ 0.210129] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127745worker2 # [ 0.214454] Transient Scheduler Attacks: Vulnerable: No microcode746lb # [ 0.223712] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'747client # [ 0.211623] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0748worker2 # [ 0.215388] Spectre V2 : Mitigation: Enhanced / Automatic IBRS749worker1 # [ 0.219662] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier750lb # [ 0.224713] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'751worker1 # [ 0.220146] active return thunk: srso_alias_return_thunk752worker2 # [ 0.216136] Speculative Return Stack Overflow: Mitigation: Safe RET753client # [ 0.211962] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto754lb # [ 0.226607] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256755worker1 # [ 0.221000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'756lb # [ 0.227434] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64757client # [ 0.213958] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl758worker1 # [ 0.222146] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'759lb # [ 0.228431] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512760client # [ 0.214958] Transient Scheduler Attacks: Vulnerable: No microcode761worker1 # [ 0.223146] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'762lb # [ 0.229435] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024763client # [ 0.216679] Spectre V2 : Mitigation: Enhanced / Automatic IBRS764worker2 # [ 0.217189] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization765lb # [ 0.230435] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8766worker1 # [ 0.224146] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'767worker2 # [ 0.218462] Spectre V2 : Enabling IBPB for BPF768client # [ 0.216958] Speculative Return Stack Overflow: Mitigation: Safe RET769worker1 # [ 0.225146] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'770lb # [ 0.230712] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.771worker2 # [ 0.219998] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier772worker1 # [ 0.226146] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'773worker2 # [ 0.220455] active return thunk: srso_alias_return_thunk774worker1 # [ 0.227145] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'775worker2 # [ 0.221364] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'776client # [ 0.217957] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization777worker1 # [ 0.228146] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256778worker2 # [ 0.222454] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'779client # [ 0.219965] Spectre V2 : Enabling IBPB for BPF780worker1 # [ 0.229146] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64781worker2 # [ 0.223454] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'782worker1 # [ 0.230146] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512783client # [ 0.220958] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier784worker2 # [ 0.224454] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'785client # [ 0.221958] active return thunk: srso_alias_return_thunk786worker1 # [ 0.231146] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024787worker2 # [ 0.225454] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'788worker1 # [ 0.232146] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8789client # [ 0.223600] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'790worker2 # [ 0.226454] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'791client # [ 0.224860] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'792worker1 # [ 0.233146] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.793worker2 # [ 0.227454] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'794client # [ 0.225703] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'795worker2 # [ 0.228455] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256796client # [ 0.226710] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'797worker2 # [ 0.229454] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64798client # [ 0.227700] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'799worker2 # [ 0.230454] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512800client # [ 0.228702] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'801worker2 # [ 0.231454] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024802client # [ 0.229750] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'803worker2 # [ 0.232454] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8804client # [ 0.230958] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256805client # [ 0.231957] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64806worker2 # [ 0.233454] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.807client # [ 0.232957] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808client # [ 0.233957] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809client # [ 0.234957] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810client # [ 0.235958] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811lb # [ 0.265617] Freeing SMP alternatives memory: 44K812lb # [ 0.266295] pid_max: default: 32768 minimum: 301813lb # [ 0.266804] LSM: initializing lsm=capability,landlock,yama,bpf,ima814lb # [ 0.267826] landlock: Up and running.815lb # [ 0.268713] Yama: becoming mindful.816lb # [ 0.269916] LSM support for eBPF active817lb # [ 0.270828] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818lb # [ 0.271749] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819worker1 # [ 0.268085] Freeing SMP alternatives memory: 44K820worker1 # [ 0.268735] pid_max: default: 32768 minimum: 301821lb # [ 0.274498] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)822worker1 # [ 0.269243] LSM: initializing lsm=capability,landlock,yama,bpf,ima823lb # [ 0.275302] Performance Events: Fam17h+ core perfctr, AMD PMU driver.824worker1 # [ 0.270247] landlock: Up and running.825worker1 # [ 0.271601] Yama: becoming mindful.826lb # [ 0.275717] ... version: 2827lb # [ 0.276461] ... bit width: 48828worker1 # [ 0.272358] LSM support for eBPF active829lb # [ 0.276715] ... generic counters: 6830worker1 # [ 0.273268] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)831lb # [ 0.277485] ... generic bitmap: 000000000000003f832lb # [ 0.277724] ... fixed-purpose counters: 0833worker2 # [ 0.268308] Freeing SMP alternatives memory: 44K834worker1 # [ 0.274183] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)835lb # [ 0.278408] ... fixed-purpose bitmap: 0000000000000000836worker2 # [ 0.269017] pid_max: default: 32768 minimum: 301837lb # [ 0.278715] ... value mask: 0000ffffffffffff838worker2 # [ 0.269539] LSM: initializing lsm=capability,landlock,yama,bpf,ima839lb # [ 0.279639] ... max period: 00007fffffffffff840worker1 # [ 0.276936] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)841worker2 # [ 0.270555] landlock: Up and running.842worker2 # [ 0.271455] Yama: becoming mindful.843lb # [ 0.280427] ... global_ctrl mask: 000000000000003f844worker2 # [ 0.272657] LSM support for eBPF active845worker1 # [ 0.277653] Performance Events: Fam17h+ core perfctr, AMD PMU driver.846lb # [ 0.280837] signal: max sigframe size: 3376847worker1 # [ 0.278150] ... version: 2848client # [ 0.270871] Freeing SMP alternatives memory: 44K849lb # [ 0.281681] rcu: Hierarchical SRCU implementation.850worker2 # [ 0.273418] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)851worker1 # [ 0.278863] ... bit width: 48852client # [ 0.271512] pid_max: default: 32768 minimum: 301853lb # [ 0.282295] rcu: Max phase no-delay instances is 400.854worker1 # [ 0.279148] ... generic counters: 6855worker2 # [ 0.274494] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)856client # [ 0.272066] LSM: initializing lsm=capability,landlock,yama,bpf,ima857worker1 # [ 0.279865] ... generic bitmap: 000000000000003f858client # [ 0.273055] landlock: Up and running.859worker1 # [ 0.280198] ... fixed-purpose counters: 0860client # [ 0.273958] Yama: becoming mindful.861worker1 # [ 0.280922] ... fixed-purpose bitmap: 0000000000000000862client # [ 0.275159] LSM support for eBPF active863worker2 # [ 0.277097] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)864worker1 # [ 0.281149] ... value mask: 0000ffffffffffff865client # [ 0.275884] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)866worker1 # [ 0.282084] ... max period: 00007fffffffffff867lb # [ 0.287240] smp: Bringing up secondary CPUs ...868worker2 # [ 0.277993] Performance Events: Fam17h+ core perfctr, AMD PMU driver.869client # [ 0.276998] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)870worker1 # [ 0.282880] ... global_ctrl mask: 000000000000003f871lb # [ 0.287729] smp: Brought up 1 node, 1 CPU872worker2 # [ 0.278459] ... version: 2873worker1 # [ 0.283260] signal: max sigframe size: 3376874worker2 # [ 0.279139] ... bit width: 48875lb # [ 0.288401] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)876worker1 # [ 0.284085] rcu: Hierarchical SRCU implementation.877worker2 # [ 0.279457] ... generic counters: 6878client # [ 0.279415] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)879worker1 # [ 0.284761] rcu: Max phase no-delay instances is 400.880worker2 # [ 0.280184] ... generic bitmap: 000000000000003f881worker2 # [ 0.280467] ... fixed-purpose counters: 0882lb # [ 0.288885] Memory: 1454576K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109320K reserved, 0K cma-reserved)883client # [ 0.280546] Performance Events: Fam17h+ core perfctr, AMD PMU driver.884worker2 # [ 0.281166] ... fixed-purpose bitmap: 0000000000000000885lb # [ 0.290176] devtmpfs: initialized886client # [ 0.280963] ... version: 2887worker2 # [ 0.281457] ... value mask: 0000ffffffffffff888lb # [ 0.290964] x86/mm: Memory block size: 128MB889client # [ 0.281695] ... bit width: 48890worker2 # [ 0.282349] ... max period: 00007fffffffffff891client # [ 0.281960] ... generic counters: 6892worker2 # [ 0.283131] ... global_ctrl mask: 000000000000003f893client # [ 0.282662] ... generic bitmap: 000000000000003f894lb # [ 0.292700] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)895worker2 # [ 0.283565] signal: max sigframe size: 3376896worker1 # [ 0.289784] smp: Bringing up secondary CPUs ...897client # [ 0.282965] ... fixed-purpose counters: 0898worker2 # [ 0.284367] rcu: Hierarchical SRCU implementation.899client # [ 0.283675] ... fixed-purpose bitmap: 0000000000000000900worker1 # [ 0.290163] smp: Brought up 1 node, 1 CPU901lb # [ 0.293598] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).902worker2 # [ 0.285055] rcu: Max phase no-delay instances is 400.903client # [ 0.283960] ... value mask: 0000ffffffffffff904worker1 # [ 0.290839] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)905lb # [ 0.294784] pinctrl core: initialized pinctrl subsystem906client # [ 0.284824] ... max period: 00007fffffffffff907lb # [ 0.295801] PM: RTC time: 16:31:22, date: 2026-09-25908client # [ 0.285631] ... global_ctrl mask: 000000000000003f909worker1 # [ 0.291389] Memory: 1454584K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)910client # [ 0.286056] signal: max sigframe size: 3376911worker1 # [ 0.292374] devtmpfs: initialized912client # [ 0.286881] rcu: Hierarchical SRCU implementation.913worker1 # [ 0.293279] x86/mm: Memory block size: 128MB914client # [ 0.287564] rcu: Max phase no-delay instances is 400.915lb # [ 0.299228] NET: Registered PF_NETLINK/PF_ROUTE protocol family916worker2 # [ 0.290062] smp: Bringing up secondary CPUs ...917worker2 # [ 0.290470] smp: Brought up 1 node, 1 CPU918lb # [ 0.300212] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations919worker1 # [ 0.295079] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)920worker2 # [ 0.291199] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)921lb # [ 0.300734] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations922worker1 # [ 0.296118] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).923worker1 # [ 0.297225] pinctrl core: initialized pinctrl subsystem924lb # [ 0.301988] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations925worker2 # [ 0.291654] Memory: 1454628K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)926client # [ 0.292498] smp: Bringing up secondary CPUs ...927worker1 # [ 0.298318] PM: RTC time: 16:31:22, date: 2026-09-25928lb # [ 0.302725] audit: initializing netlink subsys (disabled)929worker2 # [ 0.292706] devtmpfs: initialized930client # [ 0.292973] smp: Brought up 1 node, 1 CPU931worker2 # [ 0.293512] x86/mm: Memory block size: 128MB932client # [ 0.293651] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)933lb # [ 0.303858] thermal_sys: Registered thermal governor 'fair_share'934lb # [ 0.303860] thermal_sys: Registered thermal governor 'bang_bang'935lb # [ 0.304716] thermal_sys: Registered thermal governor 'step_wise'936worker2 # [ 0.295376] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)937client # [ 0.294142] Memory: 1454628K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)938worker1 # [ 0.301876] NET: Registered PF_NETLINK/PF_ROUTE protocol family939lb # [ 0.305689] thermal_sys: Registered thermal governor 'user_space'940client # [ 0.295238] devtmpfs: initialized941worker2 # [ 0.296367] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).942worker1 # [ 0.302723] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations943client # [ 0.296049] x86/mm: Memory block size: 128MB944lb # [ 0.306402] audit: type=2000 audit(1790353883.126:1): state=initialized audit_enabled=0 res=1945worker2 # [ 0.297557] pinctrl core: initialized pinctrl subsystem946lb # [ 0.307718] thermal_sys: Registered thermal governor 'power_allocator'947worker1 # [ 0.303171] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations948worker2 # [ 0.298664] PM: RTC time: 16:31:22, date: 2026-09-25949lb # [ 0.307731] cpuidle: using governor menu950client # [ 0.297840] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)951worker1 # [ 0.304432] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations952lb # [ 0.309925] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5953client # [ 0.298848] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).954worker1 # [ 0.305157] audit: initializing netlink subsys (disabled)955client # [ 0.300045] pinctrl core: initialized pinctrl subsystem956lb # [ 0.310949] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]957worker1 # [ 0.306150] audit: type=2000 audit(1790353883.132:1): state=initialized audit_enabled=0 res=1958worker2 # [ 0.302183] NET: Registered PF_NETLINK/PF_ROUTE protocol family959client # [ 0.301156] PM: RTC time: 16:31:22, date: 2026-09-25960lb # [ 0.311718] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry961worker1 # [ 0.307282] thermal_sys: Registered thermal governor 'fair_share'962lb # [ 0.312727] PCI: Using configuration type 1 for base access963worker2 # [ 0.302965] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations964worker1 # [ 0.307284] thermal_sys: Registered thermal governor 'bang_bang'965worker2 # [ 0.303479] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations966worker1 # [ 0.308150] thermal_sys: Registered thermal governor 'step_wise'967lb # [ 0.313822] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.968worker1 # [ 0.309149] thermal_sys: Registered thermal governor 'user_space'969client # [ 0.304916] NET: Registered PF_NETLINK/PF_ROUTE protocol family970worker2 # [ 0.304744] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations971worker1 # [ 0.310124] thermal_sys: Registered thermal governor 'power_allocator'972worker2 # [ 0.305466] audit: initializing netlink subsys (disabled)973worker1 # [ 0.310946] cpuidle: using governor menu974client # [ 0.306230] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations975worker2 # [ 0.306636] thermal_sys: Registered thermal governor 'fair_share'976client # [ 0.306984] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations977worker1 # [ 0.313230] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5978worker2 # [ 0.306638] thermal_sys: Registered thermal governor 'bang_bang'979worker2 # [ 0.307458] thermal_sys: Registered thermal governor 'step_wise'980client # [ 0.308250] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations981worker1 # [ 0.314402] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]982worker2 # [ 0.308448] thermal_sys: Registered thermal governor 'user_space'983client # [ 0.308970] audit: initializing netlink subsys (disabled)984worker1 # [ 0.315152] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry985lb # [ 0.320947] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages986client # [ 0.310095] thermal_sys: Registered thermal governor 'fair_share'987worker2 # [ 0.309214] audit: type=2000 audit(1790353883.138:1): state=initialized audit_enabled=0 res=1988worker1 # [ 0.316160] PCI: Using configuration type 1 for base access989lb # [ 0.321715] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page990client # [ 0.310097] thermal_sys: Registered thermal governor 'bang_bang'991worker2 # [ 0.310459] thermal_sys: Registered thermal governor 'power_allocator'992lb # [ 0.322697] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages993worker1 # [ 0.317252] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.994client # [ 0.310961] thermal_sys: Registered thermal governor 'step_wise'995worker2 # [ 0.310473] cpuidle: using governor menu996client # [ 0.311894] thermal_sys: Registered thermal governor 'user_space'997worker2 # [ 0.312681] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5998client # [ 0.312635] audit: type=2000 audit(1790353883.150:1): state=initialized audit_enabled=0 res=1999worker2 # [ 0.313746] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1000client # [ 0.313963] thermal_sys: Registered thermal governor 'power_allocator'1001client # [ 0.313987] cpuidle: using governor menu1002worker2 # [ 0.314460] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1003lb # [ 0.327717] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1004worker2 # [ 0.315469] PCI: Using configuration type 1 for base access1005client # [ 0.316198] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51006worker2 # [ 0.316594] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1007worker1 # [ 0.324177] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1008client # [ 0.317209] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1009worker1 # [ 0.325148] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1010client # [ 0.317963] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1011client # [ 0.318971] PCI: Using configuration type 1 for base access1012client # [ 0.320135] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1013lb # [ 0.336120] ACPI: Added _OSI(Module Device)1014worker1 # [ 0.330149] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1015lb # [ 0.336717] ACPI: Added _OSI(Processor Device)1016worker1 # [ 0.331150] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1017worker2 # [ 0.323702] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1018lb # [ 0.337483] ACPI: Added _OSI(Processor Aggregator Device)1019worker2 # [ 0.324458] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1020client # [ 0.327180] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1021client # [ 0.327961] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1022worker2 # [ 0.329458] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1023worker2 # [ 0.330459] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1024lb # [ 0.344270] ACPI: 1 ACPI AML tables successfully acquired and loaded1025client # [ 0.332961] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1026worker1 # [ 0.342435] ACPI: Added _OSI(Module Device)1027client # [ 0.333961] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1028worker1 # [ 0.343150] ACPI: Added _OSI(Processor Device)1029lb # [ 0.348118] ACPI: Interpreter enabled1030worker1 # [ 0.343867] ACPI: Added _OSI(Processor Aggregator Device)1031lb # [ 0.348730] ACPI: PM: (supports S0 S3 S4 S5)1032lb # [ 0.349478] ACPI: Using IOAPIC for interrupt routing1033worker2 # [ 0.339105] ACPI: Added _OSI(Module Device)1034worker2 # [ 0.339458] ACPI: Added _OSI(Processor Device)1035lb # [ 0.351770] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1036worker2 # [ 0.340209] ACPI: Added _OSI(Processor Aggregator Device)1037lb # [ 0.352716] PCI: Using E820 reservations for host bridge windows1038worker1 # [ 0.349510] ACPI: 1 ACPI AML tables successfully acquired and loaded1039lb # [ 0.355854] ACPI: Enabled 2 GPEs in block 00 to 3F1040client # [ 0.344410] ACPI: Added _OSI(Module Device)1041client # [ 0.344972] ACPI: Added _OSI(Processor Device)1042worker1 # [ 0.353550] ACPI: Interpreter enabled1043client # [ 0.345709] ACPI: Added _OSI(Processor Aggregator Device)1044worker1 # [ 0.354164] ACPI: PM: (supports S0 S3 S4 S5)1045worker1 # [ 0.354908] ACPI: Using IOAPIC for interrupt routing1046worker2 # [ 0.347031] ACPI: 1 ACPI AML tables successfully acquired and loaded1047worker1 # [ 0.357227] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1048worker2 # [ 0.350397] ACPI: Interpreter enabled1049worker1 # [ 0.358149] PCI: Using E820 reservations for host bridge windows1050worker2 # [ 0.350947] ACPI: PM: (supports S0 S3 S4 S5)1051lb # [ 0.364483] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1052worker2 # [ 0.351457] ACPI: Using IOAPIC for interrupt routing1053client # [ 0.352543] ACPI: 1 ACPI AML tables successfully acquired and loaded1054lb # [ 0.365467] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1055worker2 # [ 0.352359] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1056worker1 # [ 0.361286] ACPI: Enabled 2 GPEs in block 00 to 3F1057lb # [ 0.366794] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1058lb # [ 0.367837] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1059client # [ 0.356380] ACPI: Interpreter enabled1060lb # [ 0.369069] PCI host bridge to bus 0000:001061client # [ 0.356976] ACPI: PM: (supports S0 S3 S4 S5)1062lb # [ 0.369720] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1063client # [ 0.357727] ACPI: Using IOAPIC for interrupt routing1064worker2 # [ 0.357457] PCI: Using E820 reservations for host bridge windows1065lb # [ 0.370715] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1066worker2 # [ 0.358564] ACPI: Enabled 2 GPEs in block 00 to 3F1067lb # [ 0.371715] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1068worker1 # [ 0.368656] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1069lb # [ 0.372715] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1070client # [ 0.360024] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1071lb # [ 0.373715] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1072client # [ 0.360961] PCI: Using E820 reservations for host bridge windows1073worker1 # [ 0.369154] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1074lb # [ 0.374715] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1075worker1 # [ 0.370228] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1076lb # [ 0.375715] pci_bus 0000:00: root bus resource [bus 00-ff]1077worker1 # [ 0.371270] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1078client # [ 0.364036] ACPI: Enabled 2 GPEs in block 00 to 3F1079lb # [ 0.376755] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1080worker1 # [ 0.372498] PCI host bridge to bus 0000:001081worker1 # [ 0.373153] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1082worker2 # [ 0.367243] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1083lb # [ 0.378183] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1084worker1 # [ 0.374149] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1085worker1 # [ 0.375148] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1086worker2 # [ 0.368688] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1087worker1 # [ 0.376149] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1088worker2 # [ 0.369535] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1089worker1 # [ 0.377148] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1090worker2 # [ 0.370580] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1091client # [ 0.371119] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1092worker2 # [ 0.371820] PCI host bridge to bus 0000:001093worker1 # [ 0.378149] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1094client # [ 0.371967] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1095worker1 # [ 0.379150] pci_bus 0000:00: root bus resource [bus 00-ff]1096worker2 # [ 0.372461] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1097client # [ 0.373038] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1098worker2 # [ 0.373457] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1099lb # [ 0.380767] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1100worker1 # [ 0.380183] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1101client # [ 0.374082] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1102lb # [ 0.381727] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1103worker2 # [ 0.374457] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1104client # [ 0.375351] PCI host bridge to bus 0000:001105lb # [ 0.382670] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1106worker1 # [ 0.381637] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1107worker2 # [ 0.375457] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1108client # [ 0.375965] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1109worker2 # [ 0.376457] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1110lb # [ 0.383670] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1111client # [ 0.376961] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1112worker2 # [ 0.377457] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1113client # [ 0.377960] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1114worker2 # [ 0.378457] pci_bus 0000:00: root bus resource [bus 00-ff]1115lb # [ 0.385510] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1116client # [ 0.378960] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1117worker2 # [ 0.379412] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1118client # [ 0.379960] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1119lb # [ 0.388752] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1120worker2 # [ 0.380887] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1121client # [ 0.380960] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1122lb # [ 0.389599] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1123client # [ 0.381960] pci_bus 0000:00: root bus resource [bus 00-ff]1124worker1 # [ 0.384231] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1125lb # [ 0.390475] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1126worker1 # [ 0.385161] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1127client # [ 0.382880] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1128lb # [ 0.390721] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1129worker1 # [ 0.386125] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1130client # [ 0.384413] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1131worker1 # [ 0.387351] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1132lb # [ 0.392466] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1133worker1 # [ 0.389093] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1134lb # [ 0.394754] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1135worker2 # [ 0.383508] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1136lb # [ 0.395629] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1137worker2 # [ 0.384469] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1138lb # [ 0.396497] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1139worker2 # [ 0.385472] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1140worker1 # [ 0.392188] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1141worker1 # [ 0.393065] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1142worker2 # [ 0.386840] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1143lb # [ 0.398154] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1144worker1 # [ 0.393955] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1145client # [ 0.387118] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1146worker1 # [ 0.395017] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1147client # [ 0.388045] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1148worker2 # [ 0.388265] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1149client # [ 0.389023] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1150worker1 # [ 0.396956] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1151lb # [ 0.401001] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1152client # [ 0.390281] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1153worker2 # [ 0.390489] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1154lb # [ 0.402379] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1155worker2 # [ 0.391374] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1156lb # [ 0.402737] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1157worker1 # [ 0.399186] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1158client # [ 0.391684] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1159worker2 # [ 0.392264] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1160worker1 # [ 0.400008] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1161worker2 # [ 0.393409] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1162lb # [ 0.404264] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1163worker1 # [ 0.400966] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1164client # [ 0.394002] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1165worker2 # [ 0.395300] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1166worker1 # [ 0.402617] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1167lb # [ 0.405757] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1168client # [ 0.394880] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1169lb # [ 0.406635] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1170client # [ 0.395708] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1171lb # [ 0.407538] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1172worker2 # [ 0.397492] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1173client # [ 0.395967] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1174worker2 # [ 0.398359] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1175lb # [ 0.408592] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1176worker1 # [ 0.405185] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1177worker2 # [ 0.399254] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1178client # [ 0.397561] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1179worker1 # [ 0.406090] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1180lb # [ 0.410287] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1181worker1 # [ 0.406940] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1182worker2 # [ 0.400901] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1183client # [ 0.399000] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1184lb # [ 0.411735] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1185worker1 # [ 0.408569] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1186client # [ 0.399918] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1187lb # [ 0.412697] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1188client # [ 0.400708] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1189worker2 # [ 0.403628] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1190worker1 # [ 0.411002] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1191lb # [ 0.414175] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1192client # [ 0.401565] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1193worker2 # [ 0.404464] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1194worker1 # [ 0.411763] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1195worker2 # [ 0.405447] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1196lb # [ 0.415728] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1197worker1 # [ 0.412173] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1198worker2 # [ 0.406935] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1199worker1 # [ 0.413156] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1200client # [ 0.403994] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1201lb # [ 0.416738] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1202client # [ 0.404808] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1203worker1 # [ 0.414750] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1204client # [ 0.405736] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1205lb # [ 0.418317] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1206worker2 # [ 0.408498] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1207worker2 # [ 0.409321] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1208worker1 # [ 0.416162] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1209client # [ 0.407315] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1210worker2 # [ 0.410234] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1211lb # [ 0.419727] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1212worker1 # [ 0.417111] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1213lb # [ 0.420674] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1214worker2 # [ 0.411265] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1215client # [ 0.409003] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1216worker1 # [ 0.418565] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1217lb # [ 0.422134] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1218client # [ 0.409880] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1219worker2 # [ 0.413165] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1220lb # [ 0.423761] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1221client # [ 0.410716] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1222worker2 # [ 0.415324] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1223worker1 # [ 0.420191] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1224client # [ 0.410967] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1225worker2 # [ 0.416227] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1226lb # [ 0.424737] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1227worker1 # [ 0.421172] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1228client # [ 0.412466] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1229worker2 # [ 0.417923] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1230lb # [ 0.426356] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1231worker1 # [ 0.422761] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1232client # [ 0.413833] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1233worker2 # [ 0.419470] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1234client # [ 0.414734] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1235worker1 # [ 0.424162] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1236lb # [ 0.427734] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1237worker1 # [ 0.425103] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1238worker2 # [ 0.420467] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1239lb # [ 0.428581] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1240client # [ 0.416362] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1241lb # [ 0.429507] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1242worker1 # [ 0.426553] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1243worker2 # [ 0.422013] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1244client # [ 0.418005] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1245lb # [ 0.431190] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1246worker2 # [ 0.423258] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1247client # [ 0.419012] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1248worker1 # [ 0.428472] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1249worker2 # [ 0.424230] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1250worker1 # [ 0.429172] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1251client # [ 0.420575] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1252worker2 # [ 0.425867] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1253lb # [ 0.433216] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1254worker1 # [ 0.430730] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1255lb # [ 0.434377] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1256client # [ 0.421830] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1257worker2 # [ 0.427469] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1258lb # [ 0.434737] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1259client # [ 0.422731] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1260worker2 # [ 0.428418] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1261worker1 # [ 0.432200] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1262lb # [ 0.436269] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1263worker1 # [ 0.433103] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1264client # [ 0.424405] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1265worker1 # [ 0.433892] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1266worker2 # [ 0.429980] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1267lb # [ 0.437421] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1268client # [ 0.425973] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1269lb # [ 0.437910] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1270worker2 # [ 0.431472] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1271worker1 # [ 0.434693] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1272client # [ 0.426982] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1273worker2 # [ 0.432347] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1274lb # [ 0.439347] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1275worker2 # [ 0.433219] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1276client # [ 0.428495] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1277worker1 # [ 0.436187] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1278lb # [ 0.439906] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1279worker1 # [ 0.437075] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1280worker2 # [ 0.434049] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1281lb # [ 0.441367] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1282client # [ 0.430014] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1283worker1 # [ 0.437953] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1284client # [ 0.430867] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1285worker1 # [ 0.439563] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1286client # [ 0.431820] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1287worker2 # [ 0.435488] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1288worker2 # [ 0.436464] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1289worker1 # [ 0.440785] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1290client # [ 0.433399] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1291worker2 # [ 0.437480] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1292worker1 # [ 0.441345] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1293worker2 # [ 0.439056] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1294worker1 # [ 0.443557] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1295client # [ 0.435199] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1296worker2 # [ 0.440050] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1297client # [ 0.435967] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1298worker1 # [ 0.444354] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1299worker2 # [ 0.440663] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1300client # [ 0.436937] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1301worker1 # [ 0.445799] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1302worker2 # [ 0.442021] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1303client # [ 0.438393] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1304lb # [ 0.441952] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1305worker2 # [ 0.442693] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1306client # [ 0.439505] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1307lb # [ 0.444373] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1308worker2 # [ 0.444004] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1309client # [ 0.440177] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1310lb # [ 0.445003] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1311client # [ 0.441512] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1312lb # [ 0.446009] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1313client # [ 0.442152] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1314lb # [ 0.447045] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1315client # [ 0.443541] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1316lb # [ 0.448743] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1317lb # [ 0.449626] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1318lb # [ 0.450883] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1319worker1 # [ 0.446380] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1320lb # [ 0.452415] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1321worker1 # [ 0.448775] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1322worker1 # [ 0.449460] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1323worker2 # [ 0.444723] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1324worker1 # [ 0.450443] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1325worker2 # [ 0.447042] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1326lb # [ 0.456137] ACPI: PCI: Interrupt link LNKA configured for IRQ 101327worker1 # [ 0.451400] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1328worker2 # [ 0.447692] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1329lb # [ 0.456820] ACPI: PCI: Interrupt link LNKB configured for IRQ 101330lb # [ 0.457817] ACPI: PCI: Interrupt link LNKC configured for IRQ 111331worker2 # [ 0.448747] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1332worker1 # [ 0.453130] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1333client # [ 0.444186] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1334lb # [ 0.458813] ACPI: PCI: Interrupt link LNKD configured for IRQ 111335worker1 # [ 0.453760] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1336worker2 # [ 0.449709] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1337lb # [ 0.459811] ACPI: PCI: Interrupt link LNKE configured for IRQ 101338client # [ 0.446595] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1339lb # [ 0.460810] ACPI: PCI: Interrupt link LNKF configured for IRQ 101340worker1 # [ 0.454485] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1341worker2 # [ 0.451354] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1342client # [ 0.447267] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1343lb # [ 0.461816] ACPI: PCI: Interrupt link LNKG configured for IRQ 111344worker2 # [ 0.452098] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1345worker1 # [ 0.455812] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1346lb # [ 0.462827] ACPI: PCI: Interrupt link LNKH configured for IRQ 111347client # [ 0.448245] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1348lb # [ 0.463752] ACPI: PCI: Interrupt link GSIA configured for IRQ 161349worker2 # [ 0.452818] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1350client # [ 0.449363] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1351lb # [ 0.464729] ACPI: PCI: Interrupt link GSIB configured for IRQ 171352worker2 # [ 0.454085] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1353lb # [ 0.465730] ACPI: PCI: Interrupt link GSIC configured for IRQ 181354client # [ 0.450862] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1355worker1 # [ 0.459852] ACPI: PCI: Interrupt link LNKA configured for IRQ 101356lb # [ 0.466730] ACPI: PCI: Interrupt link GSID configured for IRQ 191357client # [ 0.451650] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1358worker1 # [ 0.460259] ACPI: PCI: Interrupt link LNKB configured for IRQ 101359lb # [ 0.467728] ACPI: PCI: Interrupt link GSIE configured for IRQ 201360lb # [ 0.468727] ACPI: PCI: Interrupt link GSIF configured for IRQ 211361client # [ 0.452278] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1362worker1 # [ 0.461245] ACPI: PCI: Interrupt link LNKC configured for IRQ 111363worker2 # [ 0.457816] ACPI: PCI: Interrupt link LNKA configured for IRQ 101364lb # [ 0.469677] ACPI: PCI: Interrupt link GSIG configured for IRQ 221365worker1 # [ 0.462249] ACPI: PCI: Interrupt link LNKD configured for IRQ 111366client # [ 0.453869] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1367lb # [ 0.470472] ACPI: PCI: Interrupt link GSIH configured for IRQ 231368worker2 # [ 0.458564] ACPI: PCI: Interrupt link LNKB configured for IRQ 101369worker1 # [ 0.463243] ACPI: PCI: Interrupt link LNKE configured for IRQ 101370worker2 # [ 0.459555] ACPI: PCI: Interrupt link LNKC configured for IRQ 111371worker1 # [ 0.464264] ACPI: PCI: Interrupt link LNKF configured for IRQ 101372lb # [ 0.471777] iommu: Default domain type: Translated1373worker2 # [ 0.460555] ACPI: PCI: Interrupt link LNKD configured for IRQ 111374worker1 # [ 0.465272] ACPI: PCI: Interrupt link LNKG configured for IRQ 111375lb # [ 0.472600] iommu: DMA domain TLB invalidation policy: lazy mode1376worker2 # [ 0.461585] ACPI: PCI: Interrupt link LNKE configured for IRQ 101377worker1 # [ 0.466254] ACPI: PCI: Interrupt link LNKH configured for IRQ 111378lb # [ 0.473743] ACPI: bus type USB registered1379client # [ 0.457640] ACPI: PCI: Interrupt link LNKA configured for IRQ 101380worker2 # [ 0.462556] ACPI: PCI: Interrupt link LNKF configured for IRQ 101381lb # [ 0.474509] usbcore: registered new interface driver usbfs1382worker1 # [ 0.467188] ACPI: PCI: Interrupt link GSIA configured for IRQ 161383client # [ 0.458069] ACPI: PCI: Interrupt link LNKB configured for IRQ 101384worker2 # [ 0.463558] ACPI: PCI: Interrupt link LNKG configured for IRQ 111385lb # [ 0.475404] usbcore: registered new interface driver hub1386worker1 # [ 0.468165] ACPI: PCI: Interrupt link GSIB configured for IRQ 171387client # [ 0.459058] ACPI: PCI: Interrupt link LNKC configured for IRQ 111388lb # [ 0.475732] usbcore: registered new device driver usb1389worker2 # [ 0.464558] ACPI: PCI: Interrupt link LNKH configured for IRQ 111390worker1 # [ 0.469155] ACPI: PCI: Interrupt link GSIC configured for IRQ 181391client # [ 0.460062] ACPI: PCI: Interrupt link LNKD configured for IRQ 111392worker2 # [ 0.465494] ACPI: PCI: Interrupt link GSIA configured for IRQ 161393worker1 # [ 0.470154] ACPI: PCI: Interrupt link GSID configured for IRQ 191394lb # [ 0.477489] NetLabel: Initializing1395client # [ 0.461061] ACPI: PCI: Interrupt link LNKE configured for IRQ 101396worker2 # [ 0.466469] ACPI: PCI: Interrupt link GSIB configured for IRQ 171397lb # [ 0.477716] NetLabel: domain hash size = 1281398worker1 # [ 0.471127] ACPI: PCI: Interrupt link GSIE configured for IRQ 201399client # [ 0.462055] ACPI: PCI: Interrupt link LNKF configured for IRQ 101400lb # [ 0.478413] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1401worker2 # [ 0.467471] ACPI: PCI: Interrupt link GSIC configured for IRQ 181402worker1 # [ 0.471904] ACPI: PCI: Interrupt link GSIF configured for IRQ 211403client # [ 0.463059] ACPI: PCI: Interrupt link LNKG configured for IRQ 111404lb # [ 0.478751] NetLabel: unlabeled traffic allowed by default1405worker2 # [ 0.468473] ACPI: PCI: Interrupt link GSID configured for IRQ 191406worker1 # [ 0.472163] ACPI: PCI: Interrupt link GSIG configured for IRQ 221407client # [ 0.464062] ACPI: PCI: Interrupt link LNKH configured for IRQ 111408lb # [ 0.479604] PCI: Using ACPI for IRQ routing1409worker2 # [ 0.469460] ACPI: PCI: Interrupt link GSIE configured for IRQ 201410worker1 # [ 0.473129] ACPI: PCI: Interrupt link GSIH configured for IRQ 231411client # [ 0.464995] ACPI: PCI: Interrupt link GSIA configured for IRQ 161412worker2 # [ 0.470443] ACPI: PCI: Interrupt link GSIF configured for IRQ 211413client # [ 0.465953] ACPI: PCI: Interrupt link GSIB configured for IRQ 171414worker1 # [ 0.474929] iommu: Default domain type: Translated1415worker2 # [ 0.471234] ACPI: PCI: Interrupt link GSIG configured for IRQ 221416client # [ 0.466716] ACPI: PCI: Interrupt link GSIC configured for IRQ 181417worker2 # [ 0.471473] ACPI: PCI: Interrupt link GSIH configured for IRQ 231418worker1 # [ 0.475156] iommu: DMA domain TLB invalidation policy: lazy mode1419client # [ 0.466972] ACPI: PCI: Interrupt link GSID configured for IRQ 191420worker1 # [ 0.476410] ACPI: bus type USB registered1421client # [ 0.467920] ACPI: PCI: Interrupt link GSIE configured for IRQ 201422worker2 # [ 0.473448] iommu: Default domain type: Translated1423worker1 # [ 0.477205] usbcore: registered new interface driver usbfs1424client # [ 0.468728] ACPI: PCI: Interrupt link GSIF configured for IRQ 211425worker1 # [ 0.478123] usbcore: registered new interface driver hub1426worker2 # [ 0.474281] iommu: DMA domain TLB invalidation policy: lazy mode1427client # [ 0.469686] ACPI: PCI: Interrupt link GSIG configured for IRQ 221428worker1 # [ 0.478831] usbcore: registered new device driver usb1429worker2 # [ 0.475489] ACPI: bus type USB registered1430client # [ 0.469975] ACPI: PCI: Interrupt link GSIH configured for IRQ 231431worker2 # [ 0.476256] usbcore: registered new interface driver usbfs1432worker1 # [ 0.480280] NetLabel: Initializing1433worker2 # [ 0.477201] usbcore: registered new interface driver hub1434client # [ 0.472001] iommu: Default domain type: Translated1435worker1 # [ 0.480902] NetLabel: domain hash size = 1281436worker2 # [ 0.477473] usbcore: registered new device driver usb1437client # [ 0.472821] iommu: DMA domain TLB invalidation policy: lazy mode1438worker1 # [ 0.481149] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1439client # [ 0.474008] ACPI: bus type USB registered1440worker2 # [ 0.479201] NetLabel: Initializing1441worker1 # [ 0.482190] NetLabel: unlabeled traffic allowed by default1442worker2 # [ 0.479463] NetLabel: domain hash size = 1281443worker1 # [ 0.483099] PCI: Using ACPI for IRQ routing1444client # [ 0.474738] usbcore: registered new interface driver usbfs1445worker2 # [ 0.480220] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1446client # [ 0.474978] usbcore: registered new interface driver hub1447worker2 # [ 0.480496] NetLabel: unlabeled traffic allowed by default1448client # [ 0.475888] usbcore: registered new device driver usb1449worker2 # [ 0.481466] PCI: Using ACPI for IRQ routing1450client # [ 0.477525] NetLabel: Initializing1451client # [ 0.477961] NetLabel: domain hash size = 1281452client # [ 0.478756] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453client # [ 0.479763] NetLabel: unlabeled traffic allowed by default1454client # [ 0.480709] PCI: Using ACPI for IRQ routing1455lb # [ 0.523985] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456lb # [ 0.524711] pci 0000:00:01.0: vgaarb: bridge control possible1457lb # [ 0.524711] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458lb # [ 0.524720] vgaarb: loaded1459lb # [ 0.525508] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460lb # [ 0.525717] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461lb # [ 0.530825] clocksource: Switched to clocksource kvm-clock1462lb # [ 0.534253] VFS: Disk quotas dquot_6.6.01463lb # [ 0.534967] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464lb # [ 0.536380] pnp: PnP ACPI init1465worker1 # [ 0.529553] pci 0000:00:01.0: vgaarb: setting as boot VGA device1466lb # [ 0.537130] ACPI: IRQ 4 override to edge(!), high(!)1467worker1 # [ 0.530145] pci 0000:00:01.0: vgaarb: bridge control possible1468lb # [ 0.538074] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1469worker1 # [ 0.530145] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1470worker1 # [ 0.530154] vgaarb: loaded1471worker2 # [ 0.528954] pci 0000:00:01.0: vgaarb: setting as boot VGA device1472lb # [ 0.539662] pnp: PnP ACPI: found 5 devices1473worker2 # [ 0.529453] pci 0000:00:01.0: vgaarb: bridge control possible1474worker1 # [ 0.530900] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01475client # [ 0.525259] pci 0000:00:01.0: vgaarb: setting as boot VGA device1476worker1 # [ 0.531149] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1477worker2 # [ 0.529453] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1478client # [ 0.525956] pci 0000:00:01.0: vgaarb: bridge control possible1479worker2 # [ 0.529462] vgaarb: loaded1480worker2 # [ 0.530227] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01481client # [ 0.525956] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1482client # [ 0.525966] vgaarb: loaded1483worker2 # [ 0.530460] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1484client # [ 0.526755] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01485client # [ 0.527636] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1486worker1 # [ 0.535237] clocksource: Switched to clocksource kvm-clock1487lb # [ 0.547232] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1488worker1 # [ 0.537023] VFS: Disk quotas dquot_6.6.01489worker2 # [ 0.534569] clocksource: Switched to clocksource kvm-clock1490lb # [ 0.548728] clocksource: Switched to clocksource acpi_pm1491lb # [ 0.549744] NET: Registered PF_INET protocol family1492worker1 # [ 0.539568] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1493client # [ 0.532053] clocksource: Switched to clocksource kvm-clock1494lb # [ 0.550940] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1495worker2 # [ 0.538057] VFS: Disk quotas dquot_6.6.01496worker1 # [ 0.540909] pnp: PnP ACPI init1497worker2 # [ 0.538783] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1498worker1 # [ 0.541757] ACPI: IRQ 4 override to edge(!), high(!)1499worker2 # [ 0.540214] pnp: PnP ACPI init1500worker1 # [ 0.542821] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1501client # [ 0.535708] VFS: Disk quotas dquot_6.6.01502worker2 # [ 0.541056] ACPI: IRQ 4 override to edge(!), high(!)1503worker1 # [ 0.544427] pnp: PnP ACPI: found 5 devices1504client # [ 0.536407] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1505worker2 # [ 0.542017] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1506client # [ 0.537714] pnp: PnP ACPI init1507worker2 # [ 0.543568] pnp: PnP ACPI: found 5 devices1508client # [ 0.538540] ACPI: IRQ 4 override to edge(!), high(!)1509client # [ 0.539498] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1510client # [ 0.541129] pnp: PnP ACPI: found 5 devices1511worker1 # [ 0.552010] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1512worker1 # [ 0.553534] clocksource: Switched to clocksource acpi_pm1513worker2 # [ 0.551177] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1514worker1 # [ 0.554539] NET: Registered PF_INET protocol family1515worker2 # [ 0.552668] clocksource: Switched to clocksource acpi_pm1516worker1 # [ 0.555727] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1517worker2 # [ 0.553692] NET: Registered PF_INET protocol family1518lb # [ 0.567034] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1519client # [ 0.548853] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1520lb # [ 0.568564] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1521worker2 # [ 0.555075] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1522client # [ 0.550292] clocksource: Switched to clocksource acpi_pm1523lb # [ 0.569866] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1524client # [ 0.551262] NET: Registered PF_INET protocol family1525lb # [ 0.571190] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1526client # [ 0.552479] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1527lb # [ 0.572490] TCP: Hash tables configured (established 16384 bind 16384)1528lb # [ 0.573731] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1529lb # [ 0.575118] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1530lb # [ 0.576356] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1531lb # [ 0.577612] NET: Registered PF_UNIX/PF_LOCAL protocol family1532lb # [ 0.578616] NET: Registered PF_XDP protocol family1533lb # [ 0.579508] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1534lb # [ 0.580568] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1535lb # [ 0.581614] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1536lb # [ 0.582783] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1537worker1 # [ 0.572350] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1538lb # [ 0.583940] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1539lb # [ 0.585032] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1540worker1 # [ 0.573899] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1541worker2 # [ 0.571433] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1542lb # [ 0.586737] ACPI: \_SB_.GSIA: Enabled at IRQ 161543worker1 # [ 0.575238] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1544worker2 # [ 0.573098] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1545worker1 # [ 0.576606] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1546worker2 # [ 0.574438] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1547lb # [ 0.589031] ACPI: \_SB_.GSIB: Enabled at IRQ 171548client # [ 0.569466] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1549worker1 # [ 0.577914] TCP: Hash tables configured (established 16384 bind 16384)1550worker2 # [ 0.575792] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1551client # [ 0.570968] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1552worker1 # [ 0.579036] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1553worker2 # [ 0.577088] TCP: Hash tables configured (established 16384 bind 16384)1554lb # [ 0.591221] ACPI: \_SB_.GSIC: Enabled at IRQ 181555worker1 # [ 0.580320] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1556client # [ 0.572283] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1557worker2 # [ 0.578208] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1558worker1 # [ 0.581390] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1559client # [ 0.573608] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1560lb # [ 0.593220] ACPI: \_SB_.GSID: Enabled at IRQ 191561worker2 # [ 0.579492] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1562worker1 # [ 0.582609] NET: Registered PF_UNIX/PF_LOCAL protocol family1563client # [ 0.574810] TCP: Hash tables configured (established 16384 bind 16384)1564worker2 # [ 0.580568] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1565worker1 # [ 0.583579] NET: Registered PF_XDP protocol family1566lb # [ 0.594976] PCI: CLS 0 bytes, default 641567client # [ 0.575906] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1568worker1 # [ 0.584411] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1569worker2 # [ 0.581767] NET: Registered PF_UNIX/PF_LOCAL protocol family1570client # [ 0.577149] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1571worker2 # [ 0.582713] NET: Registered PF_XDP protocol family1572worker1 # [ 0.585383] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1573lb # [ 0.595916] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1574worker2 # [ 0.583552] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1575client # [ 0.578225] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1576worker1 # [ 0.586335] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1577lb # [ 0.597614] Trying to unpack rootfs image as initramfs...1578client # [ 0.579394] NET: Registered PF_UNIX/PF_LOCAL protocol family1579worker2 # [ 0.584552] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1580worker1 # [ 0.587380] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1581client # [ 0.580307] NET: Registered PF_XDP protocol family1582worker2 # [ 0.585567] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1583worker1 # [ 0.588459] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1584client # [ 0.581147] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1585worker2 # [ 0.586691] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1586worker1 # [ 0.589614] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1587client # [ 0.582162] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1588worker2 # [ 0.587803] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1589worker1 # [ 0.591335] ACPI: \_SB_.GSIA: Enabled at IRQ 161590client # [ 0.583210] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1591worker2 # [ 0.588937] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1592client # [ 0.584305] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1593worker2 # [ 0.590643] ACPI: \_SB_.GSIA: Enabled at IRQ 161594client # [ 0.585443] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1595worker1 # [ 0.593704] ACPI: \_SB_.GSIB: Enabled at IRQ 171596client # [ 0.586512] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1597worker2 # [ 0.592942] ACPI: \_SB_.GSIB: Enabled at IRQ 171598worker1 # [ 0.596171] ACPI: \_SB_.GSIC: Enabled at IRQ 181599client # [ 0.588306] ACPI: \_SB_.GSIA: Enabled at IRQ 161600worker2 # [ 0.595251] ACPI: \_SB_.GSIC: Enabled at IRQ 181601worker1 # [ 0.598203] ACPI: \_SB_.GSID: Enabled at IRQ 191602client # [ 0.591014] ACPI: \_SB_.GSIB: Enabled at IRQ 171603worker2 # [ 0.597272] ACPI: \_SB_.GSID: Enabled at IRQ 191604worker1 # [ 0.599935] PCI: CLS 0 bytes, default 641605client # [ 0.593012] ACPI: \_SB_.GSIC: Enabled at IRQ 181606worker2 # [ 0.599006] PCI: CLS 0 bytes, default 641607worker1 # [ 0.600968] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1608worker1 # [ 0.602736] Trying to unpack rootfs image as initramfs...1609client # [ 0.594992] ACPI: \_SB_.GSID: Enabled at IRQ 191610worker2 # [ 0.599878] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1611worker2 # [ 0.601630] Trying to unpack rootfs image as initramfs...1612client # [ 0.596767] PCI: CLS 0 bytes, default 641613client # [ 0.597637] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614client # [ 0.599438] Trying to unpack rootfs image as initramfs...1615lb # [ 0.642673] Initialise system trusted keyrings1616lb # [ 0.643672] workingset: timestamp_bits=40 max_order=19 bucket_order=01617worker1 # [ 0.646838] Initialise system trusted keyrings1618worker2 # [ 0.646752] Initialise system trusted keyrings1619worker1 # [ 0.650172] workingset: timestamp_bits=40 max_order=19 bucket_order=01620worker2 # [ 0.647723] workingset: timestamp_bits=40 max_order=19 bucket_order=01621client # [ 0.643267] Initialise system trusted keyrings1622client # [ 0.646261] workingset: timestamp_bits=40 max_order=19 bucket_order=01623lb # [ 0.667378] Key type asymmetric registered1624lb # [ 0.668137] Asymmetric key parser 'x509' registered1625lb # [ 0.669028] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1626lb # [ 0.674374] io scheduler mq-deadline registered1627lb # [ 0.675192] io scheduler kyber registered1628lb # [ 0.677836] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1629lb # [ 0.679196] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630worker1 # [ 0.672178] Key type asymmetric registered1631worker1 # [ 0.672925] Asymmetric key parser 'x509' registered1632lb # [ 0.684910] Linux agpgart interface v0.1031633worker2 # [ 0.671088] Key type asymmetric registered1634worker1 # [ 0.673828] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1635worker2 # [ 0.671843] Asymmetric key parser 'x509' registered1636lb # [ 0.685700] ACPI: bus type drm_connector registered1637worker2 # [ 0.672697] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1638client # [ 0.669338] Key type asymmetric registered1639lb # [ 0.688720] usbcore: registered new interface driver usbserial_generic1640client # [ 0.670103] Asymmetric key parser 'x509' registered1641lb # [ 0.689811] usbserial: USB Serial support registered for generic1642worker1 # [ 0.679253] io scheduler mq-deadline registered1643client # [ 0.670982] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1644worker1 # [ 0.680013] io scheduler kyber registered1645worker2 # [ 0.678088] io scheduler mq-deadline registered1646worker2 # [ 0.678914] io scheduler kyber registered1647client # [ 0.674324] io scheduler mq-deadline registered1648lb # [ 0.693296] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1649client # [ 0.675132] io scheduler kyber registered1650lb # [ 0.693296] Please enable it if your BIOS has the CPPC option.1651worker1 # [ 0.683800] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1652lb # [ 0.695540] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1653worker2 # [ 0.682558] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1654worker1 # [ 0.685126] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1655client # [ 0.677862] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1656lb # [ 0.697449] drop_monitor: Initializing network drop monitor service1657worker2 # [ 0.683857] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1658client # [ 0.679142] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1659lb # [ 0.698655] NET: Registered PF_INET6 protocol family1660worker1 # [ 0.688725] Linux agpgart interface v0.1031661worker1 # [ 0.689508] ACPI: bus type drm_connector registered1662worker2 # [ 0.687750] Linux agpgart interface v0.1031663worker2 # [ 0.688556] ACPI: bus type drm_connector registered1664lb # [ 0.703785] Segment Routing with IPv61665worker1 # [ 0.692539] usbcore: registered new interface driver usbserial_generic1666client # [ 0.685010] Linux agpgart interface v0.1031667lb # [ 0.704489] In-situ OAM (IOAM) with IPv61668client # [ 0.685808] ACPI: bus type drm_connector registered1669worker1 # [ 0.693649] usbserial: USB Serial support registered for generic1670worker2 # [ 0.691411] usbcore: registered new interface driver usbserial_generic1671worker2 # [ 0.692544] usbserial: USB Serial support registered for generic1672lb # [ 0.707564] IPI shorthand broadcast: enabled1673worker1 # [ 0.696122] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1674worker1 # [ 0.696122] Please enable it if your BIOS has the CPPC option.1675worker2 # [ 0.695007] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1676worker1 # [ 0.698380] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1677client # [ 0.690652] usbcore: registered new interface driver usbserial_generic1678worker2 # [ 0.695007] Please enable it if your BIOS has the CPPC option.1679client # [ 0.691728] usbserial: USB Serial support registered for generic1680worker2 # [ 0.697281] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1681worker1 # [ 0.702252] drop_monitor: Initializing network drop monitor service1682worker1 # [ 0.703500] NET: Registered PF_INET6 protocol family1683client # [ 0.695227] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1684client # [ 0.695227] Please enable it if your BIOS has the CPPC option.1685lb # [ 0.715812] sched_clock: Marking stable (581014599, 134255968)->(787717591, -72447024)1686client # [ 0.697516] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1687worker2 # [ 0.703124] drop_monitor: Initializing network drop monitor service1688worker2 # [ 0.704321] NET: Registered PF_INET6 protocol family1689lb # [ 0.719396] registered taskstats version 11690worker1 # [ 0.708620] Segment Routing with IPv61691lb # [ 0.720402] Loading compiled-in X.509 certificates1692client # [ 0.701364] drop_monitor: Initializing network drop monitor service1693worker2 # [ 0.707418] Segment Routing with IPv61694client # [ 0.702541] NET: Registered PF_INET6 protocol family1695worker1 # [ 0.712152] In-situ OAM (IOAM) with IPv61696worker2 # [ 0.710036] In-situ OAM (IOAM) with IPv61697worker1 # [ 0.713277] IPI shorthand broadcast: enabled1698client # [ 0.705660] Segment Routing with IPv61699worker2 # [ 0.711109] IPI shorthand broadcast: enabled1700client # [ 0.708232] In-situ OAM (IOAM) with IPv61701client # [ 0.709375] IPI shorthand broadcast: enabled1702worker1 # [ 0.721582] sched_clock: Marking stable (585015634, 136088055)->(799426982, -78323293)1703worker2 # [ 0.719302] sched_clock: Marking stable (582014403, 136965715)->(793373053, -74392935)1704worker1 # [ 0.726211] registered taskstats version 11705client # [ 0.717723] sched_clock: Marking stable (584014813, 133187124)->(792414938, -75213001)1706worker2 # [ 0.724217] registered taskstats version 11707worker1 # [ 0.727192] Loading compiled-in X.509 certificates1708worker2 # [ 0.725216] Loading compiled-in X.509 certificates1709lb # [ 0.739279] Demotion targets for Node 0: null1710lb # [ 0.740124] Key type .fscrypt registered1711lb # [ 0.740849] Key type fscrypt-provisioning registered1712lb # [ 0.741850] ima: No TPM chip found, activating TPM-bypass!1713client # [ 0.723301] registered taskstats version 11714client # [ 0.724291] Loading compiled-in X.509 certificates1715lb # [ 0.746288] ima: Allocated hash algorithm: sha11716lb # [ 0.747076] ima: No architecture policies found1717lb # [ 0.749291] PM: Magic number: 10:74:5411718lb # [ 0.750842] RAS: Correctable Errors collector initialized.1719worker1 # [ 0.745117] Demotion targets for Node 0: null1720worker1 # [ 0.746073] Key type .fscrypt registered1721worker2 # [ 0.743995] Demotion targets for Node 0: null1722worker2 # [ 0.744918] Key type .fscrypt registered1723lb # [ 0.759983] clk: Disabling unused clocks1724client # [ 0.740854] Demotion targets for Node 0: null1725worker1 # [ 0.749108] Key type fscrypt-provisioning registered1726worker1 # [ 0.750043] ima: No TPM chip found, activating TPM-bypass!1727worker2 # [ 0.747985] Key type fscrypt-provisioning registered1728worker1 # [ 0.751010] ima: Allocated hash algorithm: sha11729worker2 # [ 0.748957] ima: No TPM chip found, activating TPM-bypass!1730client # [ 0.744245] Key type .fscrypt registered1731worker2 # [ 0.749945] ima: Allocated hash algorithm: sha11732client # [ 0.744936] Key type fscrypt-provisioning registered1733worker1 # [ 0.753135] ima: No architecture policies found1734lb # [ 0.764289] PM: genpd: Disabling unused power domains1735client # [ 0.745914] ima: No TPM chip found, activating TPM-bypass!1736worker2 # [ 0.752013] ima: No architecture policies found1737worker2 # [ 0.753030] PM: Magic number: 10:74:5411738worker1 # [ 0.756120] PM: Magic number: 10:74:5411739client # [ 0.748220] ima: Allocated hash algorithm: sha11740client # [ 0.749024] ima: No architecture policies found1741worker2 # [ 0.754617] RAS: Correctable Errors collector initialized.1742worker1 # [ 0.757693] RAS: Correctable Errors collector initialized.1743client # [ 0.753222] PM: Magic number: 10:74:5411744client # [ 0.754948] RAS: Correctable Errors collector initialized.1745worker1 # [ 0.765804] clk: Disabling unused clocks1746worker2 # [ 0.764689] clk: Disabling unused clocks1747worker2 # [ 0.766999] PM: genpd: Disabling unused power domains1748worker1 # [ 0.770124] PM: genpd: Disabling unused power domains1749client # [ 0.762909] clk: Disabling unused clocks1750client # [ 0.767225] PM: genpd: Disabling unused power domains1751lb # [ 0.920095] Freeing initrd memory: 29064K1752lb # [ 0.923367] Freeing unused decrypted memory: 2028K1753lb # [ 0.925967] Freeing unused kernel image (initmem) memory: 3652K1754lb # [ 0.927062] Write protecting the kernel read-only data: 32768k1755lb # [ 0.928958] Freeing unused kernel image (text/rodata gap) memory: 1184K1756lb # [ 0.930458] Freeing unused kernel image (rodata/data gap) memory: 720K1757worker2 # [ 0.924060] Freeing initrd memory: 29060K1758worker2 # [ 0.927235] Freeing unused decrypted memory: 2028K1759worker1 # [ 0.930315] Freeing initrd memory: 29056K1760worker2 # [ 0.929815] Freeing unused kernel image (initmem) memory: 3652K1761worker1 # [ 0.933558] Freeing unused decrypted memory: 2028K1762worker2 # [ 0.931061] Write protecting the kernel read-only data: 32768k1763worker2 # [ 0.932957] Freeing unused kernel image (text/rodata gap) memory: 1184K1764worker1 # [ 0.936180] Freeing unused kernel image (initmem) memory: 3652K1765worker2 # [ 0.934514] Freeing unused kernel image (rodata/data gap) memory: 720K1766worker1 # [ 0.937307] Write protecting the kernel read-only data: 32768k1767worker1 # [ 0.939223] Freeing unused kernel image (text/rodata gap) memory: 1184K1768worker1 # [ 0.940759] Freeing unused kernel image (rodata/data gap) memory: 720K1769client # [ 0.938514] Freeing initrd memory: 29056K1770client # [ 0.941790] Freeing unused decrypted memory: 2028K1771client # [ 0.944547] Freeing unused kernel image (initmem) memory: 3652K1772client # [ 0.945724] Write protecting the kernel read-only data: 32768k1773client # [ 0.947681] Freeing unused kernel image (text/rodata gap) memory: 1184K1774client # [ 0.949271] Freeing unused kernel image (rodata/data gap) memory: 720K1775lb # [ 0.981437] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776lb # [ 0.982524] Run /init as init process1777lb # [ 0.993062] systemd[1]: Inserted module 'autofs4'1778worker2 # [ 0.985343] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779worker2 # [ 0.986482] Run /init as init process1780worker1 # [ 0.991694] x86/mm: Checked W+X mappings: passed, no W+X pages found.1781worker1 # [ 0.992818] Run /init as init process1782worker2 # [ 0.997191] systemd[1]: Inserted module 'autofs4'1783lb # [ 1.012848] fuse: init (API version 7.45)1784worker1 # [ 1.003394] systemd[1]: Inserted module 'autofs4'1785client # [ 1.000355] x86/mm: Checked W+X mappings: passed, no W+X pages found.1786lb # [ 1.020200] ACPI: \_SB_.GSIG: Enabled at IRQ 221787client # [ 1.001526] Run /init as init process1788lb # [ 1.022575] ACPI: \_SB_.GSIH: Enabled at IRQ 231789lb # [ 1.025907] ACPI: \_SB_.GSIE: Enabled at IRQ 201790lb # [ 1.027986] ACPI: \_SB_.GSIF: Enabled at IRQ 211791worker2 # [ 1.017053] fuse: init (API version 7.45)1792client # [ 1.012603] systemd[1]: Inserted module 'autofs4'1793lb # [ 1.033031] virtiofs virtio5: discovered new tag: nix-store1794worker1 # [ 1.023164] fuse: init (API version 7.45)1795lb # [ 1.034649] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1796worker2 # [ 1.024477] ACPI: \_SB_.GSIG: Enabled at IRQ 221797worker2 # [ 1.026919] ACPI: \_SB_.GSIH: Enabled at IRQ 231798worker1 # [ 1.030547] ACPI: \_SB_.GSIG: Enabled at IRQ 221799lb # [ 1.041600] virtiofs virtio6: discovered new tag: shared1800lb # [ 1.043185] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1801worker2 # [ 1.029880] ACPI: \_SB_.GSIE: Enabled at IRQ 201802worker1 # [ 1.033029] ACPI: \_SB_.GSIH: Enabled at IRQ 231803worker2 # [ 1.031892] ACPI: \_SB_.GSIF: Enabled at IRQ 211804lb # [ 1.046279] virtiofs virtio7: discovered new tag: xchg1805worker1 # [ 1.036215] ACPI: \_SB_.GSIE: Enabled at IRQ 201806lb # [ 1.047808] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1807worker1 # [ 1.038341] ACPI: \_SB_.GSIF: Enabled at IRQ 211808worker2 # [ 1.036994] virtiofs virtio5: discovered new tag: nix-store1809client # [ 1.032394] fuse: init (API version 7.45)1810worker2 # [ 1.038559] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1811worker1 # [ 1.043543] virtiofs virtio5: discovered new tag: nix-store1812worker1 # [ 1.045107] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1813client # [ 1.039840] ACPI: \_SB_.GSIG: Enabled at IRQ 221814worker2 # [ 1.045255] virtiofs virtio6: discovered new tag: shared1815worker2 # [ 1.046717] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1816client # [ 1.042642] ACPI: \_SB_.GSIH: Enabled at IRQ 231817worker1 # [ 1.052369] virtiofs virtio6: discovered new tag: shared1818worker2 # [ 1.050130] virtiofs virtio7: discovered new tag: xchg1819client # [ 1.045993] ACPI: \_SB_.GSIE: Enabled at IRQ 201820worker1 # [ 1.054073] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1821worker2 # [ 1.051647] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1822client # [ 1.048325] ACPI: \_SB_.GSIF: Enabled at IRQ 211823worker1 # [ 1.057313] virtiofs virtio7: discovered new tag: xchg1824lb # [ 1.070049] systemd[1]: Successfully made /usr/ read-only.1825worker1 # [ 1.058829] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1826client # [ 1.054267] virtiofs virtio5: discovered new tag: nix-store1827client # [ 1.055906] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1828client # [ 1.063008] virtiofs virtio6: discovered new tag: shared1829client # [ 1.064605] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1830client # [ 1.068162] virtiofs virtio7: discovered new tag: xchg1831worker2 # [ 1.074751] systemd[1]: Successfully made /usr/ read-only.1832client # [ 1.069701] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833worker1 # [ 1.079885] systemd[1]: Successfully made /usr/ read-only.1834client # [ 1.091369] systemd[1]: Successfully made /usr/ read-only.1835lb # [ 1.406301] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836lb # [ 1.418135] systemd[1]: Detected virtualization kvm.1837lb # [ 1.420320] systemd[1]: Detected architecture x86-64.1838lb # [ 1.422429] systemd[1]: Running in initrd.1839lb # [ 1.424847] systemd[1]: Initializing machine ID from random generator.1840worker2 # [ 1.410248] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1841lb # [ 1.427750] systemd[1]: Hostname set to <lb>.1842worker2 # [ 1.415313] systemd[1]: Detected virtualization kvm.1843worker2 # [ 1.416200] systemd[1]: Detected architecture x86-64.1844worker2 # [ 1.417111] systemd[1]: Running in initrd.1845worker2 # [ 1.418196] systemd[1]: Initializing machine ID from random generator.1846worker2 # [ 1.419520] systemd[1]: Hostname set to <worker2>.1847worker1 # [ 1.416530] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848worker1 # [ 1.428807] systemd[1]: Detected virtualization kvm.1849worker1 # [ 1.431012] systemd[1]: Detected architecture x86-64.1850worker1 # [ 1.433218] systemd[1]: Running in initrd.1851worker1 # [ 1.435635] systemd[1]: Initializing machine ID from random generator.1852worker1 # [ 1.438641] systemd[1]: Hostname set to <worker1>.1853client # [ 1.428138] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854client # [ 1.440304] systemd[1]: Detected virtualization kvm.1855client # [ 1.442422] systemd[1]: Detected architecture x86-64.1856client # [ 1.444526] systemd[1]: Running in initrd.1857client # [ 1.447022] systemd[1]: Initializing machine ID from random generator.1858client # [ 1.449942] systemd[1]: Hostname set to <client>.1859worker2 # [ 1.620167] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860lb # [ 1.649209] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861worker1 # [ 1.645424] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862worker2 # [ 1.671356] systemd[1]: Queued start job for default target Initrd Default Target.1863client # [ 1.668395] systemd[1]: bpf-restrict-fs: LSM BPF program attached1864worker2 # [ 1.675226] systemd[1]: Created slice Slice /system/modprobe.1865worker2 # [ 1.676435] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866worker2 # [ 1.677908] systemd[1]: Expecting device /dev/disk/by-label/nixos...1867worker2 # [ 1.679152] systemd[1]: Reached target Path Units.1868worker2 # [ 1.680086] systemd[1]: Reached target Slice Units.1869worker1 # [ 1.682870] systemd[1]: Queued start job for default target Initrd Default Target.1870worker2 # [ 1.681010] systemd[1]: Reached target Swaps.1871worker2 # [ 1.681807] systemd[1]: Reached target Timer Units.1872worker2 # [ 1.682791] systemd[1]: Listening on D-Bus System Message Bus Socket.1873worker2 # [ 1.684083] systemd[1]: Listening on Journal Socket (/dev/log).1874worker1 # [ 1.687404] systemd[1]: Created slice Slice /system/modprobe.1875worker2 # [ 1.685288] systemd[1]: Listening on Journal Sockets.1876worker2 # [ 1.686360] systemd[1]: Listening on udev Control Socket.1877worker1 # [ 1.688643] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1878worker2 # [ 1.687460] systemd[1]: Listening on udev Kernel Socket.1879worker1 # [ 1.690154] systemd[1]: Expecting device /dev/disk/by-label/nixos...1880lb # [ 1.701554] systemd[1]: Queued start job for default target Initrd Default Target.1881worker2 # [ 1.688452] systemd[1]: Reached target Socket Units.1882worker1 # [ 1.691287] systemd[1]: Reached target Path Units.1883worker1 # [ 1.692185] systemd[1]: Reached target Slice Units.1884worker1 # [ 1.693114] systemd[1]: Reached target Swaps.1885worker2 # [ 1.690280] systemd[1]: Starting Create List of Static Device Nodes...1886worker1 # [ 1.693943] systemd[1]: Reached target Timer Units.1887lb # [ 1.705569] systemd[1]: Created slice Slice /system/modprobe.1888worker1 # [ 1.694947] systemd[1]: Listening on D-Bus System Message Bus Socket.1889lb # [ 1.706728] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1890worker1 # [ 1.696234] systemd[1]: Listening on Journal Socket (/dev/log).1891worker2 # [ 1.694052] systemd[1]: Starting Load Kernel Module configfs...1892lb # [ 1.708077] systemd[1]: Expecting device /dev/disk/by-label/nixos...1893worker1 # [ 1.697457] systemd[1]: Listening on Journal Sockets.1894lb # [ 1.709180] systemd[1]: Reached target Path Units.1895worker1 # [ 1.698509] systemd[1]: Listening on udev Control Socket.1896lb # [ 1.710009] systemd[1]: Reached target Slice Units.1897worker1 # [ 1.699561] systemd[1]: Listening on udev Kernel Socket.1898lb # [ 1.710888] systemd[1]: Reached target Swaps.1899lb # [ 1.711690] systemd[1]: Reached target Timer Units.1900worker1 # [ 1.700563] systemd[1]: Reached target Socket Units.1901lb # [ 1.712642] systemd[1]: Listening on D-Bus System Message Bus Socket.1902worker1 # [ 1.702329] systemd[1]: Starting Create List of Static Device Nodes...1903lb # [ 1.713833] systemd[1]: Listening on Journal Socket (/dev/log).1904lb # [ 1.714974] systemd[1]: Listening on Journal Sockets.1905lb # [ 1.715923] systemd[1]: Listening on udev Control Socket.1906lb # [ 1.716918] systemd[1]: Listening on udev Kernel Socket.1907lb # [ 1.717858] systemd[1]: Reached target Socket Units.1908worker1 # [ 1.707040] systemd[1]: Starting Load Kernel Module configfs...1909lb # [ 1.719605] systemd[1]: Starting Create List of Static Device Nodes...1910lb # [ 1.723347] systemd[1]: Starting Load Kernel Module configfs...1911worker2 # [ 1.702052] systemd[1]: Starting Journal Service...1912worker1 # [ 1.715248] systemd[1]: Starting Journal Service...1913client # [ 1.722122] systemd[1]: Queued start job for default target Initrd Default Target.1914worker2 # [ 1.729155] systemd[1]: Starting Load Kernel Modules...1915lb # [ 1.731875] systemd[1]: Starting Journal Service...1916client # [ 1.726580] systemd[1]: Created slice Slice /system/modprobe.1917client # [ 1.727771] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1918worker2 # [ 1.733074] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1919client # [ 1.729243] systemd[1]: Expecting device /dev/disk/by-label/nixos...1920client # [ 1.730387] systemd[1]: Reached target Path Units.1921client # [ 1.731308] systemd[1]: Reached target Slice Units.1922client # [ 1.732272] systemd[1]: Reached target Swaps.1923worker1 # [ 1.740197] systemd[1]: Starting Load Kernel Modules...1924client # [ 1.733073] systemd[1]: Reached target Timer Units.1925client # [ 1.734087] systemd[1]: Listening on D-Bus System Message Bus Socket.1926worker2 # [ 1.739751] systemd[1]: Starting Coldplug All udev Devices...1927client # [ 1.735363] systemd[1]: Listening on Journal Socket (/dev/log).1928client # [ 1.736542] systemd[1]: Listening on Journal Sockets.1929lb # [ 1.755836] systemd[1]: Starting Load Kernel Modules...1930worker1 # [ 1.744203] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1931client # [ 1.737508] systemd[1]: Listening on udev Control Socket.1932client # [ 1.738541] systemd[1]: Listening on udev Kernel Socket.1933client # [ 1.739497] systemd[1]: Reached target Socket Units.1934client # [ 1.741391] systemd[1]: Starting Create List of Static Device Nodes...1935lb # [ 1.760392] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1936worker1 # [ 1.752291] systemd[1]: Starting Coldplug All udev Devices...1937client # [ 1.745052] systemd[1]: Starting Load Kernel Module configfs...1938worker2 # [ 1.751689] systemd-journald[66]: Collecting audit messages is disabled.1939worker2 # [ 1.754337] systemd[1]: Finished Create List of Static Device Nodes.1940lb # [ 1.768341] systemd[1]: Starting Coldplug All udev Devices...1941worker2 # [ 1.760575] systemd[1]: modprobe@configfs.service: Deactivated successfully.1942worker1 # [ 1.763403] systemd-journald[66]: Collecting audit messages is disabled.1943worker1 # [ 1.765067] systemd[1]: Finished Create List of Static Device Nodes.1944lb # [ 1.780651] systemd-journald[66]: Collecting audit messages is disabled.1945worker2 # [ 1.767334] systemd[1]: Finished Load Kernel Module configfs.1946worker1 # [ 1.771553] systemd[1]: modprobe@configfs.service: Deactivated successfully.1947lb # [ 1.783324] systemd[1]: Finished Create List of Static Device Nodes.1948client # [ 1.754903] systemd[1]: Starting Journal Service...1949lb # [ 1.786838] systemd[1]: modprobe@configfs.service: Deactivated successfully.1950worker2 # [ 1.772389] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1951worker1 # [ 1.778533] systemd[1]: Finished Load Kernel Module configfs.1952lb # [ 1.795668] systemd[1]: Finished Load Kernel Module configfs.1953worker1 # [ 1.784521] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1954worker2 # [ 1.784054] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1955client # [ 1.781387] systemd[1]: Starting Load Kernel Modules...1956lb # [ 1.801699] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1957client # [ 1.785308] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1958worker1 # [ 1.796185] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1959worker2 # [ 1.796127] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1960lb # [ 1.811338] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1961client # [ 1.793392] systemd[1]: Starting Coldplug All udev Devices...1962worker1 # [ 1.803987] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1963worker2 # [ 1.804016] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1964client # [ 1.803890] systemd-journald[66]: Collecting audit messages is disabled.1965lb # [ 1.822550] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1966worker1 # [ 1.814285] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1967client # [ 1.806935] systemd[1]: Finished Create List of Static Device Nodes.1968lb # [ 1.829301] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1969client # [ 1.812874] systemd[1]: modprobe@configfs.service: Deactivated successfully.1970client # [ 1.819642] systemd[1]: Finished Load Kernel Module configfs.1971worker2 # [ 1.825076] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1972client # [ 1.826650] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1973worker1 # [ 1.837662] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1974worker2 # [ 1.835183] systemd[1]: Starting Create Static Device Nodes in /dev...1975lb # [ 1.852830] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1976worker1 # [ 1.846302] systemd[1]: Starting Create Static Device Nodes in /dev...1977client # [ 1.838275] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1978worker2 # [ 1.847617] systemd[1]: Finished Load Kernel Modules.1979lb # [ 1.861471] systemd[1]: Starting Create Static Device Nodes in /dev...1980worker1 # [ 1.857169] systemd[1]: Finished Load Kernel Modules.1981worker2 # [ 1.855521] systemd[1]: Starting Apply Kernel Variables...1982client # [ 1.851043] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1983lb # [ 1.872339] systemd[1]: Finished Load Kernel Modules.1984worker1 # [ 1.867184] systemd[1]: Starting Apply Kernel Variables...1985lb # [ 1.880527] systemd[1]: Starting Apply Kernel Variables...1986client # [ 1.862414] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1987worker2 # [ 1.737242] systemd-modules-load[68]: Inserted module 'dm_mod'1988worker2 # [ 1.739301] systemd-modules-load[68]: Inserted module 'virtio_balloon'1989worker2 # [ 1.877142] systemd[1]: Finished Create Static Device Nodes in /dev.1990worker2 # [ 1.741295] systemd-modules-load[68]: Inserted module 'virtio_gpu'1991worker2 # [ 1.882177] systemd[1]: Started Journal Service.1992worker1 # [ 1.748284] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.885685] systemd[1]: Finished Create Static Device Nodes in /dev.1993worker1 # 1994worker1 # [ 1.750729] systemd-modules-load[68]: Inserted module 'virtio_balloon'1995lb # [ 1.898833] systemd[1]: Finished Create Static Device Nodes in /dev.1996worker1 # [ 1.754327] systemd-modules-load[68]: Inserted module 'virtio_gpu'1997worker1 # [ 1.892225] systemd[1]: Started Journal Service.1998client # [ 1.884319] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1999lb # [ 1.904463] systemd[1]: Started Journal Service.2000lb # [ 1.771173] systemd-modules-load[67]: Inserted module 'dm_mod'
loading 2,661 hidden lines…
4662worker1 # [ 52.418372] nix-daemon[870]: reaped child process 968, status = succeeded4663lb # [ 52.437747] niks3-server[915]: 2026/09/25 16:32:15 INFO Received push request method=POST path=/api/pushes4664worker1 # [ 52.432815] nix-grpc-daemon[929]: time=2026-09-25T16:32:14.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4665worker1 # [ 52.434471] nix-grpc-daemon[929]: time=2026-09-25T16:32:14.732Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4666lb # [ 52.474204] niks3-server[915]: 2026/09/25 16:32:15 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4667lb # [ 52.485182] niks3-server[915]: 2026/09/25 16:32:15 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4668lb # [ 52.489918] niks3-server[915]: 2026/09/25 16:32:15 INFO Signed narinfos id=10 count=14669worker1 # [ 52.479997] nix-grpc-daemon[929]: time=2026-09-25T16:32:14.778Z level=INFO msg="Uploading 1 narinfos"4670lb # [ 52.499587] niks3-server[915]: 2026/09/25 16:32:15 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4671lb # [ 52.506516] niks3-server[915]: 2026/09/25 16:32:15 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4672lb # [ 52.517165] niks3-server[915]: 2026/09/25 16:32:15 INFO Received complete push request method=POST path=/api/pushes/10/complete4673lb # [ 52.524577] niks3-server[915]: 2026/09/25 16:32:15 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4674worker1 # [ 52.511517] nix-grpc-daemon[929]: time=2026-09-25T16:32:14.809Z level=INFO msg="Upload complete. (215ms)"4675worker1 # [ 52.517746] nix-grpc-daemon[511]: ts=2026-09-25T16:32:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04676worker1 # [ 52.532087] nix-grpc-daemon[511]: ts=2026-09-25T16:32:14Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04677worker1 # [ 52.536593] nix-grpc-daemon[511]: ts=2026-09-25T16:32:14Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=14678worker1 # [ 52.541226] nix-grpc-daemon[511]: ts=2026-09-25T16:32:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764679worker1 # [ 52.550524] nix-daemon[870]: reaped child process 947, status = succeeded4680worker1 # [ 52.552314] nix-grpc-daemon[511]: ts=2026-09-25T16:32:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764681worker1 # [ 52.657665] nix-grpc-daemon[929]: time=2026-09-25T16:32:14.956Z level=INFO msg="All 1 paths already cached"4682client # copying 1 paths to 'lb:50051'...4683client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4684worker1 # [ 52.667935] nix-grpc-daemon[511]: ts=2026-09-25T16:32:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764685worker1 # [ 52.676492] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4686worker1 # [ 52.785740] nix-daemon[870]: accepted connection from pid 974, user nix-grpc-daemon (trusted)4687worker1 # [ 52.801062] nix-daemon[870]: reaped child process 977, status = succeeded4688lb # [ 52.818171] niks3-server[915]: 2026/09/25 16:32:15 INFO Received push request method=POST path=/api/pushes4689worker1 # [ 52.812979] nix-grpc-daemon[929]: time=2026-09-25T16:32:15.111Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4690worker1 # [ 52.814481] nix-grpc-daemon[929]: time=2026-09-25T16:32:15.113Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4691lb # [ 52.847628] niks3-server[915]: 2026/09/25 16:32:16 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4692lb # [ 52.853146] niks3-server[915]: 2026/09/25 16:32:16 INFO Signed narinfos id=11 count=14693worker1 # [ 52.843211] nix-grpc-daemon[929]: time=2026-09-25T16:32:15.141Z level=INFO msg="Uploading 1 narinfos"4694lb # [ 52.861090] niks3-server[915]: 2026/09/25 16:32:16 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4695lb # [ 52.870656] niks3-server[915]: 2026/09/25 16:32:16 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4696lb # [ 52.881116] niks3-server[915]: 2026/09/25 16:32:16 INFO Received complete push request method=POST path=/api/pushes/11/complete4697worker1 # [ 52.879270] nix-grpc-daemon[929]: time=2026-09-25T16:32:15.177Z level=INFO msg="Upload complete. (188ms)"4698lb # [ 52.892812] niks3-server[915]: 2026/09/25 16:32:16 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4699worker1 # [ 52.883726] nix-grpc-daemon[511]: ts=2026-09-25T16:32:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=6244700worker1 # [ 52.889989] nix-grpc-daemon[511]: ts=2026-09-25T16:32:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764701worker1 # [ 52.899516] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4702worker1 # [ 52.910183] nix-daemon[870]: reaped child process 972, status = succeeded4703worker1 # [ 52.913449] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4704client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4705client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4706client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4707client # farm-b> LOG-farm-b-sig4708worker1 # [ 54.158671] nix-daemon[870]: reaped child process 981, status = succeeded4709worker1 # [ 54.235663] nix-daemon[870]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4710worker1 # [ 54.247161] nix-daemon[870]: reaped child process 1000, status = succeeded4711lb # [ 54.267931] niks3-server[915]: 2026/09/25 16:32:17 INFO Received push request method=POST path=/api/pushes4712worker1 # [ 54.263965] nix-grpc-daemon[929]: time=2026-09-25T16:32:16.562Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4713worker1 # [ 54.265456] nix-grpc-daemon[929]: time=2026-09-25T16:32:16.564Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4714lb # [ 54.303227] niks3-server[915]: 2026/09/25 16:32:17 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4715lb # [ 54.305743] niks3-server[915]: 2026/09/25 16:32:17 INFO Signed narinfos id=12 count=14716worker1 # [ 54.292861] nix-grpc-daemon[929]: time=2026-09-25T16:32:16.591Z level=INFO msg="Uploading 1 narinfos"4717lb # [ 54.309310] niks3-server[915]: 2026/09/25 16:32:17 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4718lb # [ 54.312072] niks3-server[915]: 2026/09/25 16:32:17 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4719lb # [ 54.317953] niks3-server[915]: 2026/09/25 16:32:17 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4720lb # [ 54.322276] niks3-server[915]: 2026/09/25 16:32:17 INFO Received complete push request method=POST path=/api/pushes/12/complete4721worker1 # [ 54.316515] nix-grpc-daemon[929]: time=2026-09-25T16:32:16.615Z level=INFO msg="Upload complete. (195ms)"4722lb # [ 54.329886] niks3-server[915]: 2026/09/25 16:32:17 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4723worker1 # [ 54.320709] nix-grpc-daemon[511]: ts=2026-09-25T16:32:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04724worker1 # [ 54.329924] nix-grpc-daemon[511]: ts=2026-09-25T16:32:16Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04725worker1 # [ 54.332699] nix-grpc-daemon[511]: ts=2026-09-25T16:32:16Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=14726worker1 # [ 54.334452] nix-grpc-daemon[511]: ts=2026-09-25T16:32:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764727worker1 # [ 54.339215] nix-daemon[870]: reaped child process 979, status = succeeded4728worker1 # [ 54.342945] nix-grpc-daemon[511]: ts=2026-09-25T16:32:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764729worker1 # [ 54.348191] nix-grpc-daemon[929]: time=2026-09-25T16:32:16.646Z level=INFO msg="All 2 paths already cached"4730client # copying 1 paths to 'lb:50051'...4731worker1 # [ 54.353287] nix-grpc-daemon[511]: ts=2026-09-25T16:32:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764732client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4733worker1 # [ 54.364312] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4734worker1 # [ 54.447138] nix-daemon[870]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4735worker1 # [ 54.460170] nix-daemon[870]: reaped child process 1009, status = succeeded4736lb # [ 54.479758] niks3-server[915]: 2026/09/25 16:32:17 INFO Received push request method=POST path=/api/pushes4737worker1 # [ 54.474794] nix-grpc-daemon[929]: time=2026-09-25T16:32:16.773Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4738worker1 # [ 54.476246] nix-grpc-daemon[929]: time=2026-09-25T16:32:16.774Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4739lb # [ 54.506937] niks3-server[915]: 2026/09/25 16:32:17 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4740worker1 # [ 54.505527] nix-grpc-daemon[929]: time=2026-09-25T16:32:16.803Z level=INFO msg="Uploading 1 narinfos"4741lb # [ 54.518939] niks3-server[915]: 2026/09/25 16:32:17 INFO Signed narinfos id=13 count=14742lb # [ 54.523576] niks3-server[915]: 2026/09/25 16:32:17 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4743lb # [ 54.532143] niks3-server[915]: 2026/09/25 16:32:17 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4744lb # [ 54.542477] niks3-server[915]: 2026/09/25 16:32:17 INFO Received complete push request method=POST path=/api/pushes/13/complete4745lb # [ 54.550905] niks3-server[915]: 2026/09/25 16:32:17 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4746worker1 # [ 54.537564] nix-grpc-daemon[929]: time=2026-09-25T16:32:16.836Z level=INFO msg="Upload complete. (167ms)"4747worker1 # [ 54.541904] nix-grpc-daemon[511]: ts=2026-09-25T16:32:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=7524748worker1 # [ 54.548131] nix-grpc-daemon[511]: ts=2026-09-25T16:32:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764749worker1 # [ 54.556708] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4750worker1 # [ 54.562294] nix-daemon[870]: reaped child process 1004, status = succeeded4751worker1 # [ 54.566992] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4752client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4753client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4754client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4755client # farm-top> LOG-farm-top-sig4756worker1 # [ 56.004458] nix-daemon[870]: reaped child process 1013, status = succeeded4757worker1 # [ 56.089611] nix-daemon[870]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)4758worker1 # [ 56.101549] nix-daemon[870]: reaped child process 1032, status = succeeded4759lb # [ 56.121816] niks3-server[915]: 2026/09/25 16:32:19 INFO Received push request method=POST path=/api/pushes4760worker1 # [ 56.116985] nix-grpc-daemon[929]: time=2026-09-25T16:32:18.415Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4761worker1 # [ 56.118576] nix-grpc-daemon[929]: time=2026-09-25T16:32:18.417Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4762lb # [ 56.159195] niks3-server[915]: 2026/09/25 16:32:19 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4763lb # [ 56.175616] niks3-server[915]: 2026/09/25 16:32:19 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4764worker1 # [ 56.170642] nix-grpc-daemon[929]: time=2026-09-25T16:32:18.468Z level=INFO msg="Uploading 1 narinfos"4765lb # [ 56.184120] niks3-server[915]: 2026/09/25 16:32:19 INFO Signed narinfos id=14 count=14766lb # [ 56.186117] niks3-server[915]: 2026/09/25 16:32:19 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4767lb # [ 56.190536] niks3-server[915]: 2026/09/25 16:32:19 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4768lb # [ 56.201394] niks3-server[915]: 2026/09/25 16:32:19 INFO Received complete push request method=POST path=/api/pushes/14/complete4769worker1 # [ 56.193263] nix-grpc-daemon[929]: time=2026-09-25T16:32:18.491Z level=INFO msg="Upload complete. (218ms)"4770lb # [ 56.208056] niks3-server[915]: 2026/09/25 16:32:19 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo4771worker1 # [ 56.201218] nix-grpc-daemon[511]: ts=2026-09-25T16:32:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04772worker1 # [ 56.216750] nix-daemon[870]: reaped child process 1011, status = succeeded4773client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.52 seconds)4774client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath4775client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.11 seconds)4776client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4777worker1 # [ 56.445434] nix-grpc-daemon[511]: ts=2026-09-25T16:32:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:449464778worker1 # [ 56.451646] nix-grpc-daemon[511]: ts=2026-09-25T16:32:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:449464779client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.12 seconds)4780client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4781worker2 # [ 56.563788] nix-grpc-daemon[803]: ts=2026-09-25T16:32:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:431164782client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4783client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a4784client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b4785client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4786worker2 # [ 56.589243] nix-grpc-daemon[803]: ts=2026-09-25T16:32:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:431164787worker2 # [ 56.595704] nix-grpc-daemon[803]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...4788worker2 # [ 56.609833] nix-grpc-daemon[803]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...4789worker2 # [ 56.621879] nix-grpc-daemon[803]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...4790client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.20 seconds)4791client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4792client # finding garbage collector roots...4793client # removing stale temporary roots file "/nix/var/nix/temproots/873"4794client # removing stale temporary roots file "/nix/var/nix/temproots/819"4795client # removing stale temporary roots file "/nix/var/nix/temproots/860"4796client # removing stale temporary roots file "/nix/var/nix/temproots/896"4797client # removing stale temporary roots file "/nix/var/nix/temproots/852"4798client # removing stale temporary roots file "/nix/var/nix/temproots/804"4799client # removing stale temporary roots file "/nix/var/nix/temproots/840"4800client # removing stale temporary roots file "/nix/var/nix/temproots/881"4801client # deleting specified paths...4802client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'4803client # deleting unused links...4804client # note: hard linking is currently saving 0.0 KiB4805worker1 # [ 56.897991] nix-grpc-daemon[511]: ts=2026-09-25T16:32:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364764806worker2 # [ 56.902788] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:552204807worker2 # [ 56.912749] nix-grpc-daemon[873]: time=2026-09-25T16:32:20.086Z level=INFO msg="All 1 paths already cached"4808worker2 # [ 56.919087] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:552204809worker1 # [ 56.930083] nix-grpc-daemon[511]: ts=2026-09-25T16:32:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364764810client # copying 3 paths...4811worker2 # [ 56.936708] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:552204812client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...4813worker1 # [ 56.956383] nix-grpc-daemon[511]: ts=2026-09-25T16:32:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364764814worker1 # [ 56.961915] nix-grpc-daemon[511]: ts=2026-09-25T16:32:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_out=2324815worker2 # [ 56.960279] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:552204816worker2 # [ 56.965740] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=0 paths=1 nar_bytes_out=1764817worker1 # [ 56.971136] nix-grpc-daemon[511]: ts=2026-09-25T16:32:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364764818worker1 # [ 56.975760] nix-grpc-daemon[511]: ts=2026-09-25T16:32:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_out=1284819client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...4820client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...4821client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.34 seconds)4822(finished: subtest: outputs read back through any worker carry the cache signature, in 6.30 seconds)4823subtest: two clients wanting the same drv build it once4824client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4825client # [ 57.024534] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4826client # Running as unit: dedup1.service; invocation ID: c3a551215c6f43fdb1ecf510431578454827client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)4828client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4829lb # [ 57.069102] envoy[922]: conn peer=[2001:db8:1::1]:44916 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9447 tx=4312 ms=41774830lb # [ 57.072829] envoy[922]: conn peer=[2001:db8:1::1]:44926 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=476 ms=564831lb # [ 57.076302] envoy[922]: conn peer=[2001:db8:1::1]:44938 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4165 tx=1902 ms=15594832lb # [ 57.079836] envoy[922]: conn peer=[2001:db8:1::1]:44948 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9567 tx=4389 ms=53634833lb # [ 57.083323] envoy[922]: conn peer=[2001:db8:1::1]:58902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2872 ms=1014834lb # [ 57.086869] envoy[922]: conn peer=[2001:db8:1::1]:58932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=284835lb # [ 57.090281] envoy[922]: conn peer=[2001:db8:1::1]:58916 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=364836lb # [ 57.093655] envoy[922]: conn peer=[2001:db8:1::1]:58906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=424837worker1 # [ 57.154725] nix-grpc-daemon[511]: ts=2026-09-25T16:32:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364764838client # [ 57.155812] nix[949]: don't know how to build these paths:4839client # [ 57.156763] nix[949]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4840worker1 # [ 57.169650] nix-grpc-daemon[511]: ts=2026-09-25T16:32:19Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04841worker2 # [ 57.168988] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552204842worker1 # [ 57.173997] nix-grpc-daemon[511]: ts=2026-09-25T16:32:19Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14843worker2 # [ 57.181109] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:552204844client # [ 57.189139] nix[949]: copying 1 paths to 'lb:50051'...4845client # [ 57.189947] nix[949]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4846worker2 # [ 57.191761] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:552204847worker2 # [ 57.202899] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4848worker2 # [ 57.311771] nix-daemon[861]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4849worker2 # [ 57.325203] nix-daemon[861]: reaped child process 1034, status = succeeded4850lb # [ 57.348889] niks3-server[915]: 2026/09/25 16:32:20 INFO Received push request method=POST path=/api/pushes4851worker2 # [ 57.340327] nix-grpc-daemon[873]: time=2026-09-25T16:32:20.514Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4852worker2 # [ 57.341767] nix-grpc-daemon[873]: time=2026-09-25T16:32:20.515Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4853lb # [ 57.380358] niks3-server[915]: 2026/09/25 16:32:20 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4854worker2 # [ 57.368478] nix-grpc-daemon[873]: time=2026-09-25T16:32:20.542Z level=INFO msg="Uploading 1 narinfos"4855lb # [ 57.385278] niks3-server[915]: 2026/09/25 16:32:20 INFO Signed narinfos id=15 count=14856lb # [ 57.390751] niks3-server[915]: 2026/09/25 16:32:20 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4857lb # [ 57.400553] niks3-server[915]: 2026/09/25 16:32:20 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4858lb # [ 57.409966] niks3-server[915]: 2026/09/25 16:32:20 INFO Received complete push request method=POST path=/api/pushes/15/complete4859worker2 # [ 57.403370] nix-grpc-daemon[873]: time=2026-09-25T16:32:20.577Z level=INFO msg="Upload complete. (187ms)"4860lb # [ 57.421432] niks3-server[915]: 2026/09/25 16:32:20 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4861worker2 # [ 57.406212] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=0 paths=1 nar_bytes_in=4324862worker2 # [ 57.410720] nix-grpc-daemon[803]: ts=2026-09-25T16:32:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552204863worker2 # [ 57.422173] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4864worker2 # [ 57.426437] nix-daemon[861]: reaped child process 1029, status = succeeded4865worker2 # [ 57.431140] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4866client # [ 57.429452] nix[949]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4867client # [ 58.077090] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4868client # Running as unit: dedup2.service; invocation ID: 93d7fa666fbc4b8aa5fa6ab2c3614fbf4869client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.06 seconds)4870worker2 # [ 58.236488] nix-grpc-daemon[803]: ts=2026-09-25T16:32:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:552204871client # [ 58.248348] nix[965]: this derivation will be built:4872worker1 # [ 58.257895] nix-grpc-daemon[511]: ts=2026-09-25T16:32:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364764873worker1 # [ 58.268554] nix-grpc-daemon[511]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...4874client # [ 58.278285] nix[965]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4875worker1 # [ 58.291122] nix-grpc-daemon[511]: ts=2026-09-25T16:32:20Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04876worker2 # [ 58.297105] nix-grpc-daemon[803]: ts=2026-09-25T16:32:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552204877client # [ 58.296456] nix[965]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4878client: must succeed: sleep 24879client: (finished: must succeed: sleep 2, in 2.04 seconds)4880client: waiting for success: ! systemctl is-active dedup1 dedup24881worker2 # [ 61.451706] nix-daemon[861]: reaped child process 1038, status = succeeded4882worker2 # [ 61.523857] nix-daemon[861]: accepted connection from pid 1068, user nix-grpc-daemon (trusted)4883worker2 # [ 61.536143] nix-daemon[861]: reaped child process 1071, status = succeeded4884lb # [ 61.560242] niks3-server[915]: 2026/09/25 16:32:24 INFO Received push request method=POST path=/api/pushes4885worker2 # [ 61.551665] nix-grpc-daemon[873]: time=2026-09-25T16:32:24.725Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4886worker2 # [ 61.553534] nix-grpc-daemon[873]: time=2026-09-25T16:32:24.727Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4887lb # [ 61.605163] niks3-server[915]: 2026/09/25 16:32:24 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4888lb # [ 61.613600] niks3-server[915]: 2026/09/25 16:32:24 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4889lb # [ 61.618529] niks3-server[915]: 2026/09/25 16:32:24 INFO Signed narinfos id=16 count=14890worker2 # [ 61.605095] nix-grpc-daemon[873]: time=2026-09-25T16:32:24.779Z level=INFO msg="Uploading 1 narinfos"4891lb # [ 61.626205] niks3-server[915]: 2026/09/25 16:32:24 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4892lb # [ 61.639152] niks3-server[915]: 2026/09/25 16:32:24 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4893lb # [ 61.641903] niks3-server[915]: 2026/09/25 16:32:24 INFO Received complete push request method=POST path=/api/pushes/16/complete4894lb # [ 61.651153] niks3-server[915]: 2026/09/25 16:32:24 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4895worker2 # [ 61.637695] nix-grpc-daemon[873]: time=2026-09-25T16:32:24.811Z level=INFO msg="Upload complete. (194ms)"4896worker2 # [ 61.644522] nix-grpc-daemon[803]: ts=2026-09-25T16:32:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04897worker2 # [ 61.652278] nix-grpc-daemon[803]: ts=2026-09-25T16:32:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04898worker2 # [ 61.666966] nix-daemon[861]: reaped child process 1036, status = succeeded4899client # [ 61.687127] systemd[1]: dedup1.service: Deactivated successfully.4900client # [ 61.691353] systemd[1]: dedup1.service: Consumed 114ms CPU time over 4.665s wall clock time, 43.5M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.4901client # [ 61.697414] systemd[1]: dedup2.service: Deactivated successfully.4902client # [ 61.700651] systemd[1]: dedup2.service: Consumed 142ms CPU time over 3.623s wall clock time, 37.4M memory peak, 6.7K incoming IP traffic, 6K outgoing IP traffic.4903client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4904client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4905client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.03 seconds)4906worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4907worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4908worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4909worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4910worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4911worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4912worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4913worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4914(finished: subtest: two clients wanting the same drv build it once, in 5.72 seconds)4915subtest: --timeout and --max-silent-time reach the builder4916worker2 # [ 62.864453] nix-grpc-daemon[803]: ts=2026-09-25T16:32:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:552204917worker1 # [ 62.877626] nix-grpc-daemon[511]: ts=2026-09-25T16:32:25Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=04918worker1 # [ 62.883563] nix-grpc-daemon[511]: ts=2026-09-25T16:32:25Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=14919worker1 # [ 62.887884] nix-grpc-daemon[511]: ts=2026-09-25T16:32:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764920worker1 # [ 62.899286] nix-grpc-daemon[511]: ts=2026-09-25T16:32:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764921worker1 # [ 62.909721] nix-grpc-daemon[511]: ts=2026-09-25T16:32:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764922worker1 # [ 62.920503] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4923worker1 # [ 63.017744] nix-daemon[870]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4924worker1 # [ 63.031394] nix-daemon[870]: reaped child process 1067, status = succeeded4925lb # [ 63.049572] niks3-server[915]: 2026/09/25 16:32:26 INFO Received push request method=POST path=/api/pushes4926worker1 # [ 63.044959] nix-grpc-daemon[929]: time=2026-09-25T16:32:25.343Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4927worker1 # [ 63.046959] nix-grpc-daemon[929]: time=2026-09-25T16:32:25.345Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"4928lb # [ 63.079209] niks3-server[915]: 2026/09/25 16:32:26 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4929worker1 # [ 63.071472] nix-grpc-daemon[929]: time=2026-09-25T16:32:25.369Z level=INFO msg="Uploading 1 narinfos"4930lb # [ 63.085256] niks3-server[915]: 2026/09/25 16:32:26 INFO Signed narinfos id=17 count=14931lb # [ 63.092636] niks3-server[915]: 2026/09/25 16:32:26 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst4932lb # [ 63.104142] niks3-server[915]: 2026/09/25 16:32:26 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls4933lb # [ 63.110347] niks3-server[915]: 2026/09/25 16:32:26 INFO Received complete push request method=POST path=/api/pushes/17/complete4934worker1 # [ 63.107748] nix-grpc-daemon[929]: time=2026-09-25T16:32:25.406Z level=INFO msg="Upload complete. (175ms)"4935lb # [ 63.121223] niks3-server[915]: 2026/09/25 16:32:26 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo4936worker1 # [ 63.112096] nix-grpc-daemon[511]: ts=2026-09-25T16:32:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=4244937worker1 # [ 63.118202] nix-grpc-daemon[511]: ts=2026-09-25T16:32:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764938worker1 # [ 63.130177] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4939worker1 # [ 63.137695] nix-daemon[870]: reaped child process 1062, status = succeeded4940worker1 # [ 63.142517] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4941lb # [ 67.069869] envoy[922]: conn peer=[2001:db8:1::1]:58942 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1615 tx=1785 ms=34254942lb # [ 67.073703] envoy[922]: conn peer=[2001:db8:1::1]:58940 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1521 ms=45074943worker1 # [ 69.154320] nix-grpc-daemon[511]: ts=2026-09-25T16:32:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04944worker1 # [ 69.167357] nix-daemon[870]: reaped child process 1069, status = succeeded4945worker1 # [ 69.168341] nix-daemon[870]: reaped child process 1071, status = succeeded4946worker2 # [ 69.309299] nix-grpc-daemon[803]: ts=2026-09-25T16:32:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:552204947worker1 # [ 69.321223] nix-grpc-daemon[511]: ts=2026-09-25T16:32:31Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=04948worker1 # [ 69.325824] nix-grpc-daemon[511]: ts=2026-09-25T16:32:31Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=14949worker2 # [ 69.324311] nix-grpc-daemon[803]: ts=2026-09-25T16:32:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552204950worker2 # [ 69.336486] nix-grpc-daemon[803]: ts=2026-09-25T16:32:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:552204951worker2 # [ 69.347424] nix-grpc-daemon[803]: ts=2026-09-25T16:32:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:552204952worker2 # [ 69.358171] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4953worker2 # [ 69.460655] nix-daemon[861]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)4954worker2 # [ 69.475344] nix-daemon[861]: reaped child process 1087, status = succeeded4955lb # [ 69.496822] niks3-server[915]: 2026/09/25 16:32:32 INFO Received push request method=POST path=/api/pushes4956worker2 # [ 69.488421] nix-grpc-daemon[873]: time=2026-09-25T16:32:32.662Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4957worker2 # [ 69.490325] nix-grpc-daemon[873]: time=2026-09-25T16:32:32.663Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"4958lb # [ 69.533679] niks3-server[915]: 2026/09/25 16:32:32 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst4959lb # [ 69.541402] niks3-server[915]: 2026/09/25 16:32:32 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4960worker2 # [ 69.531487] nix-grpc-daemon[873]: time=2026-09-25T16:32:32.705Z level=INFO msg="Uploading 1 narinfos"4961lb # [ 69.547955] niks3-server[915]: 2026/09/25 16:32:32 INFO Signed narinfos id=18 count=14962lb # [ 69.559839] niks3-server[915]: 2026/09/25 16:32:32 INFO Received complete push request method=POST path=/api/pushes/18/complete4963lb # [ 69.569981] niks3-server[915]: 2026/09/25 16:32:32 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls4964lb # [ 69.573330] niks3-server[915]: 2026/09/25 16:32:32 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo4965worker2 # [ 69.560597] nix-grpc-daemon[873]: time=2026-09-25T16:32:32.734Z level=INFO msg="Upload complete. (191ms)"4966worker2 # [ 69.565476] nix-grpc-daemon[803]: ts=2026-09-25T16:32:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=0 paths=1 nar_bytes_in=4244967worker2 # [ 69.571312] nix-grpc-daemon[803]: ts=2026-09-25T16:32:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552204968worker2 # [ 69.583107] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4969worker2 # [ 69.590203] nix-daemon[861]: reaped child process 1082, status = succeeded4970worker2 # [ 69.598531] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)4971worker2 # [ 75.411325] nix-grpc-daemon[803]: ts=2026-09-25T16:32:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=5 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04972worker2 # [ 75.423445] nix-daemon[861]: reaped child process 1089, status = succeeded4973worker2 # [ 75.424472] nix-daemon[861]: reaped child process 1091, status = succeeded4974worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4975worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4976worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4977worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4978(finished: subtest: --timeout and --max-silent-time reach the builder, in 12.90 seconds)4979subtest: a failing build is counted with its own reason4980worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4981worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4982worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4983worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4984worker2 # [ 75.847514] nix-grpc-daemon[803]: ts=2026-09-25T16:32:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:552204985worker1 # [ 75.861616] nix-grpc-daemon[511]: ts=2026-09-25T16:32:38Z level=debug event=want client=9 drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv cached=04986worker1 # [ 75.867721] nix-grpc-daemon[511]: ts=2026-09-25T16:32:38Z level=debug event=assigned drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv worker=192.168.1.3:50051 clients=14987worker1 # [ 75.872152] nix-grpc-daemon[511]: ts=2026-09-25T16:32:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364764988worker1 # [ 75.883880] nix-grpc-daemon[511]: ts=2026-09-25T16:32:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364764989worker1 # [ 75.894813] nix-grpc-daemon[511]: ts=2026-09-25T16:32:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364764990worker1 # [ 75.905331] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4991worker1 # [ 76.006196] nix-daemon[870]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4992worker1 # [ 76.018944] nix-daemon[870]: reaped child process 1107, status = succeeded4993lb # [ 76.038457] niks3-server[915]: 2026/09/25 16:32:39 INFO Received push request method=POST path=/api/pushes4994worker1 # [ 76.034167] nix-grpc-daemon[929]: time=2026-09-25T16:32:38.332Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4995worker1 # [ 76.036157] nix-grpc-daemon[929]: time=2026-09-25T16:32:38.334Z level=INFO msg="Uploading j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv (400B)"4996lb # [ 76.069307] niks3-server[915]: 2026/09/25 16:32:39 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4997worker1 # [ 76.063179] nix-grpc-daemon[929]: time=2026-09-25T16:32:38.361Z level=INFO msg="Uploading 1 narinfos"4998lb # [ 76.075780] niks3-server[915]: 2026/09/25 16:32:39 INFO Signed narinfos id=19 count=14999lb # [ 76.083142] niks3-server[915]: 2026/09/25 16:32:39 INFO Registered completed upload object_key=nar/1awa0khyixxmb53bhxv3vk1qy0fd3xzskbp3b8g2fzdmcz3cvv97.nar.zst5000lb # [ 76.092885] niks3-server[915]: 2026/09/25 16:32:39 INFO Registered completed upload object_key=j5sh943zaa0qjagii0xkivjfb4c2c5p0.ls5001lb # [ 76.103742] niks3-server[915]: 2026/09/25 16:32:39 INFO Received complete push request method=POST path=/api/pushes/19/complete5002worker1 # [ 76.098178] nix-grpc-daemon[929]: time=2026-09-25T16:32:38.396Z level=INFO msg="Upload complete. (180ms)"5003lb # [ 76.112488] niks3-server[915]: 2026/09/25 16:32:39 INFO Registered completed upload object_key=j5sh943zaa0qjagii0xkivjfb4c2c5p0.narinfo5004worker1 # [ 76.102589] nix-grpc-daemon[511]: ts=2026-09-25T16:32:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=4005005worker1 # [ 76.108359] nix-grpc-daemon[511]: ts=2026-09-25T16:32:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364765006worker1 # [ 76.118582] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5007worker1 # [ 76.130160] nix-daemon[870]: reaped child process 1102, status = succeeded5008worker1 # [ 76.133263] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5009lb # [ 77.070536] envoy[922]: conn peer=[2001:db8:1::1]:33914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=957 ms=63015010lb # [ 77.074370] envoy[922]: conn peer=[2001:db8:1::1]:33920 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=935 ms=61195011worker1 # [ 77.743784] nix-grpc-daemon[511]: ts=2026-09-25T16:32:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv assign_id=11 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05012worker1 # [ 77.771766] nix-daemon[870]: reaped child process 1109, status = succeeded5013worker1 # [ 77.774625] nix-daemon[870]: reaped child process 1111, status = succeeded5014worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5015worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5016worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5017worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5018worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5019worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5020worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5021worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5022worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5023worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5024worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5025worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5026worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5027worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5028worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5029worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5030(finished: subtest: a failing build is counted with its own reason, in 2.49 seconds)5031subtest: the first client leaving does not fail the second5032client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5033client # [ 78.154609] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5034client # Running as unit: share1.service; invocation ID: c9b6d0bd81974a698be8d1d62f9e77fe5035client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5036client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5037worker1 # [ 78.297214] nix-grpc-daemon[511]: ts=2026-09-25T16:32:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364765038client # [ 78.298427] nix[1055]: don't know how to build these paths:5039client # [ 78.299298] nix[1055]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5040worker1 # [ 78.310924] nix-grpc-daemon[511]: ts=2026-09-25T16:32:40Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05041worker1 # [ 78.315648] nix-grpc-daemon[511]: ts=2026-09-25T16:32:40Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15042worker2 # [ 78.310898] nix-grpc-daemon[803]: ts=2026-09-25T16:32:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552205043worker2 # [ 78.322981] nix-grpc-daemon[803]: ts=2026-09-25T16:32:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:552205044client # [ 78.331077] nix[1055]: copying 1 paths to 'lb:50051'...5045client # [ 78.331881] nix[1055]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5046worker2 # [ 78.333602] nix-grpc-daemon[803]: ts=2026-09-25T16:32:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:552205047worker2 # [ 78.344856] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)5048worker2 # [ 78.445788] nix-daemon[861]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)5049worker2 # [ 78.457670] nix-daemon[861]: reaped child process 1135, status = succeeded5050lb # [ 78.482242] niks3-server[915]: 2026/09/25 16:32:41 INFO Received push request method=POST path=/api/pushes5051worker2 # [ 78.474270] nix-grpc-daemon[873]: time=2026-09-25T16:32:41.648Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5052worker2 # [ 78.476240] nix-grpc-daemon[873]: time=2026-09-25T16:32:41.649Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5053lb # [ 78.510216] niks3-server[915]: 2026/09/25 16:32:41 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5054lb # [ 78.520811] niks3-server[915]: 2026/09/25 16:32:41 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign5055lb # [ 78.525276] niks3-server[915]: 2026/09/25 16:32:41 INFO Signed narinfos id=20 count=15056worker2 # [ 78.512269] nix-grpc-daemon[873]: time=2026-09-25T16:32:41.686Z level=INFO msg="Uploading 1 narinfos"5057lb # [ 78.535766] niks3-server[915]: 2026/09/25 16:32:41 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5058lb # [ 78.546412] niks3-server[915]: 2026/09/25 16:32:41 INFO Received complete push request method=POST path=/api/pushes/20/complete5059lb # [ 78.556410] niks3-server[915]: 2026/09/25 16:32:41 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5060worker2 # [ 78.539345] nix-grpc-daemon[873]: time=2026-09-25T16:32:41.713Z level=INFO msg="Upload complete. (183ms)"5061worker2 # [ 78.543229] nix-grpc-daemon[803]: ts=2026-09-25T16:32:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=0 paths=1 nar_bytes_in=4325062worker2 # [ 78.548269] nix-grpc-daemon[803]: ts=2026-09-25T16:32:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552205063worker2 # [ 78.559110] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)5064worker2 # [ 78.567354] nix-daemon[861]: reaped child process 1130, status = succeeded5065worker2 # [ 78.571532] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)5066client # [ 78.569908] nix[1055]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5067client # [ 79.222897] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5068client # Running as unit: share2.service; invocation ID: e87450d0ed5d4fc2ae161b7504aecffb5069client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5070client: must succeed: sleep 2; systemctl kill -s INT share15071client # [ 79.366084] nix[1071]: don't know how to build these paths:5072client # [ 79.366949] nix[1071]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5073worker1 # [ 79.368963] nix-grpc-daemon[511]: ts=2026-09-25T16:32:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364765074worker1 # [ 79.380262] nix-grpc-daemon[511]: ts=2026-09-25T16:32:41Z level=debug event=want client=11 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05075worker2 # [ 79.379493] nix-grpc-daemon[803]: ts=2026-09-25T16:32:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552205076client # [ 79.380930] nix[1071]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5077client # [ 81.374341] systemd[1]: share1.service: Sent signal SIGINT to main process 1055 (nix) on client request.5078client # [ 81.381755] nix[1055]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5079client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.05 seconds)5080client: waiting for success: ! systemctl is-active share15081client # [ 81.389309] nix[1055]: error: interrupted by the user5082client # [ 81.428907] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5083client # [ 81.430553] systemd[1]: share1.service: Failed with result 'exit-code'.5084client # [ 81.432485] systemd[1]: share1.service: Consumed 112ms CPU time over 3.276s wall clock time, 41.5M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.5085client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5086client: waiting for success: ! systemctl is-active share25087worker2 # [ 82.799710] nix-daemon[861]: reaped child process 1139, status = succeeded5088worker2 # [ 82.878769] nix-daemon[861]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5089worker2 # [ 82.890863] nix-daemon[861]: reaped child process 1172, status = succeeded5090lb # [ 82.915270] niks3-server[915]: 2026/09/25 16:32:46 INFO Received push request method=POST path=/api/pushes5091worker2 # [ 82.906998] nix-grpc-daemon[873]: time=2026-09-25T16:32:46.080Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5092worker2 # [ 82.909052] nix-grpc-daemon[873]: time=2026-09-25T16:32:46.082Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5093lb # [ 82.966341] niks3-server[915]: 2026/09/25 16:32:46 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5094lb # [ 82.972390] niks3-server[915]: 2026/09/25 16:32:46 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5095lb # [ 82.979830] niks3-server[915]: 2026/09/25 16:32:46 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign5096lb # [ 82.984261] niks3-server[915]: 2026/09/25 16:32:46 INFO Signed narinfos id=21 count=15097worker2 # [ 82.970755] nix-grpc-daemon[873]: time=2026-09-25T16:32:46.144Z level=INFO msg="Uploading 1 narinfos"5098lb # [ 82.996806] niks3-server[915]: 2026/09/25 16:32:46 INFO Received complete push request method=POST path=/api/pushes/21/complete5099lb # [ 82.999215] niks3-server[915]: 2026/09/25 16:32:46 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5100lb # [ 83.004930] niks3-server[915]: 2026/09/25 16:32:46 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5101worker2 # [ 82.987271] nix-grpc-daemon[873]: time=2026-09-25T16:32:46.161Z level=INFO msg="Upload complete. (205ms)"5102worker2 # [ 82.993982] nix-grpc-daemon[803]: ts=2026-09-25T16:32:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05103worker2 # [ 83.002268] nix-grpc-daemon[803]: ts=2026-09-25T16:32:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=3 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05104worker2 # [ 83.015430] nix-daemon[861]: reaped child process 1137, status = succeeded5105client # [ 83.022761] systemd[1]: share2.service: Deactivated successfully.5106client # [ 83.026844] systemd[1]: share2.service: Consumed 117ms CPU time over 3.802s wall clock time, 35.4M memory peak, 6.5K incoming IP traffic, 6K outgoing IP traffic.5107client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5108client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5109client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5110(finished: subtest: the first client leaving does not fail the second, in 5.64 seconds)5111subtest: build hook: nix-daemon with builders = grpc://lb5112client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5113client # [ 83.833374] systemd[1]: Started Nix Daemon.5114client # [ 83.913850] nix-daemon[1123]: accepted connection from pid 1120, user root (trusted)5115worker1 # [ 84.023829] nix-grpc-daemon[511]: ts=2026-09-25T16:32:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:364765116worker2 # [ 84.024967] nix-grpc-daemon[803]: ts=2026-09-25T16:32:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:552205117worker1 # [ 84.038692] nix-grpc-daemon[511]: ts=2026-09-25T16:32:46Z level=debug event=want client=12 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05118worker1 # [ 84.044142] nix-grpc-daemon[511]: ts=2026-09-25T16:32:46Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15119worker1 # [ 84.050354] nix-grpc-daemon[511]: ts=2026-09-25T16:32:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364765120worker1 # [ 84.058103] nix-grpc-daemon[511]: ts=2026-09-25T16:32:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364765121worker1 # [ 84.068878] nix-grpc-daemon[511]: ts=2026-09-25T16:32:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364765122worker1 # [ 84.074987] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5123worker1 # [ 84.173511] nix-daemon[870]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5124worker1 # [ 84.187295] nix-daemon[870]: reaped child process 1162, status = succeeded5125lb # [ 84.208435] niks3-server[915]: 2026/09/25 16:32:47 INFO Received push request method=POST path=/api/pushes5126worker1 # [ 84.203493] nix-grpc-daemon[929]: time=2026-09-25T16:32:46.501Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5127worker1 # [ 84.205498] nix-grpc-daemon[929]: time=2026-09-25T16:32:46.503Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5128lb # [ 84.239971] niks3-server[915]: 2026/09/25 16:32:47 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5129lb # [ 84.247220] niks3-server[915]: 2026/09/25 16:32:47 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign5130lb # [ 84.251226] niks3-server[915]: 2026/09/25 16:32:47 INFO Signed narinfos id=22 count=15131worker1 # [ 84.241712] nix-grpc-daemon[929]: time=2026-09-25T16:32:46.540Z level=INFO msg="Uploading 1 narinfos"5132lb # [ 84.260495] niks3-server[915]: 2026/09/25 16:32:47 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5133lb # [ 84.270366] niks3-server[915]: 2026/09/25 16:32:47 INFO Received complete push request method=POST path=/api/pushes/22/complete5134lb # [ 84.280749] niks3-server[915]: 2026/09/25 16:32:47 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5135worker1 # [ 84.266961] nix-grpc-daemon[929]: time=2026-09-25T16:32:46.565Z level=INFO msg="Upload complete. (187ms)"5136worker1 # [ 84.270825] nix-grpc-daemon[511]: ts=2026-09-25T16:32:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=5125137worker1 # [ 84.275603] nix-grpc-daemon[511]: ts=2026-09-25T16:32:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364765138worker1 # [ 84.284871] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5139worker1 # [ 84.290695] nix-daemon[870]: reaped child process 1157, status = succeeded5140worker1 # [ 84.294796] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5141worker1 # [ 85.530458] nix-daemon[870]: reaped child process 1166, status = succeeded5142worker1 # [ 85.610653] nix-daemon[870]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5143worker1 # [ 85.622808] nix-daemon[870]: reaped child process 1185, status = succeeded5144lb # [ 85.646787] niks3-server[915]: 2026/09/25 16:32:48 INFO Received push request method=POST path=/api/pushes5145worker1 # [ 85.642068] nix-grpc-daemon[929]: time=2026-09-25T16:32:47.940Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5146worker1 # [ 85.643890] nix-grpc-daemon[929]: time=2026-09-25T16:32:47.942Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5147lb # [ 85.688818] niks3-server[915]: 2026/09/25 16:32:48 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign5148lb # [ 85.692881] niks3-server[915]: 2026/09/25 16:32:48 INFO Signed narinfos id=23 count=15149worker1 # [ 85.682873] nix-grpc-daemon[929]: time=2026-09-25T16:32:47.981Z level=INFO msg="Uploading 1 narinfos"5150lb # [ 85.703113] niks3-server[915]: 2026/09/25 16:32:48 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5151lb # [ 85.707604] niks3-server[915]: 2026/09/25 16:32:48 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5152lb # [ 85.716571] niks3-server[915]: 2026/09/25 16:32:48 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5153lb # [ 85.723173] niks3-server[915]: 2026/09/25 16:32:48 INFO Received complete push request method=POST path=/api/pushes/23/complete5154worker1 # [ 85.719375] nix-grpc-daemon[929]: time=2026-09-25T16:32:48.017Z level=INFO msg="Upload complete. (216ms)"5155lb # [ 85.736474] niks3-server[915]: 2026/09/25 16:32:48 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5156worker1 # [ 85.726924] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=13 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05157worker2 # [ 85.734729] nix-grpc-daemon[803]: ts=2026-09-25T16:32:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:552205158worker1 # [ 85.742622] nix-daemon[870]: reaped child process 1164, status = succeeded5159worker2 # [ 85.745150] nix-grpc-daemon[803]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5160worker1 # [ 85.774151] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364765161worker1 # [ 85.779615] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_out=1285162worker2 # [ 85.944728] nix-grpc-daemon[803]: ts=2026-09-25T16:32:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:552205163worker2 # [ 85.951682] nix-grpc-daemon[803]: ts=2026-09-25T16:32:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:552205164worker2 # [ 85.961776] nix-grpc-daemon[873]: time=2026-09-25T16:32:49.135Z level=INFO msg="All 1 paths already cached"5165worker1 # [ 85.979175] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=debug event=want client=13 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05166worker1 # [ 85.984769] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15167worker1 # [ 85.990315] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364765168worker1 # [ 86.001657] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364765169worker1 # [ 86.014555] nix-grpc-daemon[929]: time=2026-09-25T16:32:48.313Z level=INFO msg="All 1 paths already cached"5170worker1 # [ 86.021759] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364765171worker1 # [ 86.031435] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5172worker1 # [ 86.135982] nix-daemon[870]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)5173worker1 # [ 86.149965] nix-daemon[870]: reaped child process 1194, status = succeeded5174lb # [ 86.170134] niks3-server[915]: 2026/09/25 16:32:49 INFO Received push request method=POST path=/api/pushes5175worker1 # [ 86.164663] nix-grpc-daemon[929]: time=2026-09-25T16:32:48.463Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5176worker1 # [ 86.166665] nix-grpc-daemon[929]: time=2026-09-25T16:32:48.464Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5177lb # [ 86.196187] niks3-server[915]: 2026/09/25 16:32:49 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5178lb # [ 86.207111] niks3-server[915]: 2026/09/25 16:32:49 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign5179lb # [ 86.212514] niks3-server[915]: 2026/09/25 16:32:49 INFO Signed narinfos id=24 count=15180worker1 # [ 86.203106] nix-grpc-daemon[929]: time=2026-09-25T16:32:48.501Z level=INFO msg="Uploading 1 narinfos"5181lb # [ 86.227773] niks3-server[915]: 2026/09/25 16:32:49 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5182lb # [ 86.233243] niks3-server[915]: 2026/09/25 16:32:49 INFO Received complete push request method=POST path=/api/pushes/24/complete5183worker1 # [ 86.232228] nix-grpc-daemon[929]: time=2026-09-25T16:32:48.530Z level=INFO msg="Upload complete. (188ms)"5184lb # [ 86.245801] niks3-server[915]: 2026/09/25 16:32:49 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5185worker1 # [ 86.237656] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=6245186worker1 # [ 86.243952] nix-grpc-daemon[511]: ts=2026-09-25T16:32:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364765187worker1 # [ 86.256259] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5188worker1 # [ 86.265683] nix-daemon[870]: reaped child process 1189, status = succeeded5189worker1 # [ 86.270313] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5190lb # [ 87.071523] envoy[922]: conn peer=[2001:db8:1::1]:42596 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2879 tx=1269 ms=19165191lb # [ 87.075413] envoy[922]: conn peer=[2001:db8:1::1]:42600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=772 ms=31095192lb # [ 87.078944] envoy[922]: conn peer=[2001:db8:1::1]:42602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1470 tx=1400 ms=36405193lb # [ 87.082618] envoy[922]: conn peer=[2001:db8:1::1]:38660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=265194lb # [ 87.086523] envoy[922]: conn peer=[2001:db8:1::1]:38650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3392 tx=1946 ms=17785195worker1 # [ 87.306730] nix-daemon[870]: reaped child process 1198, status = succeeded5196worker1 # [ 87.389511] nix-daemon[870]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)5197worker1 # [ 87.400410] nix-daemon[870]: reaped child process 1217, status = succeeded5198lb # [ 87.423411] niks3-server[915]: 2026/09/25 16:32:50 INFO Received push request method=POST path=/api/pushes5199worker1 # [ 87.418168] nix-grpc-daemon[929]: time=2026-09-25T16:32:49.716Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5200worker1 # [ 87.419685] nix-grpc-daemon[929]: time=2026-09-25T16:32:49.718Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5201lb # [ 87.466346] niks3-server[915]: 2026/09/25 16:32:50 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign5202lb # [ 87.473862] niks3-server[915]: 2026/09/25 16:32:50 INFO Signed narinfos id=25 count=15203worker1 # [ 87.464505] nix-grpc-daemon[929]: time=2026-09-25T16:32:49.762Z level=INFO msg="Uploading 1 narinfos"5204lb # [ 87.482945] niks3-server[915]: 2026/09/25 16:32:50 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5205lb # [ 87.487610] niks3-server[915]: 2026/09/25 16:32:50 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5206lb # [ 87.497312] niks3-server[915]: 2026/09/25 16:32:50 INFO Received complete push request method=POST path=/api/pushes/25/complete5207lb # [ 87.504223] niks3-server[915]: 2026/09/25 16:32:50 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5208lb # [ 87.512858] niks3-server[915]: 2026/09/25 16:32:50 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5209worker1 # [ 87.499573] nix-grpc-daemon[929]: time=2026-09-25T16:32:49.798Z level=INFO msg="Upload complete. (221ms)"5210worker1 # [ 87.507601] nix-grpc-daemon[511]: ts=2026-09-25T16:32:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=14 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05211worker1 # [ 87.517142] nix-grpc-daemon[511]: ts=2026-09-25T16:32:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364765212worker1 # [ 87.527572] nix-grpc-daemon[511]: ts=2026-09-25T16:32:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364765213worker1 # [ 87.531553] nix-grpc-daemon[511]: ts=2026-09-25T16:32:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_out=1765214worker1 # [ 87.536563] nix-daemon[870]: reaped child process 1196, status = succeeded5215worker2 # [ 87.687488] nix-grpc-daemon[803]: ts=2026-09-25T16:32:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:552205216worker1 # [ 87.693521] nix-grpc-daemon[511]: ts=2026-09-25T16:32:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364765217worker1 # [ 87.703801] nix-grpc-daemon[929]: time=2026-09-25T16:32:50.002Z level=INFO msg="All 2 paths already cached"5218worker1 # [ 87.718697] nix-grpc-daemon[511]: ts=2026-09-25T16:32:50Z level=debug event=want client=14 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05219worker1 # [ 87.724730] nix-grpc-daemon[511]: ts=2026-09-25T16:32:50Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15220worker1 # [ 87.729517] nix-grpc-daemon[511]: ts=2026-09-25T16:32:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364765221worker1 # [ 87.740349] nix-grpc-daemon[511]: ts=2026-09-25T16:32:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364765222worker1 # [ 87.752677] nix-grpc-daemon[929]: time=2026-09-25T16:32:50.051Z level=INFO msg="All 2 paths already cached"5223worker1 # [ 87.760643] nix-grpc-daemon[511]: ts=2026-09-25T16:32:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364765224worker1 # [ 87.770155] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5225worker1 # [ 87.872792] nix-daemon[870]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5226worker1 # [ 87.887058] nix-daemon[870]: reaped child process 1226, status = succeeded5227lb # [ 87.907095] niks3-server[915]: 2026/09/25 16:32:51 INFO Received push request method=POST path=/api/pushes5228worker1 # [ 87.902335] nix-grpc-daemon[929]: time=2026-09-25T16:32:50.200Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5229worker1 # [ 87.904024] nix-grpc-daemon[929]: time=2026-09-25T16:32:50.202Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5230lb # [ 87.936813] niks3-server[915]: 2026/09/25 16:32:51 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign5231worker1 # [ 87.930816] nix-grpc-daemon[929]: time=2026-09-25T16:32:50.229Z level=INFO msg="Uploading 1 narinfos"5232lb # [ 87.945162] niks3-server[915]: 2026/09/25 16:32:51 INFO Signed narinfos id=26 count=15233lb # [ 87.950981] niks3-server[915]: 2026/09/25 16:32:51 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5234lb # [ 87.960636] niks3-server[915]: 2026/09/25 16:32:51 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5235lb # [ 87.971173] niks3-server[915]: 2026/09/25 16:32:51 INFO Received complete push request method=POST path=/api/pushes/26/complete5236lb # [ 87.979611] niks3-server[915]: 2026/09/25 16:32:51 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5237worker1 # [ 87.969814] nix-grpc-daemon[929]: time=2026-09-25T16:32:50.268Z level=INFO msg="Upload complete. (187ms)"5238worker1 # [ 87.972961] nix-grpc-daemon[511]: ts=2026-09-25T16:32:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_in=7525239worker1 # [ 87.977793] nix-grpc-daemon[511]: ts=2026-09-25T16:32:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364765240worker1 # [ 87.990110] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5241worker1 # [ 87.993905] nix-daemon[870]: reaped child process 1221, status = succeeded5242worker1 # [ 87.999212] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5243worker1 # [ 89.214156] nix-daemon[870]: reaped child process 1230, status = succeeded5244worker1 # [ 89.292916] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5245worker1 # [ 89.304196] nix-daemon[870]: reaped child process 1249, status = succeeded5246lb # [ 89.326514] niks3-server[915]: 2026/09/25 16:32:52 INFO Received push request method=POST path=/api/pushes5247worker1 # [ 89.321832] nix-grpc-daemon[929]: time=2026-09-25T16:32:51.620Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5248worker1 # [ 89.323325] nix-grpc-daemon[929]: time=2026-09-25T16:32:51.621Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5249lb # [ 89.370873] niks3-server[915]: 2026/09/25 16:32:52 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5250lb # [ 89.378917] niks3-server[915]: 2026/09/25 16:32:52 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign5251worker1 # [ 89.371695] nix-grpc-daemon[929]: time=2026-09-25T16:32:51.670Z level=INFO msg="Uploading 1 narinfos"5252lb # [ 89.384439] niks3-server[915]: 2026/09/25 16:32:52 INFO Signed narinfos id=27 count=15253lb # [ 89.392134] niks3-server[915]: 2026/09/25 16:32:52 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5254lb # [ 89.396475] niks3-server[915]: 2026/09/25 16:32:52 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5255lb # [ 89.404185] niks3-server[915]: 2026/09/25 16:32:52 INFO Received complete push request method=POST path=/api/pushes/27/complete5256lb # [ 89.411775] niks3-server[915]: 2026/09/25 16:32:52 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5257worker1 # [ 89.398445] nix-grpc-daemon[929]: time=2026-09-25T16:32:51.696Z level=INFO msg="Upload complete. (192ms)"5258worker1 # [ 89.406542] nix-grpc-daemon[511]: ts=2026-09-25T16:32:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=15 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05259worker2 # [ 89.407549] nix-grpc-daemon[803]: ts=2026-09-25T16:32:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:552205260worker1 # [ 89.421771] nix-daemon[870]: reaped child process 1228, status = succeeded5261worker2 # [ 89.419825] nix-grpc-daemon[803]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5262worker2 # [ 89.429863] nix-grpc-daemon[803]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5263worker2 # [ 89.451420] nix-grpc-daemon[803]: ts=2026-09-25T16:32:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:552205264worker2 # [ 89.456432] nix-grpc-daemon[803]: ts=2026-09-25T16:32:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=0 paths=1 nar_bytes_out=2325265client # [ 89.533711] nix-daemon[1123]: reaped child process 1125, status = succeeded5266client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.79 seconds)5267client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5268client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5269client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5270client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5271client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5272client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5273(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.84 seconds)5274subtest: an input only one worker has still reaches the builder5275client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5276client # this derivation will be built:5277client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5278client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5279client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.22 seconds)5280client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5281client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5282worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5283worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5284worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5285worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5286client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25287client # [ 90.156699] nix-daemon[1123]: accepted connection from pid 1205, user root (trusted)5288client # this derivation will be built:5289client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5290worker1 # [ 90.273635] nix-grpc-daemon[511]: ts=2026-09-25T16:32:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:364765291client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5292worker1 # [ 90.279684] nix-grpc-daemon[511]: ts=2026-09-25T16:32:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364765293worker1 # [ 90.384703] nix-daemon[870]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5294worker1 # [ 90.396524] nix-daemon[870]: reaped child process 1259, status = succeeded5295lb # [ 90.417172] niks3-server[915]: 2026/09/25 16:32:53 INFO Received push request method=POST path=/api/pushes5296worker1 # [ 90.412266] nix-grpc-daemon[929]: time=2026-09-25T16:32:52.710Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5297worker1 # [ 90.414196] nix-grpc-daemon[929]: time=2026-09-25T16:32:52.712Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5298lb # [ 90.449352] niks3-server[915]: 2026/09/25 16:32:53 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign5299lb # [ 90.452718] niks3-server[915]: 2026/09/25 16:32:53 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5300worker1 # [ 90.449028] nix-grpc-daemon[929]: time=2026-09-25T16:32:52.747Z level=INFO msg="Uploading 1 narinfos"5301lb # [ 90.463513] niks3-server[915]: 2026/09/25 16:32:53 INFO Signed narinfos id=28 count=15302lb # [ 90.467983] niks3-server[915]: 2026/09/25 16:32:53 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5303lb # [ 90.480657] niks3-server[915]: 2026/09/25 16:32:53 INFO Received complete push request method=POST path=/api/pushes/28/complete5304worker1 # [ 90.475738] nix-grpc-daemon[929]: time=2026-09-25T16:32:52.774Z level=INFO msg="Upload complete. (197ms)"5305lb # [ 90.491314] niks3-server[915]: 2026/09/25 16:32:53 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5306worker1 # [ 90.495550] nix-grpc-daemon[511]: ts=2026-09-25T16:32:52Z level=debug event=want client=15 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05307worker2 # [ 90.495263] nix-grpc-daemon[803]: ts=2026-09-25T16:32:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552205308worker1 # [ 90.499995] nix-grpc-daemon[511]: ts=2026-09-25T16:32:52Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15309worker2 # [ 90.505332] nix-grpc-daemon[803]: ts=2026-09-25T16:32:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:552205310client # copying 2 paths to 'lb:50051'...5311client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5312worker2 # [ 90.519704] nix-grpc-daemon[803]: ts=2026-09-25T16:32:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:552205313client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5314worker2 # [ 90.531644] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)5315worker2 # [ 90.641333] nix-daemon[861]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5316worker2 # [ 90.654754] nix-daemon[861]: reaped child process 1185, status = succeeded5317lb # [ 90.677709] niks3-server[915]: 2026/09/25 16:32:53 INFO Received push request method=POST path=/api/pushes5318worker2 # [ 90.670610] nix-grpc-daemon[873]: time=2026-09-25T16:32:53.844Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5319worker2 # [ 90.673169] nix-grpc-daemon[873]: time=2026-09-25T16:32:53.846Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5320worker2 # [ 90.675234] nix-grpc-daemon[873]: time=2026-09-25T16:32:53.846Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5321lb # [ 90.719514] niks3-server[915]: 2026/09/25 16:32:53 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5322lb # [ 90.726424] niks3-server[915]: 2026/09/25 16:32:53 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5323lb # [ 90.738760] niks3-server[915]: 2026/09/25 16:32:53 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5324lb # [ 90.744884] niks3-server[915]: 2026/09/25 16:32:53 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign5325lb # [ 90.748574] niks3-server[915]: 2026/09/25 16:32:53 INFO Signed narinfos id=29 count=25326worker2 # [ 90.735400] nix-grpc-daemon[873]: time=2026-09-25T16:32:53.908Z level=INFO msg="Uploading 2 narinfos"5327lb # [ 90.756256] niks3-server[915]: 2026/09/25 16:32:53 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5328lb # [ 90.762976] niks3-server[915]: 2026/09/25 16:32:53 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5329lb # [ 90.765728] niks3-server[915]: 2026/09/25 16:32:53 INFO Received complete push request method=POST path=/api/pushes/29/complete5330worker2 # [ 90.755885] nix-grpc-daemon[873]: time=2026-09-25T16:32:53.929Z level=INFO msg="Upload complete. (208ms)"5331lb # [ 90.774308] niks3-server[915]: 2026/09/25 16:32:53 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5332worker2 # [ 90.761569] nix-grpc-daemon[803]: ts=2026-09-25T16:32:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=0 paths=2 nar_bytes_in=9765333worker2 # [ 90.767100] nix-grpc-daemon[803]: ts=2026-09-25T16:32:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552205334worker2 # [ 90.779114] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)5335worker2 # [ 90.787179] nix-daemon[861]: reaped child process 1180, status = succeeded5336worker2 # [ 90.796528] nix-grpc-daemon[803]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5337worker2 # [ 90.803514] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)5338client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5339client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5340client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5341worker2 # [ 92.218889] nix-daemon[861]: reaped child process 1190, status = succeeded5342worker2 # [ 92.293616] nix-daemon[861]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5343worker2 # [ 92.304988] nix-daemon[861]: reaped child process 1209, status = succeeded5344lb # [ 92.328761] niks3-server[915]: 2026/09/25 16:32:55 INFO Received push request method=POST path=/api/pushes5345worker2 # [ 92.321647] nix-grpc-daemon[873]: time=2026-09-25T16:32:55.495Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5346lb # [ 92.361941] niks3-server[915]: 2026/09/25 16:32:55 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign5347worker2 # [ 92.354610] nix-grpc-daemon[873]: time=2026-09-25T16:32:55.528Z level=INFO msg="Uploading 1 narinfos"5348lb # [ 92.370709] niks3-server[915]: 2026/09/25 16:32:55 INFO Signed narinfos id=30 count=15349lb # [ 92.378450] niks3-server[915]: 2026/09/25 16:32:55 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5350lb # [ 92.384709] niks3-server[915]: 2026/09/25 16:32:55 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5351lb # [ 92.393703] niks3-server[915]: 2026/09/25 16:32:55 INFO Received complete push request method=POST path=/api/pushes/30/complete5352worker2 # [ 92.384527] nix-grpc-daemon[873]: time=2026-09-25T16:32:55.558Z level=INFO msg="Upload complete. (173ms)"5353lb # [ 92.406159] niks3-server[915]: 2026/09/25 16:32:55 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5354worker2 # [ 92.389820] nix-grpc-daemon[803]: ts=2026-09-25T16:32:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=16 outputs=1 inputs=1 inputs_fetched=1 input_bytes=120 input_s=05355client # copying 1 paths...5356worker2 # [ 92.396938] nix-grpc-daemon[803]: ts=2026-09-25T16:32:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:552205357client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5358worker2 # [ 92.406859] nix-daemon[861]: reaped child process 1187, status = succeeded5359worker1 # [ 92.421710] nix-grpc-daemon[511]: ts=2026-09-25T16:32:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364765360worker1 # [ 92.432707] nix-grpc-daemon[511]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5361worker1 # [ 92.445347] nix-grpc-daemon[511]: ts=2026-09-25T16:32:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_out=1205362client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.46 seconds)5363client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25364client # [ 92.517748] nix-daemon[1123]: reaped child process 1208, status = succeeded5365client # [ 92.600919] nix-daemon[1123]: accepted connection from pid 1227, user root (trusted)5366client # this derivation will be built:5367client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5368worker2 # [ 92.709769] nix-grpc-daemon[803]: ts=2026-09-25T16:32:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:552205369client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5370worker1 # [ 92.717310] nix-grpc-daemon[511]: ts=2026-09-25T16:32:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364765371worker1 # [ 92.727189] nix-grpc-daemon[929]: time=2026-09-25T16:32:55.025Z level=INFO msg="All 1 paths already cached"5372worker1 # [ 92.741200] nix-grpc-daemon[511]: ts=2026-09-25T16:32:55Z level=debug event=want client=16 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05373worker1 # [ 92.746789] nix-grpc-daemon[511]: ts=2026-09-25T16:32:55Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15374worker1 # [ 92.752880] nix-grpc-daemon[511]: ts=2026-09-25T16:32:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364765375worker1 # [ 92.763730] nix-grpc-daemon[511]: ts=2026-09-25T16:32:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364765376client # copying 1 paths to 'lb:50051'...5377client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5378worker1 # [ 92.780205] nix-grpc-daemon[511]: ts=2026-09-25T16:32:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364765379worker1 # [ 92.791177] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5380worker1 # [ 92.806169] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5)5381worker1 # [ 92.891701] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 2/5)5382worker1 # [ 92.944692] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 301 ms (attempt 3/5)5383worker1 # [ 93.247893] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 586 ms (attempt 4/5)5384worker1 # [ 93.836902] nix-grpc-daemon[511]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5385worker1 # [ 93.841275] nix-grpc-daemon[511]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5386worker1 # [ 93.846983] nix-grpc-daemon[511]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5387worker1 # [ 93.965447] nix-daemon[870]: accepted connection from pid 1273, user nix-grpc-daemon (trusted)5388worker1 # [ 93.982410] nix-daemon[870]: reaped child process 1276, status = succeeded5389lb # [ 93.999913] niks3-server[915]: 2026/09/25 16:32:57 INFO Received push request method=POST path=/api/pushes5390worker1 # [ 93.995418] nix-grpc-daemon[929]: time=2026-09-25T16:32:56.293Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5391worker1 # [ 93.996959] nix-grpc-daemon[929]: time=2026-09-25T16:32:56.295Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5392lb # [ 94.031779] niks3-server[915]: 2026/09/25 16:32:57 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign5393worker1 # [ 94.026741] nix-grpc-daemon[929]: time=2026-09-25T16:32:56.325Z level=INFO msg="Uploading 1 narinfos"5394lb # [ 94.039245] niks3-server[915]: 2026/09/25 16:32:57 INFO Signed narinfos id=31 count=15395lb # [ 94.043835] niks3-server[915]: 2026/09/25 16:32:57 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5396lb # [ 94.057889] niks3-server[915]: 2026/09/25 16:32:57 INFO Received complete push request method=POST path=/api/pushes/31/complete5397lb # [ 94.065426] niks3-server[915]: 2026/09/25 16:32:57 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5398worker1 # [ 94.059561] nix-grpc-daemon[929]: time=2026-09-25T16:32:56.358Z level=INFO msg="Upload complete. (193ms)"5399lb # [ 94.074153] niks3-server[915]: 2026/09/25 16:32:57 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5400worker1 # [ 94.064422] nix-grpc-daemon[511]: ts=2026-09-25T16:32:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=1 paths=1 nar_bytes_in=5605401worker1 # [ 94.071490] nix-grpc-daemon[511]: ts=2026-09-25T16:32:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364765402worker1 # [ 94.082479] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5403worker1 # [ 94.092632] nix-daemon[870]: reaped child process 1266, status = succeeded5404worker1 # [ 94.097694] nix-daemon[870]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5405client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5406client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5407client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5408worker1 # [ 94.733906] nix-daemon[870]: reaped child process 1280, status = succeeded5409worker1 # [ 94.815454] nix-daemon[870]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5410worker1 # [ 94.828310] nix-daemon[870]: reaped child process 1299, status = succeeded5411lb # [ 94.847304] niks3-server[915]: 2026/09/25 16:32:58 INFO Received push request method=POST path=/api/pushes5412worker1 # [ 94.842587] nix-grpc-daemon[929]: time=2026-09-25T16:32:57.141Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5413lb # [ 94.877114] niks3-server[915]: 2026/09/25 16:32:58 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign5414lb # [ 94.883645] niks3-server[915]: 2026/09/25 16:32:58 INFO Signed narinfos id=32 count=15415worker1 # [ 94.875629] nix-grpc-daemon[929]: time=2026-09-25T16:32:57.174Z level=INFO msg="Uploading 1 narinfos"5416lb # [ 94.892870] niks3-server[915]: 2026/09/25 16:32:58 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5417lb # [ 94.896603] niks3-server[915]: 2026/09/25 16:32:58 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5418lb # [ 94.905820] niks3-server[915]: 2026/09/25 16:32:58 INFO Received complete push request method=POST path=/api/pushes/32/complete5419lb # [ 94.914717] niks3-server[915]: 2026/09/25 16:32:58 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5420worker1 # [ 94.901171] nix-grpc-daemon[929]: time=2026-09-25T16:32:57.199Z level=INFO msg="Upload complete. (197ms)"5421worker1 # [ 94.906870] nix-grpc-daemon[511]: ts=2026-09-25T16:32:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=17 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05422client # copying 1 paths...5423worker1 # [ 94.920477] nix-daemon[870]: reaped child process 1278, status = succeeded5424worker2 # [ 94.914448] nix-grpc-daemon[803]: ts=2026-09-25T16:32:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:552205425worker2 # [ 94.922703] nix-grpc-daemon[803]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5426client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5427worker1 # [ 94.952602] nix-grpc-daemon[511]: ts=2026-09-25T16:32:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364765428worker1 # [ 94.957939] nix-grpc-daemon[511]: ts=2026-09-25T16:32:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36476 duration_s=0 paths=1 nar_bytes_out=1205429client # [ 95.021494] nix-daemon[1123]: reaped child process 1230, status = succeeded5430client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.52 seconds)5431client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25432client # [ 95.121097] nix-daemon[1123]: accepted connection from pid 1249, user root (trusted)5433client # this derivation will be built:5434client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5435worker2 # [ 95.239159] nix-grpc-daemon[803]: ts=2026-09-25T16:32:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:552205436client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5437worker1 # [ 95.250733] nix-grpc-daemon[511]: ts=2026-09-25T16:32:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364765438worker1 # [ 95.261298] nix-grpc-daemon[929]: time=2026-09-25T16:32:57.559Z level=INFO msg="All 1 paths already cached"5439worker1 # [ 95.275769] nix-grpc-daemon[511]: ts=2026-09-25T16:32:57Z level=debug event=want client=17 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05440worker2 # [ 95.275098] nix-grpc-daemon[803]: ts=2026-09-25T16:32:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552205441worker1 # [ 95.281375] nix-grpc-daemon[511]: ts=2026-09-25T16:32:57Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15442worker2 # [ 95.287951] nix-grpc-daemon[803]: ts=2026-09-25T16:32:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:552205443worker2 # [ 95.301310] nix-grpc-daemon[873]: time=2026-09-25T16:32:58.474Z level=INFO msg="All 1 paths already cached"5444client # copying 1 paths to 'lb:50051'...5445client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5446worker2 # [ 95.308706] nix-grpc-daemon[803]: ts=2026-09-25T16:32:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:552205447worker2 # [ 95.319571] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)5448worker2 # [ 95.416599] nix-daemon[861]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)5449worker2 # [ 95.429186] nix-daemon[861]: reaped child process 1217, status = succeeded5450lb # [ 95.454118] niks3-server[915]: 2026/09/25 16:32:58 INFO Received push request method=POST path=/api/pushes5451worker2 # [ 95.445068] nix-grpc-daemon[873]: time=2026-09-25T16:32:58.618Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5452worker2 # [ 95.446414] nix-grpc-daemon[873]: time=2026-09-25T16:32:58.618Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5453lb # [ 95.480682] niks3-server[915]: 2026/09/25 16:32:58 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign5454lb # [ 95.482961] niks3-server[915]: 2026/09/25 16:32:58 INFO Signed narinfos id=33 count=15455worker2 # [ 95.466696] nix-grpc-daemon[873]: time=2026-09-25T16:32:58.640Z level=INFO msg="Uploading 1 narinfos"5456lb # [ 95.485520] niks3-server[915]: 2026/09/25 16:32:58 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5457lb # [ 95.491387] niks3-server[915]: 2026/09/25 16:32:58 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5458lb # [ 95.493218] niks3-server[915]: 2026/09/25 16:32:58 INFO Received complete push request method=POST path=/api/pushes/33/complete5459worker2 # [ 95.483733] nix-grpc-daemon[873]: time=2026-09-25T16:32:58.657Z level=INFO msg="Upload complete. (152ms)"5460worker2 # [ 95.485823] nix-grpc-daemon[803]: ts=2026-09-25T16:32:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=0 paths=1 nar_bytes_in=5605461lb # [ 95.503349] niks3-server[915]: 2026/09/25 16:32:58 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5462worker2 # [ 95.489486] nix-grpc-daemon[803]: ts=2026-09-25T16:32:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:552205463worker2 # [ 95.495694] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)5464worker2 # [ 95.497984] nix-daemon[861]: reaped child process 1212, status = succeeded5465worker2 # [ 95.503178] nix-daemon[861]: accepted connection from pid 803, user nix-grpc-daemon (trusted)5466client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5467client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5468client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5469worker2 # [ 96.917427] nix-daemon[861]: reaped child process 1221, status = succeeded5470worker2 # [ 96.993643] nix-daemon[861]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)5471worker2 # [ 97.006292] nix-daemon[861]: reaped child process 1240, status = succeeded5472lb # [ 97.029827] niks3-server[915]: 2026/09/25 16:33:00 INFO Received push request method=POST path=/api/pushes5473worker2 # [ 97.022435] nix-grpc-daemon[873]: time=2026-09-25T16:33:00.196Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5474lb # [ 97.064157] niks3-server[915]: 2026/09/25 16:33:00 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5475lb # [ 97.069088] niks3-server[915]: 2026/09/25 16:33:00 INFO Signed narinfos id=34 count=15476worker2 # [ 97.056180] nix-grpc-daemon[873]: time=2026-09-25T16:33:00.230Z level=INFO msg="Uploading 1 narinfos"5477lb # [ 97.074627] envoy[922]: conn peer=[2001:db8:1::1]:38668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2160 ms=15945478lb # [ 97.079163] envoy[922]: conn peer=[2001:db8:1::1]:38674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=175479lb # [ 97.083983] envoy[922]: conn peer=[2001:db8:1::1]:38690 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=215480lb # [ 97.086846] envoy[922]: conn peer=[2001:db8:1::1]:38682 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4285 tx=2368 ms=17865481lb # [ 97.089559] envoy[922]: conn peer=[2001:db8:1::1]:38692 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3833 tx=2024 ms=21835482lb # [ 97.092305] envoy[922]: conn peer=[2001:db8:1::1]:38700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=460 tx=308 ms=375483lb # [ 97.096193] envoy[922]: conn peer=[2001:db8:1::1]:38706 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3723 tx=2108 ms=22555484lb # [ 97.099169] envoy[922]: conn peer=[2001:db8:1::1]:40670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=175485lb # [ 97.102344] niks3-server[915]: 2026/09/25 16:33:00 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5486lb # [ 97.109237] niks3-server[915]: 2026/09/25 16:33:00 INFO Received complete push request method=POST path=/api/pushes/34/complete5487lb # [ 97.111699] niks3-server[915]: 2026/09/25 16:33:00 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5488lb # [ 97.115566] niks3-server[915]: 2026/09/25 16:33:00 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5489worker2 # [ 97.098544] nix-grpc-daemon[873]: time=2026-09-25T16:33:00.272Z level=INFO msg="Upload complete. (188ms)"5490client # copying 1 paths...5491worker2 # [ 97.104716] nix-grpc-daemon[803]: ts=2026-09-25T16:33:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=18 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05492worker2 # [ 97.113968] nix-grpc-daemon[803]: ts=2026-09-25T16:33:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:552205493client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5494worker2 # [ 97.125118] nix-daemon[861]: reaped child process 1219, status = succeeded5495worker2 # [ 97.131901] nix-grpc-daemon[803]: ts=2026-09-25T16:33:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:552205496worker2 # [ 97.135914] nix-grpc-daemon[803]: ts=2026-09-25T16:33:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55220 duration_s=0 paths=1 nar_bytes_out=1205497client # [ 97.203775] nix-daemon[1123]: reaped child process 1252, status = succeeded5498client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.18 seconds)5499(finished: subtest: an input only one worker has still reaches the builder, in 7.61 seconds)5500subtest: upload whose reference the worker lacks is completed from the cache5501client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5502client # this derivation will be built:5503client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5504client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5505client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5506client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5507client # this derivation will be built:5508client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5509client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5510client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.23 seconds)5511client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5512client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5513worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5514worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5515worker2 # [ 97.947316] systemd[1]: Stopping Nix worker-protocol over gRPC...5516worker2 # [ 97.954109] nix-grpc-daemon[803]: ts=2026-09-25T16:33:01Z level=info event=signal_exit5517worker1 # [ 99.956597] nix-grpc-daemon[511]: ts=2026-09-25T16:33:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:364525518worker1 # [ 99.960262] nix-grpc-daemon[511]: ts=2026-09-25T16:33:02Z level=info event=worker_gone addr=192.168.1.4:500515519worker2 # [ 99.948621] nix-grpc-daemon[803]: ts=2026-09-25T16:33:03Z level=info event=scheduler_restarting5520worker2 # [ 99.967824] nix-grpc-daemon[803]: ts=2026-09-25T16:33:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5521worker2 # [ 99.987736] nix-daemon[861]: reaped child process 863, status = succeeded5522worker2 # [ 100.162450] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5523worker2 # [ 100.166299] systemd[1]: Stopped Nix worker-protocol over gRPC.5524worker2 # [ 100.170738] systemd[1]: nix-grpc-daemon.service: Consumed 2.012s CPU time over 1min 7.372s wall clock time, 65.8M memory peak, 174K incoming IP traffic, 342.4K outgoing IP traffic.5525lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux5526lb: must succeed: curl -sf localhost:9901/clusters5527lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5528lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds)5529client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25530worker1 # [ 100.383982] nix-grpc-daemon[511]: ts=2026-09-25T16:33:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364765531worker1 # [ 100.390697] nix-grpc-daemon[511]: ts=2026-09-25T16:33:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364765532client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5533client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5534worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5535worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5536worker2 # [ 100.467515] systemd[1]: Starting scheduler-token.service...5537worker2 # [ 100.502305] systemd[1]: scheduler-token.service: Deactivated successfully.5538worker2 # [ 100.503972] systemd[1]: Finished scheduler-token.service.5539worker2 # [ 100.505364] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 36ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5540worker2 # [ 100.509509] systemd[1]: Starting Nix worker-protocol over gRPC...5541worker2 # [ 100.584291] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:03Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5542worker2 # [ 100.588712] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5543worker2 # [ 100.591639] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15544worker2 # [ 100.595410] nix-grpc-daemon[1264]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5545worker2 # [ 100.596744] nix-grpc-daemon[1264]: E0000 00:00:1790353983.765665 1264 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5546worker2 # [ 100.598960] nix-grpc-daemon[1264]: E0000 00:00:1790353983.769293 1264 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5547worker2 # [ 100.601681] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:03Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5548worker2 # [ 100.605121] systemd[1]: Started Nix worker-protocol over gRPC.5549worker2 # [ 100.613081] nix-grpc-daemon[1264]: E0000 00:00:1790353983.784763 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5550worker2 # [ 100.624088] nix-grpc-daemon[1264]: E0000 00:00:1790353983.798167 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5551worker2 # [ 100.627129] nix-grpc-daemon[1264]: E0000 00:00:1790353983.800470 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5552worker2 # [ 100.638343] nix-grpc-daemon[1264]: E0000 00:00:1790353983.812381 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5553worker2 # [ 100.647760] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:03Z level=info event=scheduler_connected addr=lb:500515554worker1 # [ 100.655060] nix-grpc-daemon[511]: ts=2026-09-25T16:33:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:364525555worker1 # [ 100.662275] nix-grpc-daemon[511]: ts=2026-09-25T16:33:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05556worker1 # [ 100.676910] systemd[1]: Stopping Nix worker-protocol over gRPC...5557worker1 # [ 100.806635] nix-grpc-daemon[511]: ts=2026-09-25T16:33:03Z level=info event=signal_exit5558lb # [ 101.598868] niks3-server[915]: 2026/09/25 16:33:04 INFO lead: released remote=192.168.1.3:439505559worker2 # [ 102.585319] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:05Z level=info event=scheduler_restarting addr=lb:500515560worker1 # [ 102.589688] nix-grpc-daemon[511]: ts=2026-09-25T16:33:04Z level=info event=scheduler_restarting5561lb # [ 102.600441] envoy[922]: [2026-09-25 16:33:05.764][922][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5562worker1 # [ 102.592638] nix-grpc-daemon[511]: ts=2026-09-25T16:33:04Z level=info event=scheduler_restarting addr=lb:500515563worker1 # [ 102.597957] nix-grpc-daemon[511]: ts=2026-09-25T16:33:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:364525564worker1 # [ 102.603506] nix-grpc-daemon[511]: ts=2026-09-25T16:33:04Z level=info event=worker_gone addr=192.168.1.4:500515565worker1 # [ 102.606924] nix-grpc-daemon[511]: ts=2026-09-25T16:33:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:364525566worker1 # [ 102.610700] nix-grpc-daemon[511]: ts=2026-09-25T16:33:04Z level=info event=worker_gone addr=192.168.1.3:500515567worker1 # [ 102.634248] nix-daemon[870]: reaped child process 873, status = succeeded5568worker2 # [ 102.785761] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:05Z level=info event=scheduler_connected addr=lb:500515569worker2 # [ 102.789833] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5570worker1 # [ 102.805391] nix-grpc-daemon[511]: ts=2026-09-25T16:33:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5571worker2 # [ 102.990650] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_connected addr=lb:500515572worker2 # [ 102.994599] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5573worker2 # [ 103.195615] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_connected addr=lb:500515574worker2 # [ 103.199556] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5575worker2 # [ 103.400749] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_connected addr=lb:500515576worker2 # [ 103.404618] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5577worker2 # [ 103.606180] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_connected addr=lb:500515578worker2 # [ 103.609924] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5579worker2 # [ 103.811342] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_connected addr=lb:500515580worker2 # [ 103.815250] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5581worker2 # [ 104.016291] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:07Z level=info event=scheduler_connected addr=lb:500515582worker2 # [ 104.020225] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5583worker1 # [ 104.129856] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5584worker1 # [ 104.133579] systemd[1]: Stopped Nix worker-protocol over gRPC.5585worker1 # [ 104.136889] systemd[1]: nix-grpc-daemon.service: Consumed 2.420s CPU time over 1min 35.625s wall clock time, 66.6M memory peak, 443.8K incoming IP traffic, 435.7K outgoing IP traffic.5586lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux5587lb: must succeed: curl -sf localhost:9901/clusters5588lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5589lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.05 seconds)5590client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25591worker2 # [ 104.221332] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:07Z level=info event=scheduler_connected addr=lb:500515592worker2 # [ 104.226278] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5593worker2 # [ 104.343155] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346245594worker2 # [ 104.354954] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5595worker2 # [ 104.371247] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 1/5)5596worker2 # [ 104.388452] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 123 ms (attempt 2/5)5597worker2 # [ 104.427511] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:07Z level=info event=scheduler_connected addr=lb:500515598worker2 # [ 104.431325] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5599worker2 # [ 104.513553] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 3/5)5600worker2 # [ 104.582215] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 358 ms (attempt 4/5)5601worker2 # [ 104.633087] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:07Z level=info event=scheduler_connected addr=lb:500515602worker2 # [ 104.636817] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5603worker2 # [ 104.838178] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_connected addr=lb:500515604worker2 # [ 104.842348] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5605worker2 # [ 104.942786] nix-grpc-daemon[1264]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5606worker2 # [ 104.946317] nix-grpc-daemon[1264]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5607client # copying 2 paths...5608client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5609worker2 # [ 104.968131] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346245610client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5611worker2 # [ 104.980365] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5612worker2 # [ 105.004437] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=niks3_push_started pid=13245613worker2 # [ 105.044802] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_connected addr=lb:500515614worker2 # [ 105.046385] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5615worker2 # [ 105.110214] nix-grpc-daemon[1324]: time=2026-09-25T16:33:08.282Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5616worker2 # [ 105.195097] nix-daemon[861]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5617worker2 # [ 105.206803] nix-daemon[861]: reaped child process 1340, status = succeeded5618lb # [ 105.231184] niks3-server[915]: 2026/09/25 16:33:08 INFO Received push request method=POST path=/api/pushes5619worker2 # [ 105.223492] nix-grpc-daemon[1324]: time=2026-09-25T16:33:08.397Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5620worker2 # [ 105.225080] nix-grpc-daemon[1324]: time=2026-09-25T16:33:08.399Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5621worker2 # [ 105.227724] nix-grpc-daemon[1324]: time=2026-09-25T16:33:08.401Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5622worker2 # [ 105.246922] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_connected addr=lb:500515623worker2 # [ 105.248539] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5624lb # [ 105.286100] niks3-server[915]: 2026/09/25 16:33:08 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5625worker2 # [ 105.279067] nix-grpc-daemon[1324]: time=2026-09-25T16:33:08.452Z level=INFO msg="Uploading 2 narinfos"5626lb # [ 105.296142] niks3-server[915]: 2026/09/25 16:33:08 INFO Signed narinfos id=35 count=25627lb # [ 105.301831] niks3-server[915]: 2026/09/25 16:33:08 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5628lb # [ 105.307730] niks3-server[915]: 2026/09/25 16:33:08 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5629lb # [ 105.323363] niks3-server[915]: 2026/09/25 16:33:08 INFO Received complete push request method=POST path=/api/pushes/35/complete5630lb # [ 105.326976] niks3-server[915]: 2026/09/25 16:33:08 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5631lb # [ 105.329035] niks3-server[915]: 2026/09/25 16:33:08 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5632lb # [ 105.333613] niks3-server[915]: 2026/09/25 16:33:08 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5633worker2 # [ 105.316227] nix-grpc-daemon[1324]: time=2026-09-25T16:33:08.490Z level=INFO msg="Upload complete. (206ms)"5634lb # [ 105.336620] niks3-server[915]: 2026/09/25 16:33:08 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5635worker2 # [ 105.321625] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=0 paths=2 nar_bytes_in=3125636worker2 # [ 105.334612] nix-daemon[861]: reaped child process 1322, status = succeeded5637client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.15 seconds)5638worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5639worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5640worker1 # [ 105.418866] systemd[1]: Starting Nix worker-protocol over gRPC...5641worker2 # [ 105.449729] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_connected addr=lb:500515642worker2 # [ 105.453734] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5643worker1 # [ 105.508338] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5644worker1 # [ 105.510492] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5645worker1 # [ 105.513773] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15646worker1 # [ 105.518057] nix-grpc-daemon[1324]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5647worker1 # [ 105.519388] nix-grpc-daemon[1324]: E0000 00:00:1790353987.816452 1324 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5648worker1 # [ 105.523152] nix-grpc-daemon[1324]: E0000 00:00:1790353987.821623 1324 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5649worker1 # [ 105.526733] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:07Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5650worker1 # [ 105.529910] systemd[1]: Started Nix worker-protocol over gRPC.5651worker1 # [ 105.534798] nix-grpc-daemon[1324]: E0000 00:00:1790353987.833402 1366 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5652lb # [ 105.548854] niks3-server[915]: 2026/09/25 16:33:08 INFO lead: acquired remote=192.168.1.3:369145653worker1 # [ 105.540252] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:07Z level=info event=scheduler_take_over5654lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux5655lb: must succeed: curl -sf localhost:9901/clusters5656worker1 # [ 105.549356] nix-grpc-daemon[1324]: E0000 00:00:1790353987.847907 1366 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5657worker1 # [ 105.553129] nix-grpc-daemon[1324]: E0000 00:00:1790353987.850073 1366 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5658worker1 # [ 105.563672] nix-grpc-daemon[1324]: E0000 00:00:1790353987.861733 1366 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5659worker1 # [ 105.577649] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:07Z level=info event=scheduler_connected addr=lb:500515660worker1 # [ 105.579540] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5661lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5662worker2 # [ 105.655335] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_connected addr=lb:500515663worker2 # [ 105.658934] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5664worker1 # [ 105.780591] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=scheduler_connected addr=lb:500515665worker1 # [ 105.784576] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5666worker2 # [ 105.857445] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:09Z level=info event=scheduler_connected addr=lb:500515667worker2 # [ 105.861329] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5668worker1 # [ 105.985470] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=scheduler_connected addr=lb:500515669worker1 # [ 105.989493] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5670worker2 # [ 106.062337] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:09Z level=info event=scheduler_connected addr=lb:500515671worker2 # [ 106.066277] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5672worker1 # [ 106.190657] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=scheduler_connected addr=lb:500515673worker1 # [ 106.194595] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5674worker2 # [ 106.267210] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:09Z level=info event=scheduler_connected addr=lb:500515675worker1 # [ 106.282174] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=oidc_jwks provider=mock keys=15676worker1 # [ 106.286224] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:522705677worker1 # [ 106.290124] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05678worker1 # [ 106.395523] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=scheduler_connected addr=lb:500515679worker1 # [ 106.400476] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:522705680worker1 # [ 106.404076] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05681lb: must succeed: curl -sf localhost:9901/clusters5682lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5683lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 1.11 seconds)5684(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.44 seconds)5685subtest: a niks3 restart does not cost the leader its role5686lb: must succeed: curl -sf localhost:9901/clusters5687lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5688lb: must succeed: curl -sf localhost:9901/clusters5689lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5690lb # [ 106.741800] systemd[1]: Stopping niks3 server...5691lb # [ 106.742850] niks3-server[915]: 2026/09/25 16:33:09 INFO Shutdown signal received, draining in-flight requests timeout=10s5692lb # [ 106.747101] niks3-server[915]: 2026/09/25 16:33:09 INFO lead: released remote=192.168.1.3:369145693worker2 # [ 106.731597] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:09Z level=info event=scheduler_lock_lost status=2005694worker1 # [ 106.739503] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:09Z level=info event=scheduler_lock_lost status=2005695lb # [ 106.764754] systemd[1]: niks3.service: Deactivated successfully.5696lb # [ 106.766331] systemd[1]: Stopped niks3 server.5697lb # [ 106.767150] systemd[1]: niks3.service: Consumed 761ms CPU time over 1min 30.618s wall clock time, 29.2M memory peak, 99.1K incoming IP traffic, 142.6K outgoing IP traffic.5698lb # [ 106.774468] systemd[1]: Starting niks3 server...5699lb # [ 106.823363] niks3-server[1130]: 2026/09/25 16:33:09 goose: no migrations to run. current version: 202609231200005700lb # [ 106.829173] niks3-server[1130]: 2026/09/25 16:33:09 OK 1_commit_pending_closure.sql (4.28ms)5701lb # [ 106.832628] niks3-server[1130]: 2026/09/25 16:33:09 OK 2_object_stats_trigger.sql (2.08ms)5702lb # [ 106.836021] niks3-server[1130]: 2026/09/25 16:33:10 OK 3_commit_push.sql (3.03ms)5703lb # [ 106.837061] niks3-server[1130]: 2026/09/25 16:33:10 goose: up to current file version: 35704lb # [ 106.838369] niks3-server[1130]: 2026/09/25 16:33:10 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5705lb # [ 106.849227] niks3-server[1130]: 2026/09/25 16:33:10 INFO Read proxy enabled — serving cache objects from S35706lb # [ 106.850463] niks3-server[1130]: 2026/09/25 16:33:10 INFO Using socket-activated listener address=0.0.0.0:57515707lb # [ 106.852846] niks3-server[1130]: 2026/09/25 16:33:10 INFO systemd watchdog enabled interval=15s5708lb # [ 106.854174] systemd[1]: Started niks3 server.5709lb # [ 106.855683] niks3-server[1130]: 2026/09/25 16:33:10 INFO Starting HTTP server address=0.0.0.0:57515710lb: waiting for unit niks3.service5711lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5712lb: must succeed: curl -sf localhost:9901/clusters5713lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5714worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5715worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5716lb: must succeed: curl -sf localhost:9901/clusters5717lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5718lb: must succeed: curl -sf localhost:9901/clusters5719lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5720worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5721worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5722client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25723lb # [ 107.073462] envoy[922]: conn peer=[2001:db8:1::1]:40680 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3725 tx=2086 ms=19075724lb # [ 107.075080] envoy[922]: conn peer=[2001:db8:1::1]:40684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=135725lb # [ 107.076540] envoy[922]: conn peer=[2001:db8:1::4]:55260 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2974 tx=1021 ms=669755726lb # [ 107.078369] envoy[922]: conn peer=[2001:db8:1::1]:40688 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=205727lb # [ 107.079801] envoy[922]: conn peer=192.168.1.3:48438 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2370 tx=1122 ms=694175728lb # [ 107.081459] envoy[922]: conn peer=[2001:db8:1::1]:47442 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=305 ms=9925729worker2 # [ 107.170692] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:346245730client # don't know how to build these paths:5731client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5732worker2 # [ 107.179444] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346245733worker1 # [ 107.210169] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:09Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05734worker1 # [ 107.214110] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:09Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15735worker2 # [ 107.217981] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346245736worker2 # [ 107.229180] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346245737client # copying 1 paths to 'lb:50051'...5738client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5739worker2 # [ 107.243290] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346245740worker2 # [ 107.254115] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5741worker2 # [ 107.355569] nix-daemon[861]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5742worker2 # [ 107.368551] nix-daemon[861]: reaped child process 1350, status = succeeded5743lb # [ 107.391059] niks3-server[1130]: 2026/09/25 16:33:10 INFO Received push request method=POST path=/api/pushes5744worker2 # [ 107.389633] nix-grpc-daemon[1324]: time=2026-09-25T16:33:10.563Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5745worker2 # [ 107.391864] nix-grpc-daemon[1324]: time=2026-09-25T16:33:10.565Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5746lb # [ 107.429124] niks3-server[1130]: 2026/09/25 16:33:10 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5747lb # [ 107.447524] niks3-server[1130]: 2026/09/25 16:33:10 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5748lb # [ 107.455413] niks3-server[1130]: 2026/09/25 16:33:10 INFO Signed narinfos id=36 count=15749worker2 # [ 107.441801] nix-grpc-daemon[1324]: time=2026-09-25T16:33:10.614Z level=INFO msg="Uploading 1 narinfos"5750lb # [ 107.464053] niks3-server[1130]: 2026/09/25 16:33:10 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5751lb # [ 107.466802] niks3-server[1130]: 2026/09/25 16:33:10 INFO Received complete push request method=POST path=/api/pushes/36/complete5752lb # [ 107.471410] niks3-server[1130]: 2026/09/25 16:33:10 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5753worker2 # [ 107.461367] nix-grpc-daemon[1324]: time=2026-09-25T16:33:10.635Z level=INFO msg="Upload complete. (195ms)"5754worker2 # [ 107.465823] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=0 paths=1 nar_bytes_in=5365755worker2 # [ 107.471825] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346245756worker2 # [ 107.485099] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5757worker2 # [ 107.494251] nix-daemon[861]: reaped child process 1345, status = succeeded5758client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5759worker2 # [ 107.500511] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5760client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5761lb # [ 107.756555] niks3-server[1130]: 2026/09/25 16:33:10 INFO lead: acquired remote=192.168.1.3:369245762client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5763client # farm-a> LOG-farm-a-after-niks3-restart5764worker2 # [ 108.526899] nix-daemon[861]: reaped child process 1354, status = succeeded5765worker2 # [ 108.604868] nix-daemon[861]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5766worker2 # [ 108.617797] nix-daemon[861]: reaped child process 1375, status = succeeded5767lb # [ 108.640701] niks3-server[1130]: 2026/09/25 16:33:11 INFO Received push request method=POST path=/api/pushes5768worker2 # [ 108.635364] nix-grpc-daemon[1324]: time=2026-09-25T16:33:11.809Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5769worker2 # [ 108.636992] nix-grpc-daemon[1324]: time=2026-09-25T16:33:11.810Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5770lb # [ 108.687920] niks3-server[1130]: 2026/09/25 16:33:11 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5771lb # [ 108.699756] niks3-server[1130]: 2026/09/25 16:33:11 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5772lb # [ 108.705237] niks3-server[1130]: 2026/09/25 16:33:11 INFO Signed narinfos id=37 count=15773worker2 # [ 108.688953] nix-grpc-daemon[1324]: time=2026-09-25T16:33:11.862Z level=INFO msg="Uploading 1 narinfos"5774lb # [ 108.708985] niks3-server[1130]: 2026/09/25 16:33:11 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5775lb # [ 108.723436] niks3-server[1130]: 2026/09/25 16:33:11 INFO Received complete push request method=POST path=/api/pushes/37/complete5776lb # [ 108.726269] niks3-server[1130]: 2026/09/25 16:33:11 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5777worker2 # [ 108.716717] nix-grpc-daemon[1324]: time=2026-09-25T16:33:11.890Z level=INFO msg="Upload complete. (208ms)"5778lb # [ 108.734923] niks3-server[1130]: 2026/09/25 16:33:11 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5779worker2 # [ 108.722776] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05780worker1 # [ 108.734370] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:11Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05781worker2 # [ 108.733611] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346245782worker1 # [ 108.738969] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:11Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15783worker2 # [ 108.745508] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346245784worker2 # [ 108.752974] nix-daemon[861]: reaped child process 1352, status = succeeded5785worker2 # [ 108.755591] nix-grpc-daemon[1324]: time=2026-09-25T16:33:11.929Z level=INFO msg="All 1 paths already cached"5786client # copying 1 paths to 'lb:50051'...5787client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5788worker2 # [ 108.761961] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346245789worker2 # [ 108.769293] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5790worker2 # [ 108.850376] nix-daemon[861]: accepted connection from pid 1379, user nix-grpc-daemon (trusted)5791worker2 # [ 108.863958] nix-daemon[861]: reaped child process 1382, status = succeeded5792lb # [ 108.885055] niks3-server[1130]: 2026/09/25 16:33:12 INFO Received push request method=POST path=/api/pushes5793worker2 # [ 108.876740] nix-grpc-daemon[1324]: time=2026-09-25T16:33:12.050Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5794worker2 # [ 108.878670] nix-grpc-daemon[1324]: time=2026-09-25T16:33:12.052Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5795lb # [ 108.916610] niks3-server[1130]: 2026/09/25 16:33:12 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5796lb # [ 108.921813] niks3-server[1130]: 2026/09/25 16:33:12 INFO Signed narinfos id=38 count=15797worker2 # [ 108.908879] nix-grpc-daemon[1324]: time=2026-09-25T16:33:12.082Z level=INFO msg="Uploading 1 narinfos"5798lb # [ 108.930789] niks3-server[1130]: 2026/09/25 16:33:12 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5799lb # [ 108.939869] niks3-server[1130]: 2026/09/25 16:33:12 INFO Received complete push request method=POST path=/api/pushes/38/complete5800lb # [ 108.951247] niks3-server[1130]: 2026/09/25 16:33:12 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5801lb # [ 108.955684] niks3-server[1130]: 2026/09/25 16:33:12 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5802worker2 # [ 108.942685] nix-grpc-daemon[1324]: time=2026-09-25T16:33:12.116Z level=INFO msg="Upload complete. (168ms)"5803worker2 # [ 108.947418] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=0 paths=1 nar_bytes_in=6565804worker2 # [ 108.952873] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346245805worker2 # [ 108.962724] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5806worker2 # [ 108.970740] nix-daemon[861]: reaped child process 1377, status = succeeded5807worker2 # [ 108.975300] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5808client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5809client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5810client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5811client # farm-b> LOG-farm-b-after-niks3-restart5812worker2 # [ 110.011533] nix-daemon[861]: reaped child process 1386, status = succeeded5813worker2 # [ 110.093645] nix-daemon[861]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)5814worker2 # [ 110.105057] nix-daemon[861]: reaped child process 1405, status = succeeded5815lb # [ 110.128938] niks3-server[1130]: 2026/09/25 16:33:13 INFO Received push request method=POST path=/api/pushes5816worker2 # [ 110.120690] nix-grpc-daemon[1324]: time=2026-09-25T16:33:13.294Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5817worker2 # [ 110.122246] nix-grpc-daemon[1324]: time=2026-09-25T16:33:13.296Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5818lb # [ 110.172471] niks3-server[1130]: 2026/09/25 16:33:13 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5819lb # [ 110.183910] niks3-server[1130]: 2026/09/25 16:33:13 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5820lb # [ 110.191567] niks3-server[1130]: 2026/09/25 16:33:13 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5821lb # [ 110.196446] niks3-server[1130]: 2026/09/25 16:33:13 INFO Signed narinfos id=39 count=15822worker2 # [ 110.180310] nix-grpc-daemon[1324]: time=2026-09-25T16:33:13.354Z level=INFO msg="Uploading 1 narinfos"5823lb # [ 110.199180] niks3-server[1130]: 2026/09/25 16:33:13 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5824lb # [ 110.210274] niks3-server[1130]: 2026/09/25 16:33:13 INFO Received complete push request method=POST path=/api/pushes/39/complete5825lb # [ 110.215226] niks3-server[1130]: 2026/09/25 16:33:13 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5826worker2 # [ 110.201902] nix-grpc-daemon[1324]: time=2026-09-25T16:33:13.375Z level=INFO msg="Upload complete. (219ms)"5827worker2 # [ 110.207885] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05828worker1 # [ 110.218170] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:12Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05829worker2 # [ 110.218257] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346245830worker1 # [ 110.222526] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:12Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15831worker2 # [ 110.229510] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346245832worker2 # [ 110.237174] nix-daemon[861]: reaped child process 1384, status = succeeded5833worker2 # [ 110.241352] nix-grpc-daemon[1324]: time=2026-09-25T16:33:13.413Z level=INFO msg="All 2 paths already cached"5834client # copying 1 paths to 'lb:50051'...5835client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5836worker2 # [ 110.247876] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346245837worker2 # [ 110.254391] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5838worker2 # [ 110.336834] nix-daemon[861]: accepted connection from pid 1409, user nix-grpc-daemon (trusted)5839worker2 # [ 110.349960] nix-daemon[861]: reaped child process 1412, status = succeeded5840lb # [ 110.372220] niks3-server[1130]: 2026/09/25 16:33:13 INFO Received push request method=POST path=/api/pushes5841worker2 # [ 110.363825] nix-grpc-daemon[1324]: time=2026-09-25T16:33:13.537Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5842worker2 # [ 110.365738] nix-grpc-daemon[1324]: time=2026-09-25T16:33:13.539Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5843lb # [ 110.401234] niks3-server[1130]: 2026/09/25 16:33:13 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5844lb # [ 110.409069] niks3-server[1130]: 2026/09/25 16:33:13 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5845lb # [ 110.416645] niks3-server[1130]: 2026/09/25 16:33:13 INFO Signed narinfos id=40 count=15846worker2 # [ 110.403947] nix-grpc-daemon[1324]: time=2026-09-25T16:33:13.577Z level=INFO msg="Uploading 1 narinfos"5847lb # [ 110.425190] niks3-server[1130]: 2026/09/25 16:33:13 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5848lb # [ 110.435472] niks3-server[1130]: 2026/09/25 16:33:13 INFO Received complete push request method=POST path=/api/pushes/40/complete5849lb # [ 110.444966] niks3-server[1130]: 2026/09/25 16:33:13 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5850worker2 # [ 110.432816] nix-grpc-daemon[1324]: time=2026-09-25T16:33:13.606Z level=INFO msg="Upload complete. (172ms)"5851worker2 # [ 110.436994] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=0 paths=1 nar_bytes_in=7845852worker2 # [ 110.443205] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346245853worker2 # [ 110.453461] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5854worker2 # [ 110.462377] nix-daemon[861]: reaped child process 1407, status = succeeded5855worker2 # [ 110.467556] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5856client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5857client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5858client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5859client # farm-top> LOG-farm-top-after-niks3-restart5860worker2 # [ 111.505317] nix-daemon[861]: reaped child process 1416, status = succeeded5861worker2 # [ 111.580777] nix-daemon[861]: accepted connection from pid 1432, user nix-grpc-daemon (trusted)5862worker2 # [ 111.594986] nix-daemon[861]: reaped child process 1435, status = succeeded5863lb # [ 111.616693] niks3-server[1130]: 2026/09/25 16:33:14 INFO Received push request method=POST path=/api/pushes5864worker2 # [ 111.610477] nix-grpc-daemon[1324]: time=2026-09-25T16:33:14.784Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5865worker2 # [ 111.612343] nix-grpc-daemon[1324]: time=2026-09-25T16:33:14.786Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5866lb # [ 111.657842] niks3-server[1130]: 2026/09/25 16:33:14 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5867lb # [ 111.667740] niks3-server[1130]: 2026/09/25 16:33:14 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5868lb # [ 111.674573] niks3-server[1130]: 2026/09/25 16:33:14 INFO Signed narinfos id=41 count=15869worker2 # [ 111.661882] nix-grpc-daemon[1324]: time=2026-09-25T16:33:14.835Z level=INFO msg="Uploading 1 narinfos"5870lb # [ 111.679104] niks3-server[1130]: 2026/09/25 16:33:14 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5871lb # [ 111.683141] niks3-server[1130]: 2026/09/25 16:33:14 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5872lb # [ 111.692466] niks3-server[1130]: 2026/09/25 16:33:14 INFO Received complete push request method=POST path=/api/pushes/41/complete5873lb # [ 111.696936] niks3-server[1130]: 2026/09/25 16:33:14 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5874worker2 # [ 111.683720] nix-grpc-daemon[1324]: time=2026-09-25T16:33:14.857Z level=INFO msg="Upload complete. (208ms)"5875worker2 # [ 111.689451] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05876worker2 # [ 111.704666] nix-daemon[861]: reaped child process 1414, status = succeeded5877client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 4.70 seconds)5878client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5879worker1 # [ 111.736766] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:14Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:522905880client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.11 seconds)5881(finished: subtest: a niks3 restart does not cost the leader its role, in 5.18 seconds)5882subtest: scheduler down: the other node takes the lock and keeps it5883lb: must succeed: curl -sf localhost:9901/clusters5884lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5885lb: must succeed: curl -sf localhost:9901/clusters5886lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5887lb: must succeed: curl -sf localhost:9901/clusters5888lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5889lb: must succeed: curl -sf localhost:9901/clusters5890lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5891worker1 # [ 112.000941] systemd[1]: Stopping Nix worker-protocol over gRPC...5892worker1 # [ 112.532217] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:14Z level=info event=signal_exit5893lb # [ 112.759512] niks3-server[1130]: 2026/09/25 16:33:15 INFO lead: released remote=192.168.1.3:369245894worker1 # [ 113.748706] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:16Z level=info event=scheduler_restarting5895lb # [ 113.760220] envoy[922]: [2026-09-25 16:33:16.925][922][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5896worker2 # [ 113.751992] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:16Z level=info event=scheduler_restarting addr=lb:500515897worker1 # [ 113.756162] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:16Z level=info event=scheduler_restarting addr=lb:500515898worker1 # [ 113.761864] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:522705899worker1 # [ 113.765823] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:16Z level=info event=worker_gone addr=192.168.1.4:500515900worker1 # [ 113.768921] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:522705901worker1 # [ 113.772525] nix-grpc-daemon[1324]: ts=2026-09-25T16:33:16Z level=info event=worker_gone addr=192.168.1.3:500515902worker2 # [ 113.952584] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_connected addr=lb:500515903worker2 # [ 113.956579] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5904worker2 # [ 114.157637] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_connected addr=lb:500515905worker2 # [ 114.161685] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5906worker2 # [ 114.362647] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_connected addr=lb:500515907worker2 # [ 114.366593] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5908worker2 # [ 114.567862] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_connected addr=lb:500515909worker2 # [ 114.571795] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5910worker2 # [ 114.773610] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_connected addr=lb:500515911worker2 # [ 114.777458] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5912worker1 # [ 114.971683] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5913worker1 # [ 114.975653] systemd[1]: Stopped Nix worker-protocol over gRPC.5914worker1 # [ 114.980170] systemd[1]: nix-grpc-daemon.service: Consumed 220ms CPU time over 9.555s wall clock time, 13.6M memory peak, 146.7K incoming IP traffic, 34K outgoing IP traffic.5915worker2 # [ 114.978583] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_connected addr=lb:500515916worker2 # [ 114.982787] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5917worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5918worker2 # [ 115.183838] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_connected addr=lb:500515919worker2 # [ 115.187736] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5920worker2 # [ 115.389227] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_connected addr=lb:500515921worker2 # [ 115.393271] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5922worker2 # [ 115.594340] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_connected addr=lb:500515923worker2 # [ 115.598294] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5924worker2 # [ 115.799929] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_connected addr=lb:500515925worker2 # [ 115.803724] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5926worker2 # [ 116.004978] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:19Z level=info event=scheduler_connected addr=lb:500515927worker2 # [ 116.009135] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5928worker2 # [ 116.210156] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:19Z level=info event=scheduler_connected addr=lb:500515929worker2 # [ 116.214333] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5930worker2 # [ 116.415438] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:19Z level=info event=scheduler_connected addr=lb:500515931worker2 # [ 116.419552] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5932worker2 # [ 116.620438] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:19Z level=info event=scheduler_connected addr=lb:500515933worker2 # [ 116.624335] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5934worker2 # [ 116.826299] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:19Z level=info event=scheduler_connected addr=lb:500515935worker2 # [ 116.829785] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5936worker2 # [ 117.030774] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_connected addr=lb:500515937worker2 # [ 117.034707] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5938lb # [ 117.074749] envoy[922]: conn peer=[2001:db8:1::1]:47452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9705 tx=4378 ms=45315939lb # [ 117.078511] envoy[922]: conn peer=[2001:db8:1::3]:53434 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1348 tx=372 ms=83955940worker2 # [ 117.235868] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_connected addr=lb:500515941worker2 # [ 117.239879] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5942worker2 # [ 117.440862] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_connected addr=lb:500515943worker2 # [ 117.444809] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5944worker2 # [ 117.645697] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_connected addr=lb:500515945worker2 # [ 117.649674] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5946worker2 # [ 117.738292] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:20Z level=info event=scheduler_take_over5947lb # [ 117.754904] niks3-server[1130]: 2026/09/25 16:33:20 INFO lead: acquired remote=192.168.1.4:382905948worker2 # [ 117.851640] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:21Z level=info event=scheduler_connected addr=lb:500515949worker2 # [ 117.855498] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5950worker2 # [ 118.056728] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:21Z level=info event=scheduler_connected addr=lb:500515951worker2 # [ 118.060657] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5952worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 3.20 seconds)5953worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5954worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5955worker2 # [ 118.261304] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:21Z level=info event=scheduler_connected addr=lb:500515956worker2 # [ 118.278141] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:21Z level=info event=oidc_jwks provider=mock keys=15957worker2 # [ 118.282247] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:482525958worker2 # [ 118.286208] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05959worker2 # [ 118.533174] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:21Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:482565960worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5961worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5962client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25963worker2 # [ 119.401649] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:346245964client # don't know how to build these paths:5965client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5966worker2 # [ 119.410190] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346245967worker2 # [ 119.433581] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:22Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05968worker2 # [ 119.439555] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:22Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15969worker2 # [ 119.444327] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346245970worker2 # [ 119.455061] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346245971client # copying 1 paths to 'lb:50051'...5972client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5973worker2 # [ 119.467333] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346245974worker2 # [ 119.475997] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5975worker2 # [ 119.569630] nix-daemon[861]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5976worker2 # [ 119.581453] nix-daemon[861]: reaped child process 1474, status = succeeded5977lb # [ 119.610285] niks3-server[1130]: 2026/09/25 16:33:22 INFO Received push request method=POST path=/api/pushes5978worker2 # [ 119.598938] nix-grpc-daemon[1324]: time=2026-09-25T16:33:22.772Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5979worker2 # [ 119.600838] nix-grpc-daemon[1324]: time=2026-09-25T16:33:22.774Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5980lb # [ 119.637668] niks3-server[1130]: 2026/09/25 16:33:22 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5981worker2 # [ 119.628214] nix-grpc-daemon[1324]: time=2026-09-25T16:33:22.802Z level=INFO msg="Uploading 1 narinfos"5982lb # [ 119.644296] niks3-server[1130]: 2026/09/25 16:33:22 INFO Signed narinfos id=42 count=15983lb # [ 119.658392] niks3-server[1130]: 2026/09/25 16:33:22 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5984lb # [ 119.666679] niks3-server[1130]: 2026/09/25 16:33:22 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5985lb # [ 119.675498] niks3-server[1130]: 2026/09/25 16:33:22 INFO Received complete push request method=POST path=/api/pushes/42/complete5986worker2 # [ 119.663930] nix-grpc-daemon[1324]: time=2026-09-25T16:33:22.837Z level=INFO msg="Upload complete. (176ms)"5987lb # [ 119.682290] niks3-server[1130]: 2026/09/25 16:33:22 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5988worker2 # [ 119.667933] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=0 paths=1 nar_bytes_in=5205989worker2 # [ 119.674299] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346245990worker2 # [ 119.685255] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5991worker2 # [ 119.694716] nix-daemon[861]: reaped child process 1469, status = succeeded5992worker2 # [ 119.697887] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5993client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5994client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5995client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5996client # farm-a> LOG-farm-a-on-standby5997worker2 # [ 120.712824] nix-daemon[861]: reaped child process 1478, status = succeeded5998worker2 # [ 120.784745] nix-daemon[861]: accepted connection from pid 1494, user nix-grpc-daemon (trusted)5999worker2 # [ 120.797748] nix-daemon[861]: reaped child process 1497, status = succeeded6000lb # [ 120.820358] niks3-server[1130]: 2026/09/25 16:33:23 INFO Received push request method=POST path=/api/pushes6001worker2 # [ 120.812588] nix-grpc-daemon[1324]: time=2026-09-25T16:33:23.986Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6002worker2 # [ 120.815063] nix-grpc-daemon[1324]: time=2026-09-25T16:33:23.988Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6003lb # [ 120.856566] niks3-server[1130]: 2026/09/25 16:33:24 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6004lb # [ 120.869529] niks3-server[1130]: 2026/09/25 16:33:24 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign6005lb # [ 120.875523] niks3-server[1130]: 2026/09/25 16:33:24 INFO Signed narinfos id=43 count=16006worker2 # [ 120.862149] nix-grpc-daemon[1324]: time=2026-09-25T16:33:24.036Z level=INFO msg="Uploading 1 narinfos"6007lb # [ 120.882860] niks3-server[1130]: 2026/09/25 16:33:24 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6008lb # [ 120.887143] niks3-server[1130]: 2026/09/25 16:33:24 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6009lb # [ 120.896832] niks3-server[1130]: 2026/09/25 16:33:24 INFO Received complete push request method=POST path=/api/pushes/43/complete6010lb # [ 120.903334] niks3-server[1130]: 2026/09/25 16:33:24 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6011worker2 # [ 120.885679] nix-grpc-daemon[1324]: time=2026-09-25T16:33:24.059Z level=INFO msg="Upload complete. (181ms)"6012worker2 # [ 120.892635] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06013worker2 # [ 120.907768] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:24Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06014worker2 # [ 120.911572] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:24Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16015worker2 # [ 120.916235] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346246016worker2 # [ 120.924811] nix-daemon[861]: reaped child process 1476, status = succeeded6017worker2 # [ 120.926253] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346246018worker2 # [ 120.933335] nix-grpc-daemon[1324]: time=2026-09-25T16:33:24.107Z level=INFO msg="All 1 paths already cached"6019client # copying 1 paths to 'lb:50051'...6020worker2 # [ 120.938801] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346246021client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6022worker2 # [ 120.944536] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)6023worker2 # [ 121.026246] nix-daemon[861]: accepted connection from pid 1503, user nix-grpc-daemon (trusted)6024worker2 # [ 121.039710] nix-daemon[861]: reaped child process 1506, status = succeeded6025lb # [ 121.061421] niks3-server[1130]: 2026/09/25 16:33:24 INFO Received push request method=POST path=/api/pushes6026worker2 # [ 121.053808] nix-grpc-daemon[1324]: time=2026-09-25T16:33:24.227Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6027worker2 # [ 121.055739] nix-grpc-daemon[1324]: time=2026-09-25T16:33:24.229Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6028lb # [ 121.091234] niks3-server[1130]: 2026/09/25 16:33:24 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign6029worker2 # [ 121.085304] nix-grpc-daemon[1324]: time=2026-09-25T16:33:24.259Z level=INFO msg="Uploading 1 narinfos"6030lb # [ 121.101603] niks3-server[1130]: 2026/09/25 16:33:24 INFO Signed narinfos id=44 count=16031lb # [ 121.112496] niks3-server[1130]: 2026/09/25 16:33:24 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6032lb # [ 121.126959] niks3-server[1130]: 2026/09/25 16:33:24 INFO Received complete push request method=POST path=/api/pushes/44/complete6033worker2 # [ 121.116571] nix-grpc-daemon[1324]: time=2026-09-25T16:33:24.290Z level=INFO msg="Upload complete. (167ms)"6034lb # [ 121.134903] niks3-server[1130]: 2026/09/25 16:33:24 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6035lb # [ 121.139129] niks3-server[1130]: 2026/09/25 16:33:24 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6036worker2 # [ 121.120862] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=0 paths=1 nar_bytes_in=6406037worker2 # [ 121.127309] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346246038worker2 # [ 121.135890] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)6039worker2 # [ 121.144727] nix-daemon[861]: reaped child process 1501, status = succeeded6040worker2 # [ 121.149409] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)6041client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6042client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6043client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6044client # farm-b> LOG-farm-b-on-standby6045worker2 # [ 122.163508] nix-daemon[861]: reaped child process 1510, status = succeeded6046worker2 # [ 122.237042] nix-daemon[861]: accepted connection from pid 1526, user nix-grpc-daemon (trusted)6047worker2 # [ 122.250040] nix-daemon[861]: reaped child process 1529, status = succeeded6048lb # [ 122.272704] niks3-server[1130]: 2026/09/25 16:33:25 INFO Received push request method=POST path=/api/pushes6049worker2 # [ 122.271280] nix-grpc-daemon[1324]: time=2026-09-25T16:33:25.444Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6050worker2 # [ 122.272650] nix-grpc-daemon[1324]: time=2026-09-25T16:33:25.444Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6051lb # [ 122.322715] niks3-server[1130]: 2026/09/25 16:33:25 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6052lb # [ 122.328327] niks3-server[1130]: 2026/09/25 16:33:25 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign6053lb # [ 122.333995] niks3-server[1130]: 2026/09/25 16:33:25 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6054lb # [ 122.338869] niks3-server[1130]: 2026/09/25 16:33:25 INFO Signed narinfos id=45 count=16055worker2 # [ 122.323072] nix-grpc-daemon[1324]: time=2026-09-25T16:33:25.496Z level=INFO msg="Uploading 1 narinfos"6056lb # [ 122.344694] niks3-server[1130]: 2026/09/25 16:33:25 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6057lb # [ 122.354413] niks3-server[1130]: 2026/09/25 16:33:25 INFO Received complete push request method=POST path=/api/pushes/45/complete6058lb # [ 122.360039] niks3-server[1130]: 2026/09/25 16:33:25 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6059worker2 # [ 122.342360] nix-grpc-daemon[1324]: time=2026-09-25T16:33:25.516Z level=INFO msg="Upload complete. (186ms)"6060worker2 # [ 122.349598] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06061worker2 # [ 122.361853] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:25Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06062worker2 # [ 122.366353] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:25Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16063worker2 # [ 122.371219] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346246064worker2 # [ 122.380273] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346246065worker2 # [ 122.382450] nix-daemon[861]: reaped child process 1508, status = succeeded6066worker2 # [ 122.387558] nix-grpc-daemon[1324]: time=2026-09-25T16:33:25.561Z level=INFO msg="All 2 paths already cached"6067client # copying 1 paths to 'lb:50051'...6068worker2 # [ 122.392977] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346246069client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6070worker2 # [ 122.397981] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)6071worker2 # [ 122.480919] nix-daemon[861]: accepted connection from pid 1535, user nix-grpc-daemon (trusted)6072worker2 # [ 122.493410] nix-daemon[861]: reaped child process 1538, status = succeeded6073lb # [ 122.516986] niks3-server[1130]: 2026/09/25 16:33:25 INFO Received push request method=POST path=/api/pushes6074worker2 # [ 122.508958] nix-grpc-daemon[1324]: time=2026-09-25T16:33:25.682Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6075worker2 # [ 122.510363] nix-grpc-daemon[1324]: time=2026-09-25T16:33:25.682Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6076lb # [ 122.542949] niks3-server[1130]: 2026/09/25 16:33:25 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign6077worker2 # [ 122.533369] nix-grpc-daemon[1324]: time=2026-09-25T16:33:25.707Z level=INFO msg="Uploading 1 narinfos"6078lb # [ 122.550278] niks3-server[1130]: 2026/09/25 16:33:25 INFO Signed narinfos id=46 count=16079lb # [ 122.556194] niks3-server[1130]: 2026/09/25 16:33:25 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6080lb # [ 122.565919] niks3-server[1130]: 2026/09/25 16:33:25 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6081lb # [ 122.573645] niks3-server[1130]: 2026/09/25 16:33:25 INFO Received complete push request method=POST path=/api/pushes/46/complete6082lb # [ 122.584706] niks3-server[1130]: 2026/09/25 16:33:25 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6083worker2 # [ 122.571387] nix-grpc-daemon[1324]: time=2026-09-25T16:33:25.745Z level=INFO msg="Upload complete. (167ms)"6084worker2 # [ 122.575491] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=0 paths=1 nar_bytes_in=7686085worker2 # [ 122.581160] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346246086worker2 # [ 122.589877] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)6087worker2 # [ 122.595909] nix-daemon[861]: reaped child process 1533, status = succeeded6088worker2 # [ 122.599940] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)6089client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6090client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6091client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6092client # farm-top> LOG-farm-top-on-standby6093worker2 # [ 123.414394] nix-daemon[861]: reaped child process 1542, status = succeeded6094worker2 # [ 123.488203] nix-daemon[861]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6095worker2 # [ 123.502103] nix-daemon[861]: reaped child process 1561, status = succeeded6096lb # [ 123.523968] niks3-server[1130]: 2026/09/25 16:33:26 INFO Received push request method=POST path=/api/pushes6097worker2 # [ 123.516404] nix-grpc-daemon[1324]: time=2026-09-25T16:33:26.690Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6098worker2 # [ 123.517891] nix-grpc-daemon[1324]: time=2026-09-25T16:33:26.691Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6099lb # [ 123.565690] niks3-server[1130]: 2026/09/25 16:33:26 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign6100worker2 # [ 123.553788] nix-grpc-daemon[1324]: time=2026-09-25T16:33:26.727Z level=INFO msg="Uploading 1 narinfos"6101lb # [ 123.571360] niks3-server[1130]: 2026/09/25 16:33:26 INFO Signed narinfos id=47 count=16102lb # [ 123.578953] niks3-server[1130]: 2026/09/25 16:33:26 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6103lb # [ 123.585867] niks3-server[1130]: 2026/09/25 16:33:26 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6104lb # [ 123.592278] niks3-server[1130]: 2026/09/25 16:33:26 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6105lb # [ 123.599935] niks3-server[1130]: 2026/09/25 16:33:26 INFO Received complete push request method=POST path=/api/pushes/47/complete6106worker2 # [ 123.589051] nix-grpc-daemon[1324]: time=2026-09-25T16:33:26.763Z level=INFO msg="Upload complete. (182ms)"6107lb # [ 123.607089] niks3-server[1130]: 2026/09/25 16:33:26 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6108worker2 # [ 123.598194] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06109worker2 # [ 123.612240] nix-daemon[861]: reaped child process 1540, status = succeeded6110client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 4.37 seconds)6111client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6112client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6113worker1 # [ 123.773984] systemd[1]: Starting Nix worker-protocol over gRPC...6114worker1 # [ 123.849346] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6115worker1 # [ 123.851626] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6116worker1 # [ 123.855494] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16117worker1 # [ 123.860060] nix-grpc-daemon[1407]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6118worker1 # [ 123.861395] nix-grpc-daemon[1407]: E0000 00:00:1790354006.157798 1407 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6119worker1 # [ 123.864460] nix-grpc-daemon[1407]: E0000 00:00:1790354006.162957 1407 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6120worker1 # [ 123.867057] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:26Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6121worker1 # [ 123.869801] systemd[1]: Started Nix worker-protocol over gRPC.6122worker1 # [ 123.877102] nix-grpc-daemon[1407]: E0000 00:00:1790354006.175482 1449 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6123lb: must succeed: curl -sf localhost:9901/clusters6124worker1 # [ 123.890066] nix-grpc-daemon[1407]: E0000 00:00:1790354006.188593 1449 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6125worker1 # [ 123.894124] nix-grpc-daemon[1407]: E0000 00:00:1790354006.190809 1449 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6126worker1 # [ 123.904248] nix-grpc-daemon[1407]: E0000 00:00:1790354006.202804 1449 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6127worker1 # [ 123.913662] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:26Z level=info event=scheduler_connected addr=lb:500516128worker2 # [ 123.911758] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:482526129worker2 # [ 123.918827] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06130lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6131worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6132worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6133lb: must succeed: curl -sf localhost:9901/clusters6134lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6135lb: must succeed: curl -sf localhost:9901/clusters6136lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6137worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6138worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6139(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.24 seconds)6140subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6141client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6142client # [ 124.121565] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6143client # Running as unit: rs.service; invocation ID: aa66dd83335d4e5faffdf851ade07a1e6144client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6145worker2 # [ 124.253627] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:346246146client # [ 124.259621] nix[1376]: don't know how to build these paths:6147client # [ 124.260567] nix[1376]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6148worker2 # [ 124.267362] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06149worker2 # [ 124.272955] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16150worker2 # [ 124.278787] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346246151worker2 # [ 124.289662] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:346246152client # [ 124.298032] nix[1376]: copying 1 paths to 'lb:50051'...6153client # [ 124.298954] nix[1376]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6154worker2 # [ 124.300278] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:346246155worker2 # [ 124.310053] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)6156worker2 # [ 124.397824] nix-daemon[861]: accepted connection from pid 1574, user nix-grpc-daemon (trusted)6157worker2 # [ 124.411161] nix-daemon[861]: reaped child process 1577, status = succeeded6158lb # [ 124.432622] niks3-server[1130]: 2026/09/25 16:33:27 INFO Received push request method=POST path=/api/pushes6159worker2 # [ 124.423952] nix-grpc-daemon[1324]: time=2026-09-25T16:33:27.598Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6160worker2 # [ 124.425420] nix-grpc-daemon[1324]: time=2026-09-25T16:33:27.599Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6161lb # [ 124.463530] niks3-server[1130]: 2026/09/25 16:33:27 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6162lb # [ 124.473474] niks3-server[1130]: 2026/09/25 16:33:27 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign6163worker2 # [ 124.461508] nix-grpc-daemon[1324]: time=2026-09-25T16:33:27.635Z level=INFO msg="Uploading 1 narinfos"6164lb # [ 124.478240] niks3-server[1130]: 2026/09/25 16:33:27 INFO Signed narinfos id=48 count=16165lb # [ 124.486368] niks3-server[1130]: 2026/09/25 16:33:27 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6166lb # [ 124.494838] niks3-server[1130]: 2026/09/25 16:33:27 INFO Received complete push request method=POST path=/api/pushes/48/complete6167lb # [ 124.504987] niks3-server[1130]: 2026/09/25 16:33:27 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6168worker2 # [ 124.487697] nix-grpc-daemon[1324]: time=2026-09-25T16:33:27.661Z level=INFO msg="Upload complete. (168ms)"6169worker2 # [ 124.492629] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=0 paths=1 nar_bytes_in=4166170worker2 # [ 124.499286] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:346246171worker2 # [ 124.508590] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)6172worker2 # [ 124.517328] nix-daemon[861]: reaped child process 1572, status = succeeded6173worker2 # [ 124.521912] nix-daemon[861]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)6174client # [ 124.520202] nix[1376]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6175lb: must succeed: curl -sf localhost:9901/clusters6176lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6177lb: must succeed: curl -sf localhost:9901/clusters6178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6179lb: must succeed: curl -sf localhost:9901/clusters6180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6181lb: must succeed: curl -sf localhost:9901/clusters6182lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6183worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6184worker1 # [ 126.785620] systemd-journald[283]: Received client request to rotate journal, rotating.6185worker1 # [ 126.804310] systemd-journald[283]: Vacuuming done, freed 0B of archived journals from /var/log/journal/ba9e9b0f224d4d3789babb65c081dfcf.6186worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6187worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6188worker2 # [ 126.704550] systemd[1]: Stopping Nix worker-protocol over gRPC...6189worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6190client: must succeed: sleep 5.06191lb # [ 127.076415] envoy[922]: conn peer=[2001:db8:1::1]:39990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9543 tx=4336 ms=42066192worker2 # [ 127.610468] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:30Z level=info event=draining6193client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6194worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6195worker2 # [ 131.941372] nix-daemon[861]: reaped child process 1581, status = succeeded6196worker2 # [ 132.015367] nix-daemon[861]: accepted connection from pid 1622, user nix-grpc-daemon (trusted)6197worker2 # [ 132.027769] nix-daemon[861]: reaped child process 1625, status = succeeded6198lb # [ 132.052742] niks3-server[1130]: 2026/09/25 16:33:35 INFO Received push request method=POST path=/api/pushes6199worker2 # [ 132.047441] nix-grpc-daemon[1324]: time=2026-09-25T16:33:35.221Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6200worker2 # [ 132.048991] nix-grpc-daemon[1324]: time=2026-09-25T16:33:35.223Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6201worker2 # [ 132.084287] nix-grpc-daemon[1324]: time=2026-09-25T16:33:35.258Z level=INFO msg="Uploading 1 narinfos"6202lb # [ 132.100892] niks3-server[1130]: 2026/09/25 16:33:35 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign6203lb # [ 132.103521] niks3-server[1130]: 2026/09/25 16:33:35 INFO Signed narinfos id=49 count=16204lb # [ 132.111282] niks3-server[1130]: 2026/09/25 16:33:35 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6205lb # [ 132.117186] niks3-server[1130]: 2026/09/25 16:33:35 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6206lb # [ 132.124887] niks3-server[1130]: 2026/09/25 16:33:35 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6207lb # [ 132.130179] niks3-server[1130]: 2026/09/25 16:33:35 INFO Received complete push request method=POST path=/api/pushes/49/complete6208worker2 # [ 132.119525] nix-grpc-daemon[1324]: time=2026-09-25T16:33:35.293Z level=INFO msg="Upload complete. (185ms)"6209lb # [ 132.138091] niks3-server[1130]: 2026/09/25 16:33:35 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6210worker2 # [ 132.126706] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34624 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06211worker2 # [ 132.142328] nix-daemon[861]: reaped child process 1579, status = succeeded6212client # [ 132.162265] systemd[1]: rs.service: Deactivated successfully.6213client # [ 132.165142] systemd[1]: rs.service: Consumed 121ms CPU time over 8.040s wall clock time, 43.4M memory peak, 7K incoming IP traffic, 8.3K outgoing IP traffic.6214worker2 # [ 132.612566] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:35Z level=info event=signal_exit6215lb # [ 132.754235] niks3-server[1130]: 2026/09/25 16:33:35 INFO lead: released remote=192.168.1.4:382906216worker2 # [ 133.739752] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:36Z level=info event=scheduler_restarting6217lb # [ 133.755255] envoy[922]: [2026-09-25 16:33:36.920][922][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6218worker2 # [ 133.746804] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:36Z level=info event=scheduler_restarting addr=lb:500516219worker1 # [ 133.750868] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_restarting addr=lb:500516220worker2 # [ 133.753692] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:482526221worker2 # [ 133.757318] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:36Z level=info event=worker_gone addr=192.168.1.3:500516222worker2 # [ 133.760810] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:482526223worker2 # [ 133.764323] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:36Z level=info event=worker_gone addr=192.168.1.4:500516224worker2 # [ 133.788412] nix-daemon[861]: reaped child process 1316, status = succeeded6225lb # [ 133.889568] niks3-server[1130]: 2026/09/25 16:33:37 INFO lead: acquired remote=192.168.1.3:538586226worker1 # [ 133.879864] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_take_over6227worker1 # [ 133.955238] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_connected addr=lb:500516228worker1 # [ 133.959098] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6229worker2 # [ 133.961504] nix-grpc-daemon[1264]: ts=2026-09-25T16:33:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6230worker2 # [ 134.005417] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6231worker2 # [ 134.009341] systemd[1]: Stopped Nix worker-protocol over gRPC.6232worker2 # [ 134.012736] systemd[1]: nix-grpc-daemon.service: Consumed 1.628s CPU time over 33.498s wall clock time, 46.2M memory peak, 381.9K incoming IP traffic, 267.7K outgoing IP traffic.6233worker2 # [ 134.026132] systemd[1]: Starting scheduler-token.service...6234worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.13 seconds)6235client: waiting for success: ! systemctl is-active rs6236worker2 # [ 134.069059] systemd[1]: scheduler-token.service: Deactivated successfully.6237worker2 # [ 134.071067] systemd[1]: Finished scheduler-token.service.6238worker2 # [ 134.071801] systemd[1]: scheduler-token.service: Consumed 20ms CPU time over 44ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6239worker2 # [ 134.075602] systemd[1]: Starting Nix worker-protocol over gRPC...6240client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6241client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6242client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6243client: must fail: journalctl -u rs | grep -q 'reconnecting'6244client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6245lb: must succeed: curl -sf localhost:9901/clusters6246worker2 # [ 134.150608] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6247worker2 # [ 134.152731] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6248worker2 # [ 134.155979] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16249worker1 # [ 134.159961] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_connected addr=lb:500516250worker2 # [ 134.161061] nix-grpc-daemon[1636]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6251worker1 # [ 134.163979] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6252worker2 # [ 134.162395] nix-grpc-daemon[1636]: E0000 00:00:1790354017.334285 1636 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6253worker2 # [ 134.165494] nix-grpc-daemon[1636]: E0000 00:00:1790354017.339504 1636 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6254lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6255worker2 # [ 134.169813] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6256worker2 # [ 134.172694] systemd[1]: Started Nix worker-protocol over gRPC.6257worker2 # [ 134.176065] nix-grpc-daemon[1636]: E0000 00:00:1790354017.349519 1678 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6258worker2 # [ 134.187541] nix-grpc-daemon[1636]: E0000 00:00:1790354017.361619 1678 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6259worker2 # [ 134.191137] nix-grpc-daemon[1636]: E0000 00:00:1790354017.363848 1678 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6260worker2 # [ 134.201674] nix-grpc-daemon[1636]: E0000 00:00:1790354017.375715 1678 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6261worker2 # [ 134.210864] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=scheduler_connected addr=lb:500516262worker2 # [ 134.212422] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6263worker1 # [ 134.364674] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_connected addr=lb:500516264worker1 # [ 134.368593] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6265worker2 # [ 134.413757] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=scheduler_connected addr=lb:500516266worker2 # [ 134.417686] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6267worker1 # [ 134.569478] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_connected addr=lb:500516268worker1 # [ 134.573671] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6269worker2 # [ 134.618627] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=scheduler_connected addr=lb:500516270worker2 # [ 134.622644] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6271worker1 # [ 134.775958] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=scheduler_connected addr=lb:500516272worker1 # [ 134.779443] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6273worker2 # [ 134.823690] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:37Z level=info event=scheduler_connected addr=lb:500516274worker2 # [ 134.827960] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6275worker1 # [ 134.976937] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=scheduler_connected addr=lb:500516276worker1 # [ 134.980973] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6277worker2 # [ 135.029084] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:38Z level=info event=scheduler_connected addr=lb:500516278worker2 # [ 135.033086] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6279lb: must succeed: curl -sf localhost:9901/clusters6280worker1 # [ 135.181715] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=scheduler_connected addr=lb:500516281worker1 # [ 135.185690] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6282lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6283worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6284worker2 # [ 135.235439] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:38Z level=info event=scheduler_connected addr=lb:500516285worker2 # [ 135.238664] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6286worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6287worker1 # [ 135.386552] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=scheduler_connected addr=lb:500516288worker1 # [ 135.398123] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:441126289worker1 # [ 135.402836] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06290worker2 # [ 135.436815] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:38Z level=info event=scheduler_connected addr=lb:500516291worker1 # [ 135.448130] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=oidc_jwks provider=mock keys=16292worker1 # [ 135.451161] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:441126293worker1 # [ 135.455474] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06294worker1 # [ 135.688350] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:37Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:441166295lb: must succeed: curl -sf localhost:9901/clusters6296lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6297worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6298worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6299(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.28 seconds)6300subtest: scheduler restart mid-build: build finishes, no double build6301client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6302client # [ 136.398693] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6303client # Running as unit: mid.service; invocation ID: fdd9bde5550f4e93bf322366aa305a8f6304client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6305worker2 # [ 136.543271] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505946306worker2 # [ 136.554244] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6307client # [ 136.565110] nix[1412]: don't know how to build these paths:6308client # [ 136.566056] nix[1412]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6309worker1 # [ 136.580239] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:38Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06310worker1 # [ 136.587233] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:38Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16311worker1 # [ 136.594706] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:441266312worker1 # [ 136.605734] nix-daemon[870]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6313worker1 # [ 136.621281] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5)6314worker1 # [ 136.710940] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 2/5)6315worker1 # [ 136.777982] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 3/5)6316worker1 # [ 136.799651] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 101 ms (attempt 4/5)6317worker1 # [ 136.903356] nix-grpc-daemon[1407]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6318worker1 # [ 136.906307] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6319worker1 # [ 136.923498] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:441266320client # [ 136.928935] nix[1412]: copying 1 paths to 'lb:50051'...6321client # [ 136.932130] nix[1412]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6322worker1 # [ 136.938342] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:441266323worker1 # [ 136.946780] nix-daemon[870]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6324worker1 # [ 136.971555] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:39Z level=info event=niks3_push_started pid=15296325lb # [ 137.076800] envoy[922]: conn peer=[2001:db8:1::1]:51986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1499 ms=78896326lb # [ 137.080576] envoy[922]: conn peer=[2001:db8:1::4]:38458 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=35707 tx=2236 ms=333066327worker1 # [ 137.074775] nix-grpc-daemon[1529]: time=2026-09-25T16:33:39.372Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6328worker1 # [ 137.159398] nix-daemon[870]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6329worker1 # [ 137.172830] nix-daemon[870]: reaped child process 1545, status = succeeded6330lb # [ 137.191317] niks3-server[1130]: 2026/09/25 16:33:40 INFO Received push request method=POST path=/api/pushes6331worker1 # [ 137.188826] nix-grpc-daemon[1529]: time=2026-09-25T16:33:39.487Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6332worker1 # [ 137.190329] nix-grpc-daemon[1529]: time=2026-09-25T16:33:39.488Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6333lb # [ 137.230847] niks3-server[1130]: 2026/09/25 16:33:40 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign6334lb # [ 137.238899] niks3-server[1130]: 2026/09/25 16:33:40 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6335lb # [ 137.252594] niks3-server[1130]: 2026/09/25 16:33:40 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6336lb # [ 137.261590] niks3-server[1130]: 2026/09/25 16:33:40 INFO Signed narinfos id=50 count=16337worker1 # [ 137.249471] nix-grpc-daemon[1529]: time=2026-09-25T16:33:39.547Z level=INFO msg="Uploading 1 narinfos"6338lb # [ 137.270819] niks3-server[1130]: 2026/09/25 16:33:40 INFO Received complete push request method=POST path=/api/pushes/50/complete6339lb # [ 137.275336] niks3-server[1130]: 2026/09/25 16:33:40 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6340worker1 # [ 137.270129] nix-grpc-daemon[1529]: time=2026-09-25T16:33:39.568Z level=INFO msg="Upload complete. (195ms)"6341worker1 # [ 137.275641] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44126 duration_s=0 paths=1 nar_bytes_in=4246342worker1 # [ 137.281479] nix-grpc-daemon[1407]: ts=2026-09-25T16:33:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:441266343worker1 # [ 137.291502] nix-daemon[870]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6344worker1 # [ 137.303132] nix-daemon[870]: reaped child process 1527, status = succeeded6345worker1 # [ 137.306484] nix-daemon[870]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6346client # [ 137.301352] nix[1412]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6347lb: must succeed: curl -sf localhost:9901/clusters6348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6349lb: must succeed: curl -sf localhost:9901/clusters6350lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6351worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6352worker1 # [ 138.889331] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1407 (nix-grpc-daemon) on client request.6353worker1 # [ 138.892345] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6354lb # [ 138.908842] niks3-server[1130]: 2026/09/25 16:33:42 INFO lead: released remote=192.168.1.3:538586355lb # [ 138.914356] envoy[922]: [2026-09-25 16:33:42.079][922][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}6356client # [ 138.896501] nix[1412]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6357worker2 # [ 138.897884] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6358client # [ 138.901186] nix[1412]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6359worker1 # [ 138.908541] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6360worker1 # [ 138.910191] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6361worker1 # [ 138.916159] systemd[1]: nix-grpc-daemon.service: Consumed 394ms CPU time over 15.136s wall clock time, 35.8M memory peak, 160.9K incoming IP traffic, 55.6K outgoing IP traffic.6362worker1 # [ 138.921252] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6363worker1 # [ 138.925549] systemd[1]: Starting Nix worker-protocol over gRPC...6364worker1 # [ 138.953716] nix-daemon[870]: reaped child process 1521, status = succeeded6365worker1 # [ 138.955717] nix-daemon[870]: reaped child process 1547, status = succeeded6366worker1 # [ 138.969660] nix-daemon[1549]: unexpected Nix daemon error: error: interrupted by the user6367worker1 # [ 138.976386] nix-daemon[870]: reaped child process 1549, status = failed with exit code 16368worker1 # [ 139.013548] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6369worker1 # [ 139.015622] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6370worker1 # [ 139.018749] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16371worker1 # [ 139.023056] nix-grpc-daemon[1585]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6372worker1 # [ 139.025155] nix-grpc-daemon[1585]: E0000 00:00:1790354021.321482 1585 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6373worker1 # [ 139.028385] nix-grpc-daemon[1585]: E0000 00:00:1790354021.326900 1585 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6374worker1 # [ 139.034075] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6375worker1 # [ 139.036940] systemd[1]: Started Nix worker-protocol over gRPC.6376worker1 # [ 139.041066] nix-grpc-daemon[1585]: E0000 00:00:1790354021.338873 1628 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6377lb # [ 139.053640] niks3-server[1130]: 2026/09/25 16:33:42 INFO lead: acquired remote=192.168.1.3:585046378worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.18 seconds)6379client: must succeed: sleep 5.06380worker1 # [ 139.044418] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=scheduler_take_over6381worker1 # [ 139.054300] nix-grpc-daemon[1585]: E0000 00:00:1790354021.352879 1628 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6382worker1 # [ 139.058789] nix-grpc-daemon[1585]: E0000 00:00:1790354021.355335 1628 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6383worker1 # [ 139.068328] nix-grpc-daemon[1585]: E0000 00:00:1790354021.366887 1628 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6384worker1 # [ 139.078358] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=scheduler_connected addr=lb:500516385worker1 # [ 139.084843] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:441486386worker1 # [ 139.086557] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06387worker2 # [ 139.098462] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:42Z level=info event=scheduler_connected addr=lb:500516388worker1 # [ 139.106719] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=oidc_jwks provider=mock keys=16389worker1 # [ 139.108138] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:441486390worker1 # [ 139.109750] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06391worker1 # [ 139.143480] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:441526392worker1 # [ 139.414783] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06393worker1 # [ 139.420275] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16394worker1 # [ 139.433133] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:441706395worker1 # [ 139.441373] nix-daemon[870]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)6396worker1 # [ 139.456459] nix-daemon[870]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)6397client # [ 139.462195] nix[1412]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6398worker1 # [ 139.468428] nix-daemon[870]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)6399client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6400client: waiting for success: ! systemctl is-active mid6401worker1 # [ 144.285847] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:46Z level=info event=niks3_push_started pid=16696402worker1 # [ 144.300496] nix-daemon[870]: reaped child process 1644, status = succeeded6403worker1 # [ 144.381452] nix-grpc-daemon[1669]: time=2026-09-25T16:33:46.679Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6404worker1 # [ 144.466815] nix-daemon[870]: accepted connection from pid 1682, user nix-grpc-daemon (trusted)6405worker1 # [ 144.478127] nix-daemon[870]: reaped child process 1685, status = succeeded6406lb # [ 144.499715] niks3-server[1130]: 2026/09/25 16:33:47 INFO Received push request method=POST path=/api/pushes6407worker1 # [ 144.498090] nix-grpc-daemon[1669]: time=2026-09-25T16:33:46.796Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6408worker1 # [ 144.499602] nix-grpc-daemon[1669]: time=2026-09-25T16:33:46.798Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6409lb # [ 144.551242] niks3-server[1130]: 2026/09/25 16:33:47 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign6410worker1 # [ 144.549430] nix-grpc-daemon[1669]: time=2026-09-25T16:33:46.847Z level=INFO msg="Uploading 1 narinfos"6411lb # [ 144.562128] niks3-server[1130]: 2026/09/25 16:33:47 INFO Signed narinfos id=51 count=16412lb # [ 144.568797] niks3-server[1130]: 2026/09/25 16:33:47 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6413lb # [ 144.573107] niks3-server[1130]: 2026/09/25 16:33:47 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6414lb # [ 144.586535] niks3-server[1130]: 2026/09/25 16:33:47 INFO Received complete push request method=POST path=/api/pushes/51/complete6415lb # [ 144.592699] niks3-server[1130]: 2026/09/25 16:33:47 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6416lb # [ 144.595760] niks3-server[1130]: 2026/09/25 16:33:47 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6417worker1 # [ 144.587755] nix-grpc-daemon[1669]: time=2026-09-25T16:33:46.886Z level=INFO msg="Upload complete. (203ms)"6418worker1 # [ 144.594477] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44170 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06419worker1 # [ 144.611578] nix-daemon[870]: reaped child process 1642, status = succeeded6420client # [ 144.627889] systemd[1]: mid.service: Deactivated successfully.6421client # [ 144.631574] systemd[1]: mid.service: Consumed 139ms CPU time over 8.232s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.2K outgoing IP traffic.6422client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6423client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6424client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6425(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.01 seconds)6426subtest: interrupting the client stops the build on the worker6427client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6428client # [ 145.382447] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6429client # Running as unit: intr.service; invocation ID: 495bef564ef041c2a32bb0cf23caf8446430client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.02 seconds)6431worker1 # [ 145.509319] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:441706432client # [ 145.510846] nix[1448]: don't know how to build these paths:6433client # [ 145.511814] nix[1448]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6434worker1 # [ 145.524209] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:47Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06435worker1 # [ 145.528784] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:47Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16436worker2 # [ 145.526377] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946437worker2 # [ 145.536268] nix-grpc-daemon[1636]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 1/5)6438worker2 # [ 145.549619] nix-grpc-daemon[1636]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 180 ms (attempt 2/5)6439worker2 # [ 145.731776] nix-grpc-daemon[1636]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 241 ms (attempt 3/5)6440worker2 # [ 145.998646] nix-grpc-daemon[1636]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 574 ms (attempt 4/5)6441worker2 # [ 146.577726] nix-grpc-daemon[1636]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6442worker2 # [ 146.581862] nix-grpc-daemon[1636]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6443worker2 # [ 146.601182] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505946444client # [ 146.606778] nix[1448]: copying 1 paths to 'lb:50051'...6445worker2 # [ 146.608202] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505946446client # [ 146.608676] nix[1448]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6447worker2 # [ 146.620376] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6448worker2 # [ 146.633931] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:49Z level=info event=niks3_push_started pid=17286449worker2 # [ 146.722366] nix-grpc-daemon[1728]: time=2026-09-25T16:33:49.895Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6450worker2 # [ 146.808464] nix-daemon[861]: accepted connection from pid 1741, user nix-grpc-daemon (trusted)6451worker2 # [ 146.819851] nix-daemon[861]: reaped child process 1744, status = succeeded6452lb # [ 146.844480] niks3-server[1130]: 2026/09/25 16:33:50 INFO Received push request method=POST path=/api/pushes6453worker2 # [ 146.838551] nix-grpc-daemon[1728]: time=2026-09-25T16:33:50.012Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6454worker2 # [ 146.840177] nix-grpc-daemon[1728]: time=2026-09-25T16:33:50.014Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6455lb # [ 146.886259] niks3-server[1130]: 2026/09/25 16:33:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6456lb # [ 146.892654] niks3-server[1130]: 2026/09/25 16:33:50 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign6457lb # [ 146.900061] niks3-server[1130]: 2026/09/25 16:33:50 INFO Signed narinfos id=52 count=16458worker2 # [ 146.886685] nix-grpc-daemon[1728]: time=2026-09-25T16:33:50.060Z level=INFO msg="Uploading 1 narinfos"6459lb # [ 146.905662] niks3-server[1130]: 2026/09/25 16:33:50 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6460lb # [ 146.918311] niks3-server[1130]: 2026/09/25 16:33:50 INFO Received complete push request method=POST path=/api/pushes/52/complete6461lb # [ 146.927136] niks3-server[1130]: 2026/09/25 16:33:50 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6462worker2 # [ 146.910051] nix-grpc-daemon[1728]: time=2026-09-25T16:33:50.083Z level=INFO msg="Upload complete. (184ms)"6463worker2 # [ 146.914551] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 paths=1 nar_bytes_in=4246464worker2 # [ 146.919550] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946465worker2 # [ 146.928719] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6466worker2 # [ 146.934557] nix-daemon[861]: reaped child process 1726, status = succeeded6467worker2 # [ 146.939240] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6468client # [ 146.937394] nix[1448]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6469lb # [ 147.077271] envoy[922]: conn peer=[2001:db8:1::3]:53118 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2204 tx=607 ms=149916470lb # [ 147.081089] envoy[922]: conn peer=[2001:db8:1::1]:43172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3878 tx=1919 ms=80706471client: must succeed: systemctl kill -s INT intr6472client # [ 148.820252] systemd[1]: intr.service: Sent signal SIGINT to main process 1448 (nix) on client request.6473worker2 # [ 148.824188] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:51Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6474client # [ 148.827573] nix[1448]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6475client # [ 148.833185] nix[1448]: error: interrupted by the user6476client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6477client # [ 148.862744] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6478client # [ 148.865590] systemd[1]: intr.service: Failed with result 'exit-code'.6479client # [ 148.869543] systemd[1]: intr.service: Consumed 116ms CPU time over 3.485s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6480worker2 # [ 148.957136] nix-daemon[861]: reaped child process 1746, status = succeeded6481worker2 # [ 148.958669] nix-daemon[1748]: unexpected Nix daemon error: error: interrupted by the user6482worker2 # [ 148.963406] nix-daemon[861]: reaped child process 1748, status = failed with exit code 16483(finished: subtest: interrupting the client stops the build on the worker, in 4.65 seconds)6484subtest: a deploy mid-build drains: the build finishes, the new generation takes over6485client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6486client # [ 150.059994] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6487client # Running as unit: sw.service; invocation ID: 7d1026ee567d4eeea08bb3889a930a7a6488client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6489worker2 # [ 150.197561] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505946490client # [ 150.202183] nix[1469]: don't know how to build these paths:6491client # [ 150.203129] nix[1469]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6492worker1 # [ 150.217195] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:52Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06493worker1 # [ 150.223822] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:52Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16494worker1 # [ 150.228246] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:441706495worker1 # [ 150.237836] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5)6496worker1 # [ 150.338328] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 2/5)6497worker1 # [ 150.438241] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 3/5)6498worker1 # [ 150.485307] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 290 ms (attempt 4/5)6499worker1 # [ 150.782894] nix-grpc-daemon[1585]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6500worker1 # [ 150.785836] nix-grpc-daemon[1585]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6501worker1 # [ 150.802674] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:441706502client # [ 150.808510] nix[1469]: copying 1 paths to 'lb:50051'...6503client # [ 150.812137] nix[1469]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6504worker1 # [ 150.817667] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:441706505worker1 # [ 150.828939] nix-daemon[870]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)6506worker1 # [ 150.929995] nix-daemon[870]: accepted connection from pid 1725, user nix-grpc-daemon (trusted)6507worker1 # [ 150.943704] nix-daemon[870]: reaped child process 1728, status = succeeded6508lb # [ 150.964086] niks3-server[1130]: 2026/09/25 16:33:54 INFO Received push request method=POST path=/api/pushes6509worker1 # [ 150.959959] nix-grpc-daemon[1669]: time=2026-09-25T16:33:53.258Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6510worker1 # [ 150.962351] nix-grpc-daemon[1669]: time=2026-09-25T16:33:53.260Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6511lb # [ 150.996771] niks3-server[1130]: 2026/09/25 16:33:54 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6512lb # [ 151.002565] niks3-server[1130]: 2026/09/25 16:33:54 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6513worker1 # [ 150.995106] nix-grpc-daemon[1669]: time=2026-09-25T16:33:53.293Z level=INFO msg="Uploading 1 narinfos"6514lb # [ 151.007635] niks3-server[1130]: 2026/09/25 16:33:54 INFO Signed narinfos id=53 count=16515lb # [ 151.016307] niks3-server[1130]: 2026/09/25 16:33:54 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6516lb # [ 151.022858] niks3-server[1130]: 2026/09/25 16:33:54 INFO Received complete push request method=POST path=/api/pushes/53/complete6517worker1 # [ 151.015922] nix-grpc-daemon[1669]: time=2026-09-25T16:33:53.314Z level=INFO msg="Upload complete. (175ms)"6518lb # [ 151.030522] niks3-server[1130]: 2026/09/25 16:33:54 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6519worker1 # [ 151.020363] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44170 duration_s=0 paths=1 nar_bytes_in=4166520worker1 # [ 151.025146] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:441706521worker1 # [ 151.034743] nix-daemon[870]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)6522worker1 # [ 151.039766] nix-daemon[870]: reaped child process 1723, status = succeeded6523worker1 # [ 151.044293] nix-daemon[870]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)6524client # [ 151.038982] nix[1469]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6525worker1: must succeed: systemctl reload nix-grpc-daemon.service6526worker1 # [ 152.479165] systemd[1]: Reloading Nix worker-protocol over gRPC...6527worker1 # [ 152.505283] systemd[1]: Reloaded Nix worker-protocol over gRPC.6528worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6529worker1: must succeed: pgrep -f 'read -t [9]0 x'6530worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6531client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26532worker2 # [ 152.691752] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505946533client # don't know how to build these paths:6534client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6535worker2 # [ 152.699635] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505946536worker1 # [ 152.724177] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:55Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06537worker1 # [ 152.727139] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:55Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16538worker2 # [ 152.725458] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946539worker2 # [ 152.736724] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505946540client # copying 1 paths to 'lb:50051'...6541client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6542worker2 # [ 152.748060] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505946543worker2 # [ 152.760397] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6544worker2 # [ 152.859974] nix-daemon[861]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6545worker2 # [ 152.873162] nix-daemon[861]: reaped child process 1802, status = succeeded6546lb # [ 152.896799] niks3-server[1130]: 2026/09/25 16:33:56 INFO Received push request method=POST path=/api/pushes6547worker2 # [ 152.891293] nix-grpc-daemon[1728]: time=2026-09-25T16:33:56.065Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6548worker2 # [ 152.893637] nix-grpc-daemon[1728]: time=2026-09-25T16:33:56.066Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6549lb # [ 152.930066] niks3-server[1130]: 2026/09/25 16:33:56 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6550lb # [ 152.937134] niks3-server[1130]: 2026/09/25 16:33:56 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6551lb # [ 152.941988] niks3-server[1130]: 2026/09/25 16:33:56 INFO Signed narinfos id=54 count=16552worker2 # [ 152.930878] nix-grpc-daemon[1728]: time=2026-09-25T16:33:56.104Z level=INFO msg="Uploading 1 narinfos"6553lb # [ 152.952304] niks3-server[1130]: 2026/09/25 16:33:56 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6554lb # [ 152.962592] niks3-server[1130]: 2026/09/25 16:33:56 INFO Received complete push request method=POST path=/api/pushes/54/complete6555lb # [ 152.972927] niks3-server[1130]: 2026/09/25 16:33:56 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6556worker2 # [ 152.959418] nix-grpc-daemon[1728]: time=2026-09-25T16:33:56.133Z level=INFO msg="Upload complete. (186ms)"6557worker2 # [ 152.964171] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 paths=1 nar_bytes_in=5286558worker2 # [ 152.970339] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946559worker2 # [ 152.982277] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6560worker2 # [ 152.992126] nix-daemon[861]: reaped child process 1797, status = succeeded6561worker2 # [ 152.995600] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6562client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6563client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6564worker1 # [ 153.037573] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:55Z level=info event=draining6565client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6566client # farm-a> LOG-farm-a-during-drain6567worker2 # [ 154.435923] nix-daemon[861]: reaped child process 1806, status = succeeded6568worker2 # [ 154.516478] nix-daemon[861]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)6569worker2 # [ 154.530544] nix-daemon[861]: reaped child process 1825, status = succeeded6570lb # [ 154.553806] niks3-server[1130]: 2026/09/25 16:33:57 INFO Received push request method=POST path=/api/pushes6571worker2 # [ 154.543991] nix-grpc-daemon[1728]: time=2026-09-25T16:33:57.717Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6572worker2 # [ 154.545557] nix-grpc-daemon[1728]: time=2026-09-25T16:33:57.719Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6573lb # [ 154.588563] niks3-server[1130]: 2026/09/25 16:33:57 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6574lb # [ 154.607110] niks3-server[1130]: 2026/09/25 16:33:57 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6575lb # [ 154.611676] niks3-server[1130]: 2026/09/25 16:33:57 INFO Signed narinfos id=55 count=16576worker2 # [ 154.598063] nix-grpc-daemon[1728]: time=2026-09-25T16:33:57.771Z level=INFO msg="Uploading 1 narinfos"6577lb # [ 154.621911] niks3-server[1130]: 2026/09/25 16:33:57 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6578lb # [ 154.629857] niks3-server[1130]: 2026/09/25 16:33:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6579lb # [ 154.632626] niks3-server[1130]: 2026/09/25 16:33:57 INFO Received complete push request method=POST path=/api/pushes/55/complete6580lb # [ 154.637509] niks3-server[1130]: 2026/09/25 16:33:57 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6581worker2 # [ 154.624324] nix-grpc-daemon[1728]: time=2026-09-25T16:33:57.798Z level=INFO msg="Upload complete. (220ms)"6582worker1 # [ 154.638574] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:56Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06583worker1 # [ 154.640350] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:56Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16584worker2 # [ 154.630695] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06585worker2 # [ 154.640602] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946586worker2 # [ 154.653835] nix-daemon[861]: reaped child process 1804, status = succeeded6587worker2 # [ 154.657423] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505946588worker2 # [ 154.666140] nix-grpc-daemon[1728]: time=2026-09-25T16:33:57.840Z level=INFO msg="All 1 paths already cached"6589client # copying 1 paths to 'lb:50051'...6590worker2 # [ 154.673348] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505946591client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6592worker2 # [ 154.690527] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6593worker2 # [ 154.780661] nix-daemon[861]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6594worker2 # [ 154.794130] nix-daemon[861]: reaped child process 1832, status = succeeded6595lb # [ 154.816287] niks3-server[1130]: 2026/09/25 16:33:57 INFO Received push request method=POST path=/api/pushes6596worker2 # [ 154.807643] nix-grpc-daemon[1728]: time=2026-09-25T16:33:57.981Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6597worker2 # [ 154.809556] nix-grpc-daemon[1728]: time=2026-09-25T16:33:57.983Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6598lb # [ 154.845676] niks3-server[1130]: 2026/09/25 16:33:58 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6599lb # [ 154.850513] niks3-server[1130]: 2026/09/25 16:33:58 INFO Signed narinfos id=56 count=16600worker2 # [ 154.837250] nix-grpc-daemon[1728]: time=2026-09-25T16:33:58.011Z level=INFO msg="Uploading 1 narinfos"6601lb # [ 154.862666] niks3-server[1130]: 2026/09/25 16:33:58 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6602lb # [ 154.867914] niks3-server[1130]: 2026/09/25 16:33:58 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6603lb # [ 154.879209] niks3-server[1130]: 2026/09/25 16:33:58 INFO Received complete push request method=POST path=/api/pushes/56/complete6604lb # [ 154.887463] niks3-server[1130]: 2026/09/25 16:33:58 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6605worker2 # [ 154.874293] nix-grpc-daemon[1728]: time=2026-09-25T16:33:58.048Z level=INFO msg="Upload complete. (172ms)"6606worker2 # [ 154.878716] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 paths=1 nar_bytes_in=6406607worker2 # [ 154.884581] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946608worker2 # [ 154.894751] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6609worker2 # [ 154.903503] nix-daemon[861]: reaped child process 1827, status = succeeded6610worker2 # [ 154.908600] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6611client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6612client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6613client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6614client # farm-b> LOG-farm-b-during-drain6615worker2 # [ 155.549364] nix-daemon[861]: reaped child process 1836, status = succeeded6616worker2 # [ 155.624761] nix-daemon[861]: accepted connection from pid 1852, user nix-grpc-daemon (trusted)6617worker2 # [ 155.637616] nix-daemon[861]: reaped child process 1855, status = succeeded6618lb # [ 155.662707] niks3-server[1130]: 2026/09/25 16:33:58 INFO Received push request method=POST path=/api/pushes6619worker2 # [ 155.652987] nix-grpc-daemon[1728]: time=2026-09-25T16:33:58.826Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6620worker2 # [ 155.655246] nix-grpc-daemon[1728]: time=2026-09-25T16:33:58.828Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6621lb # [ 155.699559] niks3-server[1130]: 2026/09/25 16:33:58 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6622lb # [ 155.707816] niks3-server[1130]: 2026/09/25 16:33:58 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6623lb # [ 155.718981] niks3-server[1130]: 2026/09/25 16:33:58 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6624lb # [ 155.722931] niks3-server[1130]: 2026/09/25 16:33:58 INFO Signed narinfos id=57 count=16625worker2 # [ 155.709720] nix-grpc-daemon[1728]: time=2026-09-25T16:33:58.883Z level=INFO msg="Uploading 1 narinfos"6626lb # [ 155.726050] niks3-server[1130]: 2026/09/25 16:33:58 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6627lb # [ 155.737618] niks3-server[1130]: 2026/09/25 16:33:58 INFO Received complete push request method=POST path=/api/pushes/57/complete6628lb # [ 155.741928] niks3-server[1130]: 2026/09/25 16:33:58 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6629worker2 # [ 155.724655] nix-grpc-daemon[1728]: time=2026-09-25T16:33:58.898Z level=INFO msg="Upload complete. (209ms)"6630worker1 # [ 155.739295] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:58Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06631worker1 # [ 155.741144] nix-grpc-daemon[1585]: ts=2026-09-25T16:33:58Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16632worker2 # [ 155.730811] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06633worker2 # [ 155.741582] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946634worker2 # [ 155.752519] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505946635worker2 # [ 155.758881] nix-daemon[861]: reaped child process 1834, status = succeeded6636worker2 # [ 155.765165] nix-grpc-daemon[1728]: time=2026-09-25T16:33:58.939Z level=INFO msg="All 2 paths already cached"6637client # copying 1 paths to 'lb:50051'...6638worker2 # [ 155.774221] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505946639client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6640worker2 # [ 155.793134] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6641worker2 # [ 155.882171] nix-daemon[861]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)6642worker2 # [ 155.895801] nix-daemon[861]: reaped child process 1862, status = succeeded6643lb # [ 155.917459] niks3-server[1130]: 2026/09/25 16:33:59 INFO Received push request method=POST path=/api/pushes6644worker2 # [ 155.909455] nix-grpc-daemon[1728]: time=2026-09-25T16:33:59.083Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6645worker2 # [ 155.911040] nix-grpc-daemon[1728]: time=2026-09-25T16:33:59.085Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6646lb # [ 155.951241] niks3-server[1130]: 2026/09/25 16:33:59 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6647worker2 # [ 155.940782] nix-grpc-daemon[1728]: time=2026-09-25T16:33:59.114Z level=INFO msg="Uploading 1 narinfos"6648lb # [ 155.957575] niks3-server[1130]: 2026/09/25 16:33:59 INFO Signed narinfos id=58 count=16649lb # [ 155.961631] niks3-server[1130]: 2026/09/25 16:33:59 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6650lb # [ 155.970897] niks3-server[1130]: 2026/09/25 16:33:59 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6651lb # [ 155.978371] niks3-server[1130]: 2026/09/25 16:33:59 INFO Received complete push request method=POST path=/api/pushes/58/complete6652worker2 # [ 155.971708] nix-grpc-daemon[1728]: time=2026-09-25T16:33:59.145Z level=INFO msg="Upload complete. (169ms)"6653lb # [ 155.988648] niks3-server[1130]: 2026/09/25 16:33:59 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6654worker2 # [ 155.975183] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 paths=1 nar_bytes_in=7686655worker2 # [ 155.980230] nix-grpc-daemon[1636]: ts=2026-09-25T16:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946656worker2 # [ 155.989897] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6657worker2 # [ 155.995682] nix-daemon[861]: reaped child process 1857, status = succeeded6658worker2 # [ 156.000468] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6659client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6660client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6661lb # [ 157.078127] envoy[922]: conn peer=[2001:db8:1::1]:55938 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=828 ms=33446662client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6663client # farm-top> LOG-farm-top-during-drain6664worker2 # [ 157.424076] nix-daemon[861]: reaped child process 1866, status = succeeded6665worker2 # [ 157.496423] nix-daemon[861]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)6666worker2 # [ 157.508492] nix-daemon[861]: reaped child process 1885, status = succeeded6667lb # [ 157.533965] niks3-server[1130]: 2026/09/25 16:34:00 INFO Received push request method=POST path=/api/pushes6668worker2 # [ 157.527826] nix-grpc-daemon[1728]: time=2026-09-25T16:34:00.701Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6669worker2 # [ 157.529501] nix-grpc-daemon[1728]: time=2026-09-25T16:34:00.703Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6670lb # [ 157.574546] niks3-server[1130]: 2026/09/25 16:34:00 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6671lb # [ 157.586861] niks3-server[1130]: 2026/09/25 16:34:00 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6672lb # [ 157.591908] niks3-server[1130]: 2026/09/25 16:34:00 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6673lb # [ 157.598825] niks3-server[1130]: 2026/09/25 16:34:00 INFO Signed narinfos id=59 count=16674worker2 # [ 157.585464] nix-grpc-daemon[1728]: time=2026-09-25T16:34:00.759Z level=INFO msg="Uploading 1 narinfos"6675lb # [ 157.605815] niks3-server[1130]: 2026/09/25 16:34:00 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6676lb # [ 157.616234] niks3-server[1130]: 2026/09/25 16:34:00 INFO Received complete push request method=POST path=/api/pushes/59/complete6677lb # [ 157.621358] niks3-server[1130]: 2026/09/25 16:34:00 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6678worker2 # [ 157.604376] nix-grpc-daemon[1728]: time=2026-09-25T16:34:00.778Z level=INFO msg="Upload complete. (196ms)"6679worker2 # [ 157.610630] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06680worker2 # [ 157.626163] nix-daemon[861]: reaped child process 1864, status = succeeded6681client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.10 seconds)6682client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6683client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)6684worker1: must succeed: pkill -f 'read -t [9]0 x'6685worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6686client: waiting for success: ! systemctl is-active sw6687worker1 # [ 157.863504] nix-daemon[870]: reaped child process 1732, status = succeeded6688worker1 # [ 157.939368] nix-daemon[870]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6689worker1 # [ 157.951380] nix-daemon[870]: reaped child process 1782, status = succeeded6690lb # [ 157.972350] niks3-server[1130]: 2026/09/25 16:34:01 INFO Received push request method=POST path=/api/pushes6691worker1 # [ 157.968305] nix-grpc-daemon[1669]: time=2026-09-25T16:34:00.266Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6692worker1 # [ 157.969845] nix-grpc-daemon[1669]: time=2026-09-25T16:34:00.268Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6693lb # [ 158.013162] niks3-server[1130]: 2026/09/25 16:34:01 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6694lb # [ 158.020902] niks3-server[1130]: 2026/09/25 16:34:01 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6695lb # [ 158.028112] niks3-server[1130]: 2026/09/25 16:34:01 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6696lb # [ 158.034791] niks3-server[1130]: 2026/09/25 16:34:01 INFO Signed narinfos id=60 count=16697worker1 # [ 158.024505] nix-grpc-daemon[1669]: time=2026-09-25T16:34:00.322Z level=INFO msg="Uploading 1 narinfos"6698lb # [ 158.038087] niks3-server[1130]: 2026/09/25 16:34:01 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6699lb # [ 158.050676] niks3-server[1130]: 2026/09/25 16:34:01 INFO Received complete push request method=POST path=/api/pushes/60/complete6700worker1 # [ 158.042342] nix-grpc-daemon[1669]: time=2026-09-25T16:34:00.340Z level=INFO msg="Upload complete. (186ms)"6701lb # [ 158.056495] niks3-server[1130]: 2026/09/25 16:34:01 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6702worker1 # [ 158.048614] nix-grpc-daemon[1585]: ts=2026-09-25T16:34:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44170 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06703worker1 # [ 158.065068] nix-daemon[870]: reaped child process 1730, status = succeeded6704client # [ 158.079372] systemd[1]: sw.service: Deactivated successfully.6705client # [ 158.083606] systemd[1]: sw.service: Consumed 126ms CPU time over 8.022s wall clock time, 43.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6706client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6707client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6708client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6709worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6710worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.04 seconds)6711lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux6712lb: must succeed: curl -sf localhost:9901/clusters6713lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6714worker1 # [ 159.040911] nix-grpc-daemon[1585]: ts=2026-09-25T16:34:01Z level=info event=signal_exit6715lb: must succeed: curl -sf localhost:9901/clusters6716lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6717lb # [ 160.056834] niks3-server[1130]: 2026/09/25 16:34:03 INFO lead: released remote=192.168.1.3:585046718lb: must succeed: curl -sf localhost:9901/clusters6719worker1 # [ 161.045602] nix-grpc-daemon[1585]: ts=2026-09-25T16:34:03Z level=info event=scheduler_restarting6720worker2 # [ 161.041867] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:04Z level=info event=scheduler_restarting addr=lb:500516721lb # [ 161.058781] envoy[922]: [2026-09-25 16:34:04.223][922][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6722worker1 # [ 161.052553] nix-grpc-daemon[1585]: ts=2026-09-25T16:34:03Z level=info event=scheduler_restarting addr=lb:500516723worker1 # [ 161.060146] nix-grpc-daemon[1585]: ts=2026-09-25T16:34:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:441486724worker1 # [ 161.064092] nix-grpc-daemon[1585]: ts=2026-09-25T16:34:03Z level=info event=worker_gone addr=192.168.1.4:500516725worker1 # [ 161.067887] nix-grpc-daemon[1585]: ts=2026-09-25T16:34:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:441486726worker1 # [ 161.071314] nix-grpc-daemon[1585]: ts=2026-09-25T16:34:03Z level=info event=worker_gone addr=192.168.1.3:500516727lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6728worker1 # [ 161.092137] nix-daemon[870]: reaped child process 1640, status = succeeded6729worker2 # [ 161.246176] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:04Z level=info event=scheduler_connected addr=lb:500516730worker2 # [ 161.250252] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6731worker1 # [ 161.266257] nix-grpc-daemon[1585]: ts=2026-09-25T16:34:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6732worker2 # [ 161.451358] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:04Z level=info event=scheduler_connected addr=lb:500516733worker2 # [ 161.455265] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6734worker2 # [ 161.656422] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:04Z level=info event=scheduler_connected addr=lb:500516735worker2 # [ 161.660294] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6736worker2 # [ 161.861437] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_connected addr=lb:500516737worker2 # [ 161.865412] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6738worker2 # [ 162.067723] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_connected addr=lb:500516739worker2 # [ 162.070926] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6740lb: must succeed: curl -sf localhost:9901/clusters6741lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6742worker2 # [ 162.268936] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_connected addr=lb:500516743worker2 # [ 162.272890] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6744worker2 # [ 162.474136] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_connected addr=lb:500516745worker2 # [ 162.478108] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6746worker2 # [ 162.679139] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_connected addr=lb:500516747worker2 # [ 162.682980] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6748worker2 # [ 162.884258] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_connected addr=lb:500516749worker2 # [ 162.888295] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6750worker1 # [ 163.067303] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6751worker1 # [ 163.071474] systemd[1]: nix-grpc-daemon.service: Consumed 583ms CPU time over 24.163s wall clock time, 36.4M memory peak, 236.2K incoming IP traffic, 68.9K outgoing IP traffic.6752worker2 # [ 163.090337] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_connected addr=lb:500516753worker2 # [ 163.093962] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6754lb: must succeed: curl -sf localhost:9901/clusters6755lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6756worker1 # [ 163.248341] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 2.6757worker1 # [ 163.257950] systemd[1]: Starting Nix worker-protocol over gRPC...6758worker2 # [ 163.295141] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_connected addr=lb:500516759worker2 # [ 163.299150] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6760worker1 # [ 163.357804] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6761worker1 # [ 163.362126] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6762worker1 # [ 163.366188] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16763worker1 # [ 163.368981] nix-grpc-daemon[1795]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6764worker1 # [ 163.370310] nix-grpc-daemon[1795]: E0000 00:00:1790354045.663398 1795 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6765worker1 # [ 163.372425] nix-grpc-daemon[1795]: E0000 00:00:1790354045.667291 1795 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6766worker1 # [ 163.375055] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:05Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6767worker1 # [ 163.378150] systemd[1]: Started Nix worker-protocol over gRPC.6768worker1 # [ 163.383071] nix-grpc-daemon[1795]: E0000 00:00:1790354045.680937 1838 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6769lb # [ 163.396992] niks3-server[1130]: 2026/09/25 16:34:06 INFO lead: acquired remote=192.168.1.3:571906770worker1 # [ 163.387126] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:05Z level=info event=scheduler_take_over6771worker1 # [ 163.395797] nix-grpc-daemon[1795]: E0000 00:00:1790354045.694340 1838 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6772worker1 # [ 163.400051] nix-grpc-daemon[1795]: E0000 00:00:1790354045.696687 1838 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6773worker1 # [ 163.410027] nix-grpc-daemon[1795]: E0000 00:00:1790354045.708565 1838 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6774worker1 # [ 163.418920] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:05Z level=info event=scheduler_connected addr=lb:500516775worker1 # [ 163.420525] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6776worker2 # [ 163.500306] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_connected addr=lb:500516777worker2 # [ 163.504227] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6778worker1 # [ 163.621897] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:05Z level=info event=scheduler_connected addr=lb:500516779worker1 # [ 163.625761] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6780worker2 # [ 163.705308] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_connected addr=lb:500516781worker2 # [ 163.709327] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6782worker1 # [ 163.826641] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=scheduler_connected addr=lb:500516783worker1 # [ 163.830670] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6784worker2 # [ 163.910296] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:07Z level=info event=scheduler_connected addr=lb:500516785worker2 # [ 163.914271] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6786worker1 # [ 164.031368] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=scheduler_connected addr=lb:500516787worker1 # [ 164.035266] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6788worker2 # [ 164.116171] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:07Z level=info event=scheduler_connected addr=lb:500516789worker2 # [ 164.119981] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6790lb: must succeed: curl -sf localhost:9901/clusters6791lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6792worker1 # [ 164.235969] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=scheduler_connected addr=lb:500516793worker1 # [ 164.240092] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6794worker2 # [ 164.321379] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:07Z level=info event=scheduler_connected addr=lb:500516795worker2 # [ 164.325434] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6796worker1 # [ 164.441874] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=scheduler_connected addr=lb:500516797worker1 # [ 164.448574] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:355086798worker1 # [ 164.453352] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06799worker2 # [ 164.526356] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:07Z level=info event=scheduler_connected addr=lb:500516800worker1 # [ 164.537162] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=oidc_jwks provider=mock keys=16801worker1 # [ 164.541321] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:355086802worker1 # [ 164.545238] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06803lb: must succeed: curl -sf localhost:9901/clusters6804lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6805worker1 # [ 166.176293] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:08Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:355146806lb: must succeed: curl -sf localhost:9901/clusters6807lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6808lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 7.40 seconds)6809lb: must succeed: curl -sf localhost:9901/clusters6810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6811worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6812worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6813(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.38 seconds)6814subtest: queue gauges show a burst while every build blocks6815lb: must succeed: curl -sf localhost:9901/clusters6816lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6817lb: must succeed: curl -sf localhost:9901/clusters6818lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6819worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6820worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6821client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6822client # [ 166.544353] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6823client # Running as unit: burst.service; invocation ID: c32f0c7fe8c64301bb7d88ea42f54e176824client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6825lb: must succeed: curl -sf localhost:9901/clusters6826lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6827lb: must succeed: curl -sf localhost:9901/clusters6828lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6829worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6830worker1 # [ 166.686913] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355186831worker1 # [ 166.696641] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6832worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6833client # [ 166.701647] nix[1525]: don't know how to build these paths:6834client # [ 166.702503] nix[1525]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6835client # [ 166.703437] nix[1525]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6836client # [ 166.705109] nix[1525]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6837client # [ 166.706290] nix[1525]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6838worker1 # [ 166.716303] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:09Z level=debug event=want client=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=06839worker1 # [ 166.720522] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:09Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.3:50051 clients=16840worker1 # [ 166.725780] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:09Z level=debug event=want client=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=06841worker2 # [ 166.721708] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946842worker1 # [ 166.729589] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:09Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.4:50051 clients=16843worker1 # [ 166.731512] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:09Z level=debug event=want client=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=06844worker1 # [ 166.733078] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:09Z level=debug event=want client=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=06845worker1 # [ 166.734737] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355186846worker1 # [ 166.738869] nix-grpc-daemon[1795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 96 ms (attempt 1/5)6847worker2 # [ 166.734532] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505946848client # [ 166.743080] nix[1525]: copying 1 paths to 'lb:50051'...6849client # [ 166.744624] nix[1525]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...6850worker2 # [ 166.744894] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505946851worker2 # [ 166.756227] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6852worker1 # [ 166.839239] nix-grpc-daemon[1795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 2/5)6853worker2 # [ 166.858206] nix-daemon[861]: accepted connection from pid 1889, user nix-grpc-daemon (trusted)6854worker2 # [ 166.870220] nix-daemon[861]: reaped child process 1892, status = succeeded6855lb # [ 166.894346] niks3-server[1130]: 2026/09/25 16:34:10 INFO Received push request method=POST path=/api/pushes6856worker2 # [ 166.885815] nix-grpc-daemon[1728]: time=2026-09-25T16:34:10.059Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6857worker2 # [ 166.887794] nix-grpc-daemon[1728]: time=2026-09-25T16:34:10.061Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"6858lb # [ 166.925751] niks3-server[1130]: 2026/09/25 16:34:10 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls6859lb # [ 166.937663] niks3-server[1130]: 2026/09/25 16:34:10 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6860worker2 # [ 166.929312] nix-grpc-daemon[1728]: time=2026-09-25T16:34:10.103Z level=INFO msg="Uploading 1 narinfos"6861lb # [ 166.945708] niks3-server[1130]: 2026/09/25 16:34:10 INFO Signed narinfos id=61 count=16862lb # [ 166.956814] niks3-server[1130]: 2026/09/25 16:34:10 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst6863lb # [ 166.961406] niks3-server[1130]: 2026/09/25 16:34:10 INFO Received complete push request method=POST path=/api/pushes/61/complete6864lb # [ 166.969070] niks3-server[1130]: 2026/09/25 16:34:10 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo6865worker2 # [ 166.951920] nix-grpc-daemon[1728]: time=2026-09-25T16:34:10.125Z level=INFO msg="Upload complete. (184ms)"6866worker2 # [ 166.956653] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 paths=1 nar_bytes_in=4166867worker2 # [ 166.962531] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946868worker2 # [ 166.972953] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6869worker2 # [ 166.984136] nix-daemon[861]: reaped child process 1887, status = succeeded6870worker2 # [ 166.987206] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6871client # [ 166.985478] nix[1525]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...6872worker1 # [ 166.995482] nix-grpc-daemon[1795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 318 ms (attempt 3/5)6873lb # [ 167.078992] envoy[922]: conn peer=[2001:db8:1::1]:55954 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9607 tx=4360 ms=49386874lb # [ 167.083672] envoy[922]: conn peer=[2001:db8:1::1]:55952 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2962 tx=1518 ms=78646875lb # [ 167.087278] envoy[922]: conn peer=[2001:db8:1::3]:47258 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1061 tx=459 ms=221826876worker1 # [ 167.315985] nix-grpc-daemon[1795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 396 ms (attempt 4/5)6877lb: must succeed: curl -sf localhost:9901/clusters6878worker1 # [ 167.713762] nix-grpc-daemon[1795]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6879worker1 # [ 167.717578] nix-grpc-daemon[1795]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6880worker1 # [ 167.735697] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355186881client # [ 167.743634] nix[1525]: copying 1 paths to 'lb:50051'...6882client # [ 167.747143] nix[1525]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...6883worker1 # [ 167.752324] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355186884lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6885lb: must succeed: curl -sf localhost:9901/clusters6886worker1 # [ 167.763505] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6887worker1 # [ 167.782892] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:10Z level=info event=niks3_push_started pid=18666888lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6889worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6890worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6891worker1 # [ 167.910297] nix-grpc-daemon[1866]: time=2026-09-25T16:34:10.206Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6892worker1 # [ 167.997135] nix-daemon[870]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)6893worker2 # [ 168.002790] nix-daemon[861]: reaped child process 1896, status = succeeded6894worker1 # [ 168.009427] nix-daemon[870]: reaped child process 1889, status = succeeded6895lb # [ 168.030555] niks3-server[1130]: 2026/09/25 16:34:11 INFO Received push request method=POST path=/api/pushes6896worker1 # [ 168.029336] nix-grpc-daemon[1866]: time=2026-09-25T16:34:10.327Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6897worker1 # [ 168.030903] nix-grpc-daemon[1866]: time=2026-09-25T16:34:10.329Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"6898lb # [ 168.069318] niks3-server[1130]: 2026/09/25 16:34:11 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6899lb # [ 168.074556] niks3-server[1130]: 2026/09/25 16:34:11 INFO Signed narinfos id=62 count=16900worker1 # [ 168.064294] nix-grpc-daemon[1866]: time=2026-09-25T16:34:10.362Z level=INFO msg="Uploading 1 narinfos"6901lb # [ 168.082099] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst6902lb # [ 168.086962] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls6903worker2 # [ 168.076751] nix-daemon[861]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6904lb # [ 168.096085] niks3-server[1130]: 2026/09/25 16:34:11 INFO Received complete push request method=POST path=/api/pushes/62/complete6905worker1 # [ 168.087240] nix-grpc-daemon[1866]: time=2026-09-25T16:34:10.385Z level=INFO msg="Upload complete. (178ms)"6906lb # [ 168.101695] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo6907worker1 # [ 168.089669] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=4166908worker2 # [ 168.088894] nix-daemon[861]: reaped child process 1920, status = succeeded6909worker1 # [ 168.093177] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355186910lb # [ 168.109975] niks3-server[1130]: 2026/09/25 16:34:11 INFO Received push request method=POST path=/api/pushes6911worker1 # [ 168.099534] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6912worker1 # [ 168.101870] nix-daemon[870]: reaped child process 1864, status = succeeded6913worker2 # [ 168.098140] nix-grpc-daemon[1728]: time=2026-09-25T16:34:11.271Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6914worker2 # [ 168.099962] nix-grpc-daemon[1728]: time=2026-09-25T16:34:11.273Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"6915worker1 # [ 168.106867] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6916client # [ 168.101628] nix[1525]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...6917lb # [ 168.132082] niks3-server[1130]: 2026/09/25 16:34:11 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6918lb # [ 168.134336] niks3-server[1130]: 2026/09/25 16:34:11 INFO Signed narinfos id=63 count=16919worker2 # [ 168.119658] nix-grpc-daemon[1728]: time=2026-09-25T16:34:11.293Z level=INFO msg="Uploading 1 narinfos"6920lb # [ 168.138812] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6921lb # [ 168.141292] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls6922lb # [ 168.146755] niks3-server[1130]: 2026/09/25 16:34:11 INFO Received complete push request method=POST path=/api/pushes/63/complete6923lb # [ 168.149449] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6924lb # [ 168.156198] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo6925worker2 # [ 168.148913] nix-grpc-daemon[1728]: time=2026-09-25T16:34:11.322Z level=INFO msg="Upload complete. (155ms)"6926worker2 # [ 168.151533] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=2 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06927worker1 # [ 168.156262] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:10Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=16928worker2 # [ 168.158328] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946929worker2 # [ 168.161774] nix-daemon[861]: reaped child process 1894, status = succeeded6930worker2 # [ 168.165463] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505946931client # [ 168.169238] nix[1525]: copying 1 paths to 'lb:50051'...6932worker2 # [ 168.171056] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505946933client # [ 168.171347] nix[1525]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...6934worker2 # [ 168.178686] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6935worker2 # [ 168.262278] nix-daemon[861]: accepted connection from pid 1924, user nix-grpc-daemon (trusted)6936worker2 # [ 168.274870] nix-daemon[861]: reaped child process 1927, status = succeeded6937lb # [ 168.298372] niks3-server[1130]: 2026/09/25 16:34:11 INFO Received push request method=POST path=/api/pushes6938worker2 # [ 168.291878] nix-grpc-daemon[1728]: time=2026-09-25T16:34:11.465Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6939worker2 # [ 168.293771] nix-grpc-daemon[1728]: time=2026-09-25T16:34:11.467Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"6940lb # [ 168.332829] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls6941lb # [ 168.336411] niks3-server[1130]: 2026/09/25 16:34:11 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6942lb # [ 168.340717] niks3-server[1130]: 2026/09/25 16:34:11 INFO Signed narinfos id=64 count=16943worker2 # [ 168.326708] nix-grpc-daemon[1728]: time=2026-09-25T16:34:11.500Z level=INFO msg="Uploading 1 narinfos"6944lb # [ 168.347340] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst6945lb # [ 168.356204] niks3-server[1130]: 2026/09/25 16:34:11 INFO Received complete push request method=POST path=/api/pushes/64/complete6946worker2 # [ 168.344858] nix-grpc-daemon[1728]: time=2026-09-25T16:34:11.518Z level=INFO msg="Upload complete. (160ms)"6947lb # [ 168.362775] niks3-server[1130]: 2026/09/25 16:34:11 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo6948worker2 # [ 168.347718] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 paths=1 nar_bytes_in=4166949worker2 # [ 168.353078] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505946950worker2 # [ 168.359693] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6951worker2 # [ 168.368270] nix-daemon[861]: reaped child process 1922, status = succeeded6952worker2 # [ 168.371089] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)6953client # [ 168.369270] nix[1525]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...6954worker1 # [ 170.336548] nix-daemon[870]: reaped child process 1893, status = succeeded6955worker2 # [ 170.411482] nix-daemon[861]: reaped child process 1931, status = succeeded6956worker1 # [ 170.419990] nix-daemon[870]: accepted connection from pid 1918, user nix-grpc-daemon (trusted)6957worker1 # [ 170.431226] nix-daemon[870]: reaped child process 1921, status = succeeded6958lb # [ 170.452484] niks3-server[1130]: 2026/09/25 16:34:13 INFO Received push request method=POST path=/api/pushes6959worker1 # [ 170.450443] nix-grpc-daemon[1866]: time=2026-09-25T16:34:12.748Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6960worker1 # [ 170.452030] nix-grpc-daemon[1866]: time=2026-09-25T16:34:12.750Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"6961lb # [ 170.493330] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6962lb # [ 170.503967] niks3-server[1130]: 2026/09/25 16:34:13 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6963worker2 # [ 170.492224] nix-daemon[861]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)6964worker1 # [ 170.499981] nix-grpc-daemon[1866]: time=2026-09-25T16:34:12.798Z level=INFO msg="Uploading 1 narinfos"6965lb # [ 170.512445] niks3-server[1130]: 2026/09/25 16:34:13 INFO Signed narinfos id=65 count=16966worker2 # [ 170.504680] nix-daemon[861]: reaped child process 1959, status = succeeded6967lb # [ 170.519098] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6968lb # [ 170.526314] niks3-server[1130]: 2026/09/25 16:34:13 INFO Received push request method=POST path=/api/pushes6969lb # [ 170.532810] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls6970lb # [ 170.535864] niks3-server[1130]: 2026/09/25 16:34:13 INFO Received complete push request method=POST path=/api/pushes/65/complete6971worker2 # [ 170.522799] nix-grpc-daemon[1728]: time=2026-09-25T16:34:13.696Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6972lb # [ 170.541588] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo6973worker1 # [ 170.527707] nix-grpc-daemon[1866]: time=2026-09-25T16:34:12.826Z level=INFO msg="Upload complete. (211ms)"6974worker2 # [ 170.527326] nix-grpc-daemon[1728]: time=2026-09-25T16:34:13.698Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"6975worker1 # [ 170.536577] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06976worker1 # [ 170.545679] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:12Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=16977lb # [ 170.562934] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6978worker1 # [ 170.549949] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355186979lb # [ 170.569320] niks3-server[1130]: 2026/09/25 16:34:13 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6980lb # [ 170.573170] niks3-server[1130]: 2026/09/25 16:34:13 INFO Signed narinfos id=66 count=16981worker2 # [ 170.558590] nix-grpc-daemon[1728]: time=2026-09-25T16:34:13.732Z level=INFO msg="Uploading 1 narinfos"6982lb # [ 170.574934] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls6983worker1 # [ 170.562110] nix-daemon[870]: reaped child process 1891, status = succeeded6984worker1 # [ 170.564150] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355186985lb # [ 170.578792] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6986lb # [ 170.585044] niks3-server[1130]: 2026/09/25 16:34:13 INFO Received complete push request method=POST path=/api/pushes/66/complete6987worker2 # [ 170.573093] nix-grpc-daemon[1728]: time=2026-09-25T16:34:13.747Z level=INFO msg="Upload complete. (193ms)"6988lb # [ 170.590509] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo6989client # [ 170.572829] nix[1525]: copying 1 paths to 'lb:50051'...6990worker1 # [ 170.578224] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355186991worker2 # [ 170.576060] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06992client # [ 170.576130] nix[1525]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...6993worker2 # [ 170.583139] nix-daemon[861]: reaped child process 1929, status = succeeded6994worker1 # [ 170.590200] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6995worker1 # [ 170.677940] nix-daemon[870]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)6996worker1 # [ 170.690648] nix-daemon[870]: reaped child process 1928, status = succeeded6997lb # [ 170.710735] niks3-server[1130]: 2026/09/25 16:34:13 INFO Received push request method=POST path=/api/pushes6998worker1 # [ 170.705921] nix-grpc-daemon[1866]: time=2026-09-25T16:34:13.004Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6999worker1 # [ 170.707481] nix-grpc-daemon[1866]: time=2026-09-25T16:34:13.006Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"7000lb # [ 170.741741] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst7001lb # [ 170.749361] niks3-server[1130]: 2026/09/25 16:34:13 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign7002worker1 # [ 170.742968] nix-grpc-daemon[1866]: time=2026-09-25T16:34:13.041Z level=INFO msg="Uploading 1 narinfos"7003lb # [ 170.755789] niks3-server[1130]: 2026/09/25 16:34:13 INFO Signed narinfos id=67 count=17004lb # [ 170.767501] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls7005lb # [ 170.773077] niks3-server[1130]: 2026/09/25 16:34:13 INFO Received complete push request method=POST path=/api/pushes/67/complete7006worker1 # [ 170.767825] nix-grpc-daemon[1866]: time=2026-09-25T16:34:13.066Z level=INFO msg="Upload complete. (169ms)"7007lb # [ 170.782279] niks3-server[1130]: 2026/09/25 16:34:13 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo7008worker1 # [ 170.771909] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=4167009worker1 # [ 170.777538] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187010worker1 # [ 170.785697] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7011worker1 # [ 170.794267] nix-daemon[870]: reaped child process 1923, status = succeeded7012worker1 # [ 170.798673] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7013client # [ 170.793531] nix[1525]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...7014worker1 # [ 172.641528] nix-daemon[870]: reaped child process 1932, status = succeeded7015worker1 # [ 172.716980] nix-daemon[870]: accepted connection from pid 1957, user nix-grpc-daemon (trusted)7016worker1 # [ 172.729443] nix-daemon[870]: reaped child process 1960, status = succeeded7017lb # [ 172.750494] niks3-server[1130]: 2026/09/25 16:34:15 INFO Received push request method=POST path=/api/pushes7018worker1 # [ 172.744958] nix-grpc-daemon[1866]: time=2026-09-25T16:34:15.043Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7019worker1 # [ 172.746454] nix-grpc-daemon[1866]: time=2026-09-25T16:34:15.045Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"7020lb # [ 172.786143] niks3-server[1130]: 2026/09/25 16:34:15 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls7021lb # [ 172.797120] niks3-server[1130]: 2026/09/25 16:34:15 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign7022worker1 # [ 172.788419] nix-grpc-daemon[1866]: time=2026-09-25T16:34:15.086Z level=INFO msg="Uploading 1 narinfos"7023lb # [ 172.801790] niks3-server[1130]: 2026/09/25 16:34:15 INFO Signed narinfos id=68 count=17024lb # [ 172.810816] niks3-server[1130]: 2026/09/25 16:34:15 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7025lb # [ 172.817817] niks3-server[1130]: 2026/09/25 16:34:15 INFO Received complete push request method=POST path=/api/pushes/68/complete7026lb # [ 172.826964] niks3-server[1130]: 2026/09/25 16:34:15 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst7027lb # [ 172.833135] niks3-server[1130]: 2026/09/25 16:34:15 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo7028worker1 # [ 172.818732] nix-grpc-daemon[1866]: time=2026-09-25T16:34:15.117Z level=INFO msg="Upload complete. (212ms)"7029worker1 # [ 172.825636] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=07030worker1 # [ 172.841251] nix-daemon[870]: reaped child process 1930, status = succeeded7031client # [ 172.856519] systemd[1]: burst.service: Deactivated successfully.7032client # [ 172.860541] systemd[1]: burst.service: Consumed 157ms CPU time over 6.316s wall clock time, 51.9M memory peak, 14K incoming IP traffic, 18.7K outgoing IP traffic.7033client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }7034client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.05 seconds)7035lb: must succeed: curl -sf localhost:9901/clusters7036lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7037lb: must succeed: curl -sf localhost:9901/clusters7038lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7039worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7040worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7041lb: must succeed: curl -sf localhost:9901/clusters7042lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7043lb: must succeed: curl -sf localhost:9901/clusters7044lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7045worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7046worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7047(finished: subtest: queue gauges show a burst while every build blocks, in 7.52 seconds)7048subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7049client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17050worker1 # [ 174.065069] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355187051worker2 # [ 174.065859] nix-grpc-daemon[1636]: ts=2026-09-25T16:34:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505947052worker1 # [ 174.109985] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=debug event=want client=2 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07053worker1 # [ 174.116388] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17054worker1 # [ 174.121215] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=debug event=want client=2 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07055worker1 # [ 174.125111] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=debug event=want client=2 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07056worker1 # [ 174.128973] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187057worker1 # [ 174.139566] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187058worker1 # [ 174.151909] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355187059worker1 # [ 174.162221] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7060worker1 # [ 174.257884] nix-daemon[870]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7061worker1 # [ 174.270473] nix-daemon[870]: reaped child process 1979, status = succeeded7062lb # [ 174.290776] niks3-server[1130]: 2026/09/25 16:34:17 INFO Received push request method=POST path=/api/pushes7063worker1 # [ 174.286880] nix-grpc-daemon[1866]: time=2026-09-25T16:34:16.585Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7064worker1 # [ 174.288874] nix-grpc-daemon[1866]: time=2026-09-25T16:34:16.586Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7065lb # [ 174.323198] niks3-server[1130]: 2026/09/25 16:34:17 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign7066lb # [ 174.326569] niks3-server[1130]: 2026/09/25 16:34:17 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7067worker1 # [ 174.322388] nix-grpc-daemon[1866]: time=2026-09-25T16:34:16.620Z level=INFO msg="Uploading 1 narinfos"7068lb # [ 174.334860] niks3-server[1130]: 2026/09/25 16:34:17 INFO Signed narinfos id=69 count=17069lb # [ 174.340990] niks3-server[1130]: 2026/09/25 16:34:17 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7070lb # [ 174.350333] niks3-server[1130]: 2026/09/25 16:34:17 INFO Received complete push request method=POST path=/api/pushes/69/complete7071worker1 # [ 174.342046] nix-grpc-daemon[1866]: time=2026-09-25T16:34:16.640Z level=INFO msg="Upload complete. (169ms)"7072lb # [ 174.356822] niks3-server[1130]: 2026/09/25 16:34:17 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7073worker1 # [ 174.345740] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=5047074worker1 # [ 174.350642] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187075worker1 # [ 174.361136] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7076worker1 # [ 174.364875] nix-daemon[870]: reaped child process 1974, status = succeeded7077worker1 # [ 174.369662] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7078worker1 # [ 175.803612] nix-daemon[870]: reaped child process 1983, status = succeeded7079worker1 # [ 175.887925] nix-daemon[870]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)7080worker1 # [ 175.900148] nix-daemon[870]: reaped child process 2002, status = succeeded7081lb # [ 175.921476] niks3-server[1130]: 2026/09/25 16:34:19 INFO Received push request method=POST path=/api/pushes7082worker1 # [ 175.917821] nix-grpc-daemon[1866]: time=2026-09-25T16:34:18.216Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7083worker1 # [ 175.919638] nix-grpc-daemon[1866]: time=2026-09-25T16:34:18.217Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7084lb # [ 175.962127] niks3-server[1130]: 2026/09/25 16:34:19 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign7085worker1 # [ 175.953715] nix-grpc-daemon[1866]: time=2026-09-25T16:34:18.252Z level=INFO msg="Uploading 1 narinfos"7086lb # [ 175.967094] niks3-server[1130]: 2026/09/25 16:34:19 INFO Signed narinfos id=70 count=17087lb # [ 175.977064] niks3-server[1130]: 2026/09/25 16:34:19 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7088lb # [ 175.985119] niks3-server[1130]: 2026/09/25 16:34:19 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7089lb # [ 175.990843] niks3-server[1130]: 2026/09/25 16:34:19 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7090lb # [ 175.999067] niks3-server[1130]: 2026/09/25 16:34:19 INFO Received complete push request method=POST path=/api/pushes/70/complete7091worker1 # [ 175.990434] nix-grpc-daemon[1866]: time=2026-09-25T16:34:18.288Z level=INFO msg="Upload complete. (210ms)"7092lb # [ 176.004411] niks3-server[1130]: 2026/09/25 16:34:19 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7093worker1 # [ 176.001183] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=07094worker1 # [ 176.009838] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:18Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17095worker1 # [ 176.014660] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187096worker1 # [ 176.018187] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:18Z level=debug event=want client=2 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07097worker1 # [ 176.028284] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187098worker1 # [ 176.031220] nix-daemon[870]: reaped child process 1981, status = succeeded7099worker1 # [ 176.035877] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355187100worker1 # [ 176.039432] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7101worker1 # [ 176.123535] nix-daemon[870]: accepted connection from pid 2008, user nix-grpc-daemon (trusted)7102worker1 # [ 176.137474] nix-daemon[870]: reaped child process 2011, status = succeeded7103lb # [ 176.155205] niks3-server[1130]: 2026/09/25 16:34:19 INFO Received push request method=POST path=/api/pushes7104worker1 # [ 176.150317] nix-grpc-daemon[1866]: time=2026-09-25T16:34:18.448Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7105worker1 # [ 176.152440] nix-grpc-daemon[1866]: time=2026-09-25T16:34:18.450Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7106lb # [ 176.183227] niks3-server[1130]: 2026/09/25 16:34:19 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign7107lb # [ 176.188471] niks3-server[1130]: 2026/09/25 16:34:19 INFO Signed narinfos id=71 count=17108worker1 # [ 176.178568] nix-grpc-daemon[1866]: time=2026-09-25T16:34:18.477Z level=INFO msg="Uploading 1 narinfos"7109lb # [ 176.198515] niks3-server[1130]: 2026/09/25 16:34:19 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7110lb # [ 176.207878] niks3-server[1130]: 2026/09/25 16:34:19 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7111lb # [ 176.215084] niks3-server[1130]: 2026/09/25 16:34:19 INFO Received complete push request method=POST path=/api/pushes/71/complete7112lb # [ 176.225669] niks3-server[1130]: 2026/09/25 16:34:19 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7113worker1 # [ 176.211988] nix-grpc-daemon[1866]: time=2026-09-25T16:34:18.510Z level=INFO msg="Upload complete. (167ms)"7114worker1 # [ 176.216319] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=5047115worker1 # [ 176.221328] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187116worker1 # [ 176.230536] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7117worker1 # [ 176.236189] nix-daemon[870]: reaped child process 2006, status = succeeded7118worker1 # [ 176.241535] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7119lb # [ 177.079876] envoy[922]: conn peer=[2001:db8:1::1]:48862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9664 tx=5201 ms=61577120worker1 # [ 177.479526] nix-daemon[870]: reaped child process 2015, status = succeeded7121worker1 # [ 177.560583] nix-daemon[870]: accepted connection from pid 2031, user nix-grpc-daemon (trusted)7122worker1 # [ 177.572831] nix-daemon[870]: reaped child process 2034, status = succeeded7123lb # [ 177.594573] niks3-server[1130]: 2026/09/25 16:34:20 INFO Received push request method=POST path=/api/pushes7124worker1 # [ 177.589940] nix-grpc-daemon[1866]: time=2026-09-25T16:34:19.888Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7125worker1 # [ 177.591440] nix-grpc-daemon[1866]: time=2026-09-25T16:34:19.890Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7126lb # [ 177.637610] niks3-server[1130]: 2026/09/25 16:34:20 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7127lb # [ 177.643536] niks3-server[1130]: 2026/09/25 16:34:20 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7128lb # [ 177.650114] niks3-server[1130]: 2026/09/25 16:34:20 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign7129lb # [ 177.654146] niks3-server[1130]: 2026/09/25 16:34:20 INFO Signed narinfos id=72 count=17130worker1 # [ 177.644466] nix-grpc-daemon[1866]: time=2026-09-25T16:34:19.942Z level=INFO msg="Uploading 1 narinfos"7131lb # [ 177.663987] niks3-server[1130]: 2026/09/25 16:34:20 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7132lb # [ 177.675469] niks3-server[1130]: 2026/09/25 16:34:20 INFO Received complete push request method=POST path=/api/pushes/72/complete7133worker1 # [ 177.666689] nix-grpc-daemon[1866]: time=2026-09-25T16:34:19.965Z level=INFO msg="Upload complete. (218ms)"7134lb # [ 177.681539] niks3-server[1130]: 2026/09/25 16:34:20 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7135worker1 # [ 177.672763] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=6 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=07136worker1 # [ 177.685633] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:19Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17137worker1 # [ 177.690230] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187138worker1 # [ 177.694238] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:19Z level=debug event=want client=2 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07139worker1 # [ 177.704276] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187140worker1 # [ 177.706512] nix-daemon[870]: reaped child process 2013, status = succeeded7141worker1 # [ 177.710931] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355187142worker1 # [ 177.714677] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7143worker1 # [ 177.799483] nix-daemon[870]: accepted connection from pid 2040, user nix-grpc-daemon (trusted)7144worker1 # [ 177.811702] nix-daemon[870]: reaped child process 2043, status = succeeded7145lb # [ 177.832398] niks3-server[1130]: 2026/09/25 16:34:20 INFO Received push request method=POST path=/api/pushes7146worker1 # [ 177.828518] nix-grpc-daemon[1866]: time=2026-09-25T16:34:20.127Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7147worker1 # [ 177.830025] nix-grpc-daemon[1866]: time=2026-09-25T16:34:20.128Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7148lb # [ 177.864822] niks3-server[1130]: 2026/09/25 16:34:21 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7149lb # [ 177.870790] niks3-server[1130]: 2026/09/25 16:34:21 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign7150lb # [ 177.877233] niks3-server[1130]: 2026/09/25 16:34:21 INFO Signed narinfos id=73 count=17151worker1 # [ 177.867580] nix-grpc-daemon[1866]: time=2026-09-25T16:34:20.166Z level=INFO msg="Uploading 1 narinfos"7152lb # [ 177.888945] niks3-server[1130]: 2026/09/25 16:34:21 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7153lb # [ 177.899136] niks3-server[1130]: 2026/09/25 16:34:21 INFO Received complete push request method=POST path=/api/pushes/73/complete7154worker1 # [ 177.893174] nix-grpc-daemon[1866]: time=2026-09-25T16:34:20.191Z level=INFO msg="Upload complete. (173ms)"7155lb # [ 177.907764] niks3-server[1130]: 2026/09/25 16:34:21 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7156worker1 # [ 177.897182] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=5047157worker1 # [ 177.903250] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187158worker1 # [ 177.912484] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7159worker1 # [ 177.918878] nix-daemon[870]: reaped child process 2038, status = succeeded7160worker1 # [ 177.923638] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7161worker1 # [ 179.338253] nix-daemon[870]: reaped child process 2047, status = succeeded7162worker1 # [ 179.415024] nix-daemon[870]: accepted connection from pid 2063, user nix-grpc-daemon (trusted)7163worker1 # [ 179.426145] nix-daemon[870]: reaped child process 2066, status = succeeded7164lb # [ 179.447473] niks3-server[1130]: 2026/09/25 16:34:22 INFO Received push request method=POST path=/api/pushes7165worker1 # [ 179.443039] nix-grpc-daemon[1866]: time=2026-09-25T16:34:21.741Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7166worker1 # [ 179.444511] nix-grpc-daemon[1866]: time=2026-09-25T16:34:21.743Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7167lb # [ 179.493297] niks3-server[1130]: 2026/09/25 16:34:22 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7168lb # [ 179.496643] niks3-server[1130]: 2026/09/25 16:34:22 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign7169lb # [ 179.501877] niks3-server[1130]: 2026/09/25 16:34:22 INFO Signed narinfos id=74 count=17170worker1 # [ 179.492033] nix-grpc-daemon[1866]: time=2026-09-25T16:34:21.790Z level=INFO msg="Uploading 1 narinfos"7171lb # [ 179.510580] niks3-server[1130]: 2026/09/25 16:34:22 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7172lb # [ 179.519596] niks3-server[1130]: 2026/09/25 16:34:22 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7173lb # [ 179.528087] niks3-server[1130]: 2026/09/25 16:34:22 INFO Received complete push request method=POST path=/api/pushes/74/complete7174lb # [ 179.535956] niks3-server[1130]: 2026/09/25 16:34:22 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7175worker1 # [ 179.522455] nix-grpc-daemon[1866]: time=2026-09-25T16:34:21.820Z level=INFO msg="Upload complete. (192ms)"7176worker1 # [ 179.532487] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=7 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=07177worker1 # [ 179.542481] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:21Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17178worker1 # [ 179.547202] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187179worker1 # [ 179.550589] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:21Z level=debug event=want client=2 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07180worker1 # [ 179.561422] nix-daemon[870]: reaped child process 2045, status = succeeded7181worker1 # [ 179.562825] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187182worker1 # [ 179.570419] nix-grpc-daemon[1866]: time=2026-09-25T16:34:21.868Z level=INFO msg="All 1 paths already cached"7183worker1 # [ 179.574884] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355187184worker1 # [ 179.579357] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7185worker1 # [ 179.664106] nix-daemon[870]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)7186worker1 # [ 179.677694] nix-daemon[870]: reaped child process 2075, status = succeeded7187lb # [ 179.696421] niks3-server[1130]: 2026/09/25 16:34:22 INFO Received push request method=POST path=/api/pushes7188worker1 # [ 179.690835] nix-grpc-daemon[1866]: time=2026-09-25T16:34:21.989Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7189worker1 # [ 179.692638] nix-grpc-daemon[1866]: time=2026-09-25T16:34:21.990Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7190lb # [ 179.724812] niks3-server[1130]: 2026/09/25 16:34:22 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7191lb # [ 179.732172] niks3-server[1130]: 2026/09/25 16:34:22 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign7192worker1 # [ 179.723962] nix-grpc-daemon[1866]: time=2026-09-25T16:34:22.022Z level=INFO msg="Uploading 1 narinfos"7193lb # [ 179.738080] niks3-server[1130]: 2026/09/25 16:34:22 INFO Signed narinfos id=75 count=17194lb # [ 179.753930] niks3-server[1130]: 2026/09/25 16:34:22 INFO Received complete push request method=POST path=/api/pushes/75/complete7195lb # [ 179.756760] niks3-server[1130]: 2026/09/25 16:34:22 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7196worker1 # [ 179.750896] nix-grpc-daemon[1866]: time=2026-09-25T16:34:22.049Z level=INFO msg="Upload complete. (166ms)"7197lb # [ 179.766260] niks3-server[1130]: 2026/09/25 16:34:22 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7198worker1 # [ 179.755104] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=6247199worker1 # [ 179.759922] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187200worker1 # [ 179.769259] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7201worker1 # [ 179.775142] nix-daemon[870]: reaped child process 2070, status = succeeded7202worker1 # [ 179.778514] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7203worker1 # [ 180.992756] nix-daemon[870]: reaped child process 2079, status = succeeded7204worker1 # [ 181.068067] nix-daemon[870]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)7205worker1 # [ 181.080295] nix-daemon[870]: reaped child process 2098, status = succeeded7206lb # [ 181.101826] niks3-server[1130]: 2026/09/25 16:34:24 INFO Received push request method=POST path=/api/pushes7207worker1 # [ 181.097085] nix-grpc-daemon[1866]: time=2026-09-25T16:34:23.395Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7208worker1 # [ 181.098616] nix-grpc-daemon[1866]: time=2026-09-25T16:34:23.397Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7209lb # [ 181.142989] niks3-server[1130]: 2026/09/25 16:34:24 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign7210lb # [ 181.147201] niks3-server[1130]: 2026/09/25 16:34:24 INFO Signed narinfos id=76 count=17211worker1 # [ 181.136999] nix-grpc-daemon[1866]: time=2026-09-25T16:34:23.435Z level=INFO msg="Uploading 1 narinfos"7212lb # [ 181.153960] niks3-server[1130]: 2026/09/25 16:34:24 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7213lb # [ 181.158652] niks3-server[1130]: 2026/09/25 16:34:24 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7214lb # [ 181.166267] niks3-server[1130]: 2026/09/25 16:34:24 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7215lb # [ 181.175389] niks3-server[1130]: 2026/09/25 16:34:24 INFO Received complete push request method=POST path=/api/pushes/76/complete7216worker1 # [ 181.166780] nix-grpc-daemon[1866]: time=2026-09-25T16:34:23.465Z level=INFO msg="Upload complete. (181ms)"7217lb # [ 181.181159] niks3-server[1130]: 2026/09/25 16:34:24 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7218worker1 # [ 181.175711] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=8 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07219worker1 # [ 181.181000] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:23Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17220worker1 # [ 181.188634] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187221worker1 # [ 181.192183] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:23Z level=debug event=want client=2 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07222worker1 # [ 181.201290] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187223worker1 # [ 181.203253] nix-daemon[870]: reaped child process 2077, status = succeeded7224worker1 # [ 181.208839] nix-grpc-daemon[1866]: time=2026-09-25T16:34:23.507Z level=INFO msg="All 1 paths already cached"7225worker1 # [ 181.213136] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355187226worker1 # [ 181.217667] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7227worker1 # [ 181.301253] nix-daemon[870]: accepted connection from pid 2104, user nix-grpc-daemon (trusted)7228worker1 # [ 181.313398] nix-daemon[870]: reaped child process 2107, status = succeeded7229lb # [ 181.334542] niks3-server[1130]: 2026/09/25 16:34:24 INFO Received push request method=POST path=/api/pushes7230worker1 # [ 181.329191] nix-grpc-daemon[1866]: time=2026-09-25T16:34:23.627Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7231worker1 # [ 181.331224] nix-grpc-daemon[1866]: time=2026-09-25T16:34:23.629Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7232lb # [ 181.364362] niks3-server[1130]: 2026/09/25 16:34:24 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign7233lb # [ 181.369546] niks3-server[1130]: 2026/09/25 16:34:24 INFO Signed narinfos id=77 count=17234worker1 # [ 181.359648] nix-grpc-daemon[1866]: time=2026-09-25T16:34:23.658Z level=INFO msg="Uploading 1 narinfos"7235lb # [ 181.377642] niks3-server[1130]: 2026/09/25 16:34:24 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7236lb # [ 181.389824] niks3-server[1130]: 2026/09/25 16:34:24 INFO Received complete push request method=POST path=/api/pushes/77/complete7237lb # [ 181.396735] niks3-server[1130]: 2026/09/25 16:34:24 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7238worker1 # [ 181.389065] nix-grpc-daemon[1866]: time=2026-09-25T16:34:23.687Z level=INFO msg="Upload complete. (167ms)"7239lb # [ 181.406888] niks3-server[1130]: 2026/09/25 16:34:24 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7240worker1 # [ 181.393061] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=6247241worker1 # [ 181.398193] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187242worker1 # [ 181.408060] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7243worker1 # [ 181.413122] nix-daemon[870]: reaped child process 2102, status = succeeded7244worker1 # [ 181.416363] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7245worker1 # [ 182.455073] nix-daemon[870]: reaped child process 2111, status = succeeded7246worker1 # [ 182.533736] nix-daemon[870]: accepted connection from pid 2127, user nix-grpc-daemon (trusted)7247worker1 # [ 182.547462] nix-daemon[870]: reaped child process 2130, status = succeeded7248lb # [ 182.566285] niks3-server[1130]: 2026/09/25 16:34:25 INFO Received push request method=POST path=/api/pushes7249worker1 # [ 182.561331] nix-grpc-daemon[1866]: time=2026-09-25T16:34:24.859Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7250worker1 # [ 182.562987] nix-grpc-daemon[1866]: time=2026-09-25T16:34:24.861Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7251lb # [ 182.610333] niks3-server[1130]: 2026/09/25 16:34:25 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7252lb # [ 182.618067] niks3-server[1130]: 2026/09/25 16:34:25 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign7253worker1 # [ 182.610873] nix-grpc-daemon[1866]: time=2026-09-25T16:34:24.909Z level=INFO msg="Uploading 1 narinfos"7254lb # [ 182.623130] niks3-server[1130]: 2026/09/25 16:34:25 INFO Signed narinfos id=78 count=17255lb # [ 182.632529] niks3-server[1130]: 2026/09/25 16:34:25 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7256lb # [ 182.643401] niks3-server[1130]: 2026/09/25 16:34:25 INFO Received complete push request method=POST path=/api/pushes/78/complete7257lb # [ 182.647592] niks3-server[1130]: 2026/09/25 16:34:25 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7258worker1 # [ 182.639138] nix-grpc-daemon[1866]: time=2026-09-25T16:34:24.937Z level=INFO msg="Upload complete. (214ms)"7259lb # [ 182.654304] niks3-server[1130]: 2026/09/25 16:34:25 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7260worker1 # [ 182.651266] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=9 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07261worker1 # [ 182.659354] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:24Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17262worker1 # [ 182.664186] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187263worker1 # [ 182.667324] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:24Z level=debug event=want client=2 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07264worker1 # [ 182.676502] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187265worker1 # [ 182.679947] nix-daemon[870]: reaped child process 2109, status = succeeded7266worker1 # [ 182.684537] nix-grpc-daemon[1866]: time=2026-09-25T16:34:24.983Z level=INFO msg="All 1 paths already cached"7267worker1 # [ 182.688781] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355187268worker1 # [ 182.692716] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7269worker1 # [ 182.777187] nix-daemon[870]: accepted connection from pid 2136, user nix-grpc-daemon (trusted)7270worker1 # [ 182.790545] nix-daemon[870]: reaped child process 2139, status = succeeded7271lb # [ 182.809528] niks3-server[1130]: 2026/09/25 16:34:25 INFO Received push request method=POST path=/api/pushes7272worker1 # [ 182.804378] nix-grpc-daemon[1866]: time=2026-09-25T16:34:25.102Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7273worker1 # [ 182.805859] nix-grpc-daemon[1866]: time=2026-09-25T16:34:25.104Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7274lb # [ 182.842897] niks3-server[1130]: 2026/09/25 16:34:26 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign7275worker1 # [ 182.835337] nix-grpc-daemon[1866]: time=2026-09-25T16:34:25.133Z level=INFO msg="Uploading 1 narinfos"7276lb # [ 182.848659] niks3-server[1130]: 2026/09/25 16:34:26 INFO Signed narinfos id=79 count=17277lb # [ 182.851913] niks3-server[1130]: 2026/09/25 16:34:26 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7278lb # [ 182.866068] niks3-server[1130]: 2026/09/25 16:34:26 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7279lb # [ 182.869391] niks3-server[1130]: 2026/09/25 16:34:26 INFO Received complete push request method=POST path=/api/pushes/79/complete7280worker1 # [ 182.865847] nix-grpc-daemon[1866]: time=2026-09-25T16:34:25.164Z level=INFO msg="Upload complete. (167ms)"7281lb # [ 182.881203] niks3-server[1130]: 2026/09/25 16:34:26 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7282worker1 # [ 182.869819] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=6247283worker1 # [ 182.874652] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187284worker1 # [ 182.886136] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7285worker1 # [ 182.891125] nix-daemon[870]: reaped child process 2134, status = succeeded7286worker1 # [ 182.894145] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7287worker1 # [ 184.109659] nix-daemon[870]: reaped child process 2143, status = succeeded7288worker1 # [ 184.186477] nix-daemon[870]: accepted connection from pid 2159, user nix-grpc-daemon (trusted)7289worker1 # [ 184.198817] nix-daemon[870]: reaped child process 2162, status = succeeded7290lb # [ 184.219926] niks3-server[1130]: 2026/09/25 16:34:27 INFO Received push request method=POST path=/api/pushes7291worker1 # [ 184.215888] nix-grpc-daemon[1866]: time=2026-09-25T16:34:26.514Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7292worker1 # [ 184.217451] nix-grpc-daemon[1866]: time=2026-09-25T16:34:26.516Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7293lb # [ 184.270237] niks3-server[1130]: 2026/09/25 16:34:27 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign7294lb # [ 184.271780] niks3-server[1130]: 2026/09/25 16:34:27 INFO Signed narinfos id=80 count=17295worker1 # [ 184.260360] nix-grpc-daemon[1866]: time=2026-09-25T16:34:26.558Z level=INFO msg="Uploading 1 narinfos"7296lb # [ 184.277289] niks3-server[1130]: 2026/09/25 16:34:27 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7297lb # [ 184.280566] niks3-server[1130]: 2026/09/25 16:34:27 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7298lb # [ 184.283228] niks3-server[1130]: 2026/09/25 16:34:27 INFO Received complete push request method=POST path=/api/pushes/80/complete7299lb # [ 184.288514] niks3-server[1130]: 2026/09/25 16:34:27 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7300lb # [ 184.291545] niks3-server[1130]: 2026/09/25 16:34:27 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7301worker1 # [ 184.277853] nix-grpc-daemon[1866]: time=2026-09-25T16:34:26.576Z level=INFO msg="Upload complete. (176ms)"7302worker1 # [ 184.287347] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=10 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07303worker1 # [ 184.294599] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:26Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17304worker1 # [ 184.298219] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187305worker1 # [ 184.300099] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:26Z level=debug event=want client=2 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07306worker1 # [ 184.306862] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187307worker1 # [ 184.308972] nix-daemon[870]: reaped child process 2141, status = succeeded7308worker1 # [ 184.312850] nix-grpc-daemon[1866]: time=2026-09-25T16:34:26.611Z level=INFO msg="All 2 paths already cached"7309worker1 # [ 184.316701] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355187310worker1 # [ 184.320415] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7311worker1 # [ 184.405167] nix-daemon[870]: accepted connection from pid 2168, user nix-grpc-daemon (trusted)7312worker1 # [ 184.417436] nix-daemon[870]: reaped child process 2171, status = succeeded7313lb # [ 184.438742] niks3-server[1130]: 2026/09/25 16:34:27 INFO Received push request method=POST path=/api/pushes7314worker1 # [ 184.433622] nix-grpc-daemon[1866]: time=2026-09-25T16:34:26.732Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7315worker1 # [ 184.435626] nix-grpc-daemon[1866]: time=2026-09-25T16:34:26.733Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7316lb # [ 184.467346] niks3-server[1130]: 2026/09/25 16:34:27 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign7317lb # [ 184.474552] niks3-server[1130]: 2026/09/25 16:34:27 INFO Signed narinfos id=81 count=17318worker1 # [ 184.464132] nix-grpc-daemon[1866]: time=2026-09-25T16:34:26.762Z level=INFO msg="Uploading 1 narinfos"7319lb # [ 184.479083] niks3-server[1130]: 2026/09/25 16:34:27 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7320lb # [ 184.489696] niks3-server[1130]: 2026/09/25 16:34:27 INFO Received complete push request method=POST path=/api/pushes/81/complete7321lb # [ 184.493445] niks3-server[1130]: 2026/09/25 16:34:27 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7322lb # [ 184.497382] niks3-server[1130]: 2026/09/25 16:34:27 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7323worker1 # [ 184.489156] nix-grpc-daemon[1866]: time=2026-09-25T16:34:26.787Z level=INFO msg="Upload complete. (163ms)"7324worker1 # [ 184.492783] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=7527325worker1 # [ 184.497652] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187326worker1 # [ 184.507477] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7327worker1 # [ 184.511128] nix-daemon[870]: reaped child process 2166, status = succeeded7328worker1 # [ 184.516047] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7329worker1 # [ 185.551169] nix-daemon[870]: reaped child process 2175, status = succeeded7330worker1 # [ 185.636285] nix-daemon[870]: accepted connection from pid 2191, user nix-grpc-daemon (trusted)7331worker1 # [ 185.650828] nix-daemon[870]: reaped child process 2194, status = succeeded7332lb # [ 185.669925] niks3-server[1130]: 2026/09/25 16:34:28 INFO Received push request method=POST path=/api/pushes7333worker1 # [ 185.666341] nix-grpc-daemon[1866]: time=2026-09-25T16:34:27.964Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7334worker1 # [ 185.667893] nix-grpc-daemon[1866]: time=2026-09-25T16:34:27.966Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7335lb # [ 185.708800] niks3-server[1130]: 2026/09/25 16:34:28 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7336lb # [ 185.715624] niks3-server[1130]: 2026/09/25 16:34:28 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign7337lb # [ 185.720515] niks3-server[1130]: 2026/09/25 16:34:28 INFO Signed narinfos id=82 count=17338worker1 # [ 185.710651] nix-grpc-daemon[1866]: time=2026-09-25T16:34:28.009Z level=INFO msg="Uploading 1 narinfos"7339lb # [ 185.727629] niks3-server[1130]: 2026/09/25 16:34:28 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7340lb # [ 185.735796] niks3-server[1130]: 2026/09/25 16:34:28 INFO Received complete push request method=POST path=/api/pushes/82/complete7341lb # [ 185.738119] niks3-server[1130]: 2026/09/25 16:34:28 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7342lb # [ 185.744705] niks3-server[1130]: 2026/09/25 16:34:28 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7343worker1 # [ 185.730588] nix-grpc-daemon[1866]: time=2026-09-25T16:34:28.029Z level=INFO msg="Upload complete. (205ms)"7344worker1 # [ 185.739062] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=11 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07345worker1 # [ 185.745522] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:28Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17346worker1 # [ 185.749577] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187347worker1 # [ 185.759302] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187348worker1 # [ 185.762583] nix-daemon[870]: reaped child process 2173, status = succeeded7349worker1 # [ 185.766583] nix-grpc-daemon[1866]: time=2026-09-25T16:34:28.065Z level=INFO msg="All 2 paths already cached"7350worker1 # [ 185.770580] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355187351worker1 # [ 185.774361] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7352worker1 # [ 185.858253] nix-daemon[870]: accepted connection from pid 2198, user nix-grpc-daemon (trusted)7353worker1 # [ 185.870584] nix-daemon[870]: reaped child process 2201, status = succeeded7354lb # [ 185.892155] niks3-server[1130]: 2026/09/25 16:34:29 INFO Received push request method=POST path=/api/pushes7355worker1 # [ 185.888375] nix-grpc-daemon[1866]: time=2026-09-25T16:34:28.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7356worker1 # [ 185.890378] nix-grpc-daemon[1866]: time=2026-09-25T16:34:28.188Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7357lb # [ 185.921683] niks3-server[1130]: 2026/09/25 16:34:29 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign7358lb # [ 185.926927] niks3-server[1130]: 2026/09/25 16:34:29 INFO Signed narinfos id=83 count=17359worker1 # [ 185.917721] nix-grpc-daemon[1866]: time=2026-09-25T16:34:28.216Z level=INFO msg="Uploading 1 narinfos"7360lb # [ 185.936129] niks3-server[1130]: 2026/09/25 16:34:29 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7361lb # [ 185.943626] niks3-server[1130]: 2026/09/25 16:34:29 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7362lb # [ 185.952593] niks3-server[1130]: 2026/09/25 16:34:29 INFO Received complete push request method=POST path=/api/pushes/83/complete7363worker1 # [ 185.942850] nix-grpc-daemon[1866]: time=2026-09-25T16:34:28.241Z level=INFO msg="Upload complete. (163ms)"7364lb # [ 185.957675] niks3-server[1130]: 2026/09/25 16:34:29 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7365worker1 # [ 185.945216] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=7527366worker1 # [ 185.950484] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187367worker1 # [ 185.958252] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7368worker1 # [ 185.963418] nix-daemon[870]: reaped child process 2196, status = succeeded7369worker1 # [ 185.967850] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7370worker1 # [ 187.408753] nix-daemon[870]: reaped child process 2205, status = succeeded7371worker1 # [ 187.487086] nix-daemon[870]: accepted connection from pid 2221, user nix-grpc-daemon (trusted)7372worker1 # [ 187.498218] nix-daemon[870]: reaped child process 2224, status = succeeded7373lb # [ 187.520340] niks3-server[1130]: 2026/09/25 16:34:30 INFO Received push request method=POST path=/api/pushes7374worker1 # [ 187.516318] nix-grpc-daemon[1866]: time=2026-09-25T16:34:29.814Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7375worker1 # [ 187.518189] nix-grpc-daemon[1866]: time=2026-09-25T16:34:29.816Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7376lb # [ 187.559861] niks3-server[1130]: 2026/09/25 16:34:30 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7377lb # [ 187.565889] niks3-server[1130]: 2026/09/25 16:34:30 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7378lb # [ 187.575123] niks3-server[1130]: 2026/09/25 16:34:30 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign7379worker1 # [ 187.567768] nix-grpc-daemon[1866]: time=2026-09-25T16:34:29.866Z level=INFO msg="Uploading 1 narinfos"7380lb # [ 187.579992] niks3-server[1130]: 2026/09/25 16:34:30 INFO Signed narinfos id=84 count=17381lb # [ 187.592242] niks3-server[1130]: 2026/09/25 16:34:30 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7382lb # [ 187.599174] niks3-server[1130]: 2026/09/25 16:34:30 INFO Received complete push request method=POST path=/api/pushes/84/complete7383lb # [ 187.603454] niks3-server[1130]: 2026/09/25 16:34:30 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7384worker1 # [ 187.593421] nix-grpc-daemon[1866]: time=2026-09-25T16:34:29.891Z level=INFO msg="Upload complete. (217ms)"7385worker1 # [ 187.602323] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=12 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07386worker1 # [ 187.609898] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:29Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17387worker1 # [ 187.614960] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187388worker1 # [ 187.626557] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187389worker1 # [ 187.632542] nix-daemon[870]: reaped child process 2203, status = succeeded7390worker1 # [ 187.639061] nix-grpc-daemon[1866]: time=2026-09-25T16:34:29.937Z level=INFO msg="All 2 paths already cached"7391worker1 # [ 187.645889] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355187392worker1 # [ 187.651853] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7393worker1 # [ 187.736759] nix-daemon[870]: accepted connection from pid 2228, user nix-grpc-daemon (trusted)7394worker1 # [ 187.750687] nix-daemon[870]: reaped child process 2231, status = succeeded7395lb # [ 187.768364] niks3-server[1130]: 2026/09/25 16:34:30 INFO Received push request method=POST path=/api/pushes7396worker1 # [ 187.763928] nix-grpc-daemon[1866]: time=2026-09-25T16:34:30.062Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7397worker1 # [ 187.765842] nix-grpc-daemon[1866]: time=2026-09-25T16:34:30.063Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7398lb # [ 187.799591] niks3-server[1130]: 2026/09/25 16:34:30 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7399lb # [ 187.807992] niks3-server[1130]: 2026/09/25 16:34:30 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign7400worker1 # [ 187.801304] nix-grpc-daemon[1866]: time=2026-09-25T16:34:30.099Z level=INFO msg="Uploading 1 narinfos"7401lb # [ 187.815355] niks3-server[1130]: 2026/09/25 16:34:30 INFO Signed narinfos id=85 count=17402lb # [ 187.821463] niks3-server[1130]: 2026/09/25 16:34:30 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7403lb # [ 187.831541] niks3-server[1130]: 2026/09/25 16:34:30 INFO Received complete push request method=POST path=/api/pushes/85/complete7404worker1 # [ 187.827849] nix-grpc-daemon[1866]: time=2026-09-25T16:34:30.126Z level=INFO msg="Upload complete. (170ms)"7405lb # [ 187.841955] niks3-server[1130]: 2026/09/25 16:34:31 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7406worker1 # [ 187.831273] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=0 paths=1 nar_bytes_in=7527407worker1 # [ 187.836587] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355187408worker1 # [ 187.846202] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7409worker1 # [ 187.853870] nix-daemon[870]: reaped child process 2226, status = succeeded7410worker1 # [ 187.858511] nix-daemon[870]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)7411worker1 # [ 189.294759] nix-daemon[870]: reaped child process 2235, status = succeeded7412worker1 # [ 189.373353] nix-daemon[870]: accepted connection from pid 2251, user nix-grpc-daemon (trusted)7413worker1 # [ 189.386342] nix-daemon[870]: reaped child process 2254, status = succeeded7414lb # [ 189.406442] niks3-server[1130]: 2026/09/25 16:34:32 INFO Received push request method=POST path=/api/pushes7415worker1 # [ 189.402372] nix-grpc-daemon[1866]: time=2026-09-25T16:34:31.700Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7416worker1 # [ 189.404543] nix-grpc-daemon[1866]: time=2026-09-25T16:34:31.702Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7417lb # [ 189.449630] niks3-server[1130]: 2026/09/25 16:34:32 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign7418worker1 # [ 189.441677] nix-grpc-daemon[1866]: time=2026-09-25T16:34:31.740Z level=INFO msg="Uploading 1 narinfos"7419lb # [ 189.454581] niks3-server[1130]: 2026/09/25 16:34:32 INFO Signed narinfos id=86 count=17420lb # [ 189.462146] niks3-server[1130]: 2026/09/25 16:34:32 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7421lb # [ 189.469690] niks3-server[1130]: 2026/09/25 16:34:32 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7422lb # [ 189.474535] niks3-server[1130]: 2026/09/25 16:34:32 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7423lb # [ 189.483236] niks3-server[1130]: 2026/09/25 16:34:32 INFO Received complete push request method=POST path=/api/pushes/86/complete7424worker1 # [ 189.479256] nix-grpc-daemon[1866]: time=2026-09-25T16:34:31.777Z level=INFO msg="Upload complete. (212ms)"7425lb # [ 189.494303] niks3-server[1130]: 2026/09/25 16:34:32 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7426worker1 # [ 189.487217] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35518 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=13 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07427worker1 # [ 189.503305] nix-daemon[870]: reaped child process 2233, status = succeeded7428client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.61 seconds)7429worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7430worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7431client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7432client # [ 189.612840] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7433client # Running as unit: gpu.service; invocation ID: 41c23845c53e41cebbefa6a6f63eed317434client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7435lb: must succeed: curl -sf localhost:9901/clusters7436lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7437lb: must succeed: curl -sf localhost:9901/clusters7438lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7439worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7440worker1 # [ 189.750732] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355187441client # [ 189.751142] nix[1599]: don't know how to build these paths:7442client # [ 189.752054] nix[1599]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7443worker1 # [ 189.757594] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355187444worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7445worker1 # [ 189.776487] nix-grpc-daemon[1795]: ts=2026-09-25T16:34:32Z level=debug event=want client=3 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07446client # [ 189.775990] nix[1599]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7447lb: must succeed: curl -sf localhost:9901/clusters7448lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7449lb: must succeed: curl -sf localhost:9901/clusters7450lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7451worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7452worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7453client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7454lb # [ 197.081954] envoy[922]: conn peer=[2001:db8:1::1]:33236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27206 tx=12350 ms=154457455client # [ 219.777293] nix[1599]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7456client # [ 219.786848] nix[1599]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7457client # [ 219.818982] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7458client # [ 219.822649] systemd[1]: gpu.service: Failed with result 'exit-code'.7459client # [ 219.826951] systemd[1]: gpu.service: Consumed 152ms CPU time over 30.207s wall clock time, 37.6M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic.7460client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.41 seconds)7461client: must succeed: journalctl -u gpu -o cat7462client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7463lb: must succeed: curl -sf localhost:9901/clusters7464lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7465lb: must succeed: curl -sf localhost:9901/clusters7466lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7467worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7468worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7469(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.52 seconds)7470subtest: low disk drains a worker and builds go to the other7471worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7472worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.90 seconds)7473worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7474worker1 # [ 221.398353] nix-grpc-daemon[1795]: ts=2026-09-25T16:35:03Z level=info event=unhealthy reason=min_free7475worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.07 seconds)7476client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27477client # don't know how to build these paths:7478client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7479worker2 # [ 222.551533] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:505947480worker2 # [ 222.557469] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505947481worker1 # [ 222.582566] nix-grpc-daemon[1795]: ts=2026-09-25T16:35:04Z level=debug event=want client=4 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07482worker2 # [ 222.580469] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505947483worker1 # [ 222.586831] nix-grpc-daemon[1795]: ts=2026-09-25T16:35:04Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17484worker2 # [ 222.592075] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505947485client # copying 1 paths to 'lb:50051'...7486client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7487worker2 # [ 222.603172] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505947488worker2 # [ 222.611916] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)7489worker2 # [ 222.711055] nix-daemon[861]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)7490worker2 # [ 222.722991] nix-daemon[861]: reaped child process 1978, status = succeeded7491lb # [ 222.748338] niks3-server[1130]: 2026/09/25 16:35:05 INFO Received push request method=POST path=/api/pushes7492worker2 # [ 222.741228] nix-grpc-daemon[1728]: time=2026-09-25T16:35:05.915Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7493worker2 # [ 222.743194] nix-grpc-daemon[1728]: time=2026-09-25T16:35:05.916Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7494lb # [ 222.782803] niks3-server[1130]: 2026/09/25 16:35:05 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign7495worker2 # [ 222.778180] nix-grpc-daemon[1728]: time=2026-09-25T16:35:05.952Z level=INFO msg="Uploading 1 narinfos"7496lb # [ 222.794412] niks3-server[1130]: 2026/09/25 16:35:05 INFO Signed narinfos id=87 count=17497lb # [ 222.809837] niks3-server[1130]: 2026/09/25 16:35:05 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7498lb # [ 222.818975] niks3-server[1130]: 2026/09/25 16:35:05 INFO Received complete push request method=POST path=/api/pushes/87/complete7499worker2 # [ 222.810229] nix-grpc-daemon[1728]: time=2026-09-25T16:35:05.984Z level=INFO msg="Upload complete. (184ms)"7500lb # [ 222.830162] niks3-server[1130]: 2026/09/25 16:35:05 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7501worker2 # [ 222.815483] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 paths=1 nar_bytes_in=5127502lb # [ 222.836214] niks3-server[1130]: 2026/09/25 16:35:06 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7503worker2 # [ 222.822275] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505947504worker2 # [ 222.832214] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)7505worker2 # [ 222.844244] nix-daemon[861]: reaped child process 1973, status = succeeded7506worker2 # [ 222.848177] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)7507client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7508client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7509client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7510client # farm-a> LOG-farm-a-drain7511worker2 # [ 223.886592] nix-daemon[861]: reaped child process 1982, status = succeeded7512worker2 # [ 223.967634] nix-daemon[861]: accepted connection from pid 1998, user nix-grpc-daemon (trusted)7513worker2 # [ 223.981611] nix-daemon[861]: reaped child process 2001, status = succeeded7514lb # [ 224.002759] niks3-server[1130]: 2026/09/25 16:35:07 INFO Received push request method=POST path=/api/pushes7515worker2 # [ 223.997621] nix-grpc-daemon[1728]: time=2026-09-25T16:35:07.171Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7516worker2 # [ 223.999474] nix-grpc-daemon[1728]: time=2026-09-25T16:35:07.173Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7517lb # [ 224.044855] niks3-server[1130]: 2026/09/25 16:35:07 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign7518lb # [ 224.050625] niks3-server[1130]: 2026/09/25 16:35:07 INFO Signed narinfos id=88 count=17519worker2 # [ 224.037203] nix-grpc-daemon[1728]: time=2026-09-25T16:35:07.211Z level=INFO msg="Uploading 1 narinfos"7520lb # [ 224.054568] niks3-server[1130]: 2026/09/25 16:35:07 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7521lb # [ 224.062728] niks3-server[1130]: 2026/09/25 16:35:07 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7522lb # [ 224.068335] niks3-server[1130]: 2026/09/25 16:35:07 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7523lb # [ 224.074057] niks3-server[1130]: 2026/09/25 16:35:07 INFO Received complete push request method=POST path=/api/pushes/88/complete7524worker2 # [ 224.065137] nix-grpc-daemon[1728]: time=2026-09-25T16:35:07.239Z level=INFO msg="Upload complete. (209ms)"7525lb # [ 224.083498] niks3-server[1130]: 2026/09/25 16:35:07 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7526worker2 # [ 224.072236] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=14 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=07527worker1 # [ 224.081905] nix-grpc-daemon[1795]: ts=2026-09-25T16:35:06Z level=debug event=want client=4 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07528worker2 # [ 224.081707] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505947529worker1 # [ 224.086482] nix-grpc-daemon[1795]: ts=2026-09-25T16:35:06Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17530worker2 # [ 224.092494] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505947531worker2 # [ 224.098240] nix-daemon[861]: reaped child process 1980, status = succeeded7532worker2 # [ 224.103849] nix-grpc-daemon[1728]: time=2026-09-25T16:35:07.277Z level=INFO msg="All 1 paths already cached"7533client # copying 1 paths to 'lb:50051'...7534client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7535worker2 # [ 224.110660] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505947536worker2 # [ 224.118104] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)7537worker2 # [ 224.200308] nix-daemon[861]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)7538worker2 # [ 224.212408] nix-daemon[861]: reaped child process 2008, status = succeeded7539lb # [ 224.236282] niks3-server[1130]: 2026/09/25 16:35:07 INFO Received push request method=POST path=/api/pushes7540worker2 # [ 224.228674] nix-grpc-daemon[1728]: time=2026-09-25T16:35:07.402Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7541worker2 # [ 224.230670] nix-grpc-daemon[1728]: time=2026-09-25T16:35:07.404Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7542lb # [ 224.265923] niks3-server[1130]: 2026/09/25 16:35:07 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7543lb # [ 224.272861] niks3-server[1130]: 2026/09/25 16:35:07 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7544lb # [ 224.277435] niks3-server[1130]: 2026/09/25 16:35:07 INFO Signed narinfos id=89 count=17545worker2 # [ 224.263444] nix-grpc-daemon[1728]: time=2026-09-25T16:35:07.437Z level=INFO msg="Uploading 1 narinfos"7546lb # [ 224.284091] niks3-server[1130]: 2026/09/25 16:35:07 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7547lb # [ 224.290081] niks3-server[1130]: 2026/09/25 16:35:07 INFO Received complete push request method=POST path=/api/pushes/89/complete7548lb # [ 224.293832] niks3-server[1130]: 2026/09/25 16:35:07 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7549worker2 # [ 224.280632] nix-grpc-daemon[1728]: time=2026-09-25T16:35:07.454Z level=INFO msg="Upload complete. (157ms)"7550worker2 # [ 224.283887] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 paths=1 nar_bytes_in=6247551worker2 # [ 224.288662] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505947552worker2 # [ 224.298147] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)7553worker2 # [ 224.300538] nix-daemon[861]: reaped child process 2003, status = succeeded7554worker2 # [ 224.305704] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)7555client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7556client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7557client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7558client # farm-b> LOG-farm-b-drain7559worker2 # [ 225.746607] nix-daemon[861]: reaped child process 2012, status = succeeded7560worker2 # [ 225.831981] nix-daemon[861]: accepted connection from pid 2028, user nix-grpc-daemon (trusted)7561worker2 # [ 225.844259] nix-daemon[861]: reaped child process 2031, status = succeeded7562lb # [ 225.868840] niks3-server[1130]: 2026/09/25 16:35:09 INFO Received push request method=POST path=/api/pushes7563worker2 # [ 225.860694] nix-grpc-daemon[1728]: time=2026-09-25T16:35:09.034Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7564worker2 # [ 225.862493] nix-grpc-daemon[1728]: time=2026-09-25T16:35:09.036Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7565lb # [ 225.913171] niks3-server[1130]: 2026/09/25 16:35:09 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7566lb # [ 225.919774] niks3-server[1130]: 2026/09/25 16:35:09 INFO Signed narinfos id=90 count=17567worker2 # [ 225.906195] nix-grpc-daemon[1728]: time=2026-09-25T16:35:09.080Z level=INFO msg="Uploading 1 narinfos"7568lb # [ 225.928410] niks3-server[1130]: 2026/09/25 16:35:09 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7569lb # [ 225.934353] niks3-server[1130]: 2026/09/25 16:35:09 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7570lb # [ 225.943327] niks3-server[1130]: 2026/09/25 16:35:09 INFO Received complete push request method=POST path=/api/pushes/90/complete7571lb # [ 225.952399] niks3-server[1130]: 2026/09/25 16:35:09 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7572worker2 # [ 225.934429] nix-grpc-daemon[1728]: time=2026-09-25T16:35:09.108Z level=INFO msg="Upload complete. (220ms)"7573lb # [ 225.955623] niks3-server[1130]: 2026/09/25 16:35:09 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7574worker2 # [ 225.940894] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=15 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07575worker1 # [ 225.952640] nix-grpc-daemon[1795]: ts=2026-09-25T16:35:08Z level=debug event=want client=4 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07576worker2 # [ 225.951321] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505947577worker1 # [ 225.957457] nix-grpc-daemon[1795]: ts=2026-09-25T16:35:08Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17578worker2 # [ 225.962199] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:505947579worker2 # [ 225.970908] nix-daemon[861]: reaped child process 2010, status = succeeded7580worker2 # [ 225.972879] nix-grpc-daemon[1728]: time=2026-09-25T16:35:09.146Z level=INFO msg="All 2 paths already cached"7581client # copying 1 paths to 'lb:50051'...7582worker2 # [ 225.978493] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:505947583client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7584worker2 # [ 225.984464] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)7585worker2 # [ 226.066949] nix-daemon[861]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)7586worker2 # [ 226.079553] nix-daemon[861]: reaped child process 2038, status = succeeded7587lb # [ 226.103078] niks3-server[1130]: 2026/09/25 16:35:09 INFO Received push request method=POST path=/api/pushes7588worker2 # [ 226.095034] nix-grpc-daemon[1728]: time=2026-09-25T16:35:09.269Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7589worker2 # [ 226.097046] nix-grpc-daemon[1728]: time=2026-09-25T16:35:09.270Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7590lb # [ 226.135317] niks3-server[1130]: 2026/09/25 16:35:09 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7591lb # [ 226.142427] niks3-server[1130]: 2026/09/25 16:35:09 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7592lb # [ 226.147242] niks3-server[1130]: 2026/09/25 16:35:09 INFO Signed narinfos id=91 count=17593worker2 # [ 226.135216] nix-grpc-daemon[1728]: time=2026-09-25T16:35:09.309Z level=INFO msg="Uploading 1 narinfos"7594lb # [ 226.162661] niks3-server[1130]: 2026/09/25 16:35:09 INFO Received complete push request method=POST path=/api/pushes/91/complete7595lb # [ 226.168717] niks3-server[1130]: 2026/09/25 16:35:09 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7596worker2 # [ 226.157913] nix-grpc-daemon[1728]: time=2026-09-25T16:35:09.331Z level=INFO msg="Upload complete. (168ms)"7597lb # [ 226.176289] niks3-server[1130]: 2026/09/25 16:35:09 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7598worker2 # [ 226.162832] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=0 paths=1 nar_bytes_in=7527599worker2 # [ 226.168724] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:505947600worker2 # [ 226.179129] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)7601worker2 # [ 226.187186] nix-daemon[861]: reaped child process 2033, status = succeeded7602worker2 # [ 226.191990] nix-daemon[861]: accepted connection from pid 1636, user nix-grpc-daemon (trusted)7603client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7604client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7605client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7606client # farm-top> LOG-farm-top-drain7607lb # [ 227.083976] envoy[922]: conn peer=[2001:db8:1::1]:57828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300537608worker2 # [ 227.228459] nix-daemon[861]: reaped child process 2042, status = succeeded7609worker2 # [ 227.311175] nix-daemon[861]: accepted connection from pid 2058, user nix-grpc-daemon (trusted)7610worker2 # [ 227.323853] nix-daemon[861]: reaped child process 2061, status = succeeded7611lb # [ 227.347611] niks3-server[1130]: 2026/09/25 16:35:10 INFO Received push request method=POST path=/api/pushes7612worker2 # [ 227.339723] nix-grpc-daemon[1728]: time=2026-09-25T16:35:10.513Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7613worker2 # [ 227.341444] nix-grpc-daemon[1728]: time=2026-09-25T16:35:10.515Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7614lb # [ 227.389604] niks3-server[1130]: 2026/09/25 16:35:10 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7615lb # [ 227.394637] niks3-server[1130]: 2026/09/25 16:35:10 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7616lb # [ 227.400484] niks3-server[1130]: 2026/09/25 16:35:10 INFO Received sign narinfos request method=POST path=/api/pushes/92/sign7617lb # [ 227.404680] niks3-server[1130]: 2026/09/25 16:35:10 INFO Signed narinfos id=92 count=17618worker2 # [ 227.391441] nix-grpc-daemon[1728]: time=2026-09-25T16:35:10.565Z level=INFO msg="Uploading 1 narinfos"7619lb # [ 227.411676] niks3-server[1130]: 2026/09/25 16:35:10 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7620lb # [ 227.423143] niks3-server[1130]: 2026/09/25 16:35:10 INFO Received complete push request method=POST path=/api/pushes/92/complete7621lb # [ 227.432862] niks3-server[1130]: 2026/09/25 16:35:10 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7622worker2 # [ 227.416258] nix-grpc-daemon[1728]: time=2026-09-25T16:35:10.590Z level=INFO msg="Upload complete. (216ms)"7623worker2 # [ 227.422610] nix-grpc-daemon[1636]: ts=2026-09-25T16:35:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50594 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=16 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07624worker2 # [ 227.438678] nix-daemon[861]: reaped child process 2040, status = succeeded7625client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.04 seconds)7626client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7627client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7628worker1: must succeed: rm /nix/.rw-store/fill7629worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7630worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7631worker1 # [ 228.404337] nix-grpc-daemon[1795]: ts=2026-09-25T16:35:10Z level=info event=healthy reason=min_free7632worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7633(finished: subtest: low disk drains a worker and builds go to the other, in 8.37 seconds)7634(finished: run the VM test script, in 229.71 seconds)7635test script finished in 229.88s7636cleanup7637kill QemuMachine (pid 49)7638client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7639client # [2026-09-25T16:35:12Z INFO virtiofsd] Client disconnected, shutting down7640client # [2026-09-25T16:35:12Z INFO virtiofsd] Client disconnected, shutting down7641client # [2026-09-25T16:35:12Z INFO virtiofsd] Client disconnected, shutting down7642kill QemuMachine (pid 50)7643lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7644lb # [2026-09-25T16:35:12Z INFO virtiofsd] Client disconnected, shutting down7645lb # [2026-09-25T16:35:12Z INFO virtiofsd] Client disconnected, shutting down7646lb # [2026-09-25T16:35:12Z INFO virtiofsd] Client disconnected, shutting down7647kill QemuMachine (pid 51)7648worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7649worker1 # [2026-09-25T16:35:12Z INFO virtiofsd] Client disconnected, shutting down7650worker1 # [2026-09-25T16:35:12Z INFO virtiofsd] Client disconnected, shutting down7651worker1 # [2026-09-25T16:35:12Z INFO virtiofsd] Client disconnected, shutting down7652kill QemuMachine (pid 52)7653worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7654worker2 # [2026-09-25T16:35:13Z INFO virtiofsd] Client disconnected, shutting down7655worker2 # [2026-09-25T16:35:13Z INFO virtiofsd] Client disconnected, shutting down7656worker2 # [2026-09-25T16:35:13Z INFO virtiofsd] Client disconnected, shutting down7657(finished: cleanup, in 1.00 seconds)7658additionally exposed symbols:7659 client, lb, worker1, worker2,7660 vlan1,7661 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh