vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #503
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.57tTPbMFOJ', fmt=raw size=107374182414worker2 # mke2fs 1.47.4 (6-Mar-2025)15worker2 # Discarding device blocks: 0/262144 done16worker2: QEMU running (pid 51)17worker1: QEMU running (pid 52)18worker1 # Disk image does not exist, creating the virtualisation disk image...19worker1 # Formatting '/build/vm-state-worker1/tmp.fsaQe8Ostj', fmt=raw size=107374182420client # Disk image does not exist, creating the virtualisation disk image...21lb # Disk image does not exist, creating the virtualisation disk image...22worker1 # mke2fs 1.47.4 (6-Mar-2025)23worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes24client: QEMU running (pid 50)25lb: QEMU running (pid 49)26client # Formatting '/build/vm-state-client/tmp.nvFkE1c5vv', fmt=raw size=107374182427client # mke2fs 1.47.4 (6-Mar-2025)28lb # Formatting '/build/vm-state-lb/tmp.qZbYB9gI9K', fmt=raw size=107374182429worker2 # Filesystem UUID: 240cadf9-9258-49a6-99ad-84947f4dec7430worker1 # Discarding device blocks: 0/262144 done31lb # mke2fs 1.47.4 (6-Mar-2025)32lb # Discarding device blocks: 0/262144 done33(finished: start all VMs, in 0.34 seconds)34worker2 # Superblock backups stored on blocks:35client # Discarding device blocks: 0/262144 done36client: waiting for unit network-addresses-eth1.service37worker2 # 32768, 98304, 163840, 22937638client # Creating filesystem with 262144 4k blocks and 65536 inodes39client # Filesystem UUID: 412e463a-2a76-41e4-924e-06e070e7f95f40client: waiting for the VM to finish booting41lb # Creating filesystem with 262144 4k blocks and 65536 inodes42client # Superblock backups stored on blocks:43client # 32768, 98304, 163840, 22937644lb # Filesystem UUID: 453f3691-3288-4ce8-9b89-15b907c7624d45worker2 # 46worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes47lb # Superblock backups stored on blocks:48client # 49lb # 32768, 98304, 163840, 22937650worker1 # Filesystem UUID: af3a589a-52a4-41cd-aa8a-247d8a5800a551worker2 # Allocating group tables: 0/8 done52client # Allocating group tables: 0/8 done53lb # 54worker1 # Superblock backups stored on blocks:55client # Writing inode tables: 0/8 done56worker2 # Writing inode tables: 0/8 done57lb # Allocating group tables: 0/8 done58worker1 # 32768, 98304, 163840, 22937659worker1 # 60worker2 # Creating journal (8192 blocks): done61lb # Writing inode tables: 0/8 done62lb # Creating journal (8192 blocks): done63worker1 # Allocating group tables: 0/8 done64worker2 # Writing superblocks and filesystem accounting information: 0/8 done65worker2 # 66lb # Writing superblocks and filesystem accounting information: 0/8 done67worker1 # Writing inode tables: 0/8 done68worker1 # Creating journal (8192 blocks): done69worker2 # Virtualisation disk image created.70lb # 71lb # Virtualisation disk image created.72client # Creating journal (8192 blocks): done73worker2 # Starting virtiofs daemons...74worker1 # Writing superblocks and filesystem accounting information: 0/8 done75worker2 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76worker1 # 77worker1 # Virtualisation disk image created.78lb # Starting virtiofs daemons...79worker2 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether80worker2 # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...81worker1 # Starting virtiofs daemons...82lb # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83lb # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84worker2 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker1 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86worker1 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87lb # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...88worker2 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...90worker1 # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...91lb # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)92lb # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93worker2 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94worker1 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)95worker1 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether96worker1 # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...97lb # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...98worker1 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)99lb # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100worker2 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether101worker2 # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...102worker1 # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether103lb # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104client # Writing superblocks and filesystem accounting information: 0/8 done105lb # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...106lb # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests107worker1 # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...108worker1 # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests109worker2 # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests110lb # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests111lb # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests112worker1 # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests113worker2 # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests114worker1 # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests115worker2 # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests116client # 117client # Virtualisation disk image created.118client # Starting virtiofs daemons...119client # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...122client # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123client # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124client # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...125client # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126client # [2026-09-22T15:43:50Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127client # [2026-09-22T15:43:50Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-22T15:43:50Z INFO virtiofsd] Client connected, servicing requests131worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker2 # 136worker2 # 137worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138client # 139client # 140client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141worker1 # 142worker1 # 143worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...144worker2 # 145worker2 # 146worker2 # 147worker2 # 148worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00149worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00150client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...151client # 152client # 153client # 154client # 155client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...157worker1 # 158worker1 # 159worker1 # 160worker1 # 161client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...162client # 163client # 164worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00165worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...166worker2 # 167worker2 # 168worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169worker1 # 170worker1 # 171lb # 172lb # 173lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00174lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...175lb # 176lb # 177lb # 178lb # 179lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181lb # 182lb # 183worker2 # Booting from ROM...184client # Booting from ROM...185worker1 # Booting from ROM...186lb # Booting from ROM...187worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6bw4n0f8bls8vkb4018phy0as0l93xnb-nixos-system-worker2-test/init regInfo=/nix/store/mnhq4clcym8k7q31bph6wbcnxjd6nnha-closure-info/registration console=ttyS0,115200n8 console=tty0189worker2 # [ 0.000000] BIOS-provided physical RAM map:190worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200worker2 # [ 0.000000] NX (Execute Disable) protection: active201client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026202worker2 # [ 0.000000] APIC: Static calls initialized203worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026204worker2 # [ 0.000000] SMBIOS 2.8 present.205worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014206worker2 # [ 0.000000] DMI: Memory slots populated: 1/1207worker2 # [ 0.000000] Hypervisor detected: KVM208worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000209client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nwrxkcyq2rabn5jfqyww8chp4fbw12l7-nixos-system-client-test/init regInfo=/nix/store/9mbchw87wm67kspwy4r01ypax3r3mych-closure-info/registration console=ttyS0,115200n8 console=tty0210worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00211client # [ 0.000000] BIOS-provided physical RAM map:212worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7hj4sk4i9w6mp1h0l38acx95n0smfk2s-nixos-system-worker1-test/init regInfo=/nix/store/1ra8h8rvaqsaasm1v4s2g2fsrrsb0g1s-closure-info/registration console=ttyS0,115200n8 console=tty0213worker1 # [ 0.000000] BIOS-provided physical RAM map:214worker2 # [ 0.000000] kvm-clock: using sched offset of 468696097 cycles215client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable216worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable217client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved218worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns219worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved220worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor221client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved222worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved223worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000224client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable225worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable226worker2 # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs227client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved228worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved229worker2 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT230client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved231worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved232worker2 # [ 0.002722] found SMP MP-table at [mem 0x000f5450-0x000f545f]233client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved234worker2 # [ 0.002740] Using GB pages for direct mapping235worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved236client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved237worker2 # [ 0.002836] RAMDISK: [mem 0x5e36f000-0x5ffcffff]238worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved239worker2 # [ 0.002845] ACPI: Early table checksum verification disabled240client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved241worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved242worker2 # [ 0.002848] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )243client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved244worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved245client # [ 0.000000] NX (Execute Disable) protection: active246worker2 # [ 0.002852] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)247worker1 # [ 0.000000] NX (Execute Disable) protection: active248client # [ 0.000000] APIC: Static calls initialized249lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026250client # [ 0.000000] SMBIOS 2.8 present.251worker1 # [ 0.000000] APIC: Static calls initialized252worker2 # [ 0.002857] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)253worker1 # [ 0.000000] SMBIOS 2.8 present.254client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014255worker2 # [ 0.002863] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)256worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014257client # [ 0.000000] DMI: Memory slots populated: 1/1258worker2 # [ 0.002865] ACPI: FACS 0x000000005FFE0000 000040259client # [ 0.000000] Hypervisor detected: KVM260worker1 # [ 0.000000] DMI: Memory slots populated: 1/1261worker1 # [ 0.000000] Hypervisor detected: KVM262client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000263worker2 # [ 0.002867] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)264worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000265client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00266lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qfr7fghmwm537hb3f4f5is3yb8phm6jx-nixos-system-lb-test/init regInfo=/nix/store/s1480xaqc00gh7hz31b4z2fwjpnxi3sa-closure-info/registration console=ttyS0,115200n8 console=tty0267worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00268worker2 # [ 0.002868] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)269client # [ 0.000000] kvm-clock: using sched offset of 457316393 cycles270lb # [ 0.000000] BIOS-provided physical RAM map:271worker1 # [ 0.000000] kvm-clock: using sched offset of 481096256 cycles272lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable273worker2 # [ 0.002870] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)274client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns275lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved276worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns277client # [ 0.000005] tsc: Detected 2400.012 MHz processor278worker2 # [ 0.002871] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)279worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor280lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved281client # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000282worker2 # [ 0.002873] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]283worker1 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000284lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable285worker2 # [ 0.002874] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]286client # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs287worker1 # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs288lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved289client # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT290worker2 # [ 0.002874] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]291worker1 # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT292lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved293client # [ 0.002734] found SMP MP-table at [mem 0x000f5450-0x000f545f]294worker2 # [ 0.002875] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]295worker1 # [ 0.002720] found SMP MP-table at [mem 0x000f5450-0x000f545f]296client # [ 0.002745] Using GB pages for direct mapping297lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved298worker1 # [ 0.002731] Using GB pages for direct mapping299worker2 # [ 0.002875] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]300client # [ 0.002864] RAMDISK: [mem 0x5e370000-0x5ffcffff]301worker1 # [ 0.002826] RAMDISK: [mem 0x5e370000-0x5ffcffff]302lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved303worker2 # [ 0.002876] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]304client # [ 0.002872] ACPI: Early table checksum verification disabled305worker1 # [ 0.002833] ACPI: Early table checksum verification disabled306client # [ 0.002874] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )307lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved308worker2 # [ 0.002876] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]309worker1 # [ 0.002836] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )310lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved311worker2 # [ 0.003104] No NUMA configuration found312client # [ 0.002878] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)313lb # [ 0.000000] NX (Execute Disable) protection: active314worker1 # [ 0.002840] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)315worker2 # [ 0.003104] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]316lb # [ 0.000000] APIC: Static calls initialized317client # [ 0.002881] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)318lb # [ 0.000000] SMBIOS 2.8 present.319worker2 # [ 0.003107] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]320worker1 # [ 0.002844] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)321worker2 # [ 0.005301] Zone ranges:322client # [ 0.002888] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)323worker2 # [ 0.005302] DMA [mem 0x0000000000001000-0x0000000000ffffff]324lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014325worker1 # [ 0.002851] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)326client # [ 0.002890] ACPI: FACS 0x000000005FFE0000 000040327worker2 # [ 0.005303] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]328lb # [ 0.000000] DMI: Memory slots populated: 1/1329worker1 # [ 0.002853] ACPI: FACS 0x000000005FFE0000 000040330worker2 # [ 0.005305] Normal empty331lb # [ 0.000000] Hypervisor detected: KVM332client # [ 0.002892] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)333worker2 # [ 0.005305] Device empty334worker1 # [ 0.002855] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)335lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000336worker2 # [ 0.005306] Movable zone start for each node337client # [ 0.002893] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)338lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00339worker2 # [ 0.005307] Early memory node ranges340worker1 # [ 0.002857] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)341lb # [ 0.000000] kvm-clock: using sched offset of 472499626 cycles342worker2 # [ 0.005307] node 0: [mem 0x0000000000001000-0x000000000009efff]343client # [ 0.002895] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)344worker1 # [ 0.002858] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)345worker2 # [ 0.005308] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]346lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns347client # [ 0.002896] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)348worker2 # [ 0.005309] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]349worker1 # [ 0.002860] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)350lb # [ 0.000005] tsc: Detected 2400.012 MHz processor351client # [ 0.002898] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]352worker2 # [ 0.005330] On node 0, zone DMA: 1 pages in unavailable ranges353lb # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000354worker1 # [ 0.002861] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]355worker2 # [ 0.005583] On node 0, zone DMA: 97 pages in unavailable ranges356client # [ 0.002899] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]357lb # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs358worker1 # [ 0.002862] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]359worker2 # [ 0.030819] On node 0, zone DMA32: 40 pages in unavailable ranges360client # [ 0.002899] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]361worker1 # [ 0.002863] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]362worker2 # [ 0.031299] ACPI: PM-Timer IO Port: 0x608363lb # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT364client # [ 0.002900] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]365worker2 # [ 0.031313] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])366lb # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]367worker1 # [ 0.002863] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]368lb # [ 0.002749] Using GB pages for direct mapping369client # [ 0.002900] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]370worker2 # [ 0.031342] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23371worker1 # [ 0.002864] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]372lb # [ 0.002860] RAMDISK: [mem 0x5e36e000-0x5ffcffff]373client # [ 0.002901] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]374worker2 # [ 0.031345] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)375lb # [ 0.002869] ACPI: Early table checksum verification disabled376worker1 # [ 0.002864] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]377worker2 # [ 0.031347] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)378lb # [ 0.002872] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )379client # [ 0.002901] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]380worker1 # [ 0.002865] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]381client # [ 0.003121] No NUMA configuration found382worker1 # [ 0.003083] No NUMA configuration found383worker2 # [ 0.031348] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)384client # [ 0.003122] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]385lb # [ 0.002876] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)386worker1 # [ 0.003084] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]387worker2 # [ 0.031349] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)388client # [ 0.003124] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]389client # [ 0.005431] Zone ranges:390lb # [ 0.002881] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)391worker1 # [ 0.003086] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]392worker2 # [ 0.031350] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)393worker1 # [ 0.005390] Zone ranges:394lb # [ 0.002888] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)395client # [ 0.005432] DMA [mem 0x0000000000001000-0x0000000000ffffff]396worker2 # [ 0.031353] ACPI: Using ACPI (MADT) for SMP configuration information397worker1 # [ 0.005391] DMA [mem 0x0000000000001000-0x0000000000ffffff]398client # [ 0.005434] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]399lb # [ 0.002891] ACPI: FACS 0x000000005FFE0000 000040400worker2 # [ 0.031354] ACPI: HPET id: 0x8086a201 base: 0xfed00000401worker1 # [ 0.005392] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]402client # [ 0.005435] Normal empty403worker2 # [ 0.031358] TSC deadline timer available404client # [ 0.005435] Device empty405worker1 # [ 0.005393] Normal empty406worker1 # [ 0.005394] Device empty407lb # [ 0.002892] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)408worker2 # [ 0.031362] CPU topo: Max. logical packages: 1409client # [ 0.005436] Movable zone start for each node410worker1 # [ 0.005395] Movable zone start for each node411client # [ 0.005437] Early memory node ranges412worker2 # [ 0.031363] CPU topo: Max. logical dies: 1413worker1 # [ 0.005395] Early memory node ranges414lb # [ 0.002894] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)415worker2 # [ 0.031363] CPU topo: Max. dies per package: 1416client # [ 0.005437] node 0: [mem 0x0000000000001000-0x000000000009efff]417worker1 # [ 0.005396] node 0: [mem 0x0000000000001000-0x000000000009efff]418worker2 # [ 0.031367] CPU topo: Max. threads per core: 1419client # [ 0.005438] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]420lb # [ 0.002895] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)421worker2 # [ 0.031368] CPU topo: Num. cores per package: 1422worker1 # [ 0.005397] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]423client # [ 0.005439] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]424worker2 # [ 0.031368] CPU topo: Num. threads per package: 1425lb # [ 0.002897] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)426worker1 # [ 0.005398] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]427client # [ 0.005459] On node 0, zone DMA: 1 pages in unavailable ranges428worker2 # [ 0.031368] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs429lb # [ 0.002898] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]430worker1 # [ 0.005417] On node 0, zone DMA: 1 pages in unavailable ranges431client # [ 0.005736] On node 0, zone DMA: 97 pages in unavailable ranges432worker2 # [ 0.031394] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()433worker1 # [ 0.005687] On node 0, zone DMA: 97 pages in unavailable ranges434lb # [ 0.002899] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]435client # [ 0.032999] On node 0, zone DMA32: 40 pages in unavailable ranges436worker2 # [ 0.031429] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]437worker1 # [ 0.032910] On node 0, zone DMA32: 40 pages in unavailable ranges438client # [ 0.033452] ACPI: PM-Timer IO Port: 0x608439lb # [ 0.002900] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]440worker1 # [ 0.033365] ACPI: PM-Timer IO Port: 0x608441client # [ 0.033461] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])442worker2 # [ 0.031431] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]443lb # [ 0.002900] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]444worker1 # [ 0.033376] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])445worker2 # [ 0.031432] [mem 0x60000000-0xafffffff] available for PCI devices446client # [ 0.033487] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23447lb # [ 0.002901] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]448worker1 # [ 0.033403] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23449worker2 # [ 0.031434] Booting paravirtualized kernel on KVM450client # [ 0.033489] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)451worker1 # [ 0.033405] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)452lb # [ 0.002901] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]453client # [ 0.033490] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)454worker2 # [ 0.031438] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns455worker1 # [ 0.033407] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)456lb # [ 0.002902] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]457client # [ 0.033491] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)458lb # [ 0.003129] No NUMA configuration found459worker1 # [ 0.033408] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)460worker2 # [ 0.035918] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1461lb # [ 0.003130] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]462client # [ 0.033492] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)463worker1 # [ 0.033409] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)464worker2 # [ 0.038002] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152465lb # [ 0.003132] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]466client # [ 0.033493] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)467worker2 # [ 0.038047] kvm-guest: PV spinlocks disabled, single CPU468lb # [ 0.005260] Zone ranges:469worker1 # [ 0.033410] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)470client # [ 0.033495] ACPI: Using ACPI (MADT) for SMP configuration information471lb # [ 0.005261] DMA [mem 0x0000000000001000-0x0000000000ffffff]472worker1 # [ 0.033412] ACPI: Using ACPI (MADT) for SMP configuration information473client # [ 0.033496] ACPI: HPET id: 0x8086a201 base: 0xfed00000474worker1 # [ 0.033413] ACPI: HPET id: 0x8086a201 base: 0xfed00000475lb # [ 0.005263] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]476client # [ 0.033499] TSC deadline timer available477lb # [ 0.005264] Normal empty478worker1 # [ 0.033416] TSC deadline timer available479lb # [ 0.005265] Device empty480client # [ 0.033503] CPU topo: Max. logical packages: 1481worker1 # [ 0.033420] CPU topo: Max. logical packages: 1482client # [ 0.033504] CPU topo: Max. logical dies: 1483lb # [ 0.005265] Movable zone start for each node484worker1 # [ 0.033421] CPU topo: Max. logical dies: 1485lb # [ 0.005266] Early memory node ranges486client # [ 0.033504] CPU topo: Max. dies per package: 1487worker1 # [ 0.033421] CPU topo: Max. dies per package: 1488worker2 # [ 0.038049] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6bw4n0f8bls8vkb4018phy0as0l93xnb-nixos-system-worker2-test/init regInfo=/nix/store/mnhq4clcym8k7q31bph6wbcnxjd6nnha-closure-info/registration console=ttyS0,115200n8 console=tty0489lb # [ 0.005266] node 0: [mem 0x0000000000001000-0x000000000009efff]490client # [ 0.033508] CPU topo: Max. threads per core: 1491worker1 # [ 0.033424] CPU topo: Max. threads per core: 1492client # [ 0.033508] CPU topo: Num. cores per package: 1493worker1 # [ 0.033425] CPU topo: Num. cores per package: 1494lb # [ 0.005267] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]495client # [ 0.033509] CPU topo: Num. threads per package: 1496worker1 # [ 0.033425] CPU topo: Num. threads per package: 1497worker2 # [ 0.038146] Unknown kernel command line parameters "regInfo=/nix/store/mnhq4clcym8k7q31bph6wbcnxjd6nnha-closure-info/registration", will be passed to user space.498lb # [ 0.005269] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]499client # [ 0.033509] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs500worker2 # [ 0.038159] random: crng init done501worker1 # [ 0.033426] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs502lb # [ 0.005288] On node 0, zone DMA: 1 pages in unavailable ranges503client # [ 0.033526] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()504worker2 # [ 0.038160] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes505lb # [ 0.005546] On node 0, zone DMA: 97 pages in unavailable ranges506worker1 # [ 0.033443] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()507client # [ 0.033556] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]508worker2 # [ 0.040255] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)509lb # [ 0.031755] On node 0, zone DMA32: 40 pages in unavailable ranges510worker1 # [ 0.033473] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]511lb # [ 0.032227] ACPI: PM-Timer IO Port: 0x608512client # [ 0.033557] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]513worker2 # [ 0.041281] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)514worker1 # [ 0.033475] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]515lb # [ 0.032240] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])516worker2 # [ 0.041313] Fallback order for Node 0: 0517client # [ 0.033559] [mem 0x60000000-0xafffffff] available for PCI devices518worker1 # [ 0.033476] [mem 0x60000000-0xafffffff] available for PCI devices519lb # [ 0.032270] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23520client # [ 0.033560] Booting paravirtualized kernel on KVM521worker2 # [ 0.041316] Built 1 zonelists, mobility grouping on. Total pages: 393078522worker1 # [ 0.033478] Booting paravirtualized kernel on KVM523worker2 # [ 0.041317] Policy zone: DMA32524lb # [ 0.032273] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)525client # [ 0.033562] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns526worker2 # [ 0.043781] mem auto-init: stack:all(zero), heap alloc:on, heap free:off527lb # [ 0.032275] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)528worker1 # [ 0.033481] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns529worker2 # [ 0.046904] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1530lb # [ 0.032276] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)531client # [ 0.038068] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1532worker1 # [ 0.037966] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1533worker2 # [ 0.050260] allocated 3145728 bytes of page_ext534client # [ 0.040195] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152535lb # [ 0.032277] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)536worker1 # [ 0.040062] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152537worker2 # [ 0.060009] ftrace: allocating 48787 entries in 192 pages538client # [ 0.040239] kvm-guest: PV spinlocks disabled, single CPU539lb # [ 0.032278] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)540worker1 # [ 0.040102] kvm-guest: PV spinlocks disabled, single CPU541worker2 # [ 0.060011] ftrace: allocated 192 pages with 2 groups542worker2 # [ 0.060830] Dynamic Preempt: lazy543lb # [ 0.032281] ACPI: Using ACPI (MADT) for SMP configuration information544worker2 # [ 0.060957] rcu: Preemptible hierarchical RCU implementation.545lb # [ 0.032282] ACPI: HPET id: 0x8086a201 base: 0xfed00000546worker2 # [ 0.060958] rcu: RCU event tracing is enabled.547lb # [ 0.032286] TSC deadline timer available548lb # [ 0.032290] CPU topo: Max. logical packages: 1549worker2 # [ 0.060959] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.550lb # [ 0.032291] CPU topo: Max. logical dies: 1551worker2 # [ 0.060960] Trampoline variant of Tasks RCU enabled.552lb # [ 0.032291] CPU topo: Max. dies per package: 1553client # [ 0.040241] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nwrxkcyq2rabn5jfqyww8chp4fbw12l7-nixos-system-client-test/init regInfo=/nix/store/9mbchw87wm67kspwy4r01ypax3r3mych-closure-info/registration console=ttyS0,115200n8 console=tty0554worker2 # [ 0.060960] Rude variant of Tasks RCU enabled.555worker1 # [ 0.040103] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7hj4sk4i9w6mp1h0l38acx95n0smfk2s-nixos-system-worker1-test/init regInfo=/nix/store/1ra8h8rvaqsaasm1v4s2g2fsrrsb0g1s-closure-info/registration console=ttyS0,115200n8 console=tty0556lb # [ 0.032294] CPU topo: Max. threads per core: 1557worker2 # [ 0.060961] Tracing variant of Tasks RCU enabled.558lb # [ 0.032295] CPU topo: Num. cores per package: 1559worker2 # [ 0.060961] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.560client # [ 0.040331] Unknown kernel command line parameters "regInfo=/nix/store/9mbchw87wm67kspwy4r01ypax3r3mych-closure-info/registration", will be passed to user space.561lb # [ 0.032295] CPU topo: Num. threads per package: 1562worker1 # [ 0.040195] Unknown kernel command line parameters "regInfo=/nix/store/1ra8h8rvaqsaasm1v4s2g2fsrrsb0g1s-closure-info/registration", will be passed to user space.563client # [ 0.040344] random: crng init done564worker1 # [ 0.040207] random: crng init done565lb # [ 0.032295] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs566worker2 # [ 0.060962] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1567client # [ 0.040344] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes568worker1 # [ 0.040208] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes569lb # [ 0.032321] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()570worker2 # [ 0.060971] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.571client # [ 0.042436] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)572worker1 # [ 0.042273] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)573lb # [ 0.032356] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]574worker2 # [ 0.060973] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.575client # [ 0.043717] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)576lb # [ 0.032358] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]577worker1 # [ 0.043658] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)578client # [ 0.043751] Fallback order for Node 0: 0579worker2 # [ 0.060974] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.580worker1 # [ 0.043689] Fallback order for Node 0: 0581lb # [ 0.032359] [mem 0x60000000-0xafffffff] available for PCI devices582client # [ 0.043753] Built 1 zonelists, mobility grouping on. Total pages: 393078583worker1 # [ 0.043692] Built 1 zonelists, mobility grouping on. Total pages: 393078584lb # [ 0.032361] Booting paravirtualized kernel on KVM585worker2 # [ 0.065324] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16586client # [ 0.043754] Policy zone: DMA32587worker1 # [ 0.043693] Policy zone: DMA32588worker2 # [ 0.065643] rcu: srcu_init: Setting srcu_struct sizes based on contention.589client # [ 0.046223] mem auto-init: stack:all(zero), heap alloc:on, heap free:off590worker1 # [ 0.046133] mem auto-init: stack:all(zero), heap alloc:on, heap free:off591lb # [ 0.032365] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns592client # [ 0.049336] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1593worker1 # [ 0.049331] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1594worker2 # [ 0.065650] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns595lb # [ 0.036828] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1596client # [ 0.052647] allocated 3145728 bytes of page_ext597worker1 # [ 0.052593] allocated 3145728 bytes of page_ext598lb # [ 0.039429] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152599client # [ 0.062483] ftrace: allocating 48787 entries in 192 pages600worker1 # [ 0.062284] ftrace: allocating 48787 entries in 192 pages601worker2 # [ 0.065762] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)602lb # [ 0.039474] kvm-guest: PV spinlocks disabled, single CPU603client # [ 0.062485] ftrace: allocated 192 pages with 2 groups604worker2 # [ 0.069273] Console: colour VGA+ 80x25605worker1 # [ 0.062285] ftrace: allocated 192 pages with 2 groups606client # [ 0.063320] Dynamic Preempt: lazy607worker1 # [ 0.063105] Dynamic Preempt: lazy608worker2 # [ 0.069276] printk: legacy console [tty0] enabled609client # [ 0.063482] rcu: Preemptible hierarchical RCU implementation.610worker2 # [ 0.098789] printk: legacy console [ttyS0] enabled611worker1 # [ 0.063222] rcu: Preemptible hierarchical RCU implementation.612client # [ 0.063482] rcu: RCU event tracing is enabled.613worker1 # [ 0.063223] rcu: RCU event tracing is enabled.614worker2 # [ 0.195967] ACPI: Core revision 20250807615client # [ 0.063483] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.616worker1 # [ 0.063223] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.617client # [ 0.063484] Trampoline variant of Tasks RCU enabled.618worker1 # [ 0.063225] Trampoline variant of Tasks RCU enabled.619client # [ 0.063485] Rude variant of Tasks RCU enabled.620worker2 # [ 0.196850] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns621worker1 # [ 0.063225] Rude variant of Tasks RCU enabled.622lb # [ 0.039475] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qfr7fghmwm537hb3f4f5is3yb8phm6jx-nixos-system-lb-test/init regInfo=/nix/store/s1480xaqc00gh7hz31b4z2fwjpnxi3sa-closure-info/registration console=ttyS0,115200n8 console=tty0623worker1 # [ 0.063225] Tracing variant of Tasks RCU enabled.624client # [ 0.063485] Tracing variant of Tasks RCU enabled.625worker2 # [ 0.198323] APIC: Switch to symmetric I/O mode setup626client # [ 0.063486] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.627worker1 # [ 0.063226] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.628worker2 # [ 0.199302] x2apic enabled629lb # [ 0.039574] Unknown kernel command line parameters "regInfo=/nix/store/s1480xaqc00gh7hz31b4z2fwjpnxi3sa-closure-info/registration", will be passed to user space.630lb # [ 0.039587] random: crng init done631client # [ 0.063486] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1632worker1 # [ 0.063227] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1633worker2 # [ 0.200077] APIC: Switched APIC routing to: physical x2apic634lb # [ 0.039588] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes635client # [ 0.063502] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.636worker1 # [ 0.063241] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.637lb # [ 0.041883] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)638worker1 # [ 0.063243] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.639client # [ 0.063504] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.640worker2 # [ 0.201891] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1641lb # [ 0.043319] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)642lb # [ 0.043351] Fallback order for Node 0: 0643client # [ 0.063505] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.644worker1 # [ 0.063244] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.645lb # [ 0.043354] Built 1 zonelists, mobility grouping on. Total pages: 393078646client # [ 0.068107] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16647worker2 # [ 0.202854] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns648worker1 # [ 0.067933] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16649lb # [ 0.043355] Policy zone: DMA32650worker1 # [ 0.068208] rcu: srcu_init: Setting srcu_struct sizes based on contention.651client # [ 0.068380] rcu: srcu_init: Setting srcu_struct sizes based on contention.652lb # [ 0.046349] mem auto-init: stack:all(zero), heap alloc:on, heap free:off653worker2 # [ 0.204486] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)654client # [ 0.068386] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns655worker1 # [ 0.068213] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns656lb # [ 0.049586] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1657lb # [ 0.053201] allocated 3145728 bytes of page_ext658worker2 # [ 0.205571] x86/cpu: User Mode Instruction Prevention (UMIP) activated659lb # [ 0.063032] ftrace: allocating 48787 entries in 192 pages660client # [ 0.068486] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)661worker1 # [ 0.068317] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)662client # [ 0.072209] Console: colour VGA+ 80x25663worker1 # [ 0.072154] Console: colour VGA+ 80x25664worker2 # [ 0.206604] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127665lb # [ 0.063034] ftrace: allocated 192 pages with 2 groups666lb # [ 0.063934] Dynamic Preempt: lazy667worker1 # [ 0.072157] printk: legacy console [tty0] enabled668client # [ 0.072212] printk: legacy console [tty0] enabled669worker2 # [ 0.207484] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0670worker1 # [ 0.104311] printk: legacy console [ttyS0] enabled671client # [ 0.103986] printk: legacy console [ttyS0] enabled672lb # [ 0.064072] rcu: Preemptible hierarchical RCU implementation.673lb # [ 0.064073] rcu: RCU event tracing is enabled.674worker1 # [ 0.199860] ACPI: Core revision 20250807675client # [ 0.200064] ACPI: Core revision 20250807676lb # [ 0.064073] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.677worker2 # [ 0.208487] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto678lb # [ 0.064075] Trampoline variant of Tasks RCU enabled.679worker1 # [ 0.200779] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns680client # [ 0.200906] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns681lb # [ 0.064075] Rude variant of Tasks RCU enabled.682worker2 # [ 0.209484] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl683lb # [ 0.064076] Tracing variant of Tasks RCU enabled.684worker1 # [ 0.202319] APIC: Switch to symmetric I/O mode setup685client # [ 0.202534] APIC: Switch to symmetric I/O mode setup686worker2 # [ 0.210484] Transient Scheduler Attacks: Vulnerable: No microcode687lb # [ 0.064076] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.688client # [ 0.203546] x2apic enabled689worker1 # [ 0.203341] x2apic enabled690lb # [ 0.064077] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1691worker2 # [ 0.211483] Spectre V2 : Mitigation: Enhanced / Automatic IBRS692client # [ 0.204302] APIC: Switched APIC routing to: physical x2apic693worker1 # [ 0.204116] APIC: Switched APIC routing to: physical x2apic694lb # [ 0.064467] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.695worker2 # [ 0.212483] Speculative Return Stack Overflow: Mitigation: Safe RET696lb # [ 0.064469] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.697client # [ 0.206208] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1698worker1 # [ 0.206659] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1699lb # [ 0.064470] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.700lb # [ 0.068899] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16701client # [ 0.207249] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns702lb # [ 0.069191] rcu: srcu_init: Setting srcu_struct sizes based on contention.703worker1 # [ 0.207771] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns704worker2 # [ 0.213483] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization705client # [ 0.208911] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)706worker2 # [ 0.215490] Spectre V2 : Enabling IBPB for BPF707lb # [ 0.069199] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns708worker1 # [ 0.209626] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)709client # [ 0.209995] x86/cpu: User Mode Instruction Prevention (UMIP) activated710lb # [ 0.069306] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)711worker2 # [ 0.216484] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier712lb # [ 0.072897] Console: colour VGA+ 80x25713client # [ 0.211845] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127714worker1 # [ 0.210719] x86/cpu: User Mode Instruction Prevention (UMIP) activated715worker2 # [ 0.217483] active return thunk: srso_alias_return_thunk716lb # [ 0.072899] printk: legacy console [tty0] enabled717lb # [ 0.102263] printk: legacy console [ttyS0] enabled718client # [ 0.212578] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0719worker1 # [ 0.212761] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127720worker2 # [ 0.218504] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'721lb # [ 0.198176] ACPI: Core revision 20250807722worker1 # [ 0.213623] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0723worker2 # [ 0.219483] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'724client # [ 0.213676] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto725lb # [ 0.199028] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns726worker2 # [ 0.220483] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'727client # [ 0.214909] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl728lb # [ 0.200589] APIC: Switch to symmetric I/O mode setup729worker1 # [ 0.214627] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto730worker2 # [ 0.221483] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'731client # [ 0.215909] Transient Scheduler Attacks: Vulnerable: No microcode732lb # [ 0.201547] x2apic enabled733worker2 # [ 0.222483] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'734client # [ 0.217629] Spectre V2 : Mitigation: Enhanced / Automatic IBRS735worker1 # [ 0.215623] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl736lb # [ 0.202287] APIC: Switched APIC routing to: physical x2apic737worker2 # [ 0.223483] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'738client # [ 0.218606] Speculative Return Stack Overflow: Mitigation: Safe RET739worker1 # [ 0.216623] Transient Scheduler Attacks: Vulnerable: No microcode740worker2 # [ 0.224483] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'741lb # [ 0.204125] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1742worker1 # [ 0.218622] Spectre V2 : Mitigation: Enhanced / Automatic IBRS743worker2 # [ 0.225484] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256744worker1 # [ 0.219622] Speculative Return Stack Overflow: Mitigation: Safe RET745worker2 # [ 0.227225] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64746lb # [ 0.205102] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns747worker2 # [ 0.228230] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512748client # [ 0.219662] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization749lb # [ 0.206815] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)750worker2 # [ 0.229237] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024751client # [ 0.220916] Spectre V2 : Enabling IBPB for BPF752lb # [ 0.207898] x86/cpu: User Mode Instruction Prevention (UMIP) activated753worker2 # [ 0.230236] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8754client # [ 0.222462] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier755lb # [ 0.208963] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127756worker1 # [ 0.220622] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization757client # [ 0.223909] active return thunk: srso_alias_return_thunk758worker2 # [ 0.231250] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759lb # [ 0.210496] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0760worker1 # [ 0.222630] Spectre V2 : Enabling IBPB for BPF761client # [ 0.224768] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'762worker1 # [ 0.223626] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier763lb # [ 0.211575] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto764client # [ 0.225909] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'765worker1 # [ 0.224623] active return thunk: srso_alias_return_thunk766client # [ 0.226909] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'767lb # [ 0.212812] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl768client # [ 0.227909] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'769lb # [ 0.213812] Transient Scheduler Attacks: Vulnerable: No microcode770worker1 # [ 0.226647] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'771lb # [ 0.215524] Spectre V2 : Mitigation: Enhanced / Automatic IBRS772client # [ 0.228909] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'773worker1 # [ 0.227623] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'774lb # [ 0.216516] Speculative Return Stack Overflow: Mitigation: Safe RET775client # [ 0.229909] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'776worker1 # [ 0.228623] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'777client # [ 0.230909] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'778worker1 # [ 0.229622] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'779client # [ 0.231910] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256780worker1 # [ 0.230622] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'781client # [ 0.232909] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64782lb # [ 0.216812] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization783worker1 # [ 0.232622] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'784client # [ 0.233909] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512785lb # [ 0.218820] Spectre V2 : Enabling IBPB for BPF786client # [ 0.234909] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024787worker1 # [ 0.233622] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'788lb # [ 0.219813] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier789client # [ 0.235909] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8790worker1 # [ 0.234623] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256791lb # [ 0.220812] active return thunk: srso_alias_return_thunk792worker1 # [ 0.235622] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64793client # [ 0.236909] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.794lb # [ 0.222481] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'795worker1 # [ 0.237458] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512796lb # [ 0.223730] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'797worker1 # [ 0.238450] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024798lb # [ 0.224572] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'799worker1 # [ 0.239622] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8800lb # [ 0.225592] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'801lb # [ 0.226580] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'802worker1 # [ 0.240623] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.803lb # [ 0.227593] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'804lb # [ 0.228644] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'805lb # [ 0.229812] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256806lb # [ 0.230812] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807lb # [ 0.231812] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808lb # [ 0.232811] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809lb # [ 0.233812] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810lb # [ 0.234811] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811worker2 # [ 0.265120] Freeing SMP alternatives memory: 44K812worker2 # [ 0.265486] pid_max: default: 32768 minimum: 301813worker2 # [ 0.266578] LSM: initializing lsm=capability,landlock,yama,bpf,ima814worker2 # [ 0.267592] landlock: Up and running.815worker2 # [ 0.268484] Yama: becoming mindful.816worker2 # [ 0.269562] LSM support for eBPF active817worker2 # [ 0.270340] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818worker2 # [ 0.271520] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819worker2 # [ 0.274065] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820worker2 # [ 0.274989] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821worker2 # [ 0.275488] ... version: 2822worker2 # [ 0.276303] ... bit width: 48823worker2 # [ 0.277072] ... generic counters: 6824worker2 # [ 0.277534] ... generic bitmap: 000000000000003f825client # [ 0.271477] Freeing SMP alternatives memory: 44K826worker2 # [ 0.278448] ... fixed-purpose counters: 0827client # [ 0.271912] pid_max: default: 32768 minimum: 301828worker2 # [ 0.279025] ... fixed-purpose bitmap: 0000000000000000829client # [ 0.273037] LSM: initializing lsm=capability,landlock,yama,bpf,ima830worker2 # [ 0.279486] ... value mask: 0000ffffffffffff831client # [ 0.274011] landlock: Up and running.832worker2 # [ 0.280421] ... max period: 00007fffffffffff833client # [ 0.274910] Yama: becoming mindful.834worker2 # [ 0.281214] ... global_ctrl mask: 000000000000003f835client # [ 0.275727] LSM support for eBPF active836worker2 # [ 0.281598] signal: max sigframe size: 3376837client # [ 0.276503] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)838worker2 # [ 0.282452] rcu: Hierarchical SRCU implementation.839worker1 # [ 0.277072] Freeing SMP alternatives memory: 44K840worker1 # [ 0.277625] pid_max: default: 32768 minimum: 301841worker2 # [ 0.283092] rcu: Max phase no-delay instances is 400.842client # [ 0.277838] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)843worker1 # [ 0.278643] LSM: initializing lsm=capability,landlock,yama,bpf,ima844worker1 # [ 0.279720] landlock: Up and running.845worker1 # [ 0.280363] Yama: becoming mindful.846client # [ 0.280148] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)847worker1 # [ 0.280835] LSM support for eBPF active848client # [ 0.281468] Performance Events: Fam17h+ core perfctr, AMD PMU driver.849worker1 # [ 0.281736] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)850client # [ 0.281918] ... version: 2851worker2 # [ 0.288019] smp: Bringing up secondary CPUs ...852client # [ 0.282639] ... bit width: 48853worker1 # [ 0.282662] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)854worker2 # [ 0.288499] smp: Brought up 1 node, 1 CPU855lb # [ 0.268767] Freeing SMP alternatives memory: 44K856client # [ 0.282912] ... generic counters: 6857lb # [ 0.269353] pid_max: default: 32768 minimum: 301858worker2 # [ 0.289155] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)859client # [ 0.283636] ... generic bitmap: 000000000000003f860lb # [ 0.269899] LSM: initializing lsm=capability,landlock,yama,bpf,ima861client # [ 0.283912] ... fixed-purpose counters: 0862worker1 # [ 0.285545] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)863lb # [ 0.270916] landlock: Up and running.864client # [ 0.284592] ... fixed-purpose bitmap: 0000000000000000865worker2 # [ 0.289600] Memory: 1454616K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)866lb # [ 0.271812] Yama: becoming mindful.867worker1 # [ 0.286124] Performance Events: Fam17h+ core perfctr, AMD PMU driver.868client # [ 0.284911] ... value mask: 0000ffffffffffff869worker2 # [ 0.290739] devtmpfs: initialized870lb # [ 0.272925] LSM support for eBPF active871worker1 # [ 0.286628] ... version: 2872client # [ 0.285781] ... max period: 00007fffffffffff873worker2 # [ 0.291605] x86/mm: Memory block size: 128MB874worker1 # [ 0.287340] ... bit width: 48875lb # [ 0.273662] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)876client # [ 0.286580] ... global_ctrl mask: 000000000000003f877worker1 # [ 0.287625] ... generic counters: 6878client # [ 0.287028] signal: max sigframe size: 3376879lb # [ 0.274855] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)880worker1 # [ 0.288342] ... generic bitmap: 000000000000003f881worker2 # [ 0.293386] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)882client # [ 0.287830] rcu: Hierarchical SRCU implementation.883worker1 # [ 0.288670] ... fixed-purpose counters: 0884client # [ 0.288513] rcu: Max phase no-delay instances is 400.885worker1 # [ 0.289379] ... fixed-purpose bitmap: 0000000000000000886worker2 # [ 0.294377] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).887lb # [ 0.277206] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)888worker1 # [ 0.289625] ... value mask: 0000ffffffffffff889worker2 # [ 0.295566] pinctrl core: initialized pinctrl subsystem890worker1 # [ 0.290530] ... max period: 00007fffffffffff891worker2 # [ 0.296627] PM: RTC time: 15:43:50, date: 2026-09-22892lb # [ 0.278392] Performance Events: Fam17h+ core perfctr, AMD PMU driver.893worker1 # [ 0.291392] ... global_ctrl mask: 000000000000003f894lb # [ 0.278821] ... version: 2895worker1 # [ 0.291735] signal: max sigframe size: 3376896lb # [ 0.279551] ... bit width: 48897lb # [ 0.279814] ... generic counters: 6898worker1 # [ 0.292557] rcu: Hierarchical SRCU implementation.899client # [ 0.293458] smp: Bringing up secondary CPUs ...900lb # [ 0.280545] ... generic bitmap: 000000000000003f901worker1 # [ 0.293280] rcu: Max phase no-delay instances is 400.902worker2 # [ 0.300090] NET: Registered PF_NETLINK/PF_ROUTE protocol family903client # [ 0.293927] smp: Brought up 1 node, 1 CPU904lb # [ 0.280814] ... fixed-purpose counters: 0905client # [ 0.294601] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)906lb # [ 0.281550] ... fixed-purpose bitmap: 0000000000000000907worker2 # [ 0.300957] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations908lb # [ 0.281814] ... value mask: 0000ffffffffffff909worker2 # [ 0.301506] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations910lb # [ 0.282724] ... max period: 00007fffffffffff911client # [ 0.295122] Memory: 1454652K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)912lb # [ 0.283538] ... global_ctrl mask: 000000000000003f913client # [ 0.296147] devtmpfs: initialized914worker2 # [ 0.302745] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations915lb # [ 0.283913] signal: max sigframe size: 3376916worker1 # [ 0.298175] smp: Bringing up secondary CPUs ...917client # [ 0.296991] x86/mm: Memory block size: 128MB918worker2 # [ 0.303495] audit: initializing netlink subsys (disabled)919lb # [ 0.284695] rcu: Hierarchical SRCU implementation.920worker1 # [ 0.298639] smp: Brought up 1 node, 1 CPU921lb # [ 0.285405] rcu: Max phase no-delay instances is 400.922worker2 # [ 0.304488] audit: type=2000 audit(1790091830.853:1): state=initialized audit_enabled=0 res=1923worker1 # [ 0.299338] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)924client # [ 0.298718] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)925worker2 # [ 0.305598] thermal_sys: Registered thermal governor 'fair_share'926client # [ 0.299837] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).927worker2 # [ 0.305600] thermal_sys: Registered thermal governor 'bang_bang'928worker1 # [ 0.299819] Memory: 1454636K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)929client # [ 0.301003] pinctrl core: initialized pinctrl subsystem930worker2 # [ 0.306487] thermal_sys: Registered thermal governor 'step_wise'931worker1 # [ 0.300841] devtmpfs: initialized932worker2 # [ 0.307444] thermal_sys: Registered thermal governor 'user_space'933client # [ 0.302120] PM: RTC time: 15:43:50, date: 2026-09-22934worker1 # [ 0.301720] x86/mm: Memory block size: 128MB935lb # [ 0.290377] smp: Bringing up secondary CPUs ...936worker2 # [ 0.308242] thermal_sys: Registered thermal governor 'power_allocator'937lb # [ 0.290829] smp: Brought up 1 node, 1 CPU938worker2 # [ 0.308500] cpuidle: using governor menu939worker1 # [ 0.303507] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)940lb # [ 0.291515] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)941worker2 # [ 0.310713] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5942client # [ 0.305635] NET: Registered PF_NETLINK/PF_ROUTE protocol family943worker1 # [ 0.304526] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).944worker1 # [ 0.305698] pinctrl core: initialized pinctrl subsystem945worker2 # [ 0.311737] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]946lb # [ 0.291994] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)947client # [ 0.306419] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations948worker1 # [ 0.306749] PM: RTC time: 15:43:50, date: 2026-09-22949lb # [ 0.293055] devtmpfs: initialized950worker2 # [ 0.312493] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry951client # [ 0.306934] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations952lb # [ 0.293833] x86/mm: Memory block size: 128MB953worker2 # [ 0.313498] PCI: Using configuration type 1 for base access954client # [ 0.308192] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations955worker2 # [ 0.314610] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.956lb # [ 0.295642] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)957client # [ 0.308923] audit: initializing netlink subsys (disabled)958worker1 # [ 0.310256] NET: Registered PF_NETLINK/PF_ROUTE protocol family959client # [ 0.310061] thermal_sys: Registered thermal governor 'fair_share'960lb # [ 0.296736] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).961worker1 # [ 0.311125] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations962client # [ 0.310063] thermal_sys: Registered thermal governor 'bang_bang'963lb # [ 0.297903] pinctrl core: initialized pinctrl subsystem964worker1 # [ 0.311659] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations965client # [ 0.310913] thermal_sys: Registered thermal governor 'step_wise'966lb # [ 0.299050] PM: RTC time: 15:43:50, date: 2026-09-22967client # [ 0.311877] thermal_sys: Registered thermal governor 'user_space'968worker1 # [ 0.312902] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations969worker1 # [ 0.313635] audit: initializing netlink subsys (disabled)970client # [ 0.312673] audit: type=2000 audit(1790091830.867:1): state=initialized audit_enabled=0 res=1971worker2 # [ 0.321501] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages972client # [ 0.313916] thermal_sys: Registered thermal governor 'power_allocator'973worker1 # [ 0.314627] audit: type=2000 audit(1790091830.871:1): state=initialized audit_enabled=0 res=1974lb # [ 0.302737] NET: Registered PF_NETLINK/PF_ROUTE protocol family975client # [ 0.313931] cpuidle: using governor menu976worker2 # [ 0.322486] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page977worker1 # [ 0.315776] thermal_sys: Registered thermal governor 'fair_share'978lb # [ 0.304089] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations979client # [ 0.316143] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5980worker1 # [ 0.315779] thermal_sys: Registered thermal governor 'bang_bang'981worker1 # [ 0.316626] thermal_sys: Registered thermal governor 'step_wise'982lb # [ 0.304834] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations983client # [ 0.317146] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]984worker1 # [ 0.317554] thermal_sys: Registered thermal governor 'user_space'985client # [ 0.317915] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry986lb # [ 0.306101] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations987worker1 # [ 0.318351] thermal_sys: Registered thermal governor 'power_allocator'988worker1 # [ 0.318638] cpuidle: using governor menu989client # [ 0.318923] PCI: Using configuration type 1 for base access990lb # [ 0.306830] audit: initializing netlink subsys (disabled)991worker2 # [ 0.327487] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages992worker2 # [ 0.328487] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page993lb # [ 0.307953] thermal_sys: Registered thermal governor 'fair_share'994worker1 # [ 0.320828] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5995client # [ 0.319998] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.996lb # [ 0.307956] thermal_sys: Registered thermal governor 'bang_bang'997worker1 # [ 0.321858] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]998lb # [ 0.308815] thermal_sys: Registered thermal governor 'step_wise'999worker1 # [ 0.322627] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1000lb # [ 0.309776] thermal_sys: Registered thermal governor 'user_space'1001worker1 # [ 0.323636] PCI: Using configuration type 1 for base access1002lb # [ 0.310551] audit: type=2000 audit(1790091830.882:1): state=initialized audit_enabled=0 res=11003lb # [ 0.311817] thermal_sys: Registered thermal governor 'power_allocator'1004worker1 # [ 0.324703] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1005client # [ 0.325232] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1006lb # [ 0.311830] cpuidle: using governor menu1007client # [ 0.325912] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1008lb # [ 0.314045] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51009client # [ 0.326912] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1010lb # [ 0.315034] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1011lb # [ 0.315817] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1012worker2 # [ 0.339743] ACPI: Added _OSI(Module Device)1013lb # [ 0.316826] PCI: Using configuration type 1 for base access1014worker2 # [ 0.340488] ACPI: Added _OSI(Processor Device)1015worker2 # [ 0.341237] ACPI: Added _OSI(Processor Aggregator Device)1016worker1 # [ 0.331639] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1017lb # [ 0.317963] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1018worker1 # [ 0.332625] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1019client # [ 0.333913] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1020lb # [ 0.323095] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1021worker1 # [ 0.337626] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1022worker2 # [ 0.346759] ACPI: 1 ACPI AML tables successfully acquired and loaded1023lb # [ 0.323815] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1024worker1 # [ 0.338626] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1025worker2 # [ 0.350914] ACPI: Interpreter enabled1026client # [ 0.342258] ACPI: Added _OSI(Module Device)1027worker2 # [ 0.351501] ACPI: PM: (supports S0 S3 S4 S5)1028client # [ 0.342914] ACPI: Added _OSI(Processor Device)1029worker2 # [ 0.352243] ACPI: Using IOAPIC for interrupt routing1030lb # [ 0.328816] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1031client # [ 0.343666] ACPI: Added _OSI(Processor Aggregator Device)1032lb # [ 0.329814] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1033worker2 # [ 0.354534] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1034worker2 # [ 0.355487] PCI: Using E820 reservations for host bridge windows1035worker1 # [ 0.349868] ACPI: Added _OSI(Module Device)1036worker1 # [ 0.350555] ACPI: Added _OSI(Processor Device)1037worker2 # [ 0.358623] ACPI: Enabled 2 GPEs in block 00 to 3F1038worker1 # [ 0.351237] ACPI: Added _OSI(Processor Aggregator Device)1039client # [ 0.352497] ACPI: 1 ACPI AML tables successfully acquired and loaded1040lb # [ 0.340257] ACPI: Added _OSI(Module Device)1041lb # [ 0.340816] ACPI: Added _OSI(Processor Device)1042lb # [ 0.341579] ACPI: Added _OSI(Processor Aggregator Device)1043client # [ 0.356172] ACPI: Interpreter enabled1044worker1 # [ 0.356873] ACPI: 1 ACPI AML tables successfully acquired and loaded1045client # [ 0.356859] ACPI: PM: (supports S0 S3 S4 S5)1046worker2 # [ 0.365998] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1047client # [ 0.357473] ACPI: Using IOAPIC for interrupt routing1048worker2 # [ 0.366492] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1049client # [ 0.357967] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1050worker2 # [ 0.367564] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1051client # [ 0.358912] PCI: Using E820 reservations for host bridge windows1052worker1 # [ 0.361042] ACPI: Interpreter enabled1053worker1 # [ 0.361640] ACPI: PM: (supports S0 S3 S4 S5)1054worker2 # [ 0.368607] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1055worker1 # [ 0.362333] ACPI: Using IOAPIC for interrupt routing1056worker2 # [ 0.369849] PCI host bridge to bus 0000:001057client # [ 0.362025] ACPI: Enabled 2 GPEs in block 00 to 3F1058worker2 # [ 0.370490] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1059worker2 # [ 0.371486] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1060lb # [ 0.350422] ACPI: 1 ACPI AML tables successfully acquired and loaded1061worker1 # [ 0.364676] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1062worker2 # [ 0.372486] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1063worker1 # [ 0.365626] PCI: Using E820 reservations for host bridge windows1064worker2 # [ 0.373486] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1065worker2 # [ 0.374486] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1066lb # [ 0.354173] ACPI: Interpreter enabled1067worker1 # [ 0.368731] ACPI: Enabled 2 GPEs in block 00 to 3F1068worker2 # [ 0.375486] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1069lb # [ 0.354829] ACPI: PM: (supports S0 S3 S4 S5)1070worker2 # [ 0.376486] pci_bus 0000:00: root bus resource [bus 00-ff]1071lb # [ 0.355517] ACPI: Using IOAPIC for interrupt routing1072worker2 # [ 0.377540] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1073lb # [ 0.355891] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1074client # [ 0.370735] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1075lb # [ 0.356815] PCI: Using E820 reservations for host bridge windows1076client # [ 0.371708] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1077worker2 # [ 0.378936] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1078client # [ 0.373499] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1079lb # [ 0.359941] ACPI: Enabled 2 GPEs in block 00 to 3F1080client # [ 0.374044] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1081worker1 # [ 0.376062] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1082client # [ 0.375262] PCI host bridge to bus 0000:001083worker1 # [ 0.376631] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1084client # [ 0.375917] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1085worker1 # [ 0.377702] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1086client # [ 0.376912] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1087worker1 # [ 0.378750] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1088worker2 # [ 0.381591] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1089client # [ 0.377912] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1090worker1 # [ 0.379964] PCI host bridge to bus 0000:001091worker2 # [ 0.382498] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1092client # [ 0.378911] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1093worker2 # [ 0.383443] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1094worker1 # [ 0.380629] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1095client # [ 0.379912] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1096worker1 # [ 0.381625] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1097client # [ 0.380912] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1098lb # [ 0.368648] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1099worker2 # [ 0.384396] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1100worker1 # [ 0.382625] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1101client # [ 0.381912] pci_bus 0000:00: root bus resource [bus 00-ff]1102lb # [ 0.369564] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1103worker1 # [ 0.383625] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1104client # [ 0.382889] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1105worker2 # [ 0.386407] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1106worker1 # [ 0.384625] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1107lb # [ 0.371161] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1108client # [ 0.384386] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1109worker1 # [ 0.385625] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1110lb # [ 0.371940] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1111worker1 # [ 0.386626] pci_bus 0000:00: root bus resource [bus 00-ff]1112lb # [ 0.373170] PCI host bridge to bus 0000:001113worker2 # [ 0.389504] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1114worker2 # [ 0.390345] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1115lb # [ 0.373819] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1116worker1 # [ 0.387658] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1117worker2 # [ 0.391190] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1118lb # [ 0.374814] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1119worker1 # [ 0.389103] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1120worker2 # [ 0.391492] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1121lb # [ 0.375814] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1122lb # [ 0.376814] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1123worker2 # [ 0.392994] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1124lb # [ 0.377814] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1125client # [ 0.387043] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1126lb # [ 0.378819] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1127client # [ 0.387995] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1128lb # [ 0.379841] pci_bus 0000:00: root bus resource [bus 00-ff]1129worker2 # [ 0.395450] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1130client # [ 0.388974] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1131worker2 # [ 0.396184] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1132lb # [ 0.380837] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1133worker2 # [ 0.396508] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1134client # [ 0.390229] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1135worker1 # [ 0.391713] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1136lb # [ 0.382330] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1137worker1 # [ 0.392637] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1138worker2 # [ 0.398077] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1139client # [ 0.391616] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1140worker1 # [ 0.393630] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1141worker1 # [ 0.394835] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1142client # [ 0.394930] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1143worker2 # [ 0.400528] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1144worker1 # [ 0.396348] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1145client # [ 0.395813] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1146worker2 # [ 0.402118] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1147client # [ 0.396666] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1148worker2 # [ 0.402508] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1149client # [ 0.396918] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1150lb # [ 0.384958] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1151worker1 # [ 0.399643] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1152lb # [ 0.385827] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1153worker2 # [ 0.404067] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1154worker1 # [ 0.400480] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1155lb # [ 0.386755] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1156client # [ 0.398450] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1157worker1 # [ 0.401348] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1158lb # [ 0.387746] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1159worker1 # [ 0.401631] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1160worker2 # [ 0.406289] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1161worker2 # [ 0.407152] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1162client # [ 0.401936] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1163lb # [ 0.389545] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1164worker2 # [ 0.407507] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1165client # [ 0.402834] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1166worker1 # [ 0.403649] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1167client # [ 0.403654] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1168worker2 # [ 0.408492] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1169worker1 # [ 0.406467] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1170lb # [ 0.391879] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1171client # [ 0.404484] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1172worker2 # [ 0.410023] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1173worker1 # [ 0.407254] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1174lb # [ 0.392692] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1175worker1 # [ 0.407646] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1176lb # [ 0.393517] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1177lb # [ 0.393821] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1178worker2 # [ 0.411499] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1179worker1 # [ 0.409158] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1180worker2 # [ 0.412496] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1181client # [ 0.406921] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1182lb # [ 0.395387] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1183client # [ 0.408597] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1184worker2 # [ 0.414071] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1185client # [ 0.408934] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1186lb # [ 0.397822] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1187worker1 # [ 0.411632] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1188lb # [ 0.398690] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1189client # [ 0.410493] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1190worker1 # [ 0.413284] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1191worker2 # [ 0.415531] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1192lb # [ 0.399602] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1193worker1 # [ 0.413648] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1194worker2 # [ 0.416508] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1195lb # [ 0.401207] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1196client # [ 0.412912] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1197worker1 # [ 0.415189] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1198worker2 # [ 0.418089] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1199client # [ 0.413749] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1200client # [ 0.414676] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1201worker2 # [ 0.419535] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1202worker1 # [ 0.416635] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1203worker1 # [ 0.417518] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1204worker2 # [ 0.420508] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1205client # [ 0.414918] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1206lb # [ 0.403868] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1207worker1 # [ 0.418358] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1208worker2 # [ 0.422023] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1209lb # [ 0.405484] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1210client # [ 0.416449] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1211worker1 # [ 0.418631] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1212lb # [ 0.405836] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1213client # [ 0.417942] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1214worker1 # [ 0.420160] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1215worker2 # [ 0.423533] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1216lb # [ 0.407403] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1217client # [ 0.418878] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1218worker2 # [ 0.424508] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1219worker1 # [ 0.421638] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1220lb # [ 0.408822] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1221client # [ 0.420280] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1222worker1 # [ 0.422635] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1223worker2 # [ 0.426022] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1224lb # [ 0.409636] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1225lb # [ 0.410585] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1226worker1 # [ 0.424172] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1227client # [ 0.421943] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1228worker2 # [ 0.428356] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1229lb # [ 0.411669] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1230worker2 # [ 0.429131] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1231client # [ 0.422874] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1232worker1 # [ 0.425638] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1233worker2 # [ 0.429507] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1234lb # [ 0.413390] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1235worker1 # [ 0.426578] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1236client # [ 0.424318] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1237lb # [ 0.414697] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1238worker2 # [ 0.431062] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1239worker1 # [ 0.428200] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1240client # [ 0.425924] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1241lb # [ 0.415624] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1242client # [ 0.426868] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1243worker1 # [ 0.429647] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1244worker2 # [ 0.433097] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1245lb # [ 0.417225] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1246worker1 # [ 0.430630] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1247client # [ 0.428283] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1248worker2 # [ 0.433493] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1249worker2 # [ 0.434457] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1250lb # [ 0.418856] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1251worker1 # [ 0.432171] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1252client # [ 0.429925] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1253lb # [ 0.419814] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1254worker2 # [ 0.436017] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1255worker1 # [ 0.433652] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1256client # [ 0.430869] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1257worker2 # [ 0.437328] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1258lb # [ 0.421872] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1259worker1 # [ 0.434636] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1260client # [ 0.432287] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1261lb # [ 0.423627] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1262worker2 # [ 0.438407] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1263worker1 # [ 0.436165] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1264lb # [ 0.424575] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1265client # [ 0.434739] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1266worker2 # [ 0.440325] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1267client # [ 0.435540] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1268worker1 # [ 0.437650] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1269lb # [ 0.425342] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1270worker2 # [ 0.441347] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1271client # [ 0.435945] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1272worker1 # [ 0.438479] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1273worker1 # [ 0.439354] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1274worker2 # [ 0.443147] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1275lb # [ 0.426857] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1276client # [ 0.437455] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1277lb # [ 0.427794] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1278worker1 # [ 0.440161] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1279lb # [ 0.429290] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1280client # [ 0.439937] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1281client # [ 0.440809] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1282worker1 # [ 0.441633] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1283lb # [ 0.430831] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1284client # [ 0.441639] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1285worker1 # [ 0.443290] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1286lb # [ 0.431708] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1287worker1 # [ 0.443647] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1288lb # [ 0.432609] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1289client # [ 0.442483] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1290worker1 # [ 0.445204] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1291client # [ 0.443623] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1292lb # [ 0.434262] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1293worker1 # [ 0.446378] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1294client # [ 0.444140] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1295worker1 # [ 0.446835] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1296worker2 # [ 0.443698] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1297client # [ 0.445627] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1298lb # [ 0.437100] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1299worker1 # [ 0.448273] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1300lb # [ 0.438448] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1301worker2 # [ 0.446282] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1302client # [ 0.446114] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1303lb # [ 0.438836] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1304worker1 # [ 0.448814] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1305client # [ 0.447730] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1306worker2 # [ 0.447468] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1307worker1 # [ 0.450326] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1308lb # [ 0.440410] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1309worker2 # [ 0.448778] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1310lb # [ 0.441382] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1311worker2 # [ 0.449734] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1312lb # [ 0.442022] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1313worker2 # [ 0.451514] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1314lb # [ 0.443352] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1315worker2 # [ 0.452428] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1316lb # [ 0.444013] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1317worker2 # [ 0.453628] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1318lb # [ 0.446384] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1319worker2 # [ 0.455200] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1320client # [ 0.448751] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1321worker2 # [ 0.458585] ACPI: PCI: Interrupt link LNKA configured for IRQ 101322worker1 # [ 0.450851] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1323client # [ 0.450631] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1324worker2 # [ 0.459596] ACPI: PCI: Interrupt link LNKB configured for IRQ 101325worker1 # [ 0.453293] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1326worker2 # [ 0.460582] ACPI: PCI: Interrupt link LNKC configured for IRQ 111327client # [ 0.451195] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1328worker2 # [ 0.461585] ACPI: PCI: Interrupt link LNKD configured for IRQ 111329worker1 # [ 0.453901] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1330client # [ 0.452404] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1331worker2 # [ 0.462581] ACPI: PCI: Interrupt link LNKE configured for IRQ 101332worker2 # [ 0.463581] ACPI: PCI: Interrupt link LNKF configured for IRQ 101333worker1 # [ 0.454919] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1334client # [ 0.453179] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1335worker2 # [ 0.464612] ACPI: PCI: Interrupt link LNKG configured for IRQ 111336worker1 # [ 0.455887] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1337worker2 # [ 0.465598] ACPI: PCI: Interrupt link LNKH configured for IRQ 111338client # [ 0.454959] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1339lb # [ 0.447032] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1340client # [ 0.455792] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1341worker1 # [ 0.457642] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1342worker2 # [ 0.466526] ACPI: PCI: Interrupt link GSIA configured for IRQ 161343lb # [ 0.449391] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1344worker1 # [ 0.458482] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1345worker2 # [ 0.467501] ACPI: PCI: Interrupt link GSIB configured for IRQ 171346client # [ 0.456947] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1347worker2 # [ 0.468461] ACPI: PCI: Interrupt link GSIC configured for IRQ 181348lb # [ 0.450115] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1349worker1 # [ 0.459652] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1350worker2 # [ 0.469255] ACPI: PCI: Interrupt link GSID configured for IRQ 191351client # [ 0.458820] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1352worker2 # [ 0.469498] ACPI: PCI: Interrupt link GSIE configured for IRQ 201353lb # [ 0.451129] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1354worker1 # [ 0.461274] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1355worker2 # [ 0.470487] ACPI: PCI: Interrupt link GSIF configured for IRQ 211356lb # [ 0.452225] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1357worker2 # [ 0.471498] ACPI: PCI: Interrupt link GSIG configured for IRQ 221358worker2 # [ 0.472500] ACPI: PCI: Interrupt link GSIH configured for IRQ 231359lb # [ 0.453749] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1360client # [ 0.463043] ACPI: PCI: Interrupt link LNKA configured for IRQ 101361worker1 # [ 0.464900] ACPI: PCI: Interrupt link LNKA configured for IRQ 101362lb # [ 0.454512] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1363worker2 # [ 0.474577] iommu: Default domain type: Translated1364client # [ 0.464024] ACPI: PCI: Interrupt link LNKB configured for IRQ 101365worker1 # [ 0.465732] ACPI: PCI: Interrupt link LNKB configured for IRQ 101366worker2 # [ 0.475411] iommu: DMA domain TLB invalidation policy: lazy mode1367client # [ 0.465015] ACPI: PCI: Interrupt link LNKC configured for IRQ 111368lb # [ 0.455124] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1369worker1 # [ 0.466733] ACPI: PCI: Interrupt link LNKC configured for IRQ 111370worker2 # [ 0.476498] ACPI: bus type USB registered1371client # [ 0.466012] ACPI: PCI: Interrupt link LNKD configured for IRQ 111372lb # [ 0.456473] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1373worker1 # [ 0.467723] ACPI: PCI: Interrupt link LNKD configured for IRQ 111374worker2 # [ 0.477253] usbcore: registered new interface driver usbfs1375client # [ 0.467011] ACPI: PCI: Interrupt link LNKE configured for IRQ 101376worker2 # [ 0.477502] usbcore: registered new interface driver hub1377worker1 # [ 0.468725] ACPI: PCI: Interrupt link LNKE configured for IRQ 101378client # [ 0.468013] ACPI: PCI: Interrupt link LNKF configured for IRQ 101379worker2 # [ 0.478384] usbcore: registered new device driver usb1380worker1 # [ 0.469722] ACPI: PCI: Interrupt link LNKF configured for IRQ 101381client # [ 0.469015] ACPI: PCI: Interrupt link LNKG configured for IRQ 111382worker1 # [ 0.470729] ACPI: PCI: Interrupt link LNKG configured for IRQ 111383worker2 # [ 0.479956] NetLabel: Initializing1384client # [ 0.470013] ACPI: PCI: Interrupt link LNKH configured for IRQ 111385worker2 # [ 0.480487] NetLabel: domain hash size = 1281386lb # [ 0.459945] ACPI: PCI: Interrupt link LNKA configured for IRQ 101387worker1 # [ 0.471726] ACPI: PCI: Interrupt link LNKH configured for IRQ 111388client # [ 0.470955] ACPI: PCI: Interrupt link GSIA configured for IRQ 161389worker2 # [ 0.481210] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1390lb # [ 0.460927] ACPI: PCI: Interrupt link LNKB configured for IRQ 101391worker1 # [ 0.472669] ACPI: PCI: Interrupt link GSIA configured for IRQ 161392client # [ 0.471929] ACPI: PCI: Interrupt link GSIB configured for IRQ 171393worker2 # [ 0.481557] NetLabel: unlabeled traffic allowed by default1394lb # [ 0.461920] ACPI: PCI: Interrupt link LNKC configured for IRQ 111395worker1 # [ 0.473641] ACPI: PCI: Interrupt link GSIB configured for IRQ 171396worker2 # [ 0.482452] PCI: Using ACPI for IRQ routing1397client # [ 0.472877] ACPI: PCI: Interrupt link GSIC configured for IRQ 181398lb # [ 0.462914] ACPI: PCI: Interrupt link LNKD configured for IRQ 111399worker1 # [ 0.474641] ACPI: PCI: Interrupt link GSIC configured for IRQ 181400client # [ 0.473616] ACPI: PCI: Interrupt link GSID configured for IRQ 191401lb # [ 0.463927] ACPI: PCI: Interrupt link LNKE configured for IRQ 101402worker1 # [ 0.475597] ACPI: PCI: Interrupt link GSID configured for IRQ 191403client # [ 0.473926] ACPI: PCI: Interrupt link GSIE configured for IRQ 201404lb # [ 0.464909] ACPI: PCI: Interrupt link LNKF configured for IRQ 101405worker1 # [ 0.476357] ACPI: PCI: Interrupt link GSIE configured for IRQ 201406client # [ 0.474878] ACPI: PCI: Interrupt link GSIF configured for IRQ 211407lb # [ 0.465912] ACPI: PCI: Interrupt link LNKG configured for IRQ 111408worker1 # [ 0.476641] ACPI: PCI: Interrupt link GSIF configured for IRQ 211409client # [ 0.475634] ACPI: PCI: Interrupt link GSIG configured for IRQ 221410worker1 # [ 0.477586] ACPI: PCI: Interrupt link GSIG configured for IRQ 221411lb # [ 0.466909] ACPI: PCI: Interrupt link LNKH configured for IRQ 111412client # [ 0.475928] ACPI: PCI: Interrupt link GSIH configured for IRQ 231413worker1 # [ 0.478338] ACPI: PCI: Interrupt link GSIH configured for IRQ 231414lb # [ 0.467850] ACPI: PCI: Interrupt link GSIA configured for IRQ 161415client # [ 0.477879] iommu: Default domain type: Translated1416lb # [ 0.468826] ACPI: PCI: Interrupt link GSIB configured for IRQ 171417worker1 # [ 0.479655] iommu: Default domain type: Translated1418client # [ 0.478510] iommu: DMA domain TLB invalidation policy: lazy mode1419worker1 # [ 0.480464] iommu: DMA domain TLB invalidation policy: lazy mode1420lb # [ 0.469829] ACPI: PCI: Interrupt link GSIC configured for IRQ 181421client # [ 0.479181] ACPI: bus type USB registered1422worker1 # [ 0.481641] ACPI: bus type USB registered1423lb # [ 0.470832] ACPI: PCI: Interrupt link GSID configured for IRQ 191424client # [ 0.479928] usbcore: registered new interface driver usbfs1425worker1 # [ 0.482374] usbcore: registered new interface driver usbfs1426lb # [ 0.471830] ACPI: PCI: Interrupt link GSIE configured for IRQ 201427client # [ 0.480796] usbcore: registered new interface driver hub1428worker1 # [ 0.482639] usbcore: registered new interface driver hub1429client # [ 0.481540] usbcore: registered new device driver usb1430lb # [ 0.472827] ACPI: PCI: Interrupt link GSIF configured for IRQ 211431worker1 # [ 0.483477] usbcore: registered new device driver usb1432lb # [ 0.473793] ACPI: PCI: Interrupt link GSIG configured for IRQ 221433client # [ 0.482766] NetLabel: Initializing1434lb # [ 0.474602] ACPI: PCI: Interrupt link GSIH configured for IRQ 231435client # [ 0.483349] NetLabel: domain hash size = 1281436worker1 # [ 0.485157] NetLabel: Initializing1437worker1 # [ 0.485626] NetLabel: domain hash size = 1281438client # [ 0.483912] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1439worker1 # [ 0.486359] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1440client # [ 0.484830] NetLabel: unlabeled traffic allowed by default1441lb # [ 0.476668] iommu: Default domain type: Translated1442client # [ 0.485559] PCI: Using ACPI for IRQ routing1443worker1 # [ 0.486662] NetLabel: unlabeled traffic allowed by default1444lb # [ 0.477406] iommu: DMA domain TLB invalidation policy: lazy mode1445worker1 # [ 0.487551] PCI: Using ACPI for IRQ routing1446lb # [ 0.478078] ACPI: bus type USB registered1447lb # [ 0.478868] usbcore: registered new interface driver usbfs1448lb # [ 0.479832] usbcore: registered new interface driver hub1449lb # [ 0.480749] usbcore: registered new device driver usb1450lb # [ 0.482393] NetLabel: Initializing1451lb # [ 0.482834] NetLabel: domain hash size = 1281452lb # [ 0.483608] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453lb # [ 0.484762] NetLabel: unlabeled traffic allowed by default1454lb # [ 0.485577] PCI: Using ACPI for IRQ routing1455worker2 # [ 0.526615] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456worker2 # [ 0.527482] pci 0000:00:01.0: vgaarb: bridge control possible1457worker2 # [ 0.527482] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458worker2 # [ 0.527490] vgaarb: loaded1459worker2 # [ 0.528297] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460worker2 # [ 0.529140] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461worker2 # [ 0.532572] clocksource: Switched to clocksource kvm-clock1462worker2 # [ 0.536100] VFS: Disk quotas dquot_6.6.01463worker2 # [ 0.536816] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464client # [ 0.528999] pci 0000:00:01.0: vgaarb: setting as boot VGA device1465worker2 # [ 0.538268] pnp: PnP ACPI init1466client # [ 0.529908] pci 0000:00:01.0: vgaarb: bridge control possible1467worker1 # [ 0.531350] pci 0000:00:01.0: vgaarb: setting as boot VGA device1468worker2 # [ 0.539035] ACPI: IRQ 4 override to edge(!), high(!)1469worker1 # [ 0.531621] pci 0000:00:01.0: vgaarb: bridge control possible1470client # [ 0.529908] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1471worker2 # [ 0.539941] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1472client # [ 0.529916] vgaarb: loaded1473worker1 # [ 0.531621] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1474worker2 # [ 0.541452] pnp: PnP ACPI: found 5 devices1475client # [ 0.530675] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01476worker1 # [ 0.531630] vgaarb: loaded1477client # [ 0.530914] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1478worker1 # [ 0.532368] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01479worker1 # [ 0.532625] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1480client # [ 0.533997] clocksource: Switched to clocksource kvm-clock1481lb # [ 0.529802] pci 0000:00:01.0: vgaarb: setting as boot VGA device1482lb # [ 0.529810] pci 0000:00:01.0: vgaarb: bridge control possible1483worker2 # [ 0.549054] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1484lb # [ 0.529810] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1485client # [ 0.537603] VFS: Disk quotas dquot_6.6.01486lb # [ 0.529818] vgaarb: loaded1487worker2 # [ 0.550543] clocksource: Switched to clocksource acpi_pm1488worker1 # [ 0.539704] clocksource: Switched to clocksource kvm-clock1489client # [ 0.538289] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1490worker2 # [ 0.551504] NET: Registered PF_INET protocol family1491lb # [ 0.530537] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01492client # [ 0.539629] pnp: PnP ACPI init1493worker1 # [ 0.541368] VFS: Disk quotas dquot_6.6.01494lb # [ 0.530814] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1495worker2 # [ 0.552676] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1496client # [ 0.540431] ACPI: IRQ 4 override to edge(!), high(!)1497client # [ 0.541428] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1498worker1 # [ 0.544020] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1499client # [ 0.543018] pnp: PnP ACPI: found 5 devices1500worker1 # [ 0.545395] pnp: PnP ACPI init1501worker1 # [ 0.546222] ACPI: IRQ 4 override to edge(!), high(!)1502lb # [ 0.534899] clocksource: Switched to clocksource kvm-clock1503worker1 # [ 0.547209] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1504lb # [ 0.536572] VFS: Disk quotas dquot_6.6.01505worker1 # [ 0.548805] pnp: PnP ACPI: found 5 devices1506lb # [ 0.539148] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1507lb # [ 0.540556] pnp: PnP ACPI init1508client # [ 0.550662] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1509lb # [ 0.541402] ACPI: IRQ 4 override to edge(!), high(!)1510client # [ 0.552111] clocksource: Switched to clocksource acpi_pm1511lb # [ 0.542398] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1512client # [ 0.553108] NET: Registered PF_INET protocol family1513lb # [ 0.543991] pnp: PnP ACPI: found 5 devices1514client # [ 0.554314] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1515worker1 # [ 0.556404] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1516worker1 # [ 0.557910] clocksource: Switched to clocksource acpi_pm1517worker2 # [ 0.569028] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1518worker1 # [ 0.558875] NET: Registered PF_INET protocol family1519worker2 # [ 0.570568] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1520worker1 # [ 0.560020] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1521worker2 # [ 0.571935] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1522worker2 # [ 0.573328] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1523worker2 # [ 0.574637] TCP: Hash tables configured (established 16384 bind 16384)1524lb # [ 0.551600] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1525lb # [ 0.553101] clocksource: Switched to clocksource acpi_pm1526worker2 # [ 0.575813] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1527lb # [ 0.554095] NET: Registered PF_INET protocol family1528worker2 # [ 0.577171] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1529lb # [ 0.555332] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1530worker2 # [ 0.578319] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1531worker2 # [ 0.579570] NET: Registered PF_UNIX/PF_LOCAL protocol family1532worker2 # [ 0.580593] NET: Registered PF_XDP protocol family1533worker2 # [ 0.581487] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1534worker2 # [ 0.582543] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1535worker2 # [ 0.583621] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1536client # [ 0.571213] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1537worker2 # [ 0.584748] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1538client # [ 0.572759] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1539worker2 # [ 0.585915] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1540client # [ 0.574071] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1541worker2 # [ 0.587035] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1542worker1 # [ 0.576223] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1543client # [ 0.575448] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1544worker2 # [ 0.588744] ACPI: \_SB_.GSIA: Enabled at IRQ 161545worker1 # [ 0.577826] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1546client # [ 0.576710] TCP: Hash tables configured (established 16384 bind 16384)1547worker1 # [ 0.579189] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1548client # [ 0.577830] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1549worker2 # [ 0.591485] ACPI: \_SB_.GSIB: Enabled at IRQ 171550client # [ 0.579092] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1551worker1 # [ 0.580560] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1552worker1 # [ 0.581784] TCP: Hash tables configured (established 16384 bind 16384)1553client # [ 0.580193] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1554worker2 # [ 0.593534] ACPI: \_SB_.GSIC: Enabled at IRQ 181555client # [ 0.581401] NET: Registered PF_UNIX/PF_LOCAL protocol family1556worker1 # [ 0.582893] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1557client # [ 0.582414] NET: Registered PF_XDP protocol family1558worker2 # [ 0.595512] ACPI: \_SB_.GSID: Enabled at IRQ 191559lb # [ 0.572167] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1560worker1 # [ 0.584240] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1561client # [ 0.583228] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1562worker1 # [ 0.585277] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1563lb # [ 0.573705] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1564client # [ 0.584209] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1565worker2 # [ 0.597384] PCI: CLS 0 bytes, default 641566worker1 # [ 0.586471] NET: Registered PF_UNIX/PF_LOCAL protocol family1567client # [ 0.585174] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1568lb # [ 0.574990] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1569worker1 # [ 0.587385] NET: Registered PF_XDP protocol family1570worker2 # [ 0.598279] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1571client # [ 0.586233] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1572lb # [ 0.576289] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1573worker1 # [ 0.588214] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1574worker2 # [ 0.599942] Trying to unpack rootfs image as initramfs...1575client # [ 0.587257] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1576lb # [ 0.577493] TCP: Hash tables configured (established 16384 bind 16384)1577worker1 # [ 0.589195] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1578client # [ 0.588299] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1579lb # [ 0.578584] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1580worker1 # [ 0.590183] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1581client # [ 0.589981] ACPI: \_SB_.GSIA: Enabled at IRQ 161582worker1 # [ 0.591242] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1583lb # [ 0.579797] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1584worker1 # [ 0.592315] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1585lb # [ 0.580851] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1586lb # [ 0.582095] NET: Registered PF_UNIX/PF_LOCAL protocol family1587worker1 # [ 0.593462] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1588client # [ 0.592351] ACPI: \_SB_.GSIB: Enabled at IRQ 171589lb # [ 0.583043] NET: Registered PF_XDP protocol family1590worker1 # [ 0.595165] ACPI: \_SB_.GSIA: Enabled at IRQ 161591lb # [ 0.583938] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1592client # [ 0.594226] ACPI: \_SB_.GSIC: Enabled at IRQ 181593lb # [ 0.584950] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1594worker1 # [ 0.597588] ACPI: \_SB_.GSIB: Enabled at IRQ 171595client # [ 0.596169] ACPI: \_SB_.GSID: Enabled at IRQ 191596lb # [ 0.586008] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1597lb # [ 0.587082] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1598client # [ 0.597905] PCI: CLS 0 bytes, default 641599worker1 # [ 0.599927] ACPI: \_SB_.GSIC: Enabled at IRQ 181600lb # [ 0.588233] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1601client # [ 0.598815] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1602lb # [ 0.589334] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1603worker1 # [ 0.601944] ACPI: \_SB_.GSID: Enabled at IRQ 191604client # [ 0.600544] Trying to unpack rootfs image as initramfs...1605lb # [ 0.591143] ACPI: \_SB_.GSIA: Enabled at IRQ 161606worker1 # [ 0.603692] PCI: CLS 0 bytes, default 641607lb # [ 0.593590] ACPI: \_SB_.GSIB: Enabled at IRQ 171608worker1 # [ 0.604659] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1609worker1 # [ 0.606435] Trying to unpack rootfs image as initramfs...1610lb # [ 0.596188] ACPI: \_SB_.GSIC: Enabled at IRQ 181611lb # [ 0.598224] ACPI: \_SB_.GSID: Enabled at IRQ 191612lb # [ 0.599967] PCI: CLS 0 bytes, default 641613lb # [ 0.600960] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614lb # [ 0.602754] Trying to unpack rootfs image as initramfs...1615worker2 # [ 0.643863] Initialise system trusted keyrings1616worker2 # [ 0.646906] workingset: timestamp_bits=40 max_order=19 bucket_order=01617client # [ 0.645938] Initialise system trusted keyrings1618worker1 # [ 0.648807] Initialise system trusted keyrings1619client # [ 0.649615] workingset: timestamp_bits=40 max_order=19 bucket_order=01620worker1 # [ 0.652482] workingset: timestamp_bits=40 max_order=19 bucket_order=01621worker2 # [ 0.669890] Key type asymmetric registered1622worker2 # [ 0.670617] Asymmetric key parser 'x509' registered1623lb # [ 0.648062] Initialise system trusted keyrings1624worker2 # [ 0.671523] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1625lb # [ 0.651691] workingset: timestamp_bits=40 max_order=19 bucket_order=01626worker2 # [ 0.674970] io scheduler mq-deadline registered1627worker2 # [ 0.675736] io scheduler kyber registered1628worker2 # [ 0.678450] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1629worker2 # [ 0.679779] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630client # [ 0.670849] Key type asymmetric registered1631client # [ 0.671587] Asymmetric key parser 'x509' registered1632worker1 # [ 0.673711] Key type asymmetric registered1633worker2 # [ 0.685274] Linux agpgart interface v0.1031634client # [ 0.672380] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1635worker1 # [ 0.674446] Asymmetric key parser 'x509' registered1636worker2 # [ 0.686022] ACPI: bus type drm_connector registered1637worker1 # [ 0.675276] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1638client # [ 0.675708] io scheduler mq-deadline registered1639client # [ 0.676448] io scheduler kyber registered1640worker2 # [ 0.689291] usbcore: registered new interface driver usbserial_generic1641worker1 # [ 0.678581] io scheduler mq-deadline registered1642worker1 # [ 0.679368] io scheduler kyber registered1643worker2 # [ 0.690398] usbserial: USB Serial support registered for generic1644client # [ 0.679127] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1645worker1 # [ 0.681995] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1646client # [ 0.680403] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1647worker2 # [ 0.692874] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1648lb # [ 0.670868] Key type asymmetric registered1649worker2 # [ 0.692874] Please enable it if your BIOS has the CPPC option.1650lb # [ 0.671560] Asymmetric key parser 'x509' registered1651worker1 # [ 0.683287] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1652worker2 # [ 0.695028] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1653lb # [ 0.674693] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1654worker2 # [ 0.698034] drop_monitor: Initializing network drop monitor service1655client # [ 0.685968] Linux agpgart interface v0.1031656worker2 # [ 0.699212] NET: Registered PF_INET6 protocol family1657client # [ 0.686735] ACPI: bus type drm_connector registered1658worker1 # [ 0.689094] Linux agpgart interface v0.1031659worker1 # [ 0.689876] ACPI: bus type drm_connector registered1660lb # [ 0.679684] io scheduler mq-deadline registered1661client # [ 0.690040] usbcore: registered new interface driver usbserial_generic1662lb # [ 0.680463] io scheduler kyber registered1663client # [ 0.691098] usbserial: USB Serial support registered for generic1664worker2 # [ 0.704311] Segment Routing with IPv61665worker1 # [ 0.692862] usbcore: registered new interface driver usbserial_generic1666worker2 # [ 0.704981] In-situ OAM (IOAM) with IPv61667worker1 # [ 0.693954] usbserial: USB Serial support registered for generic1668lb # [ 0.683822] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1669client # [ 0.693566] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1670worker2 # [ 0.707146] IPI shorthand broadcast: enabled1671client # [ 0.693566] Please enable it if your BIOS has the CPPC option.1672lb # [ 0.685110] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1673client # [ 0.695720] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1674worker1 # [ 0.696449] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1675worker1 # [ 0.696449] Please enable it if your BIOS has the CPPC option.1676worker1 # [ 0.698588] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1677client # [ 0.698670] drop_monitor: Initializing network drop monitor service1678client # [ 0.699868] NET: Registered PF_INET6 protocol family1679lb # [ 0.690457] Linux agpgart interface v0.1031680worker1 # [ 0.702563] drop_monitor: Initializing network drop monitor service1681lb # [ 0.691270] ACPI: bus type drm_connector registered1682worker1 # [ 0.703779] NET: Registered PF_INET6 protocol family1683worker2 # [ 0.715564] sched_clock: Marking stable (580015275, 134833136)->(787330733, -72482322)1684lb # [ 0.694087] usbcore: registered new interface driver usbserial_generic1685client # [ 0.704675] Segment Routing with IPv61686client # [ 0.705301] In-situ OAM (IOAM) with IPv61687worker1 # [ 0.706905] Segment Routing with IPv61688lb # [ 0.695167] usbserial: USB Serial support registered for generic1689worker2 # [ 0.719069] registered taskstats version 11690client # [ 0.706349] IPI shorthand broadcast: enabled1691worker2 # [ 0.720018] Loading compiled-in X.509 certificates1692worker1 # [ 0.709457] In-situ OAM (IOAM) with IPv61693worker1 # [ 0.710532] IPI shorthand broadcast: enabled1694lb # [ 0.699646] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1695lb # [ 0.699646] Please enable it if your BIOS has the CPPC option.1696lb # [ 0.701895] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1697lb # [ 0.705812] drop_monitor: Initializing network drop monitor service1698client # [ 0.715919] sched_clock: Marking stable (580015969, 135522357)->(791678811, -76140485)1699lb # [ 0.707010] NET: Registered PF_INET6 protocol family1700worker1 # [ 0.718930] sched_clock: Marking stable (582015150, 136408583)->(792877135, -74453402)1701lb # [ 0.710184] Segment Routing with IPv61702client # [ 0.720651] registered taskstats version 11703lb # [ 0.710895] In-situ OAM (IOAM) with IPv61704client # [ 0.721610] Loading compiled-in X.509 certificates1705worker1 # [ 0.724608] registered taskstats version 11706lb # [ 0.713949] IPI shorthand broadcast: enabled1707worker1 # [ 0.725633] Loading compiled-in X.509 certificates1708worker2 # [ 0.737862] Demotion targets for Node 0: null1709worker2 # [ 0.740895] Key type .fscrypt registered1710worker2 # [ 0.741578] Key type fscrypt-provisioning registered1711worker2 # [ 0.742583] ima: No TPM chip found, activating TPM-bypass!1712worker2 # [ 0.744865] ima: Allocated hash algorithm: sha11713lb # [ 0.722369] sched_clock: Marking stable (588016287, 133612042)->(792424338, -70796009)1714worker2 # [ 0.745660] ima: No architecture policies found1715lb # [ 0.725862] registered taskstats version 11716lb # [ 0.726867] Loading compiled-in X.509 certificates1717worker2 # [ 0.749860] PM: Magic number: 10:946:7401718worker2 # [ 0.751479] RAS: Correctable Errors collector initialized.1719client # [ 0.740563] Demotion targets for Node 0: null1720client # [ 0.741478] Key type .fscrypt registered1721worker1 # [ 0.744429] Demotion targets for Node 0: null1722worker1 # [ 0.745355] Key type .fscrypt registered1723client # [ 0.744543] Key type fscrypt-provisioning registered1724worker1 # [ 0.746098] Key type fscrypt-provisioning registered1725client # [ 0.745493] ima: No TPM chip found, activating TPM-bypass!1726worker1 # [ 0.747084] ima: No TPM chip found, activating TPM-bypass!1727client # [ 0.746407] ima: Allocated hash algorithm: sha11728worker2 # [ 0.759525] clk: Disabling unused clocks1729client # [ 0.747219] ima: No architecture policies found1730client # [ 0.749562] PM: Magic number: 10:946:7401731worker1 # [ 0.751433] ima: Allocated hash algorithm: sha11732worker1 # [ 0.752300] ima: No architecture policies found1733worker2 # [ 0.763877] PM: genpd: Disabling unused power domains1734client # [ 0.751171] RAS: Correctable Errors collector initialized.1735worker1 # [ 0.754441] PM: Magic number: 10:946:7401736worker1 # [ 0.756053] RAS: Correctable Errors collector initialized.1737lb # [ 0.745642] Demotion targets for Node 0: null1738lb # [ 0.746519] Key type .fscrypt registered1739lb # [ 0.749631] Key type fscrypt-provisioning registered1740client # [ 0.760284] clk: Disabling unused clocks1741lb # [ 0.750555] ima: No TPM chip found, activating TPM-bypass!1742lb # [ 0.751438] ima: Allocated hash algorithm: sha11743lb # [ 0.752234] ima: No architecture policies found1744worker1 # [ 0.765166] clk: Disabling unused clocks1745client # [ 0.763576] PM: genpd: Disabling unused power domains1746lb # [ 0.754638] PM: Magic number: 10:946:7401747lb # [ 0.756311] RAS: Correctable Errors collector initialized.1748worker1 # [ 0.769450] PM: genpd: Disabling unused power domains1749lb # [ 0.766402] clk: Disabling unused clocks1750lb # [ 0.768650] PM: genpd: Disabling unused power domains1751worker2 # [ 0.931889] Freeing initrd memory: 29060K1752worker2 # [ 0.935029] Freeing unused decrypted memory: 2028K1753worker2 # [ 0.937636] Freeing unused kernel image (initmem) memory: 3652K1754worker2 # [ 0.938761] Write protecting the kernel read-only data: 32768k1755worker1 # [ 0.929696] Freeing initrd memory: 29056K1756worker2 # [ 0.940629] Freeing unused kernel image (text/rodata gap) memory: 1184K1757worker2 # [ 0.942120] Freeing unused kernel image (rodata/data gap) memory: 720K1758worker1 # [ 0.932795] Freeing unused decrypted memory: 2028K1759worker1 # [ 0.935382] Freeing unused kernel image (initmem) memory: 3652K1760worker1 # [ 0.936505] Write protecting the kernel read-only data: 32768k1761client # [ 0.935642] Freeing initrd memory: 29056K1762worker1 # [ 0.938338] Freeing unused kernel image (text/rodata gap) memory: 1184K1763worker1 # [ 0.939853] Freeing unused kernel image (rodata/data gap) memory: 720K1764client # [ 0.938772] Freeing unused decrypted memory: 2028K1765client # [ 0.941321] Freeing unused kernel image (initmem) memory: 3652K1766client # [ 0.942456] Write protecting the kernel read-only data: 32768k1767client # [ 0.944337] Freeing unused kernel image (text/rodata gap) memory: 1184K1768client # [ 0.945907] Freeing unused kernel image (rodata/data gap) memory: 720K1769lb # [ 0.942294] Freeing initrd memory: 29064K1770lb # [ 0.945481] Freeing unused decrypted memory: 2028K1771lb # [ 0.948049] Freeing unused kernel image (initmem) memory: 3652K1772lb # [ 0.949126] Write protecting the kernel read-only data: 32768k1773lb # [ 0.950975] Freeing unused kernel image (text/rodata gap) memory: 1184K1774lb # [ 0.952435] Freeing unused kernel image (rodata/data gap) memory: 720K1775worker2 # [ 0.993132] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776worker2 # [ 0.994264] Run /init as init process1777worker1 # [ 0.990770] x86/mm: Checked W+X mappings: passed, no W+X pages found.1778worker1 # [ 0.991829] Run /init as init process1779worker2 # [ 1.005122] systemd[1]: Inserted module 'autofs4'1780client # [ 0.996767] x86/mm: Checked W+X mappings: passed, no W+X pages found.1781client # [ 0.997872] Run /init as init process1782worker1 # [ 1.002358] systemd[1]: Inserted module 'autofs4'1783client # [ 1.008643] systemd[1]: Inserted module 'autofs4'1784worker2 # [ 1.025044] fuse: init (API version 7.45)1785lb # [ 1.003401] x86/mm: Checked W+X mappings: passed, no W+X pages found.1786lb # [ 1.004458] Run /init as init process1787worker2 # [ 1.032502] ACPI: \_SB_.GSIG: Enabled at IRQ 221788worker1 # [ 1.022007] fuse: init (API version 7.45)1789worker2 # [ 1.035610] ACPI: \_SB_.GSIH: Enabled at IRQ 231790lb # [ 1.015269] systemd[1]: Inserted module 'autofs4'1791worker2 # [ 1.039361] ACPI: \_SB_.GSIE: Enabled at IRQ 201792worker1 # [ 1.029372] ACPI: \_SB_.GSIG: Enabled at IRQ 221793client # [ 1.028752] fuse: init (API version 7.45)1794worker2 # [ 1.041639] ACPI: \_SB_.GSIF: Enabled at IRQ 211795worker1 # [ 1.031880] ACPI: \_SB_.GSIH: Enabled at IRQ 231796worker1 # [ 1.034898] ACPI: \_SB_.GSIE: Enabled at IRQ 201797worker2 # [ 1.047440] virtiofs virtio5: discovered new tag: nix-store1798worker1 # [ 1.036870] ACPI: \_SB_.GSIF: Enabled at IRQ 211799client # [ 1.036233] ACPI: \_SB_.GSIG: Enabled at IRQ 221800worker2 # [ 1.049064] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1801client # [ 1.038697] ACPI: \_SB_.GSIH: Enabled at IRQ 231802worker1 # [ 1.041944] virtiofs virtio5: discovered new tag: nix-store1803worker1 # [ 1.043498] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1804client # [ 1.042296] ACPI: \_SB_.GSIE: Enabled at IRQ 201805worker2 # [ 1.056539] virtiofs virtio6: discovered new tag: shared1806client # [ 1.044792] ACPI: \_SB_.GSIF: Enabled at IRQ 211807lb # [ 1.035021] fuse: init (API version 7.45)1808worker2 # [ 1.058110] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1809worker2 # [ 1.061595] virtiofs virtio7: discovered new tag: xchg1810worker1 # [ 1.050666] virtiofs virtio6: discovered new tag: shared1811worker2 # [ 1.063138] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1812client # [ 1.050654] virtiofs virtio5: discovered new tag: nix-store1813worker1 # [ 1.052360] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1814lb # [ 1.042287] ACPI: \_SB_.GSIG: Enabled at IRQ 221815client # [ 1.052267] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1816worker1 # [ 1.055390] virtiofs virtio7: discovered new tag: xchg1817lb # [ 1.044727] ACPI: \_SB_.GSIH: Enabled at IRQ 231818worker1 # [ 1.056872] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1819lb # [ 1.047708] ACPI: \_SB_.GSIE: Enabled at IRQ 201820lb # [ 1.049556] ACPI: \_SB_.GSIF: Enabled at IRQ 211821client # [ 1.059480] virtiofs virtio6: discovered new tag: shared1822client # [ 1.061336] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1823lb # [ 1.054807] virtiofs virtio5: discovered new tag: nix-store1824client # [ 1.065003] virtiofs virtio7: discovered new tag: xchg1825lb # [ 1.056335] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1826client # [ 1.066509] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1827worker2 # [ 1.084659] systemd[1]: Successfully made /usr/ read-only.1828lb # [ 1.062680] virtiofs virtio6: discovered new tag: shared1829lb # [ 1.064100] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1830lb # [ 1.067181] virtiofs virtio7: discovered new tag: xchg1831worker1 # [ 1.079545] systemd[1]: Successfully made /usr/ read-only.1832lb # [ 1.068607] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833client # [ 1.089716] systemd[1]: Successfully made /usr/ read-only.1834lb # [ 1.090524] systemd[1]: Successfully made /usr/ read-only.1835worker2 # [ 1.420435] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836worker2 # [ 1.427441] systemd[1]: Detected virtualization kvm.1837worker2 # [ 1.428297] systemd[1]: Detected architecture x86-64.1838worker2 # [ 1.429132] systemd[1]: Running in initrd.1839worker1 # [ 1.415210] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1840worker2 # [ 1.430160] systemd[1]: Initializing machine ID from random generator.1841worker1 # [ 1.420012] systemd[1]: Detected virtualization kvm.1842worker2 # [ 1.431341] systemd[1]: Hostname set to <worker2>.1843worker1 # [ 1.420841] systemd[1]: Detected architecture x86-64.1844worker1 # [ 1.421676] systemd[1]: Running in initrd.1845worker1 # [ 1.422815] systemd[1]: Initializing machine ID from random generator.1846worker1 # [ 1.424083] systemd[1]: Hostname set to <worker1>.1847client # [ 1.425369] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848client # [ 1.430594] systemd[1]: Detected virtualization kvm.1849client # [ 1.431492] systemd[1]: Detected architecture x86-64.1850client # [ 1.432400] systemd[1]: Running in initrd.1851client # [ 1.433609] systemd[1]: Initializing machine ID from random generator.1852client # [ 1.434837] systemd[1]: Hostname set to <client>.1853lb # [ 1.427340] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854lb # [ 1.439403] systemd[1]: Detected virtualization kvm.1855lb # [ 1.441579] systemd[1]: Detected architecture x86-64.1856lb # [ 1.443733] systemd[1]: Running in initrd.1857lb # [ 1.446191] systemd[1]: Initializing machine ID from random generator.1858lb # [ 1.449183] systemd[1]: Hostname set to <lb>.1859worker2 # [ 1.630342] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860client # [ 1.631158] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861worker1 # [ 1.646018] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862lb # [ 1.653942] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863worker2 # [ 1.681042] systemd[1]: Queued start job for default target Initrd Default Target.1864client # [ 1.670867] systemd[1]: Queued start job for default target Initrd Default Target.1865worker2 # [ 1.684111] systemd[1]: Created slice Slice /system/modprobe.1866worker2 # [ 1.685263] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1867worker2 # [ 1.686608] systemd[1]: Expecting device /dev/disk/by-label/nixos...1868worker2 # [ 1.687665] systemd[1]: Reached target Path Units.1869client # [ 1.674929] systemd[1]: Created slice Slice /system/modprobe.1870worker2 # [ 1.688504] systemd[1]: Reached target Slice Units.1871client # [ 1.676102] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1872worker2 # [ 1.689409] systemd[1]: Reached target Swaps.1873worker2 # [ 1.690180] systemd[1]: Reached target Timer Units.1874client # [ 1.677465] systemd[1]: Expecting device /dev/disk/by-label/nixos...1875worker2 # [ 1.691115] systemd[1]: Listening on D-Bus System Message Bus Socket.1876client # [ 1.678623] systemd[1]: Reached target Path Units.1877client # [ 1.679491] systemd[1]: Reached target Slice Units.1878worker2 # [ 1.692287] systemd[1]: Listening on Journal Socket (/dev/log).1879client # [ 1.680323] systemd[1]: Reached target Swaps.1880worker2 # [ 1.693408] systemd[1]: Listening on Journal Sockets.1881client # [ 1.681086] systemd[1]: Reached target Timer Units.1882worker2 # [ 1.694386] systemd[1]: Listening on udev Control Socket.1883client # [ 1.682054] systemd[1]: Listening on D-Bus System Message Bus Socket.1884worker2 # [ 1.695369] systemd[1]: Listening on udev Kernel Socket.1885client # [ 1.683236] systemd[1]: Listening on Journal Socket (/dev/log).1886worker2 # [ 1.696295] systemd[1]: Reached target Socket Units.1887client # [ 1.684351] systemd[1]: Listening on Journal Sockets.1888client # [ 1.685338] systemd[1]: Listening on udev Control Socket.1889worker2 # [ 1.698047] systemd[1]: Starting Create List of Static Device Nodes...1890client # [ 1.686346] systemd[1]: Listening on udev Kernel Socket.1891client # [ 1.687288] systemd[1]: Reached target Socket Units.1892worker2 # [ 1.701664] systemd[1]: Starting Load Kernel Module configfs...1893client # [ 1.689104] systemd[1]: Starting Create List of Static Device Nodes...1894client # [ 1.693616] systemd[1]: Starting Load Kernel Module configfs...1895worker1 # [ 1.700612] systemd[1]: Queued start job for default target Initrd Default Target.1896lb # [ 1.691469] systemd[1]: Queued start job for default target Initrd Default Target.1897worker1 # [ 1.704821] systemd[1]: Created slice Slice /system/modprobe.1898worker1 # [ 1.705993] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1899worker1 # [ 1.707383] systemd[1]: Expecting device /dev/disk/by-label/nixos...1900lb # [ 1.695928] systemd[1]: Created slice Slice /system/modprobe.1901worker1 # [ 1.708497] systemd[1]: Reached target Path Units.1902lb # [ 1.697116] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1903worker1 # [ 1.709393] systemd[1]: Reached target Slice Units.1904worker1 # [ 1.710288] systemd[1]: Reached target Swaps.1905lb # [ 1.698539] systemd[1]: Expecting device /dev/disk/by-label/nixos...1906worker1 # [ 1.711107] systemd[1]: Reached target Timer Units.1907lb # [ 1.699704] systemd[1]: Reached target Path Units.1908worker2 # [ 1.711319] systemd[1]: Starting Journal Service...1909lb # [ 1.700570] systemd[1]: Reached target Slice Units.1910worker1 # [ 1.712146] systemd[1]: Listening on D-Bus System Message Bus Socket.1911lb # [ 1.701426] systemd[1]: Reached target Swaps.1912client # [ 1.700215] systemd[1]: Starting Journal Service...1913worker1 # [ 1.713331] systemd[1]: Listening on Journal Socket (/dev/log).1914lb # [ 1.702235] systemd[1]: Reached target Timer Units.1915worker1 # [ 1.714476] systemd[1]: Listening on Journal Sockets.1916lb # [ 1.703195] systemd[1]: Listening on D-Bus System Message Bus Socket.1917worker1 # [ 1.715442] systemd[1]: Listening on udev Control Socket.1918lb # [ 1.704425] systemd[1]: Listening on Journal Socket (/dev/log).1919worker1 # [ 1.716395] systemd[1]: Listening on udev Kernel Socket.1920lb # [ 1.705650] systemd[1]: Listening on Journal Sockets.1921worker1 # [ 1.717317] systemd[1]: Reached target Socket Units.1922lb # [ 1.706710] systemd[1]: Listening on udev Control Socket.1923worker1 # [ 1.719149] systemd[1]: Starting Create List of Static Device Nodes...1924lb # [ 1.707738] systemd[1]: Listening on udev Kernel Socket.1925lb # [ 1.708689] systemd[1]: Reached target Socket Units.1926lb # [ 1.710445] systemd[1]: Starting Create List of Static Device Nodes...1927worker1 # [ 1.723497] systemd[1]: Starting Load Kernel Module configfs...1928worker2 # [ 1.735467] systemd[1]: Starting Load Kernel Modules...1929lb # [ 1.714461] systemd[1]: Starting Load Kernel Module configfs...1930worker2 # [ 1.739975] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1931client # [ 1.728745] systemd[1]: Starting Load Kernel Modules...1932client # [ 1.732692] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1933worker2 # [ 1.746914] systemd[1]: Starting Coldplug All udev Devices...1934client # [ 1.741699] systemd[1]: Starting Coldplug All udev Devices...1935worker1 # [ 1.733214] systemd[1]: Starting Journal Service...1936lb # [ 1.722692] systemd[1]: Starting Journal Service...1937worker2 # [ 1.760251] systemd[1]: Finished Create List of Static Device Nodes.1938worker2 # [ 1.761471] systemd-journald[66]: Collecting audit messages is disabled.1939worker2 # [ 1.764692] systemd[1]: modprobe@configfs.service: Deactivated successfully.1940client # [ 1.753768] systemd-journald[66]: Collecting audit messages is disabled.1941client # [ 1.755450] systemd[1]: Finished Create List of Static Device Nodes.1942worker1 # [ 1.757570] systemd[1]: Starting Load Kernel Modules...1943lb # [ 1.747245] systemd[1]: Starting Load Kernel Modules...1944worker2 # [ 1.772101] systemd[1]: Finished Load Kernel Module configfs.1945lb # [ 1.751749] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1946client # [ 1.762208] systemd[1]: modprobe@configfs.service: Deactivated successfully.1947worker1 # [ 1.763533] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1948worker2 # [ 1.778252] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1949worker1 # [ 1.771184] systemd[1]: Starting Coldplug All udev Devices...1950lb # [ 1.759698] systemd[1]: Starting Coldplug All udev Devices...1951client # [ 1.769931] systemd[1]: Finished Load Kernel Module configfs.1952client # [ 1.774071] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1953worker2 # [ 1.788924] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1954worker1 # [ 1.783052] systemd-journald[66]: Collecting audit messages is disabled.1955lb # [ 1.771943] systemd[1]: Finished Create List of Static Device Nodes.1956worker1 # [ 1.784886] systemd[1]: Finished Create List of Static Device Nodes.1957lb # [ 1.773516] systemd-journald[66]: Collecting audit messages is disabled.1958client # [ 1.786660] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1959lb # [ 1.777478] systemd[1]: modprobe@configfs.service: Deactivated successfully.1960worker2 # [ 1.800690] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1961worker1 # [ 1.791656] systemd[1]: modprobe@configfs.service: Deactivated successfully.1962client # [ 1.792503] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1963lb # [ 1.784882] systemd[1]: Finished Load Kernel Module configfs.1964worker1 # [ 1.797863] systemd[1]: Finished Load Kernel Module configfs.1965worker2 # [ 1.810016] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1966lb # [ 1.791106] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1967worker1 # [ 1.803917] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1968client # [ 1.804746] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1969lb # [ 1.802706] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1970worker1 # [ 1.815498] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1971worker2 # [ 1.830406] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1972worker1 # [ 1.825867] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1973worker2 # [ 1.838046] systemd[1]: Starting Create Static Device Nodes in /dev...1974lb # [ 1.814827] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1975worker1 # [ 1.833524] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1976client # [ 1.832753] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1977lb # [ 1.822661] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1978worker2 # [ 1.849965] systemd[1]: Finished Load Kernel Modules.1979client # [ 1.839777] systemd[1]: Starting Create Static Device Nodes in /dev...1980worker2 # [ 1.859930] systemd[1]: Starting Apply Kernel Variables...1981client # [ 1.850622] systemd[1]: Finished Load Kernel Modules.1982lb # [ 1.845228] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1983worker1 # [ 1.858031] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1984client # [ 1.860772] systemd[1]: Starting Apply Kernel Variables...1985worker2 # [ 1.878160] systemd[1]: Finished Create Static Device Nodes in /dev.1986lb # [ 1.855850] systemd[1]: Starting Create Static Device Nodes in /dev...1987worker1 # [ 1.868649] systemd[1]: Starting Create Static Device Nodes in /dev...1988worker2 # [ 1.747590] systemd-modules-load[67]: Inserted module 'dm_mod'[ 1.883637] systemd[1]: Started Journal Service.1989worker2 # 1990worker2 # [ 1.750301] systemd-modules-load[67]: Inserted module 'virtio_balloon'1991worker2 # [ 1.752320] systemd-modules-load[67]: Inserted module 'virtio_gpu'1992lb # [ 1.866216] systemd[1]: Finished Load Kernel Modules.1993client # [ 1.878184] systemd[1]: Started Journal Service.1994worker1 # [ 1.880525] systemd[1]: Finished Load Kernel Modules.1995client # [ 1.743719] systemd-modules-load[68]: Inserted module 'dm_mod'1996client # [ 1.745992] systemd-modules-load[68]: Inserted module 'virtio_balloon'1997worker2 # [ 1.761943] systemd[1]: Finished Apply Kernel Variables.1998client # [ 1.748777] systemd-modules-load[68]: Inserted module 'virtio_gpu'1999lb # [ 1.874819] systemd[1]: Starting Apply Kernel Variables...2000worker2 # [ 1.765095] systemd[1]: Reached target Preparation for Local File Systems.
loading 2,769 hidden lines…
4770client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.09 seconds)4771lb: must succeed: curl -sf localhost:9901/clusters4772lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4773lb: must succeed: curl -sf localhost:9901/clusters4774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4775worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4776worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4777(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.79 seconds)4778subtest: inputs already in the cache are not sent to the scheduler4779lb: must succeed: curl -sf localhost:9901/clusters4780lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4781lb: must succeed: curl -sf localhost:9901/clusters4782lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4783worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4784worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4785client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24786worker1 # [ 41.105701] nix-grpc-daemon[781]: ts=2026-09-22T15:44:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504404787client # don't know how to build these paths:4788client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4789worker2 # [ 41.125789] nix-grpc-daemon[781]: ts=2026-09-22T15:44:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467624790worker2 # [ 41.139509] nix-grpc-daemon[781]: ts=2026-09-22T15:44:32Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04791worker1 # [ 41.130649] nix-grpc-daemon[781]: ts=2026-09-22T15:44:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504404792worker2 # [ 41.144974] nix-grpc-daemon[781]: ts=2026-09-22T15:44:32Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14793worker1 # [ 41.142232] nix-grpc-daemon[781]: ts=2026-09-22T15:44:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504404794worker1 # [ 41.158258] nix-grpc-daemon[859]: time=2026-09-22T15:44:31.448Z level=INFO msg="All 2 paths already cached"4795client # copying 1 paths to 'lb:50051'...4796client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4797worker1 # [ 41.163962] nix-grpc-daemon[781]: ts=2026-09-22T15:44:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504404798worker1 # [ 41.173503] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4799worker1 # [ 41.255267] nix-daemon[846]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4800worker1 # [ 41.273620] nix-daemon[846]: reaped child process 994, status = succeeded4801worker1 # [ 41.342438] nix-daemon[846]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4802worker1 # [ 41.355213] nix-daemon[846]: reaped child process 999, status = succeeded4803lb # [ 41.352664] niks3-server[981]: 2026/09/22 15:44:31 INFO Received uploads request method=POST path=/api/pending_closures4804worker1 # [ 41.369990] nix-grpc-daemon[859]: time=2026-09-22T15:44:31.660Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4805worker1 # [ 41.371578] nix-grpc-daemon[859]: time=2026-09-22T15:44:31.661Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4806lb # [ 41.385055] niks3-server[981]: 2026/09/22 15:44:31 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4807lb # [ 41.399240] niks3-server[981]: 2026/09/22 15:44:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4808lb # [ 41.404867] niks3-server[981]: 2026/09/22 15:44:31 INFO Signed narinfos id=7 count=14809worker1 # [ 41.416608] nix-grpc-daemon[859]: time=2026-09-22T15:44:31.706Z level=INFO msg="Uploading 1 narinfos"4810lb # [ 41.411868] niks3-server[981]: 2026/09/22 15:44:31 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4811lb # [ 41.424121] niks3-server[981]: 2026/09/22 15:44:31 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4812lb # [ 41.429380] niks3-server[981]: 2026/09/22 15:44:31 INFO Completed upload id=74813worker1 # [ 41.440239] nix-grpc-daemon[859]: time=2026-09-22T15:44:31.730Z level=INFO msg="Upload complete. (261ms)"4814lb # [ 41.435891] niks3-server[981]: 2026/09/22 15:44:31 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4815worker1 # [ 41.443608] nix-grpc-daemon[781]: ts=2026-09-22T15:44:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_in=7524816worker1 # [ 41.448842] nix-grpc-daemon[781]: ts=2026-09-22T15:44:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504404817worker1 # [ 41.458189] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4818worker1 # [ 41.468113] nix-daemon[846]: reaped child process 989, status = succeeded4819client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4820worker1 # [ 41.471413] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4821client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4822client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4823client # farm-top> LOG-farm-top-t1b4824worker1 # [ 42.506335] nix-daemon[846]: reaped child process 1003, status = succeeded4825worker1 # [ 42.585727] nix-daemon[846]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4826worker1 # [ 42.598368] nix-daemon[846]: reaped child process 1022, status = succeeded4827lb # [ 42.597371] niks3-server[981]: 2026/09/22 15:44:32 INFO Received uploads request method=POST path=/api/pending_closures4828worker1 # [ 42.610966] nix-grpc-daemon[859]: time=2026-09-22T15:44:32.901Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4829worker1 # [ 42.612758] nix-grpc-daemon[859]: time=2026-09-22T15:44:32.902Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4830lb # [ 42.621807] niks3-server[981]: 2026/09/22 15:44:32 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4831lb # [ 42.625641] niks3-server[981]: 2026/09/22 15:44:32 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4832lb # [ 42.628262] niks3-server[981]: 2026/09/22 15:44:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4833lb # [ 42.630392] niks3-server[981]: 2026/09/22 15:44:32 INFO Signed narinfos id=8 count=14834worker1 # [ 42.639359] nix-grpc-daemon[859]: time=2026-09-22T15:44:32.929Z level=INFO msg="Uploading 1 narinfos"4835lb # [ 42.637189] niks3-server[981]: 2026/09/22 15:44:33 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4836lb # [ 42.640282] niks3-server[981]: 2026/09/22 15:44:33 INFO Completed upload id=84837worker1 # [ 42.649239] nix-grpc-daemon[859]: time=2026-09-22T15:44:32.939Z level=INFO msg="Upload complete. (170ms)"4838lb # [ 42.642894] niks3-server[981]: 2026/09/22 15:44:33 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4839worker1 # [ 42.652074] nix-grpc-daemon[781]: ts=2026-09-22T15:44:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14840lb # [ 42.645646] niks3-server[981]: 2026/09/22 15:44:33 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4841worker1 # [ 42.658976] nix-daemon[846]: reaped child process 1001, status = succeeded4842client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.70 seconds)4843client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4844client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds)4845lb: must succeed: curl -sf localhost:9901/clusters4846lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4847lb: must succeed: curl -sf localhost:9901/clusters4848lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4849worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4850worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4851(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.03 seconds)4852subtest: outputs read back through any worker carry the cache signature4853client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24854client # don't know how to build these paths:4855client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4856worker1 # [ 43.039070] nix-grpc-daemon[781]: ts=2026-09-22T15:44:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504404857worker1 # [ 43.044155] nix-grpc-daemon[781]: ts=2026-09-22T15:44:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504404858worker2 # [ 43.081954] nix-grpc-daemon[781]: ts=2026-09-22T15:44:34Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04859worker2 # [ 43.086758] nix-grpc-daemon[781]: ts=2026-09-22T15:44:34Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.4:50051 clients=14860worker2 # [ 43.091675] nix-grpc-daemon[781]: ts=2026-09-22T15:44:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467624861worker2 # [ 43.102095] nix-grpc-daemon[781]: ts=2026-09-22T15:44:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467624862client # copying 1 paths to 'lb:50051'...4863client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4864worker2 # [ 43.113925] nix-grpc-daemon[781]: ts=2026-09-22T15:44:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467624865worker2 # [ 43.126615] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4866worker2 # [ 43.145624] nix-grpc-daemon[781]: ts=2026-09-22T15:44:34Z level=info event=niks3_push_started pid=9194867worker2 # [ 43.284567] nix-grpc-daemon[919]: time=2026-09-22T15:44:34.389Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4868worker2 # [ 43.382108] nix-daemon[866]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4869worker2 # [ 43.396403] nix-daemon[866]: reaped child process 935, status = succeeded4870worker2 # [ 43.483031] nix-daemon[866]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4871worker2 # [ 43.497345] nix-daemon[866]: reaped child process 940, status = succeeded4872lb # [ 43.482524] niks3-server[981]: 2026/09/22 15:44:33 INFO Received uploads request method=POST path=/api/pending_closures4873worker2 # [ 43.512380] nix-grpc-daemon[919]: time=2026-09-22T15:44:34.618Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4874worker2 # [ 43.515636] nix-grpc-daemon[919]: time=2026-09-22T15:44:34.619Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4875lb # [ 43.515379] niks3-server[981]: 2026/09/22 15:44:33 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4876lb # [ 43.526521] niks3-server[981]: 2026/09/22 15:44:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4877worker2 # [ 43.556030] nix-grpc-daemon[919]: time=2026-09-22T15:44:34.661Z level=INFO msg="Uploading 1 narinfos"4878lb # [ 43.534296] niks3-server[981]: 2026/09/22 15:44:33 INFO Signed narinfos id=9 count=14879lb # [ 43.547095] niks3-server[981]: 2026/09/22 15:44:33 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4880lb # [ 43.554501] niks3-server[981]: 2026/09/22 15:44:33 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4881lb # [ 43.563053] niks3-server[981]: 2026/09/22 15:44:33 INFO Completed upload id=94882worker2 # [ 43.584826] nix-grpc-daemon[919]: time=2026-09-22T15:44:34.690Z level=INFO msg="Upload complete. (295ms)"4883lb # [ 43.564436] niks3-server[981]: 2026/09/22 15:44:33 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4884worker2 # [ 43.588749] nix-grpc-daemon[781]: ts=2026-09-22T15:44:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_in=5044885worker2 # [ 43.597316] nix-grpc-daemon[781]: ts=2026-09-22T15:44:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467624886worker2 # [ 43.605739] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4887worker2 # [ 43.608171] nix-daemon[866]: reaped child process 917, status = succeeded4888worker2 # [ 43.613698] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4889client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4890client # farm-a> worker2: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4891client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4892client # farm-a> LOG-farm-a-sig4893worker2 # [ 44.660598] nix-daemon[866]: reaped child process 944, status = succeeded4894worker2 # [ 44.740585] nix-daemon[866]: accepted connection from pid 960, user nix-grpc-daemon (trusted)4895worker2 # [ 44.755198] nix-daemon[866]: reaped child process 963, status = succeeded4896lb # [ 44.739814] niks3-server[981]: 2026/09/22 15:44:35 INFO Received uploads request method=POST path=/api/pending_closures4897worker2 # [ 44.766645] nix-grpc-daemon[919]: time=2026-09-22T15:44:35.872Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4898worker2 # [ 44.770058] nix-grpc-daemon[919]: time=2026-09-22T15:44:35.874Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4899lb # [ 44.760292] niks3-server[981]: 2026/09/22 15:44:35 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4900lb # [ 44.766169] niks3-server[981]: 2026/09/22 15:44:35 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4901lb # [ 44.768654] niks3-server[981]: 2026/09/22 15:44:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4902lb # [ 44.770943] niks3-server[981]: 2026/09/22 15:44:35 INFO Signed narinfos id=10 count=14903worker2 # [ 44.794081] nix-grpc-daemon[919]: time=2026-09-22T15:44:35.899Z level=INFO msg="Uploading 1 narinfos"4904lb # [ 44.774463] niks3-server[981]: 2026/09/22 15:44:35 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4905lb # [ 44.780655] niks3-server[981]: 2026/09/22 15:44:35 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4906lb # [ 44.784995] niks3-server[981]: 2026/09/22 15:44:35 INFO Completed upload id=104907worker2 # [ 44.806768] nix-grpc-daemon[919]: time=2026-09-22T15:44:35.912Z level=INFO msg="Upload complete. (185ms)"4908lb # [ 44.787448] niks3-server[981]: 2026/09/22 15:44:35 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4909worker2 # [ 44.809755] nix-grpc-daemon[781]: ts=2026-09-22T15:44:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14910worker2 # [ 44.817459] nix-grpc-daemon[781]: ts=2026-09-22T15:44:35Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04911worker2 # [ 44.819445] nix-grpc-daemon[781]: ts=2026-09-22T15:44:35Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=14912worker2 # [ 44.824103] nix-daemon[866]: reaped child process 942, status = succeeded4913worker2 # [ 44.861202] nix-grpc-daemon[781]: ts=2026-09-22T15:44:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467624914worker2 # [ 44.871207] nix-grpc-daemon[781]: ts=2026-09-22T15:44:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467624915worker2 # [ 44.880674] nix-grpc-daemon[919]: time=2026-09-22T15:44:35.986Z level=INFO msg="All 1 paths already cached"4916client # copying 1 paths to 'lb:50051'...4917client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4918worker2 # [ 44.887632] nix-grpc-daemon[781]: ts=2026-09-22T15:44:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467624919worker2 # [ 44.894750] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4920worker2 # [ 44.986148] nix-daemon[866]: accepted connection from pid 969, user nix-grpc-daemon (trusted)4921worker2 # [ 45.002246] nix-daemon[866]: reaped child process 972, status = succeeded4922worker2 # [ 45.085102] nix-daemon[866]: accepted connection from pid 974, user nix-grpc-daemon (trusted)4923worker2 # [ 45.100717] nix-daemon[866]: reaped child process 977, status = succeeded4924lb # [ 45.082422] niks3-server[981]: 2026/09/22 15:44:35 INFO Received uploads request method=POST path=/api/pending_closures4925worker2 # [ 45.109111] nix-grpc-daemon[919]: time=2026-09-22T15:44:36.215Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4926worker2 # [ 45.110657] nix-grpc-daemon[919]: time=2026-09-22T15:44:36.216Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4927lb # [ 45.100852] niks3-server[981]: 2026/09/22 15:44:35 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4928lb # [ 45.104914] niks3-server[981]: 2026/09/22 15:44:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4929lb # [ 45.108153] niks3-server[981]: 2026/09/22 15:44:35 INFO Signed narinfos id=11 count=14930worker2 # [ 45.130100] nix-grpc-daemon[919]: time=2026-09-22T15:44:36.235Z level=INFO msg="Uploading 1 narinfos"4931lb # [ 45.112515] niks3-server[981]: 2026/09/22 15:44:35 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4932lb # [ 45.116925] niks3-server[981]: 2026/09/22 15:44:35 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4933lb # [ 45.120182] niks3-server[981]: 2026/09/22 15:44:35 INFO Completed upload id=114934worker2 # [ 45.141952] nix-grpc-daemon[919]: time=2026-09-22T15:44:36.247Z level=INFO msg="Upload complete. (238ms)"4935lb # [ 45.122471] niks3-server[981]: 2026/09/22 15:44:35 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4936worker2 # [ 45.144322] nix-grpc-daemon[781]: ts=2026-09-22T15:44:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_in=6244937worker2 # [ 45.148091] nix-grpc-daemon[781]: ts=2026-09-22T15:44:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467624938worker2 # [ 45.155311] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4939worker2 # [ 45.158276] nix-daemon[866]: reaped child process 967, status = succeeded4940client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4941client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4942worker2 # [ 45.163374] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4943client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4944client # farm-b> LOG-farm-b-sig4945worker2 # [ 46.377558] nix-daemon[866]: reaped child process 981, status = succeeded4946worker2 # [ 46.447848] nix-daemon[866]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4947worker2 # [ 46.460628] nix-daemon[866]: reaped child process 1000, status = succeeded4948lb # [ 46.445407] niks3-server[981]: 2026/09/22 15:44:36 INFO Received uploads request method=POST path=/api/pending_closures4949worker2 # [ 46.476661] nix-grpc-daemon[919]: time=2026-09-22T15:44:37.582Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4950worker2 # [ 46.478883] nix-grpc-daemon[919]: time=2026-09-22T15:44:37.584Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4951lb # [ 46.482623] niks3-server[981]: 2026/09/22 15:44:36 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4952lb # [ 46.493117] niks3-server[981]: 2026/09/22 15:44:36 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4953lb # [ 46.503554] niks3-server[981]: 2026/09/22 15:44:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4954lb # [ 46.507885] niks3-server[981]: 2026/09/22 15:44:36 INFO Signed narinfos id=12 count=14955worker2 # [ 46.532479] nix-grpc-daemon[919]: time=2026-09-22T15:44:37.638Z level=INFO msg="Uploading 1 narinfos"4956lb # [ 46.518937] niks3-server[981]: 2026/09/22 15:44:36 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4957lb # [ 46.523244] niks3-server[981]: 2026/09/22 15:44:36 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4958lb # [ 46.525943] niks3-server[981]: 2026/09/22 15:44:36 INFO Completed upload id=124959worker2 # [ 46.547839] nix-grpc-daemon[919]: time=2026-09-22T15:44:37.653Z level=INFO msg="Upload complete. (177ms)"4960lb # [ 46.528668] niks3-server[981]: 2026/09/22 15:44:36 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4961worker2 # [ 46.557931] nix-grpc-daemon[781]: ts=2026-09-22T15:44:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14962worker2 # [ 46.565927] nix-grpc-daemon[781]: ts=2026-09-22T15:44:37Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04963worker2 # [ 46.570124] nix-grpc-daemon[781]: ts=2026-09-22T15:44:37Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=14964worker2 # [ 46.578898] nix-daemon[866]: reaped child process 979, status = succeeded4965worker2 # [ 46.609312] nix-grpc-daemon[781]: ts=2026-09-22T15:44:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467624966worker2 # [ 46.621971] nix-grpc-daemon[781]: ts=2026-09-22T15:44:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467624967worker2 # [ 46.635532] nix-grpc-daemon[919]: time=2026-09-22T15:44:37.741Z level=INFO msg="All 2 paths already cached"4968client # copying 1 paths to 'lb:50051'...4969client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4970worker2 # [ 46.645285] nix-grpc-daemon[781]: ts=2026-09-22T15:44:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467624971worker2 # [ 46.653350] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4972worker2 # [ 46.749503] nix-daemon[866]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4973worker2 # [ 46.763273] nix-daemon[866]: reaped child process 1009, status = succeeded4974worker2 # [ 46.846030] nix-daemon[866]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4975worker2 # [ 46.859944] nix-daemon[866]: reaped child process 1014, status = succeeded4976lb # [ 46.845202] niks3-server[981]: 2026/09/22 15:44:37 INFO Received uploads request method=POST path=/api/pending_closures4977worker2 # [ 46.876048] nix-grpc-daemon[919]: time=2026-09-22T15:44:37.981Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4978worker2 # [ 46.877432] nix-grpc-daemon[919]: time=2026-09-22T15:44:37.981Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4979lb # [ 46.873860] niks3-server[981]: 2026/09/22 15:44:37 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4980lb # [ 46.887143] niks3-server[981]: 2026/09/22 15:44:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4981worker2 # [ 46.913947] nix-grpc-daemon[919]: time=2026-09-22T15:44:38.019Z level=INFO msg="Uploading 1 narinfos"4982lb # [ 46.892125] niks3-server[981]: 2026/09/22 15:44:37 INFO Signed narinfos id=13 count=14983lb # [ 46.895522] niks3-server[981]: 2026/09/22 15:44:37 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4984lb # [ 46.907328] niks3-server[981]: 2026/09/22 15:44:37 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4985lb # [ 46.910825] niks3-server[981]: 2026/09/22 15:44:37 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4986lb # [ 46.913650] niks3-server[981]: 2026/09/22 15:44:37 INFO Completed upload id=134987worker2 # [ 46.935260] nix-grpc-daemon[919]: time=2026-09-22T15:44:38.041Z level=INFO msg="Upload complete. (269ms)"4988worker2 # [ 46.938730] nix-grpc-daemon[781]: ts=2026-09-22T15:44:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_in=7524989worker2 # [ 46.943269] nix-grpc-daemon[781]: ts=2026-09-22T15:44:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467624990worker2 # [ 46.954486] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4991worker2 # [ 46.960335] nix-daemon[866]: reaped child process 1004, status = succeeded4992worker2 # [ 46.963143] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4993client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4994client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4995client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4996lb # [ 47.865935] envoy[926]: conn peer=[2001:db8:1::1]:48222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4363 ms=45144997lb # [ 47.871472] envoy[926]: conn peer=[2001:db8:1::1]:48232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=604998lb # [ 47.874916] envoy[926]: conn peer=[2001:db8:1::1]:48248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4195 tx=1894 ms=15644999client # farm-top> LOG-farm-top-sig5000worker2 # [ 48.180715] nix-daemon[866]: reaped child process 1018, status = succeeded5001worker2 # [ 48.253986] nix-daemon[866]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5002worker2 # [ 48.267472] nix-daemon[866]: reaped child process 1037, status = succeeded5003lb # [ 48.254873] niks3-server[981]: 2026/09/22 15:44:38 INFO Received uploads request method=POST path=/api/pending_closures5004worker2 # [ 48.283956] nix-grpc-daemon[919]: time=2026-09-22T15:44:39.389Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5005worker2 # [ 48.285716] nix-grpc-daemon[919]: time=2026-09-22T15:44:39.391Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5006lb # [ 48.288160] niks3-server[981]: 2026/09/22 15:44:38 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5007lb # [ 48.303662] niks3-server[981]: 2026/09/22 15:44:38 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5008lb # [ 48.313108] niks3-server[981]: 2026/09/22 15:44:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5009lb # [ 48.319110] niks3-server[981]: 2026/09/22 15:44:38 INFO Signed narinfos id=14 count=15010worker2 # [ 48.343498] nix-grpc-daemon[919]: time=2026-09-22T15:44:39.449Z level=INFO msg="Uploading 1 narinfos"5011lb # [ 48.324862] niks3-server[981]: 2026/09/22 15:44:38 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5012lb # [ 48.333594] niks3-server[981]: 2026/09/22 15:44:38 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5013lb # [ 48.337619] niks3-server[981]: 2026/09/22 15:44:38 INFO Completed upload id=145014lb # [ 48.339464] niks3-server[981]: 2026/09/22 15:44:38 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5015worker2 # [ 48.360615] nix-grpc-daemon[919]: time=2026-09-22T15:44:39.466Z level=INFO msg="Upload complete. (189ms)"5016worker2 # [ 48.368807] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15017worker2 # [ 48.382705] nix-daemon[866]: reaped child process 1016, status = succeeded5018client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.48 seconds)5019client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5020client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.10 seconds)5021client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5022worker1 # [ 48.588714] nix-grpc-daemon[781]: ts=2026-09-22T15:44:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:470305023client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5024client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5025client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5026client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5027worker1 # [ 48.614107] nix-grpc-daemon[781]: ts=2026-09-22T15:44:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:470305028worker1 # [ 48.620451] nix-grpc-daemon[781]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5029worker1 # [ 48.633819] nix-grpc-daemon[781]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5030worker1 # [ 48.645405] nix-grpc-daemon[781]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5031client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.19 seconds)5032client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5033worker2 # [ 48.798825] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:452845034worker2 # [ 48.840486] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:452845035client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.22 seconds)5036client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5037client # finding garbage collector roots...5038client # removing stale temporary roots file "/nix/var/nix/temproots/818"5039client # removing stale temporary roots file "/nix/var/nix/temproots/896"5040client # removing stale temporary roots file "/nix/var/nix/temproots/802"5041client # removing stale temporary roots file "/nix/var/nix/temproots/851"5042client # removing stale temporary roots file "/nix/var/nix/temproots/881"5043client # removing stale temporary roots file "/nix/var/nix/temproots/873"5044client # removing stale temporary roots file "/nix/var/nix/temproots/859"5045client # removing stale temporary roots file "/nix/var/nix/temproots/839"5046client # deleting specified paths...5047client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5048client # deleting unused links...5049client # note: hard linking is currently saving 0.0 KiB5050worker2 # [ 49.151908] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467625051worker1 # [ 49.142240] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405052worker1 # [ 49.152552] nix-grpc-daemon[859]: time=2026-09-22T15:44:39.442Z level=INFO msg="All 1 paths already cached"5053worker2 # [ 49.171607] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467625054worker1 # [ 49.166281] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:504405055client # copying 3 paths...5056worker1 # [ 49.176127] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:504405057client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5058worker1 # [ 49.191443] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:504405059worker1 # [ 49.196858] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_out=2325060worker2 # [ 49.211481] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:467625061worker2 # [ 49.216970] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_out=1765062worker2 # [ 49.222923] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:467625063worker2 # [ 49.226533] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_out=1285064client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5065client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5066client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.39 seconds)5067(finished: subtest: outputs read back through any worker carry the cache signature, in 6.38 seconds)5068subtest: two clients wanting the same drv build it once5069client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5070client # [ 49.314517] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5071client # Running as unit: dedup1.service; invocation ID: 1ddb8f3377f849f38e53dcf9bcf2a9b75072client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)5073client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5074worker1 # [ 49.439619] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504405075client # [ 49.444822] nix[949]: don't know how to build these paths:5076client # [ 49.445909] nix[949]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5077worker2 # [ 49.465807] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05078worker1 # [ 49.454950] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405079worker2 # [ 49.469938] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=15080worker1 # [ 49.466183] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405081client # [ 49.474992] nix[949]: copying 1 paths to 'lb:50051'...5082client # [ 49.475835] nix[949]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5083worker1 # [ 49.476446] nix-grpc-daemon[781]: ts=2026-09-22T15:44:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504405084worker1 # [ 49.484697] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5085worker1 # [ 49.586635] nix-daemon[846]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)5086worker1 # [ 49.601503] nix-daemon[846]: reaped child process 1032, status = succeeded5087worker1 # [ 49.680873] nix-daemon[846]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)5088worker1 # [ 49.695332] nix-daemon[846]: reaped child process 1037, status = succeeded5089lb # [ 49.694492] niks3-server[981]: 2026/09/22 15:44:40 INFO Received uploads request method=POST path=/api/pending_closures5090worker1 # [ 49.711270] nix-grpc-daemon[859]: time=2026-09-22T15:44:40.001Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5091worker1 # [ 49.713089] nix-grpc-daemon[859]: time=2026-09-22T15:44:40.003Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5092lb # [ 49.722830] niks3-server[981]: 2026/09/22 15:44:40 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5093lb # [ 49.733851] niks3-server[981]: 2026/09/22 15:44:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5094lb # [ 49.741342] niks3-server[981]: 2026/09/22 15:44:40 INFO Signed narinfos id=15 count=15095worker1 # [ 49.753164] nix-grpc-daemon[859]: time=2026-09-22T15:44:40.043Z level=INFO msg="Uploading 1 narinfos"5096lb # [ 49.748446] niks3-server[981]: 2026/09/22 15:44:40 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5097lb # [ 49.760389] niks3-server[981]: 2026/09/22 15:44:40 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5098lb # [ 49.767388] niks3-server[981]: 2026/09/22 15:44:40 INFO Completed upload id=155099worker1 # [ 49.776241] nix-grpc-daemon[859]: time=2026-09-22T15:44:40.066Z level=INFO msg="Upload complete. (278ms)"5100lb # [ 49.770724] niks3-server[981]: 2026/09/22 15:44:40 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5101worker1 # [ 49.780777] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_in=4325102worker1 # [ 49.786264] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405103worker1 # [ 49.794458] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5104worker1 # [ 49.802261] nix-daemon[846]: reaped child process 1027, status = succeeded5105worker1 # [ 49.807811] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5106client # [ 49.806215] nix[949]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5107client # [ 50.366988] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5108client # Running as unit: dedup2.service; invocation ID: 627d1f74c89644a48287d384a744d7475109client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.06 seconds)5110client # [ 50.532863] nix[965]: don't know how to build these paths:5111client # [ 50.533738] nix[965]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5112worker2 # [ 50.545252] nix-grpc-daemon[781]: ts=2026-09-22T15:44:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467625113worker2 # [ 50.555497] nix-grpc-daemon[781]: ts=2026-09-22T15:44:41Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05114worker1 # [ 50.547937] nix-grpc-daemon[781]: ts=2026-09-22T15:44:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405115client # [ 50.552354] nix[965]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5116client: must succeed: sleep 25117client: (finished: must succeed: sleep 2, in 2.04 seconds)5118client: waiting for success: ! systemctl is-active dedup1 dedup25119worker1 # [ 53.848435] nix-daemon[846]: reaped child process 1041, status = succeeded5120worker1 # [ 53.924972] nix-daemon[846]: accepted connection from pid 1071, user nix-grpc-daemon (trusted)5121worker1 # [ 53.937644] nix-daemon[846]: reaped child process 1074, status = succeeded5122lb # [ 53.937546] niks3-server[981]: 2026/09/22 15:44:44 INFO Received uploads request method=POST path=/api/pending_closures5123worker1 # [ 53.955475] nix-grpc-daemon[859]: time=2026-09-22T15:44:44.245Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5124worker1 # [ 53.956939] nix-grpc-daemon[859]: time=2026-09-22T15:44:44.247Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5125lb # [ 53.981544] niks3-server[981]: 2026/09/22 15:44:44 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5126lb # [ 53.995380] niks3-server[981]: 2026/09/22 15:44:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5127lb # [ 54.002125] niks3-server[981]: 2026/09/22 15:44:44 INFO Signed narinfos id=16 count=15128worker1 # [ 54.014944] nix-grpc-daemon[859]: time=2026-09-22T15:44:44.305Z level=INFO msg="Uploading 1 narinfos"5129lb # [ 54.012693] niks3-server[981]: 2026/09/22 15:44:44 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5130lb # [ 54.017768] niks3-server[981]: 2026/09/22 15:44:44 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5131lb # [ 54.023849] niks3-server[981]: 2026/09/22 15:44:44 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5132lb # [ 54.028809] niks3-server[981]: 2026/09/22 15:44:44 INFO Completed upload id=165133lb # [ 54.030565] niks3-server[981]: 2026/09/22 15:44:44 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5134worker1 # [ 54.039190] nix-grpc-daemon[859]: time=2026-09-22T15:44:44.329Z level=INFO msg="Upload complete. (222ms)"5135worker1 # [ 54.045815] nix-grpc-daemon[781]: ts=2026-09-22T15:44:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15136worker1 # [ 54.053998] nix-grpc-daemon[781]: ts=2026-09-22T15:44:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15137worker1 # [ 54.070495] nix-daemon[846]: reaped child process 1039, status = succeeded5138client # [ 54.079394] systemd[1]: dedup1.service: Deactivated successfully.5139client # [ 54.082230] systemd[1]: dedup1.service: Consumed 116ms CPU time over 4.764s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.5140client # [ 54.111884] systemd[1]: dedup2.service: Deactivated successfully.5141client # [ 54.113554] systemd[1]: dedup2.service: Consumed 136ms CPU time over 3.747s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.6K outgoing IP traffic.5142client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.04 seconds)5143client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5144client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.02 seconds)5145worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5146worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5147worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5148worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.02 seconds)5149worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5150worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5151worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5152worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5153(finished: subtest: two clients wanting the same drv build it once, in 5.64 seconds)5154subtest: --timeout and --max-silent-time reach the builder5155worker1 # [ 55.035200] nix-grpc-daemon[781]: ts=2026-09-22T15:44:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504405156worker2 # [ 55.059616] nix-grpc-daemon[781]: ts=2026-09-22T15:44:46Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05157worker2 # [ 55.067543] nix-grpc-daemon[781]: ts=2026-09-22T15:44:46Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=15158worker2 # [ 55.072777] nix-grpc-daemon[781]: ts=2026-09-22T15:44:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467625159worker2 # [ 55.080609] nix-grpc-daemon[781]: ts=2026-09-22T15:44:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467625160worker2 # [ 55.092124] nix-grpc-daemon[781]: ts=2026-09-22T15:44:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467625161worker2 # [ 55.103371] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5162worker2 # [ 55.204989] nix-daemon[866]: accepted connection from pid 1068, user nix-grpc-daemon (trusted)5163worker2 # [ 55.219597] nix-daemon[866]: reaped child process 1071, status = succeeded5164worker2 # [ 55.304840] nix-daemon[866]: accepted connection from pid 1073, user nix-grpc-daemon (trusted)5165worker2 # [ 55.320236] nix-daemon[866]: reaped child process 1076, status = succeeded5166lb # [ 55.305851] niks3-server[981]: 2026/09/22 15:44:45 INFO Received uploads request method=POST path=/api/pending_closures5167worker2 # [ 55.334166] nix-grpc-daemon[919]: time=2026-09-22T15:44:46.439Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5168worker2 # [ 55.335954] nix-grpc-daemon[919]: time=2026-09-22T15:44:46.441Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5169lb # [ 55.336526] niks3-server[981]: 2026/09/22 15:44:45 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5170lb # [ 55.346129] niks3-server[981]: 2026/09/22 15:44:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5171worker2 # [ 55.376735] nix-grpc-daemon[919]: time=2026-09-22T15:44:46.482Z level=INFO msg="Uploading 1 narinfos"5172lb # [ 55.354678] niks3-server[981]: 2026/09/22 15:44:45 INFO Signed narinfos id=17 count=15173lb # [ 55.361481] niks3-server[981]: 2026/09/22 15:44:45 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5174lb # [ 55.376852] niks3-server[981]: 2026/09/22 15:44:45 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5175lb # [ 55.380179] niks3-server[981]: 2026/09/22 15:44:45 INFO Completed upload id=175176worker2 # [ 55.403145] nix-grpc-daemon[919]: time=2026-09-22T15:44:46.508Z level=INFO msg="Upload complete. (288ms)"5177lb # [ 55.386210] niks3-server[981]: 2026/09/22 15:44:45 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5178worker2 # [ 55.407916] nix-grpc-daemon[781]: ts=2026-09-22T15:44:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_in=4245179worker2 # [ 55.413768] nix-grpc-daemon[781]: ts=2026-09-22T15:44:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467625180worker2 # [ 55.424503] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5181worker2 # [ 55.435137] nix-daemon[866]: reaped child process 1066, status = succeeded5182worker2 # [ 55.438092] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5183lb # [ 57.866176] envoy[926]: conn peer=[2001:db8:1::1]:59582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4385 ms=53265184lb # [ 57.869950] envoy[926]: conn peer=[2001:db8:1::1]:59590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2887 ms=1365185lb # [ 57.873647] envoy[926]: conn peer=[2001:db8:1::1]:59606 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=685186lb # [ 57.877074] envoy[926]: conn peer=[2001:db8:1::1]:59600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=755187lb # [ 57.881211] envoy[926]: conn peer=[2001:db8:1::1]:59594 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=825188lb # [ 57.884604] envoy[926]: conn peer=[2001:db8:1::1]:59608 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3024 tx=1536 ms=46245189lb # [ 57.888202] envoy[926]: conn peer=[2001:db8:1::1]:59612 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=35755190worker2 # [ 61.654698] nix-grpc-daemon[781]: ts=2026-09-22T15:44:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05191worker2 # [ 61.678846] nix-daemon[866]: reaped child process 1078, status = succeeded5192worker2 # [ 61.681616] nix-daemon[866]: reaped child process 1080, status = succeeded5193worker1 # [ 61.761453] nix-grpc-daemon[781]: ts=2026-09-22T15:44:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504405194worker2 # [ 61.781890] nix-grpc-daemon[781]: ts=2026-09-22T15:44:52Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05195worker1 # [ 61.775443] nix-grpc-daemon[781]: ts=2026-09-22T15:44:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405196worker2 # [ 61.786597] nix-grpc-daemon[781]: ts=2026-09-22T15:44:52Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=15197worker1 # [ 61.782289] nix-grpc-daemon[781]: ts=2026-09-22T15:44:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405198worker1 # [ 61.788118] nix-grpc-daemon[781]: ts=2026-09-22T15:44:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504405199worker1 # [ 61.793054] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5200worker1 # [ 61.872876] nix-daemon[846]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5201worker1 # [ 61.886677] nix-daemon[846]: reaped child process 1090, status = succeeded5202worker1 # [ 61.961403] nix-daemon[846]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)5203worker1 # [ 61.973614] nix-daemon[846]: reaped child process 1095, status = succeeded5204lb # [ 61.971818] niks3-server[981]: 2026/09/22 15:44:52 INFO Received uploads request method=POST path=/api/pending_closures5205worker1 # [ 61.989284] nix-grpc-daemon[859]: time=2026-09-22T15:44:52.279Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5206worker1 # [ 61.990569] nix-grpc-daemon[859]: time=2026-09-22T15:44:52.279Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5207lb # [ 62.007153] niks3-server[981]: 2026/09/22 15:44:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5208lb # [ 62.014297] niks3-server[981]: 2026/09/22 15:44:52 INFO Signed narinfos id=18 count=15209worker1 # [ 62.026781] nix-grpc-daemon[859]: time=2026-09-22T15:44:52.316Z level=INFO msg="Uploading 1 narinfos"5210lb # [ 62.031125] niks3-server[981]: 2026/09/22 15:44:52 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5211lb # [ 62.077653] niks3-server[981]: 2026/09/22 15:44:52 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5212lb # [ 62.096234] niks3-server[981]: 2026/09/22 15:44:52 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5213lb # [ 62.100727] niks3-server[981]: 2026/09/22 15:44:52 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5214lb # [ 62.116920] niks3-server[981]: 2026/09/22 15:44:52 INFO Completed upload id=185215worker1 # [ 62.125954] nix-grpc-daemon[859]: time=2026-09-22T15:44:52.416Z level=INFO msg="Upload complete. (328ms)"5216worker1 # [ 62.131869] nix-grpc-daemon[781]: ts=2026-09-22T15:44:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_in=4245217worker1 # [ 62.138283] nix-grpc-daemon[781]: ts=2026-09-22T15:44:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405218worker1 # [ 62.150162] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5219worker1 # [ 62.165275] nix-daemon[846]: reaped child process 1085, status = succeeded5220worker1 # [ 62.171248] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5221lb # [ 67.867181] envoy[926]: conn peer=[2001:db8:1::1]:53476 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2970 tx=953 ms=66175222worker1 # [ 67.987186] nix-grpc-daemon[781]: ts=2026-09-22T15:44:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=5 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05223worker1 # [ 68.000413] nix-daemon[846]: reaped child process 1100, status = succeeded5224worker1 # [ 68.001786] nix-daemon[846]: reaped child process 1098, status = succeeded5225(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.15 seconds)5226subtest: the first client leaving does not fail the second5227client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5228client # [ 68.097891] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5229client # Running as unit: share1.service; invocation ID: c7e95a3af84943c59265da980f4f4a1f5230client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.03 seconds)5231client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5232client # [ 68.221721] nix[1039]: don't know how to build these paths:5233worker2 # [ 68.233834] nix-grpc-daemon[781]: ts=2026-09-22T15:44:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467625234client # [ 68.222678] nix[1039]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5235worker2 # [ 68.240562] nix-grpc-daemon[781]: ts=2026-09-22T15:44:59Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05236worker2 # [ 68.242574] nix-grpc-daemon[781]: ts=2026-09-22T15:44:59Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15237worker2 # [ 68.245489] nix-grpc-daemon[781]: ts=2026-09-22T15:44:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467625238worker2 # [ 68.253125] nix-grpc-daemon[781]: ts=2026-09-22T15:44:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467625239client # [ 68.246102] nix[1039]: copying 1 paths to 'lb:50051'...5240client # [ 68.246986] nix[1039]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5241worker2 # [ 68.260199] nix-grpc-daemon[781]: ts=2026-09-22T15:44:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467625242worker2 # [ 68.265112] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5243worker2 # [ 68.352909] nix-daemon[866]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5244worker2 # [ 68.366985] nix-daemon[866]: reaped child process 1110, status = succeeded5245worker2 # [ 68.454078] nix-daemon[866]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)5246worker2 # [ 68.468384] nix-daemon[866]: reaped child process 1115, status = succeeded5247lb # [ 68.456417] niks3-server[981]: 2026/09/22 15:44:58 INFO Received uploads request method=POST path=/api/pending_closures5248worker2 # [ 68.486539] nix-grpc-daemon[919]: time=2026-09-22T15:44:59.592Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5249worker2 # [ 68.488139] nix-grpc-daemon[919]: time=2026-09-22T15:44:59.594Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5250lb # [ 68.495988] niks3-server[981]: 2026/09/22 15:44:58 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5251lb # [ 68.507212] niks3-server[981]: 2026/09/22 15:44:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5252worker2 # [ 68.535250] nix-grpc-daemon[919]: time=2026-09-22T15:44:59.641Z level=INFO msg="Uploading 1 narinfos"5253lb # [ 68.513297] niks3-server[981]: 2026/09/22 15:44:58 INFO Signed narinfos id=19 count=15254lb # [ 68.521043] niks3-server[981]: 2026/09/22 15:44:58 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5255lb # [ 68.533146] niks3-server[981]: 2026/09/22 15:44:58 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5256lb # [ 68.537403] niks3-server[981]: 2026/09/22 15:44:58 INFO Completed upload id=195257worker2 # [ 68.559034] nix-grpc-daemon[919]: time=2026-09-22T15:44:59.664Z level=INFO msg="Upload complete. (288ms)"5258lb # [ 68.539408] niks3-server[981]: 2026/09/22 15:44:58 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5259worker2 # [ 68.562296] nix-grpc-daemon[781]: ts=2026-09-22T15:44:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_in=4325260worker2 # [ 68.567831] nix-grpc-daemon[781]: ts=2026-09-22T15:44:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467625261worker2 # [ 68.579205] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5262worker2 # [ 68.588637] nix-daemon[866]: reaped child process 1105, status = succeeded5263client # [ 68.578996] nix[1039]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5264worker2 # [ 68.592886] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5265client # [ 69.137395] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5266client # Running as unit: share2.service; invocation ID: d0326efd77f3487d98a02f3533924d785267client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.04 seconds)5268worker1 # [ 69.255862] nix-grpc-daemon[781]: ts=2026-09-22T15:44:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504405269client # [ 69.258941] nix[1055]: don't know how to build these paths:5270client # [ 69.259828] nix[1055]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5271worker2 # [ 69.279737] nix-grpc-daemon[781]: ts=2026-09-22T15:45:00Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05272client # [ 69.272227] nix[1055]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5273worker2 # [ 69.285530] nix-grpc-daemon[781]: ts=2026-09-22T15:45:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467625274client: must succeed: sleep 2; systemctl kill -s INT share15275client # [ 72.317605] systemd[1]: share1.service: Sent signal SIGINT to main process 1039 (nix) on client request.5276client # [ 72.320654] nix[1039]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5277client # [ 72.323455] nix[1039]: error: interrupted by the user5278client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.03 seconds)5279client: waiting for success: ! systemctl is-active share15280client # [ 72.348344] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5281client # [ 72.349792] systemd[1]: share1.service: Failed with result 'exit-code'.5282client # [ 72.351977] systemd[1]: share1.service: Consumed 101ms CPU time over 4.253s wall clock time, 41.4M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.5283client: (finished: waiting for success: ! systemctl is-active share1, in 1.04 seconds)5284client: waiting for success: ! systemctl is-active share25285worker2 # [ 73.614237] nix-daemon[866]: reaped child process 1119, status = succeeded5286worker2 # [ 73.688464] nix-daemon[866]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5287worker2 # [ 73.701349] nix-daemon[866]: reaped child process 1158, status = succeeded5288lb # [ 73.686805] niks3-server[981]: 2026/09/22 15:45:04 INFO Received uploads request method=POST path=/api/pending_closures5289worker2 # [ 73.719351] nix-grpc-daemon[919]: time=2026-09-22T15:45:04.825Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5290worker2 # [ 73.721240] nix-grpc-daemon[919]: time=2026-09-22T15:45:04.826Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5291lb # [ 73.731590] niks3-server[981]: 2026/09/22 15:45:04 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5292lb # [ 73.744705] niks3-server[981]: 2026/09/22 15:45:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5293lb # [ 73.751702] niks3-server[981]: 2026/09/22 15:45:04 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5294lb # [ 73.756686] niks3-server[981]: 2026/09/22 15:45:04 INFO Signed narinfos id=20 count=15295worker2 # [ 73.782046] nix-grpc-daemon[919]: time=2026-09-22T15:45:04.887Z level=INFO msg="Uploading 1 narinfos"5296lb # [ 73.764571] niks3-server[981]: 2026/09/22 15:45:04 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5297lb # [ 73.772117] niks3-server[981]: 2026/09/22 15:45:04 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5298lb # [ 73.775840] niks3-server[981]: 2026/09/22 15:45:04 INFO Completed upload id=205299worker2 # [ 73.797709] nix-grpc-daemon[919]: time=2026-09-22T15:45:04.903Z level=INFO msg="Upload complete. (192ms)"5300lb # [ 73.778779] niks3-server[981]: 2026/09/22 15:45:04 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5301worker2 # [ 73.803901] nix-grpc-daemon[781]: ts=2026-09-22T15:45:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15302worker2 # [ 73.809608] nix-grpc-daemon[781]: ts=2026-09-22T15:45:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15303worker2 # [ 73.823448] nix-daemon[866]: reaped child process 1117, status = succeeded5304client # [ 73.848848] systemd[1]: share2.service: Deactivated successfully.5305client # [ 73.850545] systemd[1]: share2.service: Consumed 95ms CPU time over 4.712s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.8K outgoing IP traffic.5306client: (finished: waiting for success: ! systemctl is-active share2, in 1.06 seconds)5307client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5308client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5309(finished: subtest: the first client leaving does not fail the second, in 6.51 seconds)5310subtest: build hook: nix-daemon with builders = grpc://lb5311client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5312client # [ 74.670196] systemd[1]: Started Nix Daemon.5313client # [ 74.749974] nix-daemon[1107]: accepted connection from pid 1104, user root (trusted)5314worker2 # [ 74.871617] nix-grpc-daemon[781]: ts=2026-09-22T15:45:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:467625315worker1 # [ 74.861626] nix-grpc-daemon[781]: ts=2026-09-22T15:45:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405316worker2 # [ 74.884262] nix-grpc-daemon[781]: ts=2026-09-22T15:45:05Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05317worker1 # [ 74.877884] nix-grpc-daemon[781]: ts=2026-09-22T15:45:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405318worker2 # [ 74.888969] nix-grpc-daemon[781]: ts=2026-09-22T15:45:05Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15319worker1 # [ 74.885479] nix-grpc-daemon[781]: ts=2026-09-22T15:45:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405320worker1 # [ 74.892065] nix-grpc-daemon[781]: ts=2026-09-22T15:45:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504405321worker1 # [ 74.895696] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5322worker1 # [ 74.978106] nix-daemon[846]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)5323worker1 # [ 74.991266] nix-daemon[846]: reaped child process 1142, status = succeeded5324worker1 # [ 75.069853] nix-daemon[846]: accepted connection from pid 1144, user nix-grpc-daemon (trusted)5325worker1 # [ 75.081729] nix-daemon[846]: reaped child process 1147, status = succeeded5326lb # [ 75.081423] niks3-server[981]: 2026/09/22 15:45:05 INFO Received uploads request method=POST path=/api/pending_closures5327worker1 # [ 75.098512] nix-grpc-daemon[859]: time=2026-09-22T15:45:05.388Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5328worker1 # [ 75.099857] nix-grpc-daemon[859]: time=2026-09-22T15:45:05.388Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5329lb # [ 75.111072] niks3-server[981]: 2026/09/22 15:45:05 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5330lb # [ 75.123661] niks3-server[981]: 2026/09/22 15:45:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5331worker1 # [ 75.140599] nix-grpc-daemon[859]: time=2026-09-22T15:45:05.430Z level=INFO msg="Uploading 1 narinfos"5332lb # [ 75.131609] niks3-server[981]: 2026/09/22 15:45:05 INFO Signed narinfos id=21 count=15333lb # [ 75.138128] niks3-server[981]: 2026/09/22 15:45:05 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5334lb # [ 75.150977] niks3-server[981]: 2026/09/22 15:45:05 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5335lb # [ 75.159807] niks3-server[981]: 2026/09/22 15:45:05 INFO Completed upload id=215336worker1 # [ 75.168906] nix-grpc-daemon[859]: time=2026-09-22T15:45:05.459Z level=INFO msg="Upload complete. (267ms)"5337lb # [ 75.164682] niks3-server[981]: 2026/09/22 15:45:05 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5338worker1 # [ 75.172843] nix-grpc-daemon[781]: ts=2026-09-22T15:45:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_in=5125339worker1 # [ 75.178614] nix-grpc-daemon[781]: ts=2026-09-22T15:45:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405340worker1 # [ 75.188450] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5341worker1 # [ 75.199767] nix-daemon[846]: reaped child process 1137, status = succeeded5342worker1 # [ 75.202612] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5343worker1 # [ 76.422399] nix-daemon[846]: reaped child process 1151, status = succeeded5344worker1 # [ 76.494586] nix-daemon[846]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)5345worker1 # [ 76.506534] nix-daemon[846]: reaped child process 1170, status = succeeded5346lb # [ 76.505879] niks3-server[981]: 2026/09/22 15:45:06 INFO Received uploads request method=POST path=/api/pending_closures5347worker1 # [ 76.520981] nix-grpc-daemon[859]: time=2026-09-22T15:45:06.811Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5348worker1 # [ 76.522688] nix-grpc-daemon[859]: time=2026-09-22T15:45:06.812Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5349lb # [ 76.535370] niks3-server[981]: 2026/09/22 15:45:06 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5350lb # [ 76.556552] niks3-server[981]: 2026/09/22 15:45:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5351lb # [ 76.561203] niks3-server[981]: 2026/09/22 15:45:06 INFO Signed narinfos id=22 count=15352worker1 # [ 76.572794] nix-grpc-daemon[859]: time=2026-09-22T15:45:06.862Z level=INFO msg="Uploading 1 narinfos"5353lb # [ 76.565538] niks3-server[981]: 2026/09/22 15:45:06 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5354lb # [ 76.575288] niks3-server[981]: 2026/09/22 15:45:06 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5355lb # [ 76.578230] niks3-server[981]: 2026/09/22 15:45:06 INFO Completed upload id=225356worker1 # [ 76.587107] nix-grpc-daemon[859]: time=2026-09-22T15:45:06.877Z level=INFO msg="Upload complete. (178ms)"5357lb # [ 76.580051] niks3-server[981]: 2026/09/22 15:45:06 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5358lb # [ 76.582477] niks3-server[981]: 2026/09/22 15:45:06 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5359worker1 # [ 76.592741] nix-grpc-daemon[781]: ts=2026-09-22T15:45:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15360worker1 # [ 76.605983] nix-daemon[846]: reaped child process 1149, status = succeeded5361worker2 # [ 76.616914] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467625362worker2 # [ 76.626606] nix-grpc-daemon[781]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5363worker1 # [ 76.642731] nix-grpc-daemon[781]: ts=2026-09-22T15:45:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:504405364worker1 # [ 76.647818] nix-grpc-daemon[781]: ts=2026-09-22T15:45:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_out=1285365worker2 # [ 76.824900] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:467625366worker1 # [ 76.819478] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405367worker1 # [ 76.828858] nix-grpc-daemon[859]: time=2026-09-22T15:45:07.119Z level=INFO msg="All 1 paths already cached"5368worker2 # [ 76.855218] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05369worker2 # [ 76.859752] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15370worker1 # [ 76.849235] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405371worker1 # [ 76.860971] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405372worker1 # [ 76.872992] nix-grpc-daemon[859]: time=2026-09-22T15:45:07.163Z level=INFO msg="All 1 paths already cached"5373worker1 # [ 76.880223] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504405374worker1 # [ 76.890125] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5375worker1 # [ 76.983475] nix-daemon[846]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5376worker1 # [ 76.996187] nix-daemon[846]: reaped child process 1177, status = succeeded5377worker1 # [ 77.075482] nix-daemon[846]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)5378worker1 # [ 77.088231] nix-daemon[846]: reaped child process 1182, status = succeeded5379lb # [ 77.085038] niks3-server[981]: 2026/09/22 15:45:07 INFO Received uploads request method=POST path=/api/pending_closures5380worker1 # [ 77.098436] nix-grpc-daemon[859]: time=2026-09-22T15:45:07.388Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5381worker1 # [ 77.099948] nix-grpc-daemon[859]: time=2026-09-22T15:45:07.390Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5382lb # [ 77.100793] niks3-server[981]: 2026/09/22 15:45:07 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5383lb # [ 77.104685] niks3-server[981]: 2026/09/22 15:45:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5384worker1 # [ 77.115667] nix-grpc-daemon[859]: time=2026-09-22T15:45:07.405Z level=INFO msg="Uploading 1 narinfos"5385lb # [ 77.106378] niks3-server[981]: 2026/09/22 15:45:07 INFO Signed narinfos id=23 count=15386lb # [ 77.113505] niks3-server[981]: 2026/09/22 15:45:07 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5387lb # [ 77.115061] niks3-server[981]: 2026/09/22 15:45:07 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5388lb # [ 77.120284] niks3-server[981]: 2026/09/22 15:45:07 INFO Completed upload id=235389worker1 # [ 77.128979] nix-grpc-daemon[859]: time=2026-09-22T15:45:07.419Z level=INFO msg="Upload complete. (227ms)"5390lb # [ 77.121584] niks3-server[981]: 2026/09/22 15:45:07 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5391worker1 # [ 77.131083] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_in=6245392worker1 # [ 77.134826] nix-grpc-daemon[781]: ts=2026-09-22T15:45:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405393worker1 # [ 77.140413] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5394worker1 # [ 77.142859] nix-daemon[846]: reaped child process 1172, status = succeeded5395worker1 # [ 77.147571] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5396lb # [ 77.867432] envoy[926]: conn peer=[2001:db8:1::1]:53484 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2982 tx=974 ms=62365397lb # [ 77.869031] envoy[926]: conn peer=[2001:db8:1::1]:47518 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=771 ms=41155398lb # [ 77.870491] envoy[926]: conn peer=[2001:db8:1::1]:47522 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1197 ms=45895399lb # [ 77.871935] envoy[926]: conn peer=[2001:db8:1::1]:39866 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=325400lb # [ 77.873355] envoy[926]: conn peer=[2001:db8:1::1]:39858 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3410 tx=1962 ms=18125401worker1 # [ 77.963323] nix-daemon[846]: reaped child process 1186, status = succeeded5402worker1 # [ 78.030576] nix-daemon[846]: accepted connection from pid 1202, user nix-grpc-daemon (trusted)5403worker1 # [ 78.042891] nix-daemon[846]: reaped child process 1205, status = succeeded5404lb # [ 78.041638] niks3-server[981]: 2026/09/22 15:45:08 INFO Received uploads request method=POST path=/api/pending_closures5405worker1 # [ 78.054945] nix-grpc-daemon[859]: time=2026-09-22T15:45:08.345Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5406worker1 # [ 78.056428] nix-grpc-daemon[859]: time=2026-09-22T15:45:08.346Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5407lb # [ 78.066340] niks3-server[981]: 2026/09/22 15:45:08 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5408lb # [ 78.068993] niks3-server[981]: 2026/09/22 15:45:08 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5409lb # [ 78.071649] niks3-server[981]: 2026/09/22 15:45:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5410lb # [ 78.073550] niks3-server[981]: 2026/09/22 15:45:08 INFO Signed narinfos id=24 count=15411worker1 # [ 78.082690] nix-grpc-daemon[859]: time=2026-09-22T15:45:08.372Z level=INFO msg="Uploading 1 narinfos"5412lb # [ 78.079959] niks3-server[981]: 2026/09/22 15:45:08 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5413lb # [ 78.082662] niks3-server[981]: 2026/09/22 15:45:08 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5414lb # [ 78.085809] niks3-server[981]: 2026/09/22 15:45:08 INFO Completed upload id=245415worker1 # [ 78.095877] nix-grpc-daemon[859]: time=2026-09-22T15:45:08.386Z level=INFO msg="Upload complete. (142ms)"5416worker1 # [ 78.098348] nix-grpc-daemon[781]: ts=2026-09-22T15:45:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15417lb # [ 78.091075] niks3-server[981]: 2026/09/22 15:45:08 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5418worker2 # [ 78.117174] nix-grpc-daemon[781]: ts=2026-09-22T15:45:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467625419worker1 # [ 78.105438] nix-daemon[846]: reaped child process 1184, status = succeeded5420worker2 # [ 78.122060] nix-grpc-daemon[781]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5421worker1 # [ 78.127261] nix-grpc-daemon[781]: ts=2026-09-22T15:45:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:504405422worker1 # [ 78.129540] nix-grpc-daemon[781]: ts=2026-09-22T15:45:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_out=1765423worker1 # [ 78.325166] nix-grpc-daemon[781]: ts=2026-09-22T15:45:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:504405424worker2 # [ 78.342862] nix-grpc-daemon[781]: ts=2026-09-22T15:45:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467625425worker2 # [ 78.351911] nix-grpc-daemon[919]: time=2026-09-22T15:45:09.457Z level=INFO msg="All 2 paths already cached"5426worker2 # [ 78.365269] nix-grpc-daemon[781]: ts=2026-09-22T15:45:09Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05427worker1 # [ 78.354455] nix-grpc-daemon[781]: ts=2026-09-22T15:45:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405428worker2 # [ 78.369810] nix-grpc-daemon[781]: ts=2026-09-22T15:45:09Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15429worker1 # [ 78.364184] nix-grpc-daemon[781]: ts=2026-09-22T15:45:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405430worker1 # [ 78.374080] nix-grpc-daemon[859]: time=2026-09-22T15:45:08.664Z level=INFO msg="All 2 paths already cached"5431worker1 # [ 78.380549] nix-grpc-daemon[781]: ts=2026-09-22T15:45:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504405432worker1 # [ 78.390160] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5433worker1 # [ 78.489579] nix-daemon[846]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5434worker1 # [ 78.503556] nix-daemon[846]: reaped child process 1212, status = succeeded5435worker1 # [ 78.576515] nix-daemon[846]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)5436worker1 # [ 78.589443] nix-daemon[846]: reaped child process 1217, status = succeeded5437lb # [ 78.586890] niks3-server[981]: 2026/09/22 15:45:08 INFO Received uploads request method=POST path=/api/pending_closures5438worker1 # [ 78.604047] nix-grpc-daemon[859]: time=2026-09-22T15:45:08.894Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5439worker1 # [ 78.605864] nix-grpc-daemon[859]: time=2026-09-22T15:45:08.895Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5440lb # [ 78.615093] niks3-server[981]: 2026/09/22 15:45:08 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5441lb # [ 78.626772] niks3-server[981]: 2026/09/22 15:45:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5442lb # [ 78.631322] niks3-server[981]: 2026/09/22 15:45:09 INFO Signed narinfos id=25 count=15443worker1 # [ 78.644358] nix-grpc-daemon[859]: time=2026-09-22T15:45:08.934Z level=INFO msg="Uploading 1 narinfos"5444lb # [ 78.639207] niks3-server[981]: 2026/09/22 15:45:09 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5445lb # [ 78.647649] niks3-server[981]: 2026/09/22 15:45:09 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5446worker1 # [ 78.662247] nix-grpc-daemon[859]: time=2026-09-22T15:45:08.952Z level=INFO msg="Upload complete. (260ms)"5447lb # [ 78.653983] niks3-server[981]: 2026/09/22 15:45:09 INFO Completed upload id=255448worker1 # [ 78.665420] nix-grpc-daemon[781]: ts=2026-09-22T15:45:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_in=7525449lb # [ 78.660976] niks3-server[981]: 2026/09/22 15:45:09 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5450worker1 # [ 78.670482] nix-grpc-daemon[781]: ts=2026-09-22T15:45:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405451worker1 # [ 78.678611] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5452worker1 # [ 78.684054] nix-daemon[846]: reaped child process 1207, status = succeeded5453worker1 # [ 78.689536] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5454worker1 # [ 79.903947] nix-daemon[846]: reaped child process 1221, status = succeeded5455worker1 # [ 79.976206] nix-daemon[846]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)5456worker1 # [ 79.988938] nix-daemon[846]: reaped child process 1240, status = succeeded5457lb # [ 79.988959] niks3-server[981]: 2026/09/22 15:45:10 INFO Received uploads request method=POST path=/api/pending_closures5458worker1 # [ 80.004140] nix-grpc-daemon[859]: time=2026-09-22T15:45:10.294Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5459worker1 # [ 80.005563] nix-grpc-daemon[859]: time=2026-09-22T15:45:10.295Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5460lb # [ 80.028234] niks3-server[981]: 2026/09/22 15:45:10 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5461lb # [ 80.032604] niks3-server[981]: 2026/09/22 15:45:10 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5462lb # [ 80.038681] niks3-server[981]: 2026/09/22 15:45:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5463worker1 # [ 80.053080] nix-grpc-daemon[859]: time=2026-09-22T15:45:10.343Z level=INFO msg="Uploading 1 narinfos"5464lb # [ 80.044171] niks3-server[981]: 2026/09/22 15:45:10 INFO Signed narinfos id=26 count=15465lb # [ 80.049496] niks3-server[981]: 2026/09/22 15:45:10 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5466lb # [ 80.058654] niks3-server[981]: 2026/09/22 15:45:10 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5467lb # [ 80.062792] niks3-server[981]: 2026/09/22 15:45:10 INFO Completed upload id=265468worker1 # [ 80.073192] nix-grpc-daemon[859]: time=2026-09-22T15:45:10.363Z level=INFO msg="Upload complete. (178ms)"5469lb # [ 80.066084] niks3-server[981]: 2026/09/22 15:45:10 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5470worker1 # [ 80.079478] nix-grpc-daemon[781]: ts=2026-09-22T15:45:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15471worker2 # [ 80.095769] nix-grpc-daemon[781]: ts=2026-09-22T15:45:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467625472worker2 # [ 80.103984] nix-grpc-daemon[781]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5473worker1 # [ 80.092904] nix-daemon[846]: reaped child process 1219, status = succeeded5474worker1 # [ 80.108074] nix-grpc-daemon[781]: ts=2026-09-22T15:45:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:504405475worker1 # [ 80.112838] nix-grpc-daemon[781]: ts=2026-09-22T15:45:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_out=2325476client # [ 80.203617] nix-daemon[1107]: reaped child process 1109, status = succeeded5477client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.62 seconds)5478client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5479client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.01 seconds)5480client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5481client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5482client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5483client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5484(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.67 seconds)5485subtest: an input only one worker has still reaches the builder5486client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5487client # this derivation will be built:5488client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5489client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5490client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds)5491client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5492client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5493worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5494worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.12 seconds)5495worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5496worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5497client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25498client # [ 80.816914] nix-daemon[1107]: accepted connection from pid 1190, user root (trusted)5499client # this derivation will be built:5500client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5501client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5502worker1 # [ 80.934499] nix-grpc-daemon[781]: ts=2026-09-22T15:45:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:504405503worker1 # [ 80.940458] nix-grpc-daemon[781]: ts=2026-09-22T15:45:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405504worker1 # [ 81.041335] nix-daemon[846]: accepted connection from pid 1247, user nix-grpc-daemon (trusted)5505worker1 # [ 81.053753] nix-daemon[846]: reaped child process 1250, status = succeeded5506lb # [ 81.053163] niks3-server[981]: 2026/09/22 15:45:11 INFO Received uploads request method=POST path=/api/pending_closures5507worker1 # [ 81.069449] nix-grpc-daemon[859]: time=2026-09-22T15:45:11.359Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5508worker1 # [ 81.071266] nix-grpc-daemon[859]: time=2026-09-22T15:45:11.361Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5509lb # [ 81.082478] niks3-server[981]: 2026/09/22 15:45:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5510lb # [ 81.085408] niks3-server[981]: 2026/09/22 15:45:11 INFO Signed narinfos id=27 count=15511worker1 # [ 81.095807] nix-grpc-daemon[859]: time=2026-09-22T15:45:11.385Z level=INFO msg="Uploading 1 narinfos"5512lb # [ 81.087774] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5513lb # [ 81.091848] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5514lb # [ 81.098570] niks3-server[981]: 2026/09/22 15:45:11 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5515lb # [ 81.101936] niks3-server[981]: 2026/09/22 15:45:11 INFO Completed upload id=275516worker1 # [ 81.110926] nix-grpc-daemon[859]: time=2026-09-22T15:45:11.401Z level=INFO msg="Upload complete. (171ms)"5517lb # [ 81.105588] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5518worker2 # [ 81.138628] nix-grpc-daemon[781]: ts=2026-09-22T15:45:12Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05519worker2 # [ 81.143852] nix-grpc-daemon[781]: ts=2026-09-22T15:45:12Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15520worker2 # [ 81.148508] nix-grpc-daemon[781]: ts=2026-09-22T15:45:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467625521worker2 # [ 81.158547] nix-grpc-daemon[781]: ts=2026-09-22T15:45:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467625522client # copying 2 paths to 'lb:50051'...5523client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5524worker2 # [ 81.170625] nix-grpc-daemon[781]: ts=2026-09-22T15:45:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467625525client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5526worker2 # [ 81.181908] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5527worker2 # [ 81.277495] nix-daemon[866]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5528worker2 # [ 81.291111] nix-daemon[866]: reaped child process 1179, status = succeeded5529worker2 # [ 81.372326] nix-daemon[866]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)5530worker2 # [ 81.388545] nix-daemon[866]: reaped child process 1184, status = succeeded5531lb # [ 81.372692] niks3-server[981]: 2026/09/22 15:45:11 INFO Received uploads request method=POST path=/api/pending_closures5532lb # [ 81.383381] niks3-server[981]: 2026/09/22 15:45:11 INFO Received uploads request method=POST path=/api/pending_closures5533worker2 # [ 81.416222] nix-grpc-daemon[919]: time=2026-09-22T15:45:12.521Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5534worker2 # [ 81.417594] nix-grpc-daemon[919]: time=2026-09-22T15:45:12.521Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5535worker2 # [ 81.419209] nix-grpc-daemon[919]: time=2026-09-22T15:45:12.521Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5536lb # [ 81.419881] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5537lb # [ 81.426485] niks3-server[981]: 2026/09/22 15:45:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5538lb # [ 81.428710] niks3-server[981]: 2026/09/22 15:45:11 INFO Signed narinfos id=28 count=15539lb # [ 81.430526] niks3-server[981]: 2026/09/22 15:45:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5540lb # [ 81.433599] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5541lb # [ 81.437054] niks3-server[981]: 2026/09/22 15:45:11 INFO Signed narinfos id=29 count=25542worker2 # [ 81.459074] nix-grpc-daemon[919]: time=2026-09-22T15:45:12.564Z level=INFO msg="Uploading 3 narinfos"5543lb # [ 81.438529] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5544lb # [ 81.444238] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5545lb # [ 81.456731] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5546lb # [ 81.458488] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5547lb # [ 81.461842] niks3-server[981]: 2026/09/22 15:45:11 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5548lb # [ 81.464622] niks3-server[981]: 2026/09/22 15:45:11 INFO Completed upload id=295549lb # [ 81.466529] niks3-server[981]: 2026/09/22 15:45:11 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5550lb # [ 81.468490] niks3-server[981]: 2026/09/22 15:45:11 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5551lb # [ 81.471617] niks3-server[981]: 2026/09/22 15:45:11 INFO Completed upload id=285552worker2 # [ 81.493559] nix-grpc-daemon[919]: time=2026-09-22T15:45:12.599Z level=INFO msg="Upload complete. (299ms)"5553worker2 # [ 81.498872] nix-grpc-daemon[781]: ts=2026-09-22T15:45:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=2 nar_bytes_in=9765554worker2 # [ 81.505318] nix-grpc-daemon[781]: ts=2026-09-22T15:45:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467625555worker2 # [ 81.514105] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5556worker2 # [ 81.525183] nix-daemon[866]: reaped child process 1174, status = succeeded5557worker2 # [ 81.530399] nix-grpc-daemon[781]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5558client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5559worker2 # [ 81.537218] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5560client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5561client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5562worker2 # [ 82.352852] nix-daemon[866]: reaped child process 1189, status = succeeded5563worker2 # [ 82.426621] nix-daemon[866]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5564worker2 # [ 82.440661] nix-daemon[866]: reaped child process 1208, status = succeeded5565lb # [ 82.423650] niks3-server[981]: 2026/09/22 15:45:12 INFO Received uploads request method=POST path=/api/pending_closures5566worker2 # [ 82.455255] nix-grpc-daemon[919]: time=2026-09-22T15:45:13.559Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5567lb # [ 82.451604] niks3-server[981]: 2026/09/22 15:45:12 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5568lb # [ 82.459102] niks3-server[981]: 2026/09/22 15:45:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5569lb # [ 82.463958] niks3-server[981]: 2026/09/22 15:45:12 INFO Signed narinfos id=30 count=15570worker2 # [ 82.488887] nix-grpc-daemon[919]: time=2026-09-22T15:45:13.594Z level=INFO msg="Uploading 1 narinfos"5571lb # [ 82.479429] niks3-server[981]: 2026/09/22 15:45:12 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5572lb # [ 82.489444] niks3-server[981]: 2026/09/22 15:45:12 INFO Completed upload id=305573worker2 # [ 82.513460] nix-grpc-daemon[919]: time=2026-09-22T15:45:13.619Z level=INFO msg="Upload complete. (169ms)"5574lb # [ 82.492392] niks3-server[981]: 2026/09/22 15:45:12 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5575worker2 # [ 82.516306] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15576lb # [ 82.497112] niks3-server[981]: 2026/09/22 15:45:12 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5577client # copying 1 paths...5578worker2 # [ 82.524380] nix-daemon[866]: reaped child process 1186, status = succeeded5579worker1 # [ 82.513137] nix-grpc-daemon[781]: ts=2026-09-22T15:45:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:504405580worker1 # [ 82.522091] nix-grpc-daemon[781]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5581client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5582worker2 # [ 82.560344] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:467625583worker2 # [ 82.565576] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_out=1205584client # [ 82.660868] nix-daemon[1107]: reaped child process 1193, status = succeeded5585client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 1.93 seconds)5586client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25587client # [ 82.759097] nix-daemon[1107]: accepted connection from pid 1212, user root (trusted)5588client # this derivation will be built:5589client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5590worker1 # [ 82.871939] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:504405591client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5592worker2 # [ 82.887866] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467625593worker2 # [ 82.897218] nix-grpc-daemon[919]: time=2026-09-22T15:45:14.003Z level=INFO msg="All 1 paths already cached"5594worker2 # [ 82.911250] nix-grpc-daemon[781]: ts=2026-09-22T15:45:14Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05595worker2 # [ 82.916831] nix-grpc-daemon[781]: ts=2026-09-22T15:45:14Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15596worker1 # [ 82.906106] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405597worker1 # [ 82.918193] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:504405598client # copying 1 paths to 'lb:50051'...5599client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5600worker1 # [ 82.935436] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:504405601worker1 # [ 82.946235] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5602worker1 # [ 82.961585] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5)5603worker1 # [ 83.051525] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 2/5)5604worker1 # [ 83.069952] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 3/5)5605worker1 # [ 83.157558] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 4/5)5606worker1 # [ 83.194179] nix-grpc-daemon[781]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5607worker1 # [ 83.197362] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5608worker1 # [ 83.203346] nix-grpc-daemon[781]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5609worker1 # [ 83.314927] nix-daemon[846]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5610worker1 # [ 83.329405] nix-daemon[846]: reaped child process 1263, status = succeeded5611worker1 # [ 83.413617] nix-daemon[846]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)5612worker1 # [ 83.429226] nix-daemon[846]: reaped child process 1268, status = succeeded5613lb # [ 83.425927] niks3-server[981]: 2026/09/22 15:45:13 INFO Received uploads request method=POST path=/api/pending_closures5614worker1 # [ 83.443143] nix-grpc-daemon[859]: time=2026-09-22T15:45:13.733Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5615worker1 # [ 83.444801] nix-grpc-daemon[859]: time=2026-09-22T15:45:13.735Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5616lb # [ 83.455665] niks3-server[981]: 2026/09/22 15:45:13 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5617lb # [ 83.469652] niks3-server[981]: 2026/09/22 15:45:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5618worker1 # [ 83.486475] nix-grpc-daemon[859]: time=2026-09-22T15:45:13.776Z level=INFO msg="Uploading 1 narinfos"5619lb # [ 83.477931] niks3-server[981]: 2026/09/22 15:45:13 INFO Signed narinfos id=31 count=15620lb # [ 83.482285] niks3-server[981]: 2026/09/22 15:45:13 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5621lb # [ 83.494240] niks3-server[981]: 2026/09/22 15:45:13 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5622lb # [ 83.502051] niks3-server[981]: 2026/09/22 15:45:13 INFO Completed upload id=315623worker1 # [ 83.511052] nix-grpc-daemon[859]: time=2026-09-22T15:45:13.801Z level=INFO msg="Upload complete. (288ms)"5624lb # [ 83.508077] niks3-server[981]: 2026/09/22 15:45:13 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5625worker1 # [ 83.517158] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 paths=1 nar_bytes_in=5605626worker1 # [ 83.523292] nix-grpc-daemon[781]: ts=2026-09-22T15:45:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:504405627worker1 # [ 83.534106] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5628worker1 # [ 83.543147] nix-daemon[846]: reaped child process 1253, status = succeeded5629worker1 # [ 83.546326] nix-daemon[846]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5630client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5631client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5632client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5633worker1 # [ 84.361530] nix-daemon[846]: reaped child process 1272, status = succeeded5634worker1 # [ 84.434955] nix-daemon[846]: accepted connection from pid 1288, user nix-grpc-daemon (trusted)5635worker1 # [ 84.447429] nix-daemon[846]: reaped child process 1291, status = succeeded5636lb # [ 84.445799] niks3-server[981]: 2026/09/22 15:45:14 INFO Received uploads request method=POST path=/api/pending_closures5637worker1 # [ 84.459153] nix-grpc-daemon[859]: time=2026-09-22T15:45:14.749Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5638lb # [ 84.459736] niks3-server[981]: 2026/09/22 15:45:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5639lb # [ 84.461848] niks3-server[981]: 2026/09/22 15:45:14 INFO Signed narinfos id=32 count=15640worker1 # [ 84.470534] nix-grpc-daemon[859]: time=2026-09-22T15:45:14.760Z level=INFO msg="Uploading 1 narinfos"5641lb # [ 84.464987] niks3-server[981]: 2026/09/22 15:45:14 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5642lb # [ 84.470221] niks3-server[981]: 2026/09/22 15:45:14 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5643lb # [ 84.473671] niks3-server[981]: 2026/09/22 15:45:14 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5644lb # [ 84.476589] niks3-server[981]: 2026/09/22 15:45:14 INFO Completed upload id=325645worker1 # [ 84.485476] nix-grpc-daemon[859]: time=2026-09-22T15:45:14.775Z level=INFO msg="Upload complete. (133ms)"5646lb # [ 84.477828] niks3-server[981]: 2026/09/22 15:45:14 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5647worker1 # [ 84.487878] nix-grpc-daemon[781]: ts=2026-09-22T15:45:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50440 duration_s=0 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15648client # copying 1 paths...5649worker1 # [ 84.494648] nix-daemon[846]: reaped child process 1270, status = succeeded5650worker2 # [ 84.509876] nix-grpc-daemon[781]: ts=2026-09-22T15:45:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:467625651worker2 # [ 84.519307] nix-grpc-daemon[781]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5652client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5653worker2 # [ 84.543774] nix-grpc-daemon[781]: ts=2026-09-22T15:45:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:467625654worker2 # [ 84.549633] nix-grpc-daemon[781]: ts=2026-09-22T15:45:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_out=1205655client # [ 84.645877] nix-daemon[1107]: reaped child process 1215, status = succeeded5656client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 1.99 seconds)5657client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25658client # [ 84.745720] nix-daemon[1107]: accepted connection from pid 1234, user root (trusted)5659client # this derivation will be built:5660client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5661client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5662worker2 # [ 84.868782] nix-grpc-daemon[781]: ts=2026-09-22T15:45:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:467625663worker2 # [ 84.874533] nix-grpc-daemon[781]: ts=2026-09-22T15:45:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467625664worker2 # [ 84.879950] nix-grpc-daemon[919]: time=2026-09-22T15:45:15.984Z level=INFO msg="All 1 paths already cached"5665worker2 # [ 84.888477] nix-grpc-daemon[781]: ts=2026-09-22T15:45:15Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05666worker2 # [ 84.894113] nix-grpc-daemon[781]: ts=2026-09-22T15:45:15Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15667worker2 # [ 84.899922] nix-grpc-daemon[781]: ts=2026-09-22T15:45:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467625668worker2 # [ 84.911592] nix-grpc-daemon[781]: ts=2026-09-22T15:45:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467625669worker2 # [ 84.923533] nix-grpc-daemon[919]: time=2026-09-22T15:45:16.029Z level=INFO msg="All 1 paths already cached"5670client # copying 1 paths to 'lb:50051'...5671client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5672worker2 # [ 84.931272] nix-grpc-daemon[781]: ts=2026-09-22T15:45:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467625673worker2 # [ 84.939899] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5674worker2 # [ 85.037604] nix-daemon[866]: accepted connection from pid 1217, user nix-grpc-daemon (trusted)5675worker2 # [ 85.051316] nix-daemon[866]: reaped child process 1220, status = succeeded5676worker2 # [ 85.133606] nix-daemon[866]: accepted connection from pid 1222, user nix-grpc-daemon (trusted)5677worker2 # [ 85.145672] nix-daemon[866]: reaped child process 1225, status = succeeded5678lb # [ 85.130893] niks3-server[981]: 2026/09/22 15:45:15 INFO Received uploads request method=POST path=/api/pending_closures5679worker2 # [ 85.161366] nix-grpc-daemon[919]: time=2026-09-22T15:45:16.267Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5680worker2 # [ 85.162900] nix-grpc-daemon[919]: time=2026-09-22T15:45:16.268Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5681lb # [ 85.157407] niks3-server[981]: 2026/09/22 15:45:15 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5682lb # [ 85.173685] niks3-server[981]: 2026/09/22 15:45:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5683worker2 # [ 85.201644] nix-grpc-daemon[919]: time=2026-09-22T15:45:16.307Z level=INFO msg="Uploading 1 narinfos"5684lb # [ 85.180775] niks3-server[981]: 2026/09/22 15:45:15 INFO Signed narinfos id=33 count=15685lb # [ 85.186154] niks3-server[981]: 2026/09/22 15:45:15 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5686lb # [ 85.199305] niks3-server[981]: 2026/09/22 15:45:15 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5687lb # [ 85.204639] niks3-server[981]: 2026/09/22 15:45:15 INFO Completed upload id=335688worker2 # [ 85.229670] nix-grpc-daemon[919]: time=2026-09-22T15:45:16.335Z level=INFO msg="Upload complete. (277ms)"5689lb # [ 85.211302] niks3-server[981]: 2026/09/22 15:45:15 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5690worker2 # [ 85.236471] nix-grpc-daemon[781]: ts=2026-09-22T15:45:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_in=5605691worker2 # [ 85.243597] nix-grpc-daemon[781]: ts=2026-09-22T15:45:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467625692worker2 # [ 85.254133] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5693worker2 # [ 85.263844] nix-daemon[866]: reaped child process 1215, status = succeeded5694client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5695worker2 # [ 85.266833] nix-daemon[866]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5696client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5697client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5698worker2 # [ 86.707583] nix-daemon[866]: reaped child process 1229, status = succeeded5699worker2 # [ 86.779364] nix-daemon[866]: accepted connection from pid 1245, user nix-grpc-daemon (trusted)5700worker2 # [ 86.791146] nix-daemon[866]: reaped child process 1248, status = succeeded5701lb # [ 86.775421] niks3-server[981]: 2026/09/22 15:45:17 INFO Received uploads request method=POST path=/api/pending_closures5702worker2 # [ 86.807137] nix-grpc-daemon[919]: time=2026-09-22T15:45:17.912Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5703lb # [ 86.808505] niks3-server[981]: 2026/09/22 15:45:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5704lb # [ 86.817126] niks3-server[981]: 2026/09/22 15:45:17 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5705lb # [ 86.823974] niks3-server[981]: 2026/09/22 15:45:17 INFO Signed narinfos id=34 count=15706worker2 # [ 86.849738] nix-grpc-daemon[919]: time=2026-09-22T15:45:17.955Z level=INFO msg="Uploading 1 narinfos"5707lb # [ 86.829260] niks3-server[981]: 2026/09/22 15:45:17 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5708lb # [ 86.851646] niks3-server[981]: 2026/09/22 15:45:17 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5709lb # [ 86.854842] niks3-server[981]: 2026/09/22 15:45:17 INFO Completed upload id=345710worker2 # [ 86.876581] nix-grpc-daemon[919]: time=2026-09-22T15:45:17.982Z level=INFO msg="Upload complete. (199ms)"5711lb # [ 86.857073] niks3-server[981]: 2026/09/22 15:45:17 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5712worker2 # [ 86.881204] nix-grpc-daemon[781]: ts=2026-09-22T15:45:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15713client # copying 1 paths...5714worker2 # [ 86.893807] nix-daemon[866]: reaped child process 1227, status = succeeded5715worker1 # [ 86.877434] nix-grpc-daemon[781]: ts=2026-09-22T15:45:17Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:504405716worker1 # [ 86.886411] nix-grpc-daemon[781]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5717client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5718worker2 # [ 86.924091] nix-grpc-daemon[781]: ts=2026-09-22T15:45:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:467625719worker2 # [ 86.928633] nix-grpc-daemon[781]: ts=2026-09-22T15:45:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46762 duration_s=0 paths=1 nar_bytes_out=1205720client # [ 86.981357] nix-daemon[1107]: reaped child process 1237, status = succeeded5721client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.33 seconds)5722(finished: subtest: an input only one worker has still reaches the builder, in 6.73 seconds)5723subtest: upload whose reference the worker lacks is completed from the cache5724client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5725client # this derivation will be built:5726client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5727client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5728client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5729client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5730client # this derivation will be built:5731client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5732client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5733client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.23 seconds)5734client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5735client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5736worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5737worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.12 seconds)5738worker2 # [ 87.719090] systemd[1]: Stopping Nix worker-protocol over gRPC...5739lb # [ 87.868105] envoy[926]: conn peer=[2001:db8:1::1]:39890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=525740lb # [ 87.871811] envoy[926]: conn peer=[2001:db8:1::1]:39880 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3863 tx=2161 ms=13695741lb # [ 87.875403] envoy[926]: conn peer=[2001:db8:1::1]:39904 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4311 tx=2368 ms=18355742lb # [ 87.879372] envoy[926]: conn peer=[2001:db8:1::1]:39908 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=525743lb # [ 87.882838] envoy[926]: conn peer=[2001:db8:1::1]:39914 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3862 tx=2044 ms=16725744lb # [ 87.886930] envoy[926]: conn peer=[2001:db8:1::1]:39928 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575745lb # [ 87.890380] envoy[926]: conn peer=[2001:db8:1::1]:55550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=555746lb # [ 87.893774] envoy[926]: conn peer=[2001:db8:1::1]:55538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3711 tx=2105 ms=17165747lb # [ 87.897336] envoy[926]: conn peer=[2001:db8:1::1]:55560 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3742 tx=2126 ms=20685748lb # [ 87.900833] envoy[926]: conn peer=[2001:db8:1::1]:55568 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=145749worker2 # [ 88.092430] nix-grpc-daemon[781]: ts=2026-09-22T15:45:19Z level=info event=signal_exit5750lb # [ 88.450721] niks3-server[981]: 2026/09/22 15:45:18 INFO lead: released remote=192.168.1.4:331745751worker1 # [ 89.463846] nix-grpc-daemon[781]: ts=2026-09-22T15:45:19Z level=info event=scheduler_restarting addr=lb:500515752worker2 # [ 89.470834] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_restarting5753worker2 # [ 89.480586] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_restarting addr=lb:500515754worker2 # [ 89.484260] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:401885755worker2 # [ 89.487817] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=worker_gone addr=192.168.1.4:500515756worker2 # [ 89.491089] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:401885757worker2 # [ 89.494637] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=worker_gone addr=192.168.1.3:500515758worker2 # [ 89.514558] nix-daemon[866]: reaped child process 868, status = succeeded5759worker1 # [ 89.664193] nix-grpc-daemon[781]: ts=2026-09-22T15:45:19Z level=info event=scheduler_connected addr=lb:500515760worker1 # [ 89.667927] nix-grpc-daemon[781]: ts=2026-09-22T15:45:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5761worker2 # [ 89.687593] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5762worker1 # [ 89.868495] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_connected addr=lb:500515763worker1 # [ 89.872292] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5764worker1 # [ 90.072981] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_connected addr=lb:500515765worker1 # [ 90.076997] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5766worker2 # [ 90.244655] nix-grpc-daemon[781]: E0000 00:00:1790091921.350320 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5767worker2 # [ 90.249760] nix-grpc-daemon[781]: E0000 00:00:1790091921.350535 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5768worker1 # [ 90.278142] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_connected addr=lb:500515769worker1 # [ 90.281978] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5770worker1 # [ 90.456912] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_take_over5771lb # [ 90.448527] niks3-server[981]: 2026/09/22 15:45:20 INFO lead: acquired remote=192.168.1.3:580065772worker1 # [ 90.483172] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_connected addr=lb:500515773worker1 # [ 90.486800] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5774worker1 # [ 90.687762] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_connected addr=lb:500515775worker1 # [ 90.691773] nix-grpc-daemon[781]: ts=2026-09-22T15:45:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5776worker1 # [ 90.892575] nix-grpc-daemon[781]: ts=2026-09-22T15:45:21Z level=info event=scheduler_connected addr=lb:500515777worker1 # [ 90.896538] nix-grpc-daemon[781]: ts=2026-09-22T15:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5778worker2 # [ 90.999852] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5779worker2 # [ 91.003195] systemd[1]: Stopped Nix worker-protocol over gRPC.5780worker2 # [ 91.006816] systemd[1]: nix-grpc-daemon.service: Consumed 2.350s CPU time over 1min 13.068s wall clock time, 64.1M memory peak, 88.3K incoming IP traffic, 233K outgoing IP traffic.5781worker2 # [ 91.018299] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5782worker2 # [ 91.022548] systemd[1]: Closed Nix worker-protocol over gRPC.5783worker2 # [ 91.025556] systemd[1]: nix-grpc-daemon.socket: Consumed 282us CPU time over 1min 21.953s wall clock time, 256K memory peak, 107.2K incoming IP traffic, 91.7K outgoing IP traffic.5784lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5785lb: must succeed: curl -sf localhost:9901/clusters5786lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5787lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.03 seconds)5788client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25789worker1 # [ 91.097307] nix-grpc-daemon[781]: ts=2026-09-22T15:45:21Z level=info event=scheduler_connected addr=lb:500515790worker1 # [ 91.101177] nix-grpc-daemon[781]: ts=2026-09-22T15:45:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5791worker1 # [ 91.186807] nix-grpc-daemon[781]: ts=2026-09-22T15:45:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504405792worker1 # [ 91.193771] nix-grpc-daemon[781]: ts=2026-09-22T15:45:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:504405793client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5794client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds)5795worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5796worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5797worker2 # [ 91.288867] systemd[1]: Listening on Nix worker-protocol over gRPC.5798worker1 # [ 91.302160] nix-grpc-daemon[781]: ts=2026-09-22T15:45:21Z level=info event=scheduler_connected addr=lb:500515799worker1 # [ 91.315103] nix-grpc-daemon[781]: ts=2026-09-22T15:45:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:440645800worker1 # [ 91.318677] nix-grpc-daemon[781]: ts=2026-09-22T15:45:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05801worker1 # [ 91.339520] systemd[1]: Stopping Nix worker-protocol over gRPC...5802worker2 # [ 91.995684] systemd[1]: Starting scheduler-token.service...5803worker2 # [ 92.054862] systemd[1]: scheduler-token.service: Deactivated successfully.5804worker2 # [ 92.058114] systemd[1]: Finished scheduler-token.service.5805worker2 # [ 92.060529] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 62ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5806worker1 # [ 92.053616] nix-grpc-daemon[781]: ts=2026-09-22T15:45:22Z level=info event=signal_exit5807worker2 # [ 92.069529] systemd[1]: Starting Nix worker-protocol over gRPC...5808worker2 # [ 92.148970] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5809worker2 # [ 92.151099] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5810worker2 # [ 92.154389] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15811worker2 # [ 92.158711] nix-grpc-daemon[1272]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5812worker2 # [ 92.160065] nix-grpc-daemon[1272]: E0000 00:00:1790091923.264526 1272 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5813worker2 # [ 92.165807] nix-grpc-daemon[1272]: E0000 00:00:1790091923.269948 1272 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5814worker2 # [ 92.169117] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5815worker2 # [ 92.170793] systemd[1]: Started Nix worker-protocol over gRPC.5816worker2 # [ 92.178069] nix-grpc-daemon[1272]: E0000 00:00:1790091923.283363 1313 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5817worker2 # [ 92.192065] nix-grpc-daemon[1272]: E0000 00:00:1790091923.297958 1313 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5818worker2 # [ 92.196150] nix-grpc-daemon[1272]: E0000 00:00:1790091923.300462 1313 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5819worker2 # [ 92.207367] nix-grpc-daemon[1272]: E0000 00:00:1790091923.313317 1313 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5820worker2 # [ 92.216531] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=scheduler_connected addr=lb:500515821worker2 # [ 92.217840] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5822worker2 # [ 92.417950] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=scheduler_connected addr=lb:500515823worker2 # [ 92.421857] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5824lb # [ 92.454951] niks3-server[981]: 2026/09/22 15:45:22 INFO lead: released remote=192.168.1.3:580065825worker2 # [ 92.623076] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=scheduler_connected addr=lb:500515826worker2 # [ 92.626899] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5827worker2 # [ 92.828141] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=scheduler_connected addr=lb:500515828worker2 # [ 92.832200] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5829worker2 # [ 93.033604] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_connected addr=lb:500515830worker2 # [ 93.037623] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5831worker2 # [ 93.239440] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_connected addr=lb:500515832worker2 # [ 93.243214] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5833worker2 # [ 93.444262] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_connected addr=lb:500515834worker2 # [ 93.448159] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5835worker1 # [ 93.463704] nix-grpc-daemon[781]: ts=2026-09-22T15:45:23Z level=info event=scheduler_restarting5836worker1 # [ 93.470308] nix-grpc-daemon[781]: ts=2026-09-22T15:45:23Z level=info event=scheduler_restarting addr=lb:500515837worker1 # [ 93.474104] nix-grpc-daemon[781]: ts=2026-09-22T15:45:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:440645838worker1 # [ 93.477713] nix-grpc-daemon[781]: ts=2026-09-22T15:45:23Z level=info event=worker_gone addr=192.168.1.3:500515839worker1 # [ 93.501326] nix-daemon[846]: reaped child process 848, status = succeeded5840worker2 # [ 93.649313] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_connected addr=lb:500515841worker2 # [ 93.653284] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5842worker1 # [ 93.676316] nix-grpc-daemon[781]: ts=2026-09-22T15:45:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5843worker2 # [ 93.854332] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_connected addr=lb:500515844worker2 # [ 93.858644] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5845worker2 # [ 94.059694] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_connected addr=lb:500515846worker2 # [ 94.063676] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5847worker1 # [ 94.190233] nix-grpc-daemon[781]: E0000 00:00:1790091924.480222 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5848worker1 # [ 94.195188] nix-grpc-daemon[781]: E0000 00:00:1790091924.480350 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5849worker2 # [ 94.265606] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_connected addr=lb:500515850worker2 # [ 94.269267] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5851worker2 # [ 94.470206] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_connected addr=lb:500515852worker2 # [ 94.474166] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5853worker2 # [ 94.675218] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_connected addr=lb:500515854worker2 # [ 94.679316] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5855worker2 # [ 94.880354] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_connected addr=lb:500515856worker2 # [ 94.884639] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5857worker1 # [ 94.958286] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5858worker1 # [ 94.960952] systemd[1]: Stopped Nix worker-protocol over gRPC.5859worker1 # [ 94.964472] systemd[1]: nix-grpc-daemon.service: Consumed 2.882s CPU time over 1min 17.083s wall clock time, 67.5M memory peak, 103.7K incoming IP traffic, 192.3K outgoing IP traffic.5860worker1 # [ 94.976984] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5861worker1 # [ 94.981341] systemd[1]: Closed Nix worker-protocol over gRPC.5862worker1 # [ 94.984605] systemd[1]: nix-grpc-daemon.socket: Consumed 312us CPU time over 1min 27.493s wall clock time, 256K memory peak, 103.4K incoming IP traffic, 90.4K outgoing IP traffic.5863lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5864lb: must succeed: curl -sf localhost:9901/clusters5865lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5866lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.03 seconds)5867client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25868worker2 # [ 95.085651] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=info event=scheduler_connected addr=lb:500515869worker2 # [ 95.089636] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5870worker2 # [ 95.174664] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411385871worker2 # [ 95.186217] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5872worker2 # [ 95.203667] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)5873worker2 # [ 95.243318] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 2/5)5874worker2 # [ 95.291460] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=info event=scheduler_connected addr=lb:500515875worker2 # [ 95.295805] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5876worker2 # [ 95.338326] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 202 ms (attempt 3/5)5877worker2 # [ 95.496937] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=info event=scheduler_connected addr=lb:500515878worker2 # [ 95.501526] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5879worker2 # [ 95.542516] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 227 ms (attempt 4/5)5880worker2 # [ 95.702509] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=info event=scheduler_connected addr=lb:500515881worker2 # [ 95.706358] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5882worker2 # [ 95.772208] nix-grpc-daemon[1272]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5883worker2 # [ 95.775788] nix-grpc-daemon[1272]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5884client # copying 2 paths...5885client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5886worker2 # [ 95.798203] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:411385887client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5888worker2 # [ 95.810167] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)5889worker2 # [ 95.832428] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:26Z level=info event=niks3_push_started pid=13325890worker2 # [ 95.907501] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_connected addr=lb:500515891worker2 # [ 95.911174] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5892worker2 # [ 95.939671] nix-grpc-daemon[1332]: time=2026-09-22T15:45:27.043Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5893worker2 # [ 96.025495] nix-daemon[866]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)5894worker2 # [ 96.037596] nix-daemon[866]: reaped child process 1347, status = succeeded5895lb # [ 96.026741] niks3-server[981]: 2026/09/22 15:45:26 INFO Received uploads request method=POST path=/api/pending_closures5896lb # [ 96.036519] niks3-server[981]: 2026/09/22 15:45:26 INFO Received uploads request method=POST path=/api/pending_closures5897worker2 # [ 96.065592] nix-grpc-daemon[1332]: time=2026-09-22T15:45:27.171Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5898worker2 # [ 96.067151] nix-grpc-daemon[1332]: time=2026-09-22T15:45:27.173Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5899worker2 # [ 96.069044] nix-grpc-daemon[1332]: time=2026-09-22T15:45:27.175Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5900lb # [ 96.080505] niks3-server[981]: 2026/09/22 15:45:26 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5901worker2 # [ 96.109759] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_connected addr=lb:500515902worker2 # [ 96.111909] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5903lb # [ 96.090933] niks3-server[981]: 2026/09/22 15:45:26 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5904lb # [ 96.106635] niks3-server[981]: 2026/09/22 15:45:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5905lb # [ 96.110242] niks3-server[981]: 2026/09/22 15:45:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5906lb # [ 96.112418] niks3-server[981]: 2026/09/22 15:45:26 INFO Signed narinfos id=35 count=15907lb # [ 96.114601] niks3-server[981]: 2026/09/22 15:45:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5908lb # [ 96.117369] niks3-server[981]: 2026/09/22 15:45:26 INFO Signed narinfos id=36 count=25909lb # [ 96.118366] niks3-server[981]: 2026/09/22 15:45:26 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5910worker2 # [ 96.140133] nix-grpc-daemon[1332]: time=2026-09-22T15:45:27.245Z level=INFO msg="Uploading 3 narinfos"5911lb # [ 96.128243] niks3-server[981]: 2026/09/22 15:45:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5912lb # [ 96.135548] niks3-server[981]: 2026/09/22 15:45:26 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5913lb # [ 96.141379] niks3-server[981]: 2026/09/22 15:45:26 INFO Completed upload id=355914lb # [ 96.142896] niks3-server[981]: 2026/09/22 15:45:26 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5915lb # [ 96.145928] niks3-server[981]: 2026/09/22 15:45:26 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5916lb # [ 96.149397] niks3-server[981]: 2026/09/22 15:45:26 INFO Completed upload id=365917worker2 # [ 96.171342] nix-grpc-daemon[1332]: time=2026-09-22T15:45:27.277Z level=INFO msg="Upload complete. (232ms)"5918worker2 # [ 96.176898] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41138 duration_s=0 paths=2 nar_bytes_in=3125919client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.16 seconds)5920worker2 # [ 96.193145] nix-daemon[866]: reaped child process 1330, status = succeeded5921worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5922worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.01 seconds)5923worker1 # [ 96.221054] systemd[1]: Listening on Nix worker-protocol over gRPC.5924worker1 # [ 96.224107] systemd[1]: Starting Nix worker-protocol over gRPC...5925worker1 # [ 96.302900] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5926worker2 # [ 96.314166] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_connected addr=lb:500515927worker1 # [ 96.304800] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5928worker1 # [ 96.307861] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15929worker2 # [ 96.318531] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5930worker1 # [ 96.312054] nix-grpc-daemon[1315]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5931worker1 # [ 96.315165] nix-grpc-daemon[1315]: E0000 00:00:1790091926.602313 1315 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5932worker1 # [ 96.320499] nix-grpc-daemon[1315]: E0000 00:00:1790091926.605357 1315 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5933worker1 # [ 96.324824] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5934worker1 # [ 96.328077] systemd[1]: Started Nix worker-protocol over gRPC.5935worker1 # [ 96.328965] nix-grpc-daemon[1315]: E0000 00:00:1790091926.617725 1356 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5936lb # [ 96.320775] niks3-server[981]: 2026/09/22 15:45:26 INFO lead: acquired remote=192.168.1.3:550405937worker1 # [ 96.333133] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:26Z level=info event=scheduler_take_over5938worker1 # [ 96.343102] nix-grpc-daemon[1315]: E0000 00:00:1790091926.633241 1356 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5939worker1 # [ 96.346137] nix-grpc-daemon[1315]: E0000 00:00:1790091926.635688 1356 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5940lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5941lb: must succeed: curl -sf localhost:9901/clusters5942worker1 # [ 96.357776] nix-grpc-daemon[1315]: E0000 00:00:1790091926.648040 1356 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5943worker1 # [ 96.369645] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:26Z level=info event=scheduler_connected addr=lb:500515944worker1 # [ 96.371231] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5945lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5946worker2 # [ 96.519882] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_connected addr=lb:500515947worker2 # [ 96.524221] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5948worker1 # [ 96.571881] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:26Z level=info event=scheduler_connected addr=lb:500515949worker1 # [ 96.573688] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5950worker2 # [ 96.725193] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_connected addr=lb:500515951worker2 # [ 96.729158] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5952worker1 # [ 96.774231] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:27Z level=info event=scheduler_connected addr=lb:500515953worker1 # [ 96.776185] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5954worker2 # [ 96.930460] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:28Z level=info event=scheduler_connected addr=lb:500515955worker2 # [ 96.934474] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5956worker1 # [ 96.976474] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:27Z level=info event=scheduler_connected addr=lb:500515957worker1 # [ 96.979654] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:559305958worker1 # [ 96.981171] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05959worker2 # [ 97.135814] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:28Z level=info event=scheduler_connected addr=lb:500515960worker1 # [ 97.128741] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:27Z level=info event=oidc_jwks provider=mock keys=15961worker1 # [ 97.130242] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:27Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:559305962worker1 # [ 97.131835] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05963lb: must succeed: curl -sf localhost:9901/clusters5964lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5965lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)5966(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.46 seconds)5967subtest: a niks3 restart does not cost the leader its role5968lb: must succeed: curl -sf localhost:9901/clusters5969lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5970lb: must succeed: curl -sf localhost:9901/clusters5971lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5972lb # [ 97.504540] systemd[1]: Stopping niks3 server...5973worker2 # [ 97.526857] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:28Z level=info event=scheduler_lock_lost status=2005974worker1 # [ 97.513937] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:27Z level=info event=scheduler_lock_lost status=2005975lb # [ 97.508323] niks3-server[981]: 2026/09/22 15:45:27 INFO Shutdown signal received, draining in-flight requests timeout=10s5976lb # [ 97.513978] niks3-server[981]: 2026/09/22 15:45:27 INFO lead: released remote=192.168.1.3:550405977lb # [ 97.520907] systemd[1]: niks3.service: Deactivated successfully.5978lb # [ 97.523085] systemd[1]: Stopped niks3 server.5979lb # [ 97.523820] systemd[1]: niks3.service: Consumed 705ms CPU time over 1min 17.293s wall clock time, 28.6M memory peak, 99.1K incoming IP traffic, 140.8K outgoing IP traffic.5980lb # [ 97.532107] systemd[1]: Starting niks3 server...5981lb # [ 97.587775] niks3-server[1122]: 2026/09/22 15:45:27 goose: no migrations to run. current version: 202609200000005982lb # [ 97.594391] niks3-server[1122]: 2026/09/22 15:45:27 OK 1_commit_pending_closure.sql (5.06ms)5983lb # [ 97.598606] niks3-server[1122]: 2026/09/22 15:45:27 OK 2_object_stats_trigger.sql (2.27ms)5984lb # [ 97.599690] niks3-server[1122]: 2026/09/22 15:45:27 goose: up to current file version: 25985lb # [ 97.601032] niks3-server[1122]: 2026/09/22 15:45:27 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5986lb # [ 97.613462] niks3-server[1122]: 2026/09/22 15:45:27 INFO Read proxy enabled — serving cache objects from S35987lb # [ 97.614951] niks3-server[1122]: 2026/09/22 15:45:27 INFO Using socket-activated listener address=0.0.0.0:57515988lb # [ 97.617033] niks3-server[1122]: 2026/09/22 15:45:27 INFO systemd watchdog enabled interval=15s5989lb # [ 97.618886] systemd[1]: Started niks3 server.5990lb # [ 97.621297] niks3-server[1122]: 2026/09/22 15:45:27 INFO Starting HTTP server address=0.0.0.0:57515991lb: waiting for unit niks3.service5992lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5993lb: must succeed: curl -sf localhost:9901/clusters5994lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5995worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5996worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5997lb: must succeed: curl -sf localhost:9901/clusters5998lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5999lb: must succeed: curl -sf localhost:9901/clusters6000lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6001worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield6002worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)6003client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&26004lb # [ 97.868372] envoy[926]: conn peer=[2001:db8:1::4]:51846 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=55322 tx=2935 ms=715646005lb # [ 97.872579] envoy[926]: conn peer=[2001:db8:1::1]:55576 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=206006lb # [ 97.876080] envoy[926]: conn peer=[2001:db8:1::3]:42134 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=17036 tx=3538 ms=755846007lb # [ 97.879754] envoy[926]: conn peer=[2001:db8:1::1]:46008 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=304 ms=10126008worker2 # [ 97.946985] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:411386009client # don't know how to build these paths:6010client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6011worker2 # [ 97.955773] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411386012worker1 # [ 97.972580] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:28Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=06013worker1 # [ 97.977688] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:28Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.3:50051 clients=16014worker1 # [ 98.019799] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559426015worker1 # [ 98.028652] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6016worker1 # [ 98.046911] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)6017worker1 # [ 98.135634] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 2/5)6018worker1 # [ 98.291765] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 3/5)6019worker1 # [ 98.356611] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 701 ms (attempt 4/5)6020lb # [ 98.513421] niks3-server[1122]: 2026/09/22 15:45:28 INFO lead: acquired remote=192.168.1.3:550526021worker1 # [ 99.060705] nix-grpc-daemon[1315]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6022worker1 # [ 99.064396] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6023worker1 # [ 99.078468] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559426024client # copying 1 paths to 'lb:50051'...6025client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...6026worker1 # [ 99.089533] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559426027worker1 # [ 99.100119] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6028worker1 # [ 99.118669] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:29Z level=info event=niks3_push_started pid=13876029worker1 # [ 99.220881] nix-grpc-daemon[1387]: time=2026-09-22T15:45:29.510Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6030worker1 # [ 99.319274] nix-daemon[846]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)6031worker1 # [ 99.331052] nix-daemon[846]: reaped child process 1403, status = succeeded6032worker1 # [ 99.413494] nix-daemon[846]: accepted connection from pid 1405, user nix-grpc-daemon (trusted)6033worker1 # [ 99.426126] nix-daemon[846]: reaped child process 1408, status = succeeded6034lb # [ 99.424625] niks3-server[1122]: 2026/09/22 15:45:29 INFO Received uploads request method=POST path=/api/pending_closures6035worker1 # [ 99.440806] nix-grpc-daemon[1387]: time=2026-09-22T15:45:29.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6036worker1 # [ 99.443688] nix-grpc-daemon[1387]: time=2026-09-22T15:45:29.732Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"6037lb # [ 99.453237] niks3-server[1122]: 2026/09/22 15:45:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6038lb # [ 99.462087] niks3-server[1122]: 2026/09/22 15:45:29 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst6039lb # [ 99.475609] niks3-server[1122]: 2026/09/22 15:45:29 INFO Signed narinfos id=37 count=16040worker1 # [ 99.486505] nix-grpc-daemon[1387]: time=2026-09-22T15:45:29.776Z level=INFO msg="Uploading 1 narinfos"6041lb # [ 99.483320] niks3-server[1122]: 2026/09/22 15:45:29 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls6042lb # [ 99.487214] niks3-server[1122]: 2026/09/22 15:45:29 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6043lb # [ 99.493398] niks3-server[1122]: 2026/09/22 15:45:29 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo6044lb # [ 99.498478] niks3-server[1122]: 2026/09/22 15:45:29 INFO Completed upload id=376045worker1 # [ 99.508722] nix-grpc-daemon[1387]: time=2026-09-22T15:45:29.799Z level=INFO msg="Upload complete. (284ms)"6046worker1 # [ 99.512346] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55942 duration_s=0 paths=1 nar_bytes_in=5366047worker1 # [ 99.518194] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559426048worker1 # [ 99.527133] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6049worker1 # [ 99.529884] nix-daemon[846]: reaped child process 1385, status = succeeded6050worker1 # [ 99.534906] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6051client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...6052client # farm-a> node-a: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6053client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6054client # farm-a> LOG-farm-a-after-niks3-restart6055worker1 # [ 100.367525] nix-daemon[846]: reaped child process 1412, status = succeeded6056worker1 # [ 100.435903] nix-daemon[846]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)6057worker1 # [ 100.447907] nix-daemon[846]: reaped child process 1431, status = succeeded6058lb # [ 100.447291] niks3-server[1122]: 2026/09/22 15:45:30 INFO Received uploads request method=POST path=/api/pending_closures6059worker1 # [ 100.462541] nix-grpc-daemon[1387]: time=2026-09-22T15:45:30.752Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6060worker1 # [ 100.464170] nix-grpc-daemon[1387]: time=2026-09-22T15:45:30.754Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"6061lb # [ 100.473328] niks3-server[1122]: 2026/09/22 15:45:30 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6062lb # [ 100.477574] niks3-server[1122]: 2026/09/22 15:45:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6063lb # [ 100.479417] niks3-server[1122]: 2026/09/22 15:45:30 INFO Signed narinfos id=38 count=16064worker1 # [ 100.488944] nix-grpc-daemon[1387]: time=2026-09-22T15:45:30.778Z level=INFO msg="Uploading 1 narinfos"6065lb # [ 100.484077] niks3-server[1122]: 2026/09/22 15:45:30 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst6066lb # [ 100.487247] niks3-server[1122]: 2026/09/22 15:45:30 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls6067lb # [ 100.492949] niks3-server[1122]: 2026/09/22 15:45:30 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6068lb # [ 100.495391] niks3-server[1122]: 2026/09/22 15:45:30 INFO Completed upload id=386069worker1 # [ 100.505545] nix-grpc-daemon[1387]: time=2026-09-22T15:45:30.795Z level=INFO msg="Upload complete. (164ms)"6070lb # [ 100.498775] niks3-server[1122]: 2026/09/22 15:45:30 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6071worker1 # [ 100.508337] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55942 duration_s=0 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16072worker1 # [ 100.516099] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:30Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06073worker1 # [ 100.518698] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:30Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.3:50051 clients=16074worker1 # [ 100.523410] nix-daemon[846]: reaped child process 1410, status = succeeded6075worker1 # [ 100.561531] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559426076worker1 # [ 100.567877] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559426077worker1 # [ 100.574173] nix-grpc-daemon[1387]: time=2026-09-22T15:45:30.864Z level=INFO msg="All 1 paths already cached"6078client # copying 1 paths to 'lb:50051'...6079client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6080worker1 # [ 100.577889] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559426081worker1 # [ 100.581487] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6082worker1 # [ 100.665946] nix-daemon[846]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)6083worker1 # [ 100.678263] nix-daemon[846]: reaped child process 1440, status = succeeded6084worker1 # [ 100.760988] nix-daemon[846]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6085worker1 # [ 100.774638] nix-daemon[846]: reaped child process 1445, status = succeeded6086lb # [ 100.770706] niks3-server[1122]: 2026/09/22 15:45:31 INFO Received uploads request method=POST path=/api/pending_closures6087worker1 # [ 100.784591] nix-grpc-daemon[1387]: time=2026-09-22T15:45:31.074Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6088worker1 # [ 100.786033] nix-grpc-daemon[1387]: time=2026-09-22T15:45:31.074Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6089lb # [ 100.785577] niks3-server[1122]: 2026/09/22 15:45:31 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6090lb # [ 100.789629] niks3-server[1122]: 2026/09/22 15:45:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6091lb # [ 100.792027] niks3-server[1122]: 2026/09/22 15:45:31 INFO Signed narinfos id=39 count=16092worker1 # [ 100.801042] nix-grpc-daemon[1387]: time=2026-09-22T15:45:31.091Z level=INFO msg="Uploading 1 narinfos"6093lb # [ 100.795031] niks3-server[1122]: 2026/09/22 15:45:31 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6094lb # [ 100.800070] niks3-server[1122]: 2026/09/22 15:45:31 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6095lb # [ 100.803912] niks3-server[1122]: 2026/09/22 15:45:31 INFO Completed upload id=396096worker1 # [ 100.812905] nix-grpc-daemon[1387]: time=2026-09-22T15:45:31.103Z level=INFO msg="Upload complete. (225ms)"6097lb # [ 100.806752] niks3-server[1122]: 2026/09/22 15:45:31 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6098worker1 # [ 100.815137] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55942 duration_s=0 paths=1 nar_bytes_in=6566099worker1 # [ 100.818895] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559426100worker1 # [ 100.825400] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6101worker1 # [ 100.826789] nix-daemon[846]: reaped child process 1435, status = succeeded6102client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6103client # farm-b> node-a: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6104worker1 # [ 100.831846] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6105client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6106client # farm-b> LOG-farm-b-after-niks3-restart6107worker1 # [ 102.072475] nix-daemon[846]: reaped child process 1449, status = succeeded6108worker1 # [ 102.146987] nix-daemon[846]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)6109worker1 # [ 102.159061] nix-daemon[846]: reaped child process 1468, status = succeeded6110lb # [ 102.156480] niks3-server[1122]: 2026/09/22 15:45:32 INFO Received uploads request method=POST path=/api/pending_closures6111worker1 # [ 102.171561] nix-grpc-daemon[1387]: time=2026-09-22T15:45:32.461Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6112worker1 # [ 102.173102] nix-grpc-daemon[1387]: time=2026-09-22T15:45:32.463Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6113lb # [ 102.177763] niks3-server[1122]: 2026/09/22 15:45:32 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6114lb # [ 102.181919] niks3-server[1122]: 2026/09/22 15:45:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6115lb # [ 102.186652] niks3-server[1122]: 2026/09/22 15:45:32 INFO Signed narinfos id=40 count=16116worker1 # [ 102.195492] nix-grpc-daemon[1387]: time=2026-09-22T15:45:32.485Z level=INFO msg="Uploading 1 narinfos"6117lb # [ 102.188604] niks3-server[1122]: 2026/09/22 15:45:32 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6118lb # [ 102.192169] niks3-server[1122]: 2026/09/22 15:45:32 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6119lb # [ 102.197371] niks3-server[1122]: 2026/09/22 15:45:32 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6120lb # [ 102.200697] niks3-server[1122]: 2026/09/22 15:45:32 INFO Completed upload id=406121worker1 # [ 102.209697] nix-grpc-daemon[1387]: time=2026-09-22T15:45:32.499Z level=INFO msg="Upload complete. (171ms)"6122lb # [ 102.203279] niks3-server[1122]: 2026/09/22 15:45:32 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6123worker1 # [ 102.212509] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55942 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16124worker1 # [ 102.220403] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:32Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06125worker1 # [ 102.223288] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:32Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.3:50051 clients=16126worker1 # [ 102.226430] nix-daemon[846]: reaped child process 1447, status = succeeded6127worker1 # [ 102.264112] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559426128worker1 # [ 102.273328] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559426129worker1 # [ 102.282686] nix-grpc-daemon[1387]: time=2026-09-22T15:45:32.572Z level=INFO msg="All 2 paths already cached"6130client # copying 1 paths to 'lb:50051'...6131client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6132worker1 # [ 102.289358] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559426133worker1 # [ 102.296766] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6134worker1 # [ 102.383858] nix-daemon[846]: accepted connection from pid 1474, user nix-grpc-daemon (trusted)6135worker1 # [ 102.396757] nix-daemon[846]: reaped child process 1477, status = succeeded6136worker1 # [ 102.478981] nix-daemon[846]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)6137worker1 # [ 102.491178] nix-daemon[846]: reaped child process 1482, status = succeeded6138lb # [ 102.489852] niks3-server[1122]: 2026/09/22 15:45:32 INFO Received uploads request method=POST path=/api/pending_closures6139worker1 # [ 102.503719] nix-grpc-daemon[1387]: time=2026-09-22T15:45:32.794Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6140worker1 # [ 102.505247] nix-grpc-daemon[1387]: time=2026-09-22T15:45:32.795Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6141lb # [ 102.508118] niks3-server[1122]: 2026/09/22 15:45:32 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6142lb # [ 102.511821] niks3-server[1122]: 2026/09/22 15:45:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6143lb # [ 102.515064] niks3-server[1122]: 2026/09/22 15:45:32 INFO Signed narinfos id=41 count=16144worker1 # [ 102.524190] nix-grpc-daemon[1387]: time=2026-09-22T15:45:32.814Z level=INFO msg="Uploading 1 narinfos"6145lb # [ 102.517328] niks3-server[1122]: 2026/09/22 15:45:32 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6146lb # [ 102.523441] niks3-server[1122]: 2026/09/22 15:45:32 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6147lb # [ 102.528435] niks3-server[1122]: 2026/09/22 15:45:32 INFO Completed upload id=416148worker1 # [ 102.537425] nix-grpc-daemon[1387]: time=2026-09-22T15:45:32.827Z level=INFO msg="Upload complete. (232ms)"6149lb # [ 102.530222] niks3-server[1122]: 2026/09/22 15:45:32 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6150worker1 # [ 102.539584] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55942 duration_s=0 paths=1 nar_bytes_in=7846151worker1 # [ 102.542712] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559426152worker1 # [ 102.550090] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6153worker1 # [ 102.551382] nix-daemon[846]: reaped child process 1472, status = succeeded6154client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6155worker1 # [ 102.556534] nix-daemon[846]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)6156client # farm-top> node-a: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6157client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6158client # farm-top> LOG-farm-top-after-niks3-restart6159worker1 # [ 103.999148] nix-daemon[846]: reaped child process 1486, status = succeeded6160worker1 # [ 104.068691] nix-daemon[846]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6161worker1 # [ 104.081168] nix-daemon[846]: reaped child process 1505, status = succeeded6162lb # [ 104.079727] niks3-server[1122]: 2026/09/22 15:45:34 INFO Received uploads request method=POST path=/api/pending_closures6163worker1 # [ 104.098272] nix-grpc-daemon[1387]: time=2026-09-22T15:45:34.388Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6164worker1 # [ 104.099793] nix-grpc-daemon[1387]: time=2026-09-22T15:45:34.390Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6165lb # [ 104.119142] niks3-server[1122]: 2026/09/22 15:45:34 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6166lb # [ 104.127286] niks3-server[1122]: 2026/09/22 15:45:34 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6167lb # [ 104.140271] niks3-server[1122]: 2026/09/22 15:45:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6168lb # [ 104.144500] niks3-server[1122]: 2026/09/22 15:45:34 INFO Signed narinfos id=42 count=16169worker1 # [ 104.156186] nix-grpc-daemon[1387]: time=2026-09-22T15:45:34.446Z level=INFO msg="Uploading 1 narinfos"6170lb # [ 104.151865] niks3-server[1122]: 2026/09/22 15:45:34 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6171lb # [ 104.160970] niks3-server[1122]: 2026/09/22 15:45:34 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6172lb # [ 104.164094] niks3-server[1122]: 2026/09/22 15:45:34 INFO Completed upload id=426173worker1 # [ 104.173477] nix-grpc-daemon[1387]: time=2026-09-22T15:45:34.463Z level=INFO msg="Upload complete. (209ms)"6174lb # [ 104.168261] niks3-server[1122]: 2026/09/22 15:45:34 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6175worker1 # [ 104.181708] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55942 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16176worker1 # [ 104.195222] nix-daemon[846]: reaped child process 1484, status = succeeded6177client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 6.39 seconds)6178client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6179client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.11 seconds)6180(finished: subtest: a niks3 restart does not cost the leader its role, in 6.87 seconds)6181subtest: scheduler down: the other node takes the lock and keeps it6182lb: must succeed: curl -sf localhost:9901/clusters6183lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6184lb: must succeed: curl -sf localhost:9901/clusters6185lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6186lb: must succeed: curl -sf localhost:9901/clusters6187lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6188lb: must succeed: curl -sf localhost:9901/clusters6189lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6190worker1 # [ 104.473305] systemd[1]: Stopping Nix worker-protocol over gRPC...6191worker1 # [ 105.321878] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:35Z level=info event=signal_exit6192lb # [ 105.516958] niks3-server[1122]: 2026/09/22 15:45:35 INFO lead: released remote=192.168.1.3:550526193worker2 # [ 106.541837] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:37Z level=info event=scheduler_restarting addr=lb:500516194worker1 # [ 106.531850] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:36Z level=info event=scheduler_restarting6195worker1 # [ 106.535849] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:36Z level=info event=scheduler_restarting addr=lb:500516196worker1 # [ 106.539878] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:559306197worker1 # [ 106.544187] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:36Z level=info event=worker_gone addr=192.168.1.4:500516198worker1 # [ 106.547932] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:559306199worker1 # [ 106.552162] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:36Z level=info event=worker_gone addr=192.168.1.3:500516200worker1 # [ 106.570958] nix-daemon[846]: reaped child process 1377, status = succeeded6201worker2 # [ 106.742077] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:500516202worker2 # [ 106.746322] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6203worker1 # [ 106.740783] nix-grpc-daemon[1315]: ts=2026-09-22T15:45:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6204worker2 # [ 106.947266] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:500516205worker2 # [ 106.951306] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6206worker1 # [ 106.977607] nix-grpc-daemon[1315]: E0000 00:00:1790091937.267601 1360 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true6207worker1 # [ 106.982458] nix-grpc-daemon[1315]: E0000 00:00:1790091937.267828 1360 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6208worker2 # [ 107.152359] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:500516209worker2 # [ 107.156231] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6210worker2 # [ 107.357224] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:500516211worker2 # [ 107.361178] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6212worker2 # [ 107.563128] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:500516213worker2 # [ 107.566770] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6214worker1 # [ 107.747820] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6215worker1 # [ 107.751553] systemd[1]: Stopped Nix worker-protocol over gRPC.6216worker2 # [ 107.767947] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:500516217worker1 # [ 107.755155] systemd[1]: nix-grpc-daemon.service: Consumed 922ms CPU time over 11.526s wall clock time, 38.5M memory peak, 35.6K incoming IP traffic, 59.6K outgoing IP traffic.6218worker2 # [ 107.771898] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6219worker1 # [ 107.766420] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6220worker1 # [ 107.770606] systemd[1]: Closed Nix worker-protocol over gRPC.6221worker1 # [ 107.774253] systemd[1]: nix-grpc-daemon.socket: Consumed 293us CPU time over 11.551s wall clock time, 256K memory peak, 33.6K incoming IP traffic, 24.6K outgoing IP traffic.6222worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6223worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)6224worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6225worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6226lb # [ 107.867717] envoy[926]: conn peer=[2001:db8:1::1]:46022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9666 tx=4399 ms=62586227lb # [ 107.869349] envoy[926]: conn peer=[2001:db8:1::3]:52248 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1726 tx=569 ms=103696228worker2 # [ 107.973074] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:500516229worker2 # [ 107.977093] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6230worker2 # [ 108.177867] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:500516231worker2 # [ 108.181722] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6232worker2 # [ 108.382767] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:500516233worker2 # [ 108.386570] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6234worker2 # [ 108.533300] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_take_over6235lb # [ 108.512252] niks3-server[1122]: 2026/09/22 15:45:38 INFO lead: acquired remote=192.168.1.4:542006236worker2 # [ 108.588156] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:500516237worker2 # [ 108.591695] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6238worker2 # [ 108.792824] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:500516239worker2 # [ 108.796202] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6240worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6241worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6242worker2 # [ 108.997248] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:500516243worker2 # [ 109.001757] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6244worker2 # [ 109.202665] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:500516245worker2 # [ 109.206608] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6246worker2 # [ 109.407640] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:500516247worker2 # [ 109.422998] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:40Z level=info event=oidc_jwks provider=mock keys=16248worker2 # [ 109.425907] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:383366249worker2 # [ 109.430195] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06250worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6251worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6252client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26253worker2 # [ 110.088818] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:411386254client # don't know how to build these paths:6255client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6256worker2 # [ 110.096257] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411386257worker2 # [ 110.111909] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:41Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06258worker2 # [ 110.116799] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:41Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=16259worker2 # [ 110.156289] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411386260worker2 # [ 110.168381] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411386261client # copying 1 paths to 'lb:50051'...6262client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6263worker2 # [ 110.180773] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:411386264worker2 # [ 110.191925] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6265worker2 # [ 110.290554] nix-daemon[866]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)6266worker2 # [ 110.303047] nix-daemon[866]: reaped child process 1377, status = succeeded6267worker2 # [ 110.383973] nix-daemon[866]: accepted connection from pid 1379, user nix-grpc-daemon (trusted)6268worker2 # [ 110.397257] nix-daemon[866]: reaped child process 1382, status = succeeded6269lb # [ 110.382204] niks3-server[1122]: 2026/09/22 15:45:40 INFO Received uploads request method=POST path=/api/pending_closures6270worker2 # [ 110.412299] nix-grpc-daemon[1332]: time=2026-09-22T15:45:41.517Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6271worker2 # [ 110.413688] nix-grpc-daemon[1332]: time=2026-09-22T15:45:41.517Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6272lb # [ 110.406099] niks3-server[1122]: 2026/09/22 15:45:40 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6273lb # [ 110.416130] niks3-server[1122]: 2026/09/22 15:45:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6274lb # [ 110.418236] niks3-server[1122]: 2026/09/22 15:45:40 INFO Signed narinfos id=43 count=16275worker2 # [ 110.440109] nix-grpc-daemon[1332]: time=2026-09-22T15:45:41.545Z level=INFO msg="Uploading 1 narinfos"6276lb # [ 110.423663] niks3-server[1122]: 2026/09/22 15:45:40 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6277lb # [ 110.427584] niks3-server[1122]: 2026/09/22 15:45:40 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6278lb # [ 110.431258] niks3-server[1122]: 2026/09/22 15:45:40 INFO Completed upload id=436279worker2 # [ 110.452948] nix-grpc-daemon[1332]: time=2026-09-22T15:45:41.558Z level=INFO msg="Upload complete. (250ms)"6280lb # [ 110.433858] niks3-server[1122]: 2026/09/22 15:45:40 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6281worker2 # [ 110.454930] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41138 duration_s=0 paths=1 nar_bytes_in=5206282worker2 # [ 110.459549] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411386283worker2 # [ 110.464904] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6284worker2 # [ 110.467155] nix-daemon[866]: reaped child process 1371, status = succeeded6285worker2 # [ 110.472173] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6286client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6287client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6288client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6289client # farm-a> LOG-farm-a-on-standby6290worker2 # [ 111.889199] nix-daemon[866]: reaped child process 1386, status = succeeded6291worker2 # [ 111.962937] nix-daemon[866]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)6292worker2 # [ 111.976968] nix-daemon[866]: reaped child process 1405, status = succeeded6293lb # [ 111.960642] niks3-server[1122]: 2026/09/22 15:45:42 INFO Received uploads request method=POST path=/api/pending_closures6294worker2 # [ 111.990429] nix-grpc-daemon[1332]: time=2026-09-22T15:45:43.096Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6295worker2 # [ 111.993060] nix-grpc-daemon[1332]: time=2026-09-22T15:45:43.097Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6296lb # [ 111.990121] niks3-server[1122]: 2026/09/22 15:45:42 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6297lb # [ 112.006583] niks3-server[1122]: 2026/09/22 15:45:42 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6298lb # [ 112.017302] niks3-server[1122]: 2026/09/22 15:45:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6299lb # [ 112.023886] niks3-server[1122]: 2026/09/22 15:45:42 INFO Signed narinfos id=44 count=16300worker2 # [ 112.048810] nix-grpc-daemon[1332]: time=2026-09-22T15:45:43.154Z level=INFO msg="Uploading 1 narinfos"6301lb # [ 112.029784] niks3-server[1122]: 2026/09/22 15:45:42 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6302lb # [ 112.037430] niks3-server[1122]: 2026/09/22 15:45:42 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6303lb # [ 112.045103] niks3-server[1122]: 2026/09/22 15:45:42 INFO Completed upload id=446304worker2 # [ 112.066939] nix-grpc-daemon[1332]: time=2026-09-22T15:45:43.172Z level=INFO msg="Upload complete. (186ms)"6305lb # [ 112.046570] niks3-server[1122]: 2026/09/22 15:45:42 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6306worker2 # [ 112.070355] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41138 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16307worker2 # [ 112.082293] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:43Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06308worker2 # [ 112.084379] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:43Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16309worker2 # [ 112.090043] nix-daemon[866]: reaped child process 1384, status = succeeded6310worker2 # [ 112.125773] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411386311worker2 # [ 112.138484] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411386312worker2 # [ 112.152568] nix-grpc-daemon[1332]: time=2026-09-22T15:45:43.258Z level=INFO msg="All 1 paths already cached"6313client # copying 1 paths to 'lb:50051'...6314client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6315worker2 # [ 112.161652] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:411386316worker2 # [ 112.171138] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6317worker2 # [ 112.264898] nix-daemon[866]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)6318worker2 # [ 112.278530] nix-daemon[866]: reaped child process 1414, status = succeeded6319worker2 # [ 112.358988] nix-daemon[866]: accepted connection from pid 1416, user nix-grpc-daemon (trusted)6320worker2 # [ 112.373522] nix-daemon[866]: reaped child process 1419, status = succeeded6321lb # [ 112.355769] niks3-server[1122]: 2026/09/22 15:45:42 INFO Received uploads request method=POST path=/api/pending_closures6322worker2 # [ 112.387224] nix-grpc-daemon[1332]: time=2026-09-22T15:45:43.493Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6323worker2 # [ 112.388830] nix-grpc-daemon[1332]: time=2026-09-22T15:45:43.494Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6324lb # [ 112.387073] niks3-server[1122]: 2026/09/22 15:45:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6325worker2 # [ 112.414933] nix-grpc-daemon[1332]: time=2026-09-22T15:45:43.520Z level=INFO msg="Uploading 1 narinfos"6326lb # [ 112.393429] niks3-server[1122]: 2026/09/22 15:45:42 INFO Signed narinfos id=45 count=16327lb # [ 112.404438] niks3-server[1122]: 2026/09/22 15:45:42 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6328lb # [ 112.414358] niks3-server[1122]: 2026/09/22 15:45:42 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6329lb # [ 112.423117] niks3-server[1122]: 2026/09/22 15:45:42 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6330lb # [ 112.430744] niks3-server[1122]: 2026/09/22 15:45:42 INFO Completed upload id=456331worker2 # [ 112.452394] nix-grpc-daemon[1332]: time=2026-09-22T15:45:43.558Z level=INFO msg="Upload complete. (270ms)"6332lb # [ 112.434115] niks3-server[1122]: 2026/09/22 15:45:42 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6333worker2 # [ 112.456708] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41138 duration_s=0 paths=1 nar_bytes_in=6406334worker2 # [ 112.461697] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411386335worker2 # [ 112.472699] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6336worker2 # [ 112.476498] nix-daemon[866]: reaped child process 1409, status = succeeded6337worker2 # [ 112.481213] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6338client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6339client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6340client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6341client # farm-b> LOG-farm-b-on-standby6342worker2 # [ 113.298532] nix-daemon[866]: reaped child process 1423, status = succeeded6343worker2 # [ 113.378913] nix-daemon[866]: accepted connection from pid 1439, user nix-grpc-daemon (trusted)6344worker2 # [ 113.392937] nix-daemon[866]: reaped child process 1442, status = succeeded6345lb # [ 113.377080] niks3-server[1122]: 2026/09/22 15:45:43 INFO Received uploads request method=POST path=/api/pending_closures6346worker2 # [ 113.406656] nix-grpc-daemon[1332]: time=2026-09-22T15:45:44.512Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6347worker2 # [ 113.409059] nix-grpc-daemon[1332]: time=2026-09-22T15:45:44.514Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6348lb # [ 113.413165] niks3-server[1122]: 2026/09/22 15:45:43 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6349lb # [ 113.419389] niks3-server[1122]: 2026/09/22 15:45:43 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6350lb # [ 113.430548] niks3-server[1122]: 2026/09/22 15:45:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6351worker2 # [ 113.458883] nix-grpc-daemon[1332]: time=2026-09-22T15:45:44.564Z level=INFO msg="Uploading 1 narinfos"6352lb # [ 113.437101] niks3-server[1122]: 2026/09/22 15:45:43 INFO Signed narinfos id=46 count=16353lb # [ 113.441945] niks3-server[1122]: 2026/09/22 15:45:43 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6354lb # [ 113.453161] niks3-server[1122]: 2026/09/22 15:45:43 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6355lb # [ 113.457478] niks3-server[1122]: 2026/09/22 15:45:43 INFO Completed upload id=466356lb # [ 113.458703] niks3-server[1122]: 2026/09/22 15:45:43 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6357worker2 # [ 113.479382] nix-grpc-daemon[1332]: time=2026-09-22T15:45:44.585Z level=INFO msg="Upload complete. (191ms)"6358worker2 # [ 113.490240] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41138 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16359worker2 # [ 113.498372] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:44Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06360worker2 # [ 113.503209] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:44Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16361worker2 # [ 113.512220] nix-daemon[866]: reaped child process 1421, status = succeeded6362worker2 # [ 113.541513] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411386363worker2 # [ 113.554315] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411386364worker2 # [ 113.568131] nix-grpc-daemon[1332]: time=2026-09-22T15:45:44.673Z level=INFO msg="All 2 paths already cached"6365client # copying 1 paths to 'lb:50051'...6366client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6367worker2 # [ 113.577186] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:411386368worker2 # [ 113.585391] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6369worker2 # [ 113.677066] nix-daemon[866]: accepted connection from pid 1448, user nix-grpc-daemon (trusted)6370worker2 # [ 113.690620] nix-daemon[866]: reaped child process 1451, status = succeeded6371worker2 # [ 113.771092] nix-daemon[866]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)6372worker2 # [ 113.783978] nix-daemon[866]: reaped child process 1456, status = succeeded6373lb # [ 113.768423] niks3-server[1122]: 2026/09/22 15:45:44 INFO Received uploads request method=POST path=/api/pending_closures6374worker2 # [ 113.797845] nix-grpc-daemon[1332]: time=2026-09-22T15:45:44.903Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6375worker2 # [ 113.799453] nix-grpc-daemon[1332]: time=2026-09-22T15:45:44.905Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6376lb # [ 113.794867] niks3-server[1122]: 2026/09/22 15:45:44 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6377lb # [ 113.808716] niks3-server[1122]: 2026/09/22 15:45:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6378worker2 # [ 113.838078] nix-grpc-daemon[1332]: time=2026-09-22T15:45:44.943Z level=INFO msg="Uploading 1 narinfos"6379lb # [ 113.816334] niks3-server[1122]: 2026/09/22 15:45:44 INFO Signed narinfos id=47 count=16380lb # [ 113.820584] niks3-server[1122]: 2026/09/22 15:45:44 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6381lb # [ 113.834319] niks3-server[1122]: 2026/09/22 15:45:44 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6382worker2 # [ 113.863100] nix-grpc-daemon[1332]: time=2026-09-22T15:45:44.969Z level=INFO msg="Upload complete. (265ms)"6383lb # [ 113.841806] niks3-server[1122]: 2026/09/22 15:45:44 INFO Completed upload id=476384lb # [ 113.845322] niks3-server[1122]: 2026/09/22 15:45:44 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6385worker2 # [ 113.865782] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41138 duration_s=0 paths=1 nar_bytes_in=7686386worker2 # [ 113.871203] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411386387worker2 # [ 113.881931] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6388worker2 # [ 113.887748] nix-daemon[866]: reaped child process 1446, status = succeeded6389worker2 # [ 113.890596] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6390client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6391client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6392client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6393client # farm-top> LOG-farm-top-on-standby6394worker2 # [ 115.122833] nix-daemon[866]: reaped child process 1460, status = succeeded6395worker2 # [ 115.206590] nix-daemon[866]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)6396worker2 # [ 115.219222] nix-daemon[866]: reaped child process 1479, status = succeeded6397lb # [ 115.202986] niks3-server[1122]: 2026/09/22 15:45:45 INFO Received uploads request method=POST path=/api/pending_closures6398worker2 # [ 115.234683] nix-grpc-daemon[1332]: time=2026-09-22T15:45:46.340Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6399worker2 # [ 115.237715] nix-grpc-daemon[1332]: time=2026-09-22T15:45:46.342Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6400lb # [ 115.245978] niks3-server[1122]: 2026/09/22 15:45:45 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6401lb # [ 115.251192] niks3-server[1122]: 2026/09/22 15:45:45 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6402lb # [ 115.259476] niks3-server[1122]: 2026/09/22 15:45:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6403worker2 # [ 115.287310] nix-grpc-daemon[1332]: time=2026-09-22T15:45:46.393Z level=INFO msg="Uploading 1 narinfos"6404lb # [ 115.265202] niks3-server[1122]: 2026/09/22 15:45:45 INFO Signed narinfos id=48 count=16405lb # [ 115.271786] niks3-server[1122]: 2026/09/22 15:45:45 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6406lb # [ 115.281440] niks3-server[1122]: 2026/09/22 15:45:45 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6407lb # [ 115.285302] niks3-server[1122]: 2026/09/22 15:45:45 INFO Completed upload id=486408worker2 # [ 115.307189] nix-grpc-daemon[1332]: time=2026-09-22T15:45:46.413Z level=INFO msg="Upload complete. (207ms)"6409lb # [ 115.287268] niks3-server[1122]: 2026/09/22 15:45:45 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6410worker2 # [ 115.315452] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41138 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16411worker2 # [ 115.329118] nix-daemon[866]: reaped child process 1458, status = succeeded6412client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.39 seconds)6413client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6414client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6415worker1 # [ 115.492884] systemd[1]: Listening on Nix worker-protocol over gRPC.6416worker1 # [ 115.501778] systemd[1]: Starting Nix worker-protocol over gRPC...6417worker1 # [ 115.595345] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6418worker1 # [ 115.600337] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6419worker1 # [ 115.603192] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16420worker1 # [ 115.605220] nix-grpc-daemon[1523]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6421worker1 # [ 115.611315] nix-grpc-daemon[1523]: E0000 00:00:1790091945.893023 1523 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6422worker1 # [ 115.615963] nix-grpc-daemon[1523]: E0000 00:00:1790091945.898453 1523 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6423worker1 # [ 115.620212] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6424worker1 # [ 115.625142] systemd[1]: Started Nix worker-protocol over gRPC.6425worker1 # [ 115.627151] nix-grpc-daemon[1523]: E0000 00:00:1790091945.909909 1564 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6426worker1 # [ 115.637065] nix-grpc-daemon[1523]: E0000 00:00:1790091945.926589 1564 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6427worker1 # [ 115.639194] nix-grpc-daemon[1523]: E0000 00:00:1790091945.926726 1564 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6428lb: must succeed: curl -sf localhost:9901/clusters6429worker1 # [ 115.651622] nix-grpc-daemon[1523]: E0000 00:00:1790091945.941851 1564 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6430worker1 # [ 115.664072] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:45Z level=info event=scheduler_connected addr=lb:500516431worker2 # [ 115.681073] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:383366432worker2 # [ 115.685153] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06433lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6434worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6435worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6436lb: must succeed: curl -sf localhost:9901/clusters6437lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6438lb: must succeed: curl -sf localhost:9901/clusters6439lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6440worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6441worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6442(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.54 seconds)6443subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6444client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6445client # [ 115.899688] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6446client # Running as unit: rs.service; invocation ID: d8b63e7805ab4da8a48a065dc4de63566447client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6448client # [ 116.033513] nix[1361]: don't know how to build these paths:6449client # [ 116.034403] nix[1361]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6450worker2 # [ 116.046228] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:411386451worker2 # [ 116.056196] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:47Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06452worker2 # [ 116.061867] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:47Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16453worker2 # [ 116.103288] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411386454worker2 # [ 116.114590] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:411386455client # [ 116.111180] nix[1361]: copying 1 paths to 'lb:50051'...6456worker2 # [ 116.125075] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:411386457client # [ 116.113133] nix[1361]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6458worker2 # [ 116.138390] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6459worker2 # [ 116.233519] nix-daemon[866]: accepted connection from pid 1492, user nix-grpc-daemon (trusted)6460worker2 # [ 116.248867] nix-daemon[866]: reaped child process 1495, status = succeeded6461worker2 # [ 116.324543] nix-daemon[866]: accepted connection from pid 1497, user nix-grpc-daemon (trusted)6462worker2 # [ 116.339093] nix-daemon[866]: reaped child process 1500, status = succeeded6463lb # [ 116.321120] niks3-server[1122]: 2026/09/22 15:45:46 INFO Received uploads request method=POST path=/api/pending_closures6464worker2 # [ 116.347817] nix-grpc-daemon[1332]: time=2026-09-22T15:45:47.453Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6465worker2 # [ 116.349407] nix-grpc-daemon[1332]: time=2026-09-22T15:45:47.453Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6466lb # [ 116.334315] niks3-server[1122]: 2026/09/22 15:45:46 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6467lb # [ 116.338530] niks3-server[1122]: 2026/09/22 15:45:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6468lb # [ 116.341379] niks3-server[1122]: 2026/09/22 15:45:46 INFO Signed narinfos id=49 count=16469worker2 # [ 116.362944] nix-grpc-daemon[1332]: time=2026-09-22T15:45:47.468Z level=INFO msg="Uploading 1 narinfos"6470lb # [ 116.343582] niks3-server[1122]: 2026/09/22 15:45:46 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6471lb # [ 116.350072] niks3-server[1122]: 2026/09/22 15:45:46 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6472lb # [ 116.352668] niks3-server[1122]: 2026/09/22 15:45:46 INFO Completed upload id=496473worker2 # [ 116.375809] nix-grpc-daemon[1332]: time=2026-09-22T15:45:47.481Z level=INFO msg="Upload complete. (226ms)"6474lb # [ 116.355273] niks3-server[1122]: 2026/09/22 15:45:46 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6475worker2 # [ 116.378325] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41138 duration_s=0 paths=1 nar_bytes_in=4166476worker2 # [ 116.381368] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:411386477worker2 # [ 116.389266] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6478worker2 # [ 116.393103] nix-daemon[866]: reaped child process 1490, status = succeeded6479worker2 # [ 116.395915] nix-daemon[866]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)6480client # [ 116.382200] nix[1361]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6481lb # [ 117.869345] envoy[926]: conn peer=[2001:db8:1::1]:36134 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9543 tx=4326 ms=52346482lb: must succeed: curl -sf localhost:9901/clusters6483lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6484lb: must succeed: curl -sf localhost:9901/clusters6485lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6486lb: must succeed: curl -sf localhost:9901/clusters6487lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6488lb: must succeed: curl -sf localhost:9901/clusters6489lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6490worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6491worker1 # [ 118.483239] systemd-journald[285]: Received client request to rotate journal, rotating.6492worker1 # [ 118.502065] systemd-journald[285]: Vacuuming done, freed 0B of archived journals from /var/log/journal/aae88770b7c74fbfbbc734b8d3e78e47.6493worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.05 seconds)6494worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6495worker2 # [ 118.418406] systemd[1]: Stopping Nix worker-protocol over gRPC...6496worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6497client: must succeed: sleep 5.06498worker2 # [ 119.177942] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:50Z level=info event=draining6499client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6500worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6501worker2 # [ 123.846289] nix-daemon[866]: reaped child process 1504, status = succeeded6502worker2 # [ 123.927576] nix-daemon[866]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6503worker2 # [ 123.940937] nix-daemon[866]: reaped child process 1548, status = succeeded6504lb # [ 123.927089] niks3-server[1122]: 2026/09/22 15:45:54 INFO Received uploads request method=POST path=/api/pending_closures6505worker2 # [ 123.953496] nix-grpc-daemon[1332]: time=2026-09-22T15:45:55.059Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6506worker2 # [ 123.955309] nix-grpc-daemon[1332]: time=2026-09-22T15:45:55.061Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6507lb # [ 123.954771] niks3-server[1122]: 2026/09/22 15:45:54 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6508lb # [ 123.958620] niks3-server[1122]: 2026/09/22 15:45:54 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6509worker2 # [ 123.985791] nix-grpc-daemon[1332]: time=2026-09-22T15:45:55.091Z level=INFO msg="Uploading 1 narinfos"6510lb # [ 123.964240] niks3-server[1122]: 2026/09/22 15:45:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6511lb # [ 123.965707] niks3-server[1122]: 2026/09/22 15:45:54 INFO Signed narinfos id=50 count=16512lb # [ 123.971022] niks3-server[1122]: 2026/09/22 15:45:54 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6513lb # [ 123.979069] niks3-server[1122]: 2026/09/22 15:45:54 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6514lb # [ 123.982766] niks3-server[1122]: 2026/09/22 15:45:54 INFO Completed upload id=506515worker2 # [ 124.004508] nix-grpc-daemon[1332]: time=2026-09-22T15:45:55.110Z level=INFO msg="Upload complete. (192ms)"6516worker2 # [ 124.007325] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41138 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16517lb # [ 123.986794] niks3-server[1122]: 2026/09/22 15:45:54 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6518worker2 # [ 124.016607] nix-daemon[866]: reaped child process 1502, status = succeeded6519client # [ 124.025516] systemd[1]: rs.service: Deactivated successfully.6520client # [ 124.028968] systemd[1]: rs.service: Consumed 120ms CPU time over 8.128s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6521worker2 # [ 124.179177] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:55Z level=info event=signal_exit6522lb # [ 124.514753] niks3-server[1122]: 2026/09/22 15:45:54 INFO lead: released remote=192.168.1.4:542006523worker1 # [ 125.525737] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:55Z level=info event=scheduler_restarting addr=lb:500516524worker2 # [ 125.534950] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:56Z level=info event=scheduler_restarting6525worker2 # [ 125.545289] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:56Z level=info event=scheduler_restarting addr=lb:500516526worker2 # [ 125.548411] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:383366527worker2 # [ 125.551932] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:56Z level=info event=worker_gone addr=192.168.1.3:500516528worker2 # [ 125.555268] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:56Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:383366529worker2 # [ 125.558981] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:56Z level=info event=worker_gone addr=192.168.1.4:500516530worker2 # [ 125.578554] nix-daemon[866]: reaped child process 1324, status = succeeded6531lb # [ 125.613764] niks3-server[1122]: 2026/09/22 15:45:55 INFO lead: acquired remote=192.168.1.3:608786532worker1 # [ 125.626109] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:55Z level=info event=scheduler_take_over6533worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.11 seconds)6534client: waiting for success: ! systemctl is-active rs6535client: (finished: waiting for success: ! systemctl is-active rs, in 0.02 seconds)6536client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6537worker1 # [ 125.725500] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:56Z level=info event=scheduler_connected addr=lb:500516538worker1 # [ 125.730272] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6539client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.02 seconds)6540client: must fail: journalctl -u rs | grep -q 'reconnecting'6541worker2 # [ 125.752714] nix-grpc-daemon[1272]: ts=2026-09-22T15:45:56Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6542client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6543lb: must succeed: curl -sf localhost:9901/clusters6544worker2 # [ 125.788285] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6545worker2 # [ 125.791146] systemd[1]: Stopped Nix worker-protocol over gRPC.6546worker2 # [ 125.794455] systemd[1]: nix-grpc-daemon.service: Consumed 1.457s CPU time over 33.717s wall clock time, 39.7M memory peak, 62K incoming IP traffic, 160.6K outgoing IP traffic.6547worker2 # [ 125.808128] systemd[1]: Starting scheduler-token.service...6548lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6549worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6550worker2 # [ 125.851255] systemd[1]: scheduler-token.service: Deactivated successfully.6551worker2 # [ 125.853555] systemd[1]: Finished scheduler-token.service.6552worker2 # [ 125.855203] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 45ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6553worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6554worker2 # [ 125.858177] systemd[1]: Starting Nix worker-protocol over gRPC...6555worker2 # [ 125.934362] nix-grpc-daemon[1558]: ts=2026-09-22T15:45:57Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6556worker2 # [ 125.938798] nix-grpc-daemon[1558]: ts=2026-09-22T15:45:57Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6557worker2 # [ 125.942109] nix-grpc-daemon[1558]: ts=2026-09-22T15:45:57Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16558worker2 # [ 125.944075] nix-grpc-daemon[1558]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6559worker2 # [ 125.945460] nix-grpc-daemon[1558]: E0000 00:00:1790091957.047422 1558 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6560worker1 # [ 125.931557] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:56Z level=info event=scheduler_connected addr=lb:500516561worker2 # [ 125.952546] nix-grpc-daemon[1558]: E0000 00:00:1790091957.054973 1558 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6562worker1 # [ 125.939568] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:448546563worker2 # [ 125.955614] nix-grpc-daemon[1558]: ts=2026-09-22T15:45:57Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6564worker2 # [ 125.957486] systemd[1]: Started Nix worker-protocol over gRPC.6565worker2 # [ 125.960078] nix-grpc-daemon[1558]: E0000 00:00:1790091957.065101 1599 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6566worker1 # [ 125.945633] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06567worker2 # [ 125.971684] nix-grpc-daemon[1558]: E0000 00:00:1790091957.077620 1599 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6568worker2 # [ 125.976054] nix-grpc-daemon[1558]: E0000 00:00:1790091957.079897 1599 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6569worker2 # [ 125.985764] nix-grpc-daemon[1558]: E0000 00:00:1790091957.091718 1599 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6570worker2 # [ 125.994798] nix-grpc-daemon[1558]: ts=2026-09-22T15:45:57Z level=info event=scheduler_connected addr=lb:500516571worker1 # [ 125.990158] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:56Z level=info event=oidc_jwks provider=mock keys=16572worker1 # [ 125.994634] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:56Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:448546573worker1 # [ 125.998573] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:56Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06574lb: must succeed: curl -sf localhost:9901/clusters6575lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6576worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6577worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6578(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.05 seconds)6579subtest: scheduler restart mid-build: build finishes, no double build6580client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6581client # [ 126.927597] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6582client # Running as unit: mid.service; invocation ID: 405b112ebad9406a9f2293e792d638a36583client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.03 seconds)6584worker1 # [ 127.041878] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:448566585worker1 # [ 127.047690] nix-daemon[846]: accepted connection from pid 1523, user nix-grpc-daemon (trusted)6586client # [ 127.053872] nix[1397]: don't know how to build these paths:6587client # [ 127.054801] nix[1397]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6588worker1 # [ 127.059576] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:57Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06589worker1 # [ 127.064050] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:57Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16590worker1 # [ 127.108643] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448566591worker1 # [ 127.119516] nix-grpc-daemon[1523]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 1/5)6592worker1 # [ 127.218912] nix-grpc-daemon[1523]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 110 ms (attempt 2/5)6593worker1 # [ 127.331229] nix-grpc-daemon[1523]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 3/5)6594worker1 # [ 127.338296] nix-grpc-daemon[1523]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 4/5)6595worker1 # [ 127.416839] nix-grpc-daemon[1523]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6596worker1 # [ 127.420267] nix-grpc-daemon[1523]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6597worker1 # [ 127.433782] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448566598client # [ 127.441213] nix[1397]: copying 1 paths to 'lb:50051'...6599worker1 # [ 127.442696] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448566600client # [ 127.444175] nix[1397]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6601worker1 # [ 127.452738] nix-daemon[846]: accepted connection from pid 1523, user nix-grpc-daemon (trusted)6602worker1 # [ 127.473676] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:57Z level=info event=niks3_push_started pid=16456603worker1 # [ 127.571628] nix-grpc-daemon[1645]: time=2026-09-22T15:45:57.861Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6604worker1 # [ 127.656084] nix-daemon[846]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6605worker1 # [ 127.667399] nix-daemon[846]: reaped child process 1660, status = succeeded6606worker1 # [ 127.753063] nix-daemon[846]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)6607worker1 # [ 127.765235] nix-daemon[846]: reaped child process 1665, status = succeeded6608lb # [ 127.764625] niks3-server[1122]: 2026/09/22 15:45:58 INFO Received uploads request method=POST path=/api/pending_closures6609worker1 # [ 127.782771] nix-grpc-daemon[1645]: time=2026-09-22T15:45:58.072Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6610worker1 # [ 127.784151] nix-grpc-daemon[1645]: time=2026-09-22T15:45:58.072Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6611lb # [ 127.796561] niks3-server[1122]: 2026/09/22 15:45:58 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6612lb # [ 127.807209] niks3-server[1122]: 2026/09/22 15:45:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6613worker1 # [ 127.821051] nix-grpc-daemon[1645]: time=2026-09-22T15:45:58.111Z level=INFO msg="Uploading 1 narinfos"6614lb # [ 127.812138] niks3-server[1122]: 2026/09/22 15:45:58 INFO Signed narinfos id=51 count=16615lb # [ 127.826088] niks3-server[1122]: 2026/09/22 15:45:58 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6616lb # [ 127.832885] niks3-server[1122]: 2026/09/22 15:45:58 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6617lb # [ 127.839823] niks3-server[1122]: 2026/09/22 15:45:58 INFO Completed upload id=516618worker1 # [ 127.848995] nix-grpc-daemon[1645]: time=2026-09-22T15:45:58.139Z level=INFO msg="Upload complete. (274ms)"6619worker1 # [ 127.852978] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44856 duration_s=0 paths=1 nar_bytes_in=4246620worker1 # [ 127.857929] nix-grpc-daemon[1523]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448566621worker1 # [ 127.866361] nix-daemon[846]: accepted connection from pid 1523, user nix-grpc-daemon (trusted)6622worker1 # [ 127.871980] nix-daemon[846]: reaped child process 1643, status = succeeded6623worker1 # [ 127.876183] nix-daemon[846]: accepted connection from pid 1523, user nix-grpc-daemon (trusted)6624client # [ 127.875216] nix[1397]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6625lb # [ 127.870376] envoy[926]: conn peer=[2001:db8:1::1]:44344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=1517 ms=79776626lb # [ 127.871967] envoy[926]: conn peer=[2001:db8:1::4]:48654 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=33416 tx=1918 ms=335306627lb: must succeed: curl -sf localhost:9901/clusters6628lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6629lb: must succeed: curl -sf localhost:9901/clusters6630lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6631worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6632worker1 # [ 130.436954] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1523 (nix-grpc-daemon) on client request.6633worker1 # [ 130.439307] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6634lb # [ 130.436671] niks3-server[1122]: 2026/09/22 15:46:00 INFO lead: released remote=192.168.1.3:608786635worker1 # [ 130.450669] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6636worker1 # [ 130.454152] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6637worker1 # [ 130.455361] systemd[1]: nix-grpc-daemon.service: Consumed 429ms CPU time over 14.951s wall clock time, 35.3M memory peak, 14.5K incoming IP traffic, 23.6K outgoing IP traffic.6638client # [ 130.451489] nix[1397]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6639worker2 # [ 130.464717] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:01Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6640client # [ 130.456975] nix[1397]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6641worker1 # [ 130.468728] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6642worker1 # [ 130.470191] systemd[1]: Starting Nix worker-protocol over gRPC...6643worker1 # [ 130.501605] nix-daemon[846]: reaped child process 1667, status = succeeded6644worker1 # [ 130.504417] nix-daemon[846]: reaped child process 1635, status = succeeded6645worker1 # [ 130.511218] nix-daemon[1669]: unexpected Nix daemon error: error: interrupted by the user6646worker1 # [ 130.523778] nix-daemon[846]: reaped child process 1669, status = failed with exit code 16647worker1 # [ 130.557225] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6648worker1 # [ 130.559356] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6649worker1 # [ 130.562622] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16650worker1 # [ 130.566542] nix-grpc-daemon[1710]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6651worker1 # [ 130.569191] nix-grpc-daemon[1710]: E0000 00:00:1790091960.856855 1710 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6652worker1 # [ 130.576123] nix-grpc-daemon[1710]: E0000 00:00:1790091960.862014 1710 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6653worker1 # [ 130.579341] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6654worker1 # [ 130.582180] systemd[1]: Started Nix worker-protocol over gRPC.6655worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.18 seconds)6656lb # [ 130.573801] niks3-server[1122]: 2026/09/22 15:46:00 INFO lead: acquired remote=192.168.1.3:505546657client: must succeed: sleep 5.06658worker1 # [ 130.586197] nix-grpc-daemon[1710]: E0000 00:00:1790091960.873623 1751 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6659worker1 # [ 130.589145] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=scheduler_take_over6660worker1 # [ 130.597218] nix-grpc-daemon[1710]: E0000 00:00:1790091960.887484 1751 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6661worker1 # [ 130.602055] nix-grpc-daemon[1710]: E0000 00:00:1790091960.889731 1751 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6662worker1 # [ 130.611053] nix-grpc-daemon[1710]: E0000 00:00:1790091960.901328 1751 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6663worker1 # [ 130.619994] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=scheduler_connected addr=lb:500516664worker1 # [ 130.626650] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:448666665worker1 # [ 130.628178] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06666worker2 # [ 130.665591] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:01Z level=info event=scheduler_connected addr=lb:500516667worker1 # [ 130.657378] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=oidc_jwks provider=mock keys=16668worker1 # [ 130.658727] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:448666669worker1 # [ 130.660301] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06670worker1 # [ 130.965843] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:01Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06671worker1 # [ 130.970875] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:01Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16672worker1 # [ 131.013946] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448986673worker1 # [ 131.024156] nix-daemon[846]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6674worker1 # [ 131.037718] nix-daemon[846]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6675client # [ 131.049931] nix[1397]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6676worker1 # [ 131.052312] nix-daemon[846]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6677client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6678client: waiting for success: ! systemctl is-active mid6679worker1 # [ 135.872983] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:06Z level=info event=niks3_push_started pid=17936680worker1 # [ 135.912181] nix-daemon[846]: reaped child process 1768, status = succeeded6681worker1 # [ 135.981642] nix-grpc-daemon[1793]: time=2026-09-22T15:46:06.271Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6682worker1 # [ 136.062920] nix-daemon[846]: accepted connection from pid 1806, user nix-grpc-daemon (trusted)6683worker1 # [ 136.076417] nix-daemon[846]: reaped child process 1809, status = succeeded6684lb # [ 136.074425] niks3-server[1122]: 2026/09/22 15:46:06 INFO Received uploads request method=POST path=/api/pending_closures6685worker1 # [ 136.094252] nix-grpc-daemon[1793]: time=2026-09-22T15:46:06.384Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6686worker1 # [ 136.095777] nix-grpc-daemon[1793]: time=2026-09-22T15:46:06.386Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6687lb # [ 136.124678] niks3-server[1122]: 2026/09/22 15:46:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6688worker1 # [ 136.141492] nix-grpc-daemon[1793]: time=2026-09-22T15:46:06.431Z level=INFO msg="Uploading 1 narinfos"6689lb # [ 136.133257] niks3-server[1122]: 2026/09/22 15:46:06 INFO Signed narinfos id=52 count=16690lb # [ 136.144510] niks3-server[1122]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6691lb # [ 136.165109] niks3-server[1122]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6692lb # [ 136.169982] niks3-server[1122]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6693lb # [ 136.174339] niks3-server[1122]: 2026/09/22 15:46:06 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6694lb # [ 136.180946] niks3-server[1122]: 2026/09/22 15:46:06 INFO Completed upload id=526695worker1 # [ 136.190223] nix-grpc-daemon[1793]: time=2026-09-22T15:46:06.480Z level=INFO msg="Upload complete. (205ms)"6696lb # [ 136.183559] niks3-server[1122]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6697worker1 # [ 136.198059] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44898 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16698worker1 # [ 136.214482] nix-daemon[846]: reaped child process 1766, status = succeeded6699client # [ 136.236721] systemd[1]: mid.service: Deactivated successfully.6700client # [ 136.240570] systemd[1]: mid.service: Consumed 128ms CPU time over 9.311s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.6K outgoing IP traffic.6701client: (finished: waiting for success: ! systemctl is-active mid, in 1.06 seconds)6702client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6703client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6704(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.97 seconds)6705subtest: interrupting the client stops the build on the worker6706client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6707client # [ 136.898370] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6708client # Running as unit: intr.service; invocation ID: 645dd6b60ef24f0fb403958ed3b03d626709client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6710worker2 # [ 137.049115] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:548686711worker2 # [ 137.054052] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6712client # [ 137.047599] nix[1433]: don't know how to build these paths:6713client # [ 137.048506] nix[1433]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6714worker1 # [ 137.061540] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:07Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06715worker1 # [ 137.065821] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:07Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16716worker2 # [ 137.117154] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548686717worker2 # [ 137.126664] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)6718worker2 # [ 137.147863] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 2/5)6719worker2 # [ 137.217739] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 208 ms (attempt 3/5)6720worker2 # [ 137.428238] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 342 ms (attempt 4/5)6721worker2 # [ 137.772231] nix-grpc-daemon[1558]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6722worker2 # [ 137.775622] nix-grpc-daemon[1558]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6723worker2 # [ 137.793970] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:548686724client # [ 137.791367] nix[1433]: copying 1 paths to 'lb:50051'...6725worker2 # [ 137.805781] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:548686726client # [ 137.794174] nix[1433]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6727worker2 # [ 137.816381] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6728worker2 # [ 137.838696] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:08Z level=info event=niks3_push_started pid=16506729lb # [ 137.871437] envoy[926]: conn peer=[2001:db8:1::3]:59656 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=993 tx=376 ms=147946730lb # [ 137.875398] envoy[926]: conn peer=[2001:db8:1::1]:58494 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3934 tx=1915 ms=91796731worker2 # [ 137.942715] nix-grpc-daemon[1650]: time=2026-09-22T15:46:09.048Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6732worker2 # [ 138.031142] nix-daemon[866]: accepted connection from pid 1663, user nix-grpc-daemon (trusted)6733worker2 # [ 138.042726] nix-daemon[866]: reaped child process 1666, status = succeeded6734worker2 # [ 138.139606] nix-daemon[866]: accepted connection from pid 1671, user nix-grpc-daemon (trusted)6735worker2 # [ 138.153388] nix-daemon[866]: reaped child process 1675, status = succeeded6736lb # [ 138.137987] niks3-server[1122]: 2026/09/22 15:46:08 INFO Received uploads request method=POST path=/api/pending_closures6737worker2 # [ 138.171066] nix-grpc-daemon[1650]: time=2026-09-22T15:46:09.276Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6738worker2 # [ 138.172496] nix-grpc-daemon[1650]: time=2026-09-22T15:46:09.276Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6739lb # [ 138.173277] niks3-server[1122]: 2026/09/22 15:46:08 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6740lb # [ 138.185116] niks3-server[1122]: 2026/09/22 15:46:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6741worker2 # [ 138.214691] nix-grpc-daemon[1650]: time=2026-09-22T15:46:09.320Z level=INFO msg="Uploading 1 narinfos"6742lb # [ 138.192913] niks3-server[1122]: 2026/09/22 15:46:08 INFO Signed narinfos id=53 count=16743lb # [ 138.202326] niks3-server[1122]: 2026/09/22 15:46:08 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6744lb # [ 138.208636] niks3-server[1122]: 2026/09/22 15:46:08 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6745lb # [ 138.216918] niks3-server[1122]: 2026/09/22 15:46:08 INFO Completed upload id=536746worker2 # [ 138.238369] nix-grpc-daemon[1650]: time=2026-09-22T15:46:09.344Z level=INFO msg="Upload complete. (292ms)"6747lb # [ 138.220051] niks3-server[1122]: 2026/09/22 15:46:08 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6748worker2 # [ 138.243170] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=0 paths=1 nar_bytes_in=4246749worker2 # [ 138.249256] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548686750worker2 # [ 138.258120] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6751worker2 # [ 138.263858] nix-daemon[866]: reaped child process 1648, status = succeeded6752worker2 # [ 138.268380] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6753client # [ 138.254971] nix[1433]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6754client: must succeed: systemctl kill -s INT intr6755client # [ 139.202376] systemd[1]: intr.service: Sent signal SIGINT to main process 1433 (nix) on client request.6756client # [ 139.205928] nix[1433]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6757client: (finished: must succeed: systemctl kill -s INT intr, in 0.03 seconds)6758worker2 # [ 139.221472] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:10Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6759client # [ 139.209292] nix[1433]: error: interrupted by the user6760client # [ 139.231548] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6761client # [ 139.233043] systemd[1]: intr.service: Failed with result 'exit-code'.6762client # [ 139.235220] systemd[1]: intr.service: Consumed 128ms CPU time over 2.333s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.6763worker2 # [ 139.294827] nix-daemon[1679]: unexpected Nix daemon error:6764worker2 # [ 139.295876] nix-daemon[866]: reaped child process 1677, status = succeeded6765worker2 # [ 139.297060] nix-daemon[1679]: error: interrupted by the user6766worker2 # [ 139.304828] nix-daemon[866]: reaped child process 1679, status = failed with exit code 16767(finished: subtest: interrupting the client stops the build on the worker, in 2.42 seconds)6768subtest: a deploy mid-build drains: the build finishes, the new generation takes over6769client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6770client # [ 139.322131] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6771client # Running as unit: sw.service; invocation ID: 8a67fb8a8f0e486ba48f07496d50bc566772client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.03 seconds)6773client # [ 139.458462] nix[1453]: don't know how to build these paths:6774client # [ 139.459469] nix[1453]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6775worker2 # [ 139.468966] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:548686776worker1 # [ 139.470317] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:09Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06777worker1 # [ 139.474406] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:09Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16778worker1 # [ 139.512602] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448986779worker1 # [ 139.524443] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5)6780worker1 # [ 139.555857] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 2/5)6781worker1 # [ 139.721187] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 297 ms (attempt 3/5)6782worker1 # [ 140.025835] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 569 ms (attempt 4/5)6783worker1 # [ 140.597861] nix-grpc-daemon[1710]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6784worker1 # [ 140.601250] nix-grpc-daemon[1710]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6785worker1 # [ 140.619339] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:448986786client # [ 140.629904] nix[1453]: copying 1 paths to 'lb:50051'...6787client # [ 140.633530] nix[1453]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6788worker1 # [ 140.634807] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:448986789worker1 # [ 140.646170] nix-daemon[846]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6790worker1 # [ 140.750368] nix-daemon[846]: accepted connection from pid 1845, user nix-grpc-daemon (trusted)6791worker1 # [ 140.766258] nix-daemon[846]: reaped child process 1848, status = succeeded6792worker1 # [ 140.849240] nix-daemon[846]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)6793worker1 # [ 140.861690] nix-daemon[846]: reaped child process 1853, status = succeeded6794lb # [ 140.860679] niks3-server[1122]: 2026/09/22 15:46:11 INFO Received uploads request method=POST path=/api/pending_closures6795worker1 # [ 140.875079] nix-grpc-daemon[1793]: time=2026-09-22T15:46:11.164Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6796worker1 # [ 140.876436] nix-grpc-daemon[1793]: time=2026-09-22T15:46:11.164Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6797lb # [ 140.875746] niks3-server[1122]: 2026/09/22 15:46:11 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6798lb # [ 140.878486] niks3-server[1122]: 2026/09/22 15:46:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6799lb # [ 140.882293] niks3-server[1122]: 2026/09/22 15:46:11 INFO Signed narinfos id=54 count=16800worker1 # [ 140.891187] nix-grpc-daemon[1793]: time=2026-09-22T15:46:11.181Z level=INFO msg="Uploading 1 narinfos"6801lb # [ 140.886171] niks3-server[1122]: 2026/09/22 15:46:11 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6802lb # [ 140.894747] niks3-server[1122]: 2026/09/22 15:46:11 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6803lb # [ 140.897744] niks3-server[1122]: 2026/09/22 15:46:11 INFO Completed upload id=546804worker1 # [ 140.906727] nix-grpc-daemon[1793]: time=2026-09-22T15:46:11.197Z level=INFO msg="Upload complete. (249ms)"6805lb # [ 140.899666] niks3-server[1122]: 2026/09/22 15:46:11 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6806worker1 # [ 140.908624] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44898 duration_s=0 paths=1 nar_bytes_in=4166807worker1 # [ 140.914376] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:448986808worker1 # [ 140.918433] nix-daemon[846]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6809worker1 # [ 140.920682] nix-daemon[846]: reaped child process 1843, status = succeeded6810worker1 # [ 140.925712] nix-daemon[846]: accepted connection from pid 1710, user nix-grpc-daemon (trusted)6811client # [ 140.924884] nix[1453]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6812worker1: must succeed: systemctl reload nix-grpc-daemon.service6813worker1 # [ 141.701504] systemd[1]: Reloading Nix worker-protocol over gRPC...6814worker1 # [ 141.727687] systemd[1]: Reloaded Nix worker-protocol over gRPC.6815worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6816worker1: must succeed: pgrep -f 'read -t [9]0 x'6817worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6818client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26819worker2 # [ 141.929321] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:548686820client # don't know how to build these paths:6821client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6822worker2 # [ 141.937530] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:548686823worker1 # [ 141.941479] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:12Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06824worker1 # [ 141.943178] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:12Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16825worker2 # [ 141.996875] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548686826worker2 # [ 142.007731] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:548686827client # copying 1 paths to 'lb:50051'...6828worker2 # [ 142.018756] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:548686829client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6830worker2 # [ 142.040645] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6831worker2 # [ 142.143850] nix-daemon[866]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6832worker2 # [ 142.156851] nix-daemon[866]: reaped child process 1725, status = succeeded6833worker2 # [ 142.233591] nix-daemon[866]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6834worker2 # [ 142.246730] nix-daemon[866]: reaped child process 1730, status = succeeded6835lb # [ 142.230280] niks3-server[1122]: 2026/09/22 15:46:12 INFO Received uploads request method=POST path=/api/pending_closures6836worker2 # [ 142.256966] nix-grpc-daemon[1650]: time=2026-09-22T15:46:13.362Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6837worker2 # [ 142.258891] nix-grpc-daemon[1650]: time=2026-09-22T15:46:13.364Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6838lb # [ 142.248913] niks3-server[1122]: 2026/09/22 15:46:12 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6839lb # [ 142.256937] niks3-server[1122]: 2026/09/22 15:46:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6840worker2 # [ 142.282886] nix-grpc-daemon[1650]: time=2026-09-22T15:46:13.388Z level=INFO msg="Uploading 1 narinfos"6841lb # [ 142.261083] niks3-server[1122]: 2026/09/22 15:46:12 INFO Signed narinfos id=55 count=16842lb # [ 142.264829] niks3-server[1122]: 2026/09/22 15:46:12 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6843lb # [ 142.270128] niks3-server[1122]: 2026/09/22 15:46:12 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6844lb # [ 142.276385] niks3-server[1122]: 2026/09/22 15:46:12 INFO Completed upload id=556845worker2 # [ 142.298072] nix-grpc-daemon[1650]: time=2026-09-22T15:46:13.404Z level=INFO msg="Upload complete. (246ms)"6846lb # [ 142.278580] niks3-server[1122]: 2026/09/22 15:46:12 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6847worker2 # [ 142.300235] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=0 paths=1 nar_bytes_in=5286848worker2 # [ 142.304702] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548686849worker2 # [ 142.310384] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6850worker2 # [ 142.312799] nix-daemon[866]: reaped child process 1720, status = succeeded6851worker2 # [ 142.317895] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6852client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6853client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6854worker1 # [ 142.578348] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:12Z level=info event=draining6855client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6856client # farm-a> LOG-farm-a-during-drain6857worker2 # [ 143.356621] nix-daemon[866]: reaped child process 1734, status = succeeded6858worker2 # [ 143.436888] nix-daemon[866]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6859worker2 # [ 143.449934] nix-daemon[866]: reaped child process 1753, status = succeeded6860lb # [ 143.435629] niks3-server[1122]: 2026/09/22 15:46:13 INFO Received uploads request method=POST path=/api/pending_closures6861worker2 # [ 143.468138] nix-grpc-daemon[1650]: time=2026-09-22T15:46:14.573Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6862worker2 # [ 143.469906] nix-grpc-daemon[1650]: time=2026-09-22T15:46:14.575Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6863lb # [ 143.477913] niks3-server[1122]: 2026/09/22 15:46:13 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6864lb # [ 143.486119] niks3-server[1122]: 2026/09/22 15:46:13 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6865lb # [ 143.496484] niks3-server[1122]: 2026/09/22 15:46:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6866lb # [ 143.499822] niks3-server[1122]: 2026/09/22 15:46:13 INFO Signed narinfos id=56 count=16867worker2 # [ 143.524311] nix-grpc-daemon[1650]: time=2026-09-22T15:46:14.630Z level=INFO msg="Uploading 1 narinfos"6868lb # [ 143.508108] niks3-server[1122]: 2026/09/22 15:46:13 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6869lb # [ 143.518715] niks3-server[1122]: 2026/09/22 15:46:13 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6870lb # [ 143.522587] niks3-server[1122]: 2026/09/22 15:46:13 INFO Completed upload id=566871lb # [ 143.525087] niks3-server[1122]: 2026/09/22 15:46:13 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6872worker2 # [ 143.545943] nix-grpc-daemon[1650]: time=2026-09-22T15:46:14.651Z level=INFO msg="Upload complete. (220ms)"6873worker2 # [ 143.551861] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16874worker1 # [ 143.543849] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:13Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06875worker1 # [ 143.545583] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:13Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16876worker2 # [ 143.567685] nix-daemon[866]: reaped child process 1732, status = succeeded6877worker2 # [ 143.607274] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548686878worker2 # [ 143.620246] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:548686879worker2 # [ 143.633954] nix-grpc-daemon[1650]: time=2026-09-22T15:46:14.739Z level=INFO msg="All 1 paths already cached"6880client # copying 1 paths to 'lb:50051'...6881worker2 # [ 143.642633] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:548686882client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6883worker2 # [ 143.662134] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6884worker2 # [ 143.763523] nix-daemon[866]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6885worker2 # [ 143.776878] nix-daemon[866]: reaped child process 1760, status = succeeded6886worker2 # [ 143.858635] nix-daemon[866]: accepted connection from pid 1762, user nix-grpc-daemon (trusted)6887worker2 # [ 143.871939] nix-daemon[866]: reaped child process 1765, status = succeeded6888lb # [ 143.855995] niks3-server[1122]: 2026/09/22 15:46:14 INFO Received uploads request method=POST path=/api/pending_closures6889worker2 # [ 143.882397] nix-grpc-daemon[1650]: time=2026-09-22T15:46:14.988Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6890worker2 # [ 143.884229] nix-grpc-daemon[1650]: time=2026-09-22T15:46:14.990Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6891lb # [ 143.872202] niks3-server[1122]: 2026/09/22 15:46:14 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6892lb # [ 143.877927] niks3-server[1122]: 2026/09/22 15:46:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6893lb # [ 143.880751] niks3-server[1122]: 2026/09/22 15:46:14 INFO Signed narinfos id=57 count=16894worker2 # [ 143.902664] nix-grpc-daemon[1650]: time=2026-09-22T15:46:15.008Z level=INFO msg="Uploading 1 narinfos"6895lb # [ 143.882680] niks3-server[1122]: 2026/09/22 15:46:14 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6896lb # [ 143.888919] niks3-server[1122]: 2026/09/22 15:46:14 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6897lb # [ 143.891700] niks3-server[1122]: 2026/09/22 15:46:14 INFO Completed upload id=576898worker2 # [ 143.914497] nix-grpc-daemon[1650]: time=2026-09-22T15:46:15.020Z level=INFO msg="Upload complete. (240ms)"6899lb # [ 143.893801] niks3-server[1122]: 2026/09/22 15:46:14 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6900worker2 # [ 143.916725] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=0 paths=1 nar_bytes_in=6406901worker2 # [ 143.920169] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548686902worker2 # [ 143.926672] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6903worker2 # [ 143.928929] nix-daemon[866]: reaped child process 1755, status = succeeded6904worker2 # [ 143.934075] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6905client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6906client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6907client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6908client # farm-b> LOG-farm-b-during-drain6909worker2 # [ 145.149715] nix-daemon[866]: reaped child process 1769, status = succeeded6910worker2 # [ 145.224424] nix-daemon[866]: accepted connection from pid 1785, user nix-grpc-daemon (trusted)6911worker2 # [ 145.237947] nix-daemon[866]: reaped child process 1788, status = succeeded6912lb # [ 145.223285] niks3-server[1122]: 2026/09/22 15:46:15 INFO Received uploads request method=POST path=/api/pending_closures6913worker2 # [ 145.256352] nix-grpc-daemon[1650]: time=2026-09-22T15:46:16.362Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6914worker2 # [ 145.257967] nix-grpc-daemon[1650]: time=2026-09-22T15:46:16.363Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6915lb # [ 145.272628] niks3-server[1122]: 2026/09/22 15:46:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6916worker2 # [ 145.302686] nix-grpc-daemon[1650]: time=2026-09-22T15:46:16.408Z level=INFO msg="Uploading 1 narinfos"6917lb # [ 145.281305] niks3-server[1122]: 2026/09/22 15:46:15 INFO Signed narinfos id=58 count=16918lb # [ 145.289516] niks3-server[1122]: 2026/09/22 15:46:15 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6919lb # [ 145.296085] niks3-server[1122]: 2026/09/22 15:46:15 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6920lb # [ 145.303219] niks3-server[1122]: 2026/09/22 15:46:15 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6921lb # [ 145.310574] niks3-server[1122]: 2026/09/22 15:46:15 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6922lb # [ 145.314297] niks3-server[1122]: 2026/09/22 15:46:15 INFO Completed upload id=586923lb # [ 145.316202] niks3-server[1122]: 2026/09/22 15:46:15 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6924worker2 # [ 145.337677] nix-grpc-daemon[1650]: time=2026-09-22T15:46:16.443Z level=INFO msg="Upload complete. (196ms)"6925worker2 # [ 145.344580] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16926worker1 # [ 145.336889] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:15Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06927worker1 # [ 145.338690] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:15Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16928worker2 # [ 145.359702] nix-daemon[866]: reaped child process 1767, status = succeeded6929worker2 # [ 145.393329] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548686930worker2 # [ 145.404352] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:548686931worker2 # [ 145.415179] nix-grpc-daemon[1650]: time=2026-09-22T15:46:16.521Z level=INFO msg="All 2 paths already cached"6932client # copying 1 paths to 'lb:50051'...6933worker2 # [ 145.423656] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:548686934client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6935worker2 # [ 145.441630] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6936worker2 # [ 145.540449] nix-daemon[866]: accepted connection from pid 1792, user nix-grpc-daemon (trusted)6937worker2 # [ 145.553783] nix-daemon[866]: reaped child process 1795, status = succeeded6938worker2 # [ 145.634855] nix-daemon[866]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6939worker2 # [ 145.647728] nix-daemon[866]: reaped child process 1800, status = succeeded6940lb # [ 145.632771] niks3-server[1122]: 2026/09/22 15:46:16 INFO Received uploads request method=POST path=/api/pending_closures6941worker2 # [ 145.659490] nix-grpc-daemon[1650]: time=2026-09-22T15:46:16.765Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6942worker2 # [ 145.661451] nix-grpc-daemon[1650]: time=2026-09-22T15:46:16.767Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6943lb # [ 145.650818] niks3-server[1122]: 2026/09/22 15:46:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6944lb # [ 145.653569] niks3-server[1122]: 2026/09/22 15:46:16 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6945lb # [ 145.655935] niks3-server[1122]: 2026/09/22 15:46:16 INFO Signed narinfos id=59 count=16946worker2 # [ 145.679142] nix-grpc-daemon[1650]: time=2026-09-22T15:46:16.784Z level=INFO msg="Uploading 1 narinfos"6947lb # [ 145.657447] niks3-server[1122]: 2026/09/22 15:46:16 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6948lb # [ 145.663741] niks3-server[1122]: 2026/09/22 15:46:16 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6949lb # [ 145.667870] niks3-server[1122]: 2026/09/22 15:46:16 INFO Completed upload id=596950worker2 # [ 145.690989] nix-grpc-daemon[1650]: time=2026-09-22T15:46:16.796Z level=INFO msg="Upload complete. (235ms)"6951lb # [ 145.669495] niks3-server[1122]: 2026/09/22 15:46:16 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6952worker2 # [ 145.693140] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=0 paths=1 nar_bytes_in=7686953worker2 # [ 145.696193] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548686954worker2 # [ 145.703089] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6955worker2 # [ 145.705347] nix-daemon[866]: reaped child process 1790, status = succeeded6956worker2 # [ 145.710547] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6957client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6958client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6959client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6960client # farm-top> LOG-farm-top-during-drain6961worker2 # [ 146.725187] nix-daemon[866]: reaped child process 1804, status = succeeded6962worker2 # [ 146.799792] nix-daemon[866]: accepted connection from pid 1820, user nix-grpc-daemon (trusted)6963worker2 # [ 146.812938] nix-daemon[866]: reaped child process 1823, status = succeeded6964lb # [ 146.799362] niks3-server[1122]: 2026/09/22 15:46:17 INFO Received uploads request method=POST path=/api/pending_closures6965worker2 # [ 146.830431] nix-grpc-daemon[1650]: time=2026-09-22T15:46:17.936Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6966worker2 # [ 146.832516] nix-grpc-daemon[1650]: time=2026-09-22T15:46:17.937Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6967lb # [ 146.839597] niks3-server[1122]: 2026/09/22 15:46:17 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6968lb # [ 146.849170] niks3-server[1122]: 2026/09/22 15:46:17 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6969lb # [ 146.858993] niks3-server[1122]: 2026/09/22 15:46:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6970lb # [ 146.864329] niks3-server[1122]: 2026/09/22 15:46:17 INFO Signed narinfos id=60 count=16971worker2 # [ 146.888837] nix-grpc-daemon[1650]: time=2026-09-22T15:46:17.994Z level=INFO msg="Uploading 1 narinfos"6972lb # [ 146.870126] niks3-server[1122]: 2026/09/22 15:46:17 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6973lb # [ 146.877709] niks3-server[1122]: 2026/09/22 15:46:17 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6974lb # [ 146.881499] niks3-server[1122]: 2026/09/22 15:46:17 INFO Completed upload id=606975worker2 # [ 146.903307] nix-grpc-daemon[1650]: time=2026-09-22T15:46:18.009Z level=INFO msg="Upload complete. (184ms)"6976lb # [ 146.884067] niks3-server[1122]: 2026/09/22 15:46:17 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6977worker2 # [ 146.910192] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16978worker2 # [ 146.923715] nix-daemon[866]: reaped child process 1802, status = succeeded6979client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.17 seconds)6980client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6981client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.13 seconds)6982worker1: must succeed: pkill -f 'read -t [9]0 x'6983worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6984client: waiting for success: ! systemctl is-active sw6985worker1 # [ 147.146263] nix-daemon[846]: reaped child process 1857, status = succeeded6986worker1 # [ 147.222974] nix-daemon[846]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)6987worker1 # [ 147.234674] nix-daemon[846]: reaped child process 1903, status = succeeded6988lb # [ 147.233499] niks3-server[1122]: 2026/09/22 15:46:17 INFO Received uploads request method=POST path=/api/pending_closures6989worker1 # [ 147.250908] nix-grpc-daemon[1793]: time=2026-09-22T15:46:17.541Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6990worker1 # [ 147.252709] nix-grpc-daemon[1793]: time=2026-09-22T15:46:17.542Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6991lb # [ 147.265527] niks3-server[1122]: 2026/09/22 15:46:17 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6992lb # [ 147.280931] niks3-server[1122]: 2026/09/22 15:46:17 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6993lb # [ 147.294696] niks3-server[1122]: 2026/09/22 15:46:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6994lb # [ 147.298377] niks3-server[1122]: 2026/09/22 15:46:17 INFO Signed narinfos id=61 count=16995worker1 # [ 147.307929] nix-grpc-daemon[1793]: time=2026-09-22T15:46:17.597Z level=INFO msg="Uploading 1 narinfos"6996lb # [ 147.303436] niks3-server[1122]: 2026/09/22 15:46:17 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6997lb # [ 147.314409] niks3-server[1122]: 2026/09/22 15:46:17 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6998lb # [ 147.317201] niks3-server[1122]: 2026/09/22 15:46:17 INFO Completed upload id=616999lb # [ 147.320060] niks3-server[1122]: 2026/09/22 15:46:17 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo7000worker1 # [ 147.327560] nix-grpc-daemon[1793]: time=2026-09-22T15:46:17.617Z level=INFO msg="Upload complete. (189ms)"7001worker1 # [ 147.336212] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44898 duration_s=6 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=17002worker1 # [ 147.349581] nix-daemon[846]: reaped child process 1855, status = succeeded7003client # [ 147.365881] systemd[1]: sw.service: Deactivated successfully.7004client # [ 147.369725] systemd[1]: sw.service: Consumed 134ms CPU time over 8.046s wall clock time, 41.4M memory peak, 7.4K incoming IP traffic, 8.3K outgoing IP traffic.7005worker1 # [ 147.579180] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:17Z level=info event=signal_exit7006lb # [ 147.577380] niks3-server[1122]: 2026/09/22 15:46:17 INFO lead: released remote=192.168.1.3:505547007lb # [ 147.872675] envoy[926]: conn peer=[2001:db8:1::1]:52470 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2967 tx=830 ms=21887008lb # [ 147.876390] envoy[926]: conn peer=[2001:db8:1::1]:52500 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9581 tx=4379 ms=49947009lb # [ 147.879895] envoy[926]: conn peer=[2001:db8:1::1]:52486 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2936 tx=1538 ms=78907010client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)7011client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }7012client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)7013worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service7014worker1 # [ 148.586781] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:18Z level=info event=scheduler_restarting7015worker2 # [ 148.600497] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:19Z level=info event=scheduler_restarting addr=lb:500517016worker1 # [ 148.594386] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:18Z level=info event=scheduler_restarting addr=lb:500517017worker1 # [ 148.597924] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448667018worker1 # [ 148.601852] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:18Z level=info event=worker_gone addr=192.168.1.4:500517019worker1 # [ 148.605553] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:18Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:448667020worker1 # [ 148.609337] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:18Z level=info event=worker_gone addr=192.168.1.3:500517021worker1 # [ 148.622825] nix-daemon[846]: reaped child process 1764, status = succeeded7022worker2 # [ 148.800895] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:19Z level=info event=scheduler_connected addr=lb:500517023worker2 # [ 148.804879] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7024worker1 # [ 148.802792] nix-grpc-daemon[1710]: ts=2026-09-22T15:46:19Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"7025worker1 # [ 148.836279] nix-grpc-daemon[1710]: E0000 00:00:1790091979.126323 1755 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true7026worker1 # [ 148.841441] nix-grpc-daemon[1710]: E0000 00:00:1790091979.131546 1755 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true7027worker2 # [ 149.006071] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_connected addr=lb:500517028worker2 # [ 149.010195] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7029worker2 # [ 149.211254] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_connected addr=lb:500517030worker2 # [ 149.215156] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7031worker2 # [ 149.416379] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_connected addr=lb:500517032worker2 # [ 149.420315] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7033worker2 # [ 149.622136] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_connected addr=lb:500517034worker2 # [ 149.625890] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7035worker2 # [ 149.827299] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_connected addr=lb:500517036worker2 # [ 149.831324] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7037worker2 # [ 150.032395] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517038worker2 # [ 150.036453] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7039worker2 # [ 150.237589] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517040worker2 # [ 150.241571] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7041worker2 # [ 150.442677] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517042worker2 # [ 150.446501] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7043worker1 # [ 150.610641] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.7044worker1 # [ 150.614342] systemd[1]: nix-grpc-daemon.service: Consumed 627ms CPU time over 20.156s wall clock time, 36.5M memory peak, 24.5K incoming IP traffic, 31.9K outgoing IP traffic.7045worker1 # [ 150.632141] systemd[1]: Starting Nix worker-protocol over gRPC...7046worker2 # [ 150.648538] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517047worker2 # [ 150.652228] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7048worker1 # [ 150.716690] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"7049worker1 # [ 150.718603] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc7050worker1 # [ 150.721634] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=17051worker1 # [ 150.725837] nix-grpc-daemon[1917]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR7052worker1 # [ 150.727183] nix-grpc-daemon[1917]: E0000 00:00:1790091981.015897 1917 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7053worker1 # [ 150.733206] nix-grpc-daemon[1917]: E0000 00:00:1790091981.021207 1917 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7054worker1 # [ 150.736966] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket7055worker1 # [ 150.739673] systemd[1]: Started Nix worker-protocol over gRPC.7056worker1 # [ 150.742288] nix-grpc-daemon[1917]: E0000 00:00:1790091981.032611 1958 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7057lb # [ 150.734339] niks3-server[1122]: 2026/09/22 15:46:21 INFO lead: acquired remote=192.168.1.3:380727058worker1 # [ 150.746883] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_take_over7059worker1 # [ 150.758103] nix-grpc-daemon[1917]: E0000 00:00:1790091981.047412 1958 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7060worker1 # [ 150.760244] nix-grpc-daemon[1917]: E0000 00:00:1790091981.047519 1958 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7061worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.58 seconds)7062lb: waiting until exactly no endpoints of x86_64-linux are unhealthy7063lb: must succeed: curl -sf localhost:9901/clusters7064worker1 # [ 150.772277] nix-grpc-daemon[1917]: E0000 00:00:1790091981.062540 1958 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7065worker1 # [ 150.784809] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517066worker1 # [ 150.786344] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7067lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7068worker2 # [ 150.853339] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517069worker2 # [ 150.857130] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7070worker1 # [ 150.987526] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517071worker1 # [ 150.991386] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7072worker2 # [ 151.058179] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:22Z level=info event=scheduler_connected addr=lb:500517073worker2 # [ 151.062306] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7074worker1 # [ 151.192255] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517075worker1 # [ 151.196198] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7076worker2 # [ 151.263262] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:22Z level=info event=scheduler_connected addr=lb:500517077worker2 # [ 151.267282] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7078worker1 # [ 151.396965] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517079worker1 # [ 151.401088] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7080worker2 # [ 151.468325] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:22Z level=info event=scheduler_connected addr=lb:500517081worker2 # [ 151.472327] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7082worker1 # [ 151.603737] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_connected addr=lb:500517083worker1 # [ 151.609952] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7084worker2 # [ 151.673201] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:22Z level=info event=scheduler_connected addr=lb:500517085worker1 # [ 151.672266] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=oidc_jwks provider=mock keys=17086worker1 # [ 151.675838] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:399047087worker1 # [ 151.679740] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07088worker1 # [ 151.811448] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:22Z level=info event=scheduler_connected addr=lb:500517089worker1 # [ 151.816131] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:399047090lb: must succeed: curl -sf localhost:9901/clusters7091worker1 # [ 151.819692] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07092lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7093lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)7094lb: must succeed: curl -sf localhost:9901/clusters7095lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7096worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7097worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7098(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 12.64 seconds)7099subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7100client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17101worker1 # [ 152.095573] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:399127102worker1 # [ 152.107381] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7103worker1 # [ 152.120861] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127104worker1 # [ 152.131213] nix-grpc-daemon[1917]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)7105worker1 # [ 152.194168] nix-grpc-daemon[1917]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 199 ms (attempt 2/5)7106worker1 # [ 152.395169] nix-grpc-daemon[1917]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 174 ms (attempt 3/5)7107worker1 # [ 152.571336] nix-grpc-daemon[1917]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 581 ms (attempt 4/5)7108worker1 # [ 153.154409] nix-grpc-daemon[1917]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7109worker1 # [ 153.156293] nix-grpc-daemon[1917]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7110worker1 # [ 153.199260] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:23Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07111worker1 # [ 153.203203] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:23Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17112worker1 # [ 153.207410] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:23Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07113worker1 # [ 153.211966] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:23Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07114worker1 # [ 153.257221] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127115worker1 # [ 153.271706] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127116worker1 # [ 153.284130] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399127117worker1 # [ 153.295975] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7118worker1 # [ 153.317190] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:23Z level=info event=niks3_push_started pid=19827119worker1 # [ 153.424985] nix-grpc-daemon[1982]: time=2026-09-22T15:46:23.714Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""7120worker1 # [ 153.518144] nix-daemon[846]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)7121worker1 # [ 153.534454] nix-daemon[846]: reaped child process 1998, status = succeeded7122worker1 # [ 153.618078] nix-daemon[846]: accepted connection from pid 2000, user nix-grpc-daemon (trusted)7123worker1 # [ 153.634983] nix-daemon[846]: reaped child process 2003, status = succeeded7124lb # [ 153.632757] niks3-server[1122]: 2026/09/22 15:46:24 INFO Received uploads request method=POST path=/api/pending_closures7125worker1 # [ 153.652879] nix-grpc-daemon[1982]: time=2026-09-22T15:46:23.943Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7126worker1 # [ 153.655499] nix-grpc-daemon[1982]: time=2026-09-22T15:46:23.944Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7127lb # [ 153.673942] niks3-server[1122]: 2026/09/22 15:46:24 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7128lb # [ 153.678938] niks3-server[1122]: 2026/09/22 15:46:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7129lb # [ 153.683706] niks3-server[1122]: 2026/09/22 15:46:24 INFO Signed narinfos id=62 count=17130worker1 # [ 153.695364] nix-grpc-daemon[1982]: time=2026-09-22T15:46:23.985Z level=INFO msg="Uploading 1 narinfos"7131lb # [ 153.696649] niks3-server[1122]: 2026/09/22 15:46:24 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7132lb # [ 153.703693] niks3-server[1122]: 2026/09/22 15:46:24 INFO Completed upload id=627133worker1 # [ 153.712763] nix-grpc-daemon[1982]: time=2026-09-22T15:46:24.003Z level=INFO msg="Upload complete. (284ms)"7134worker1 # [ 153.715166] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 paths=1 nar_bytes_in=5047135lb # [ 153.707805] niks3-server[1122]: 2026/09/22 15:46:24 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7136worker1 # [ 153.723517] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127137lb # [ 153.716403] niks3-server[1122]: 2026/09/22 15:46:24 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7138worker1 # [ 153.727473] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7139worker1 # [ 153.729934] nix-daemon[846]: reaped child process 1980, status = succeeded7140worker1 # [ 153.735804] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7141worker1 # [ 154.352129] nix-daemon[846]: reaped child process 2007, status = succeeded7142worker1 # [ 154.432352] nix-daemon[846]: accepted connection from pid 2023, user nix-grpc-daemon (trusted)7143worker1 # [ 154.449079] nix-daemon[846]: reaped child process 2026, status = succeeded7144lb # [ 154.443849] niks3-server[1122]: 2026/09/22 15:46:24 INFO Received uploads request method=POST path=/api/pending_closures7145worker1 # [ 154.460467] nix-grpc-daemon[1982]: time=2026-09-22T15:46:24.750Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7146worker1 # [ 154.462035] nix-grpc-daemon[1982]: time=2026-09-22T15:46:24.752Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7147lb # [ 154.468659] niks3-server[1122]: 2026/09/22 15:46:24 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7148lb # [ 154.481462] niks3-server[1122]: 2026/09/22 15:46:24 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7149lb # [ 154.490493] niks3-server[1122]: 2026/09/22 15:46:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7150lb # [ 154.494870] niks3-server[1122]: 2026/09/22 15:46:24 INFO Signed narinfos id=63 count=17151worker1 # [ 154.504434] nix-grpc-daemon[1982]: time=2026-09-22T15:46:24.794Z level=INFO msg="Uploading 1 narinfos"7152lb # [ 154.496402] niks3-server[1122]: 2026/09/22 15:46:24 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7153lb # [ 154.503566] niks3-server[1122]: 2026/09/22 15:46:24 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7154lb # [ 154.507646] niks3-server[1122]: 2026/09/22 15:46:24 INFO Completed upload id=637155worker1 # [ 154.516614] nix-grpc-daemon[1982]: time=2026-09-22T15:46:24.806Z level=INFO msg="Upload complete. (174ms)"7156lb # [ 154.509491] niks3-server[1122]: 2026/09/22 15:46:24 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7157worker1 # [ 154.519742] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=17158worker1 # [ 154.526103] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:24Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17159worker1 # [ 154.529160] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:24Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07160worker1 # [ 154.535484] nix-daemon[846]: reaped child process 2005, status = succeeded7161worker1 # [ 154.569701] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127162worker1 # [ 154.579695] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127163worker1 # [ 154.589989] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399127164worker1 # [ 154.598136] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7165worker1 # [ 154.693139] nix-daemon[846]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)7166worker1 # [ 154.707764] nix-daemon[846]: reaped child process 2035, status = succeeded7167worker1 # [ 154.795873] nix-daemon[846]: accepted connection from pid 2037, user nix-grpc-daemon (trusted)7168worker1 # [ 154.810893] nix-daemon[846]: reaped child process 2040, status = succeeded7169lb # [ 154.809392] niks3-server[1122]: 2026/09/22 15:46:25 INFO Received uploads request method=POST path=/api/pending_closures7170worker1 # [ 154.827154] nix-grpc-daemon[1982]: time=2026-09-22T15:46:25.117Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7171worker1 # [ 154.828504] nix-grpc-daemon[1982]: time=2026-09-22T15:46:25.117Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7172lb # [ 154.827983] niks3-server[1122]: 2026/09/22 15:46:25 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7173lb # [ 154.831798] niks3-server[1122]: 2026/09/22 15:46:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7174worker1 # [ 154.843710] nix-grpc-daemon[1982]: time=2026-09-22T15:46:25.133Z level=INFO msg="Uploading 1 narinfos"7175lb # [ 154.835327] niks3-server[1122]: 2026/09/22 15:46:25 INFO Signed narinfos id=64 count=17176lb # [ 154.839197] niks3-server[1122]: 2026/09/22 15:46:25 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7177lb # [ 154.848686] niks3-server[1122]: 2026/09/22 15:46:25 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7178lb # [ 154.851838] niks3-server[1122]: 2026/09/22 15:46:25 INFO Completed upload id=647179worker1 # [ 154.860640] nix-grpc-daemon[1982]: time=2026-09-22T15:46:25.150Z level=INFO msg="Upload complete. (254ms)"7180lb # [ 154.853858] niks3-server[1122]: 2026/09/22 15:46:25 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7181worker1 # [ 154.863055] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 paths=1 nar_bytes_in=5047182worker1 # [ 154.867520] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127183worker1 # [ 154.874404] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7184worker1 # [ 154.876822] nix-daemon[846]: reaped child process 2030, status = succeeded7185worker1 # [ 154.882223] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7186worker1 # [ 155.528231] nix-daemon[846]: reaped child process 2044, status = succeeded7187worker1 # [ 155.607335] nix-daemon[846]: accepted connection from pid 2060, user nix-grpc-daemon (trusted)7188worker1 # [ 155.622411] nix-daemon[846]: reaped child process 2063, status = succeeded7189lb # [ 155.620993] niks3-server[1122]: 2026/09/22 15:46:25 INFO Received uploads request method=POST path=/api/pending_closures7190worker1 # [ 155.638349] nix-grpc-daemon[1982]: time=2026-09-22T15:46:25.928Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7191worker1 # [ 155.639877] nix-grpc-daemon[1982]: time=2026-09-22T15:46:25.930Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7192lb # [ 155.656802] niks3-server[1122]: 2026/09/22 15:46:26 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7193lb # [ 155.671132] niks3-server[1122]: 2026/09/22 15:46:26 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7194lb # [ 155.678382] niks3-server[1122]: 2026/09/22 15:46:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7195lb # [ 155.683363] niks3-server[1122]: 2026/09/22 15:46:26 INFO Signed narinfos id=65 count=17196worker1 # [ 155.695793] nix-grpc-daemon[1982]: time=2026-09-22T15:46:25.985Z level=INFO msg="Uploading 1 narinfos"7197lb # [ 155.693255] niks3-server[1122]: 2026/09/22 15:46:26 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7198lb # [ 155.702077] niks3-server[1122]: 2026/09/22 15:46:26 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7199lb # [ 155.709529] niks3-server[1122]: 2026/09/22 15:46:26 INFO Completed upload id=657200worker1 # [ 155.718692] nix-grpc-daemon[1982]: time=2026-09-22T15:46:26.008Z level=INFO msg="Upload complete. (229ms)"7201lb # [ 155.711559] niks3-server[1122]: 2026/09/22 15:46:26 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7202worker1 # [ 155.725673] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=17203worker1 # [ 155.738278] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:26Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17204worker1 # [ 155.742242] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:26Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07205worker1 # [ 155.753199] nix-daemon[846]: reaped child process 2042, status = succeeded7206worker1 # [ 155.777215] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127207worker1 # [ 155.787495] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127208worker1 # [ 155.796761] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399127209worker1 # [ 155.806224] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7210worker1 # [ 155.904755] nix-daemon[846]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)7211worker1 # [ 155.921315] nix-daemon[846]: reaped child process 2072, status = succeeded7212worker1 # [ 156.002921] nix-daemon[846]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)7213worker1 # [ 156.019806] nix-daemon[846]: reaped child process 2077, status = succeeded7214lb # [ 156.014299] niks3-server[1122]: 2026/09/22 15:46:26 INFO Received uploads request method=POST path=/api/pending_closures7215worker1 # [ 156.029447] nix-grpc-daemon[1982]: time=2026-09-22T15:46:26.319Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7216worker1 # [ 156.031376] nix-grpc-daemon[1982]: time=2026-09-22T15:46:26.321Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7217lb # [ 156.035509] niks3-server[1122]: 2026/09/22 15:46:26 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7218lb # [ 156.041352] niks3-server[1122]: 2026/09/22 15:46:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7219worker1 # [ 156.053154] nix-grpc-daemon[1982]: time=2026-09-22T15:46:26.343Z level=INFO msg="Uploading 1 narinfos"7220lb # [ 156.044252] niks3-server[1122]: 2026/09/22 15:46:26 INFO Signed narinfos id=66 count=17221lb # [ 156.048590] niks3-server[1122]: 2026/09/22 15:46:26 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7222lb # [ 156.057062] niks3-server[1122]: 2026/09/22 15:46:26 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7223lb # [ 156.060539] niks3-server[1122]: 2026/09/22 15:46:26 INFO Completed upload id=667224worker1 # [ 156.069409] nix-grpc-daemon[1982]: time=2026-09-22T15:46:26.359Z level=INFO msg="Upload complete. (253ms)"7225lb # [ 156.062167] niks3-server[1122]: 2026/09/22 15:46:26 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7226worker1 # [ 156.071667] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 paths=1 nar_bytes_in=5047227worker1 # [ 156.076194] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127228worker1 # [ 156.082935] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7229worker1 # [ 156.085249] nix-daemon[846]: reaped child process 2067, status = succeeded7230worker1 # [ 156.090597] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7231worker1 # [ 157.309251] nix-daemon[846]: reaped child process 2081, status = succeeded7232worker1 # [ 157.387814] nix-daemon[846]: accepted connection from pid 2097, user nix-grpc-daemon (trusted)7233worker1 # [ 157.404083] nix-daemon[846]: reaped child process 2100, status = succeeded7234lb # [ 157.402780] niks3-server[1122]: 2026/09/22 15:46:27 INFO Received uploads request method=POST path=/api/pending_closures7235worker1 # [ 157.418883] nix-grpc-daemon[1982]: time=2026-09-22T15:46:27.708Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7236worker1 # [ 157.422234] nix-grpc-daemon[1982]: time=2026-09-22T15:46:27.710Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7237lb # [ 157.437150] niks3-server[1122]: 2026/09/22 15:46:27 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7238lb # [ 157.453172] niks3-server[1122]: 2026/09/22 15:46:27 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7239lb # [ 157.458317] niks3-server[1122]: 2026/09/22 15:46:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7240worker1 # [ 157.474310] nix-grpc-daemon[1982]: time=2026-09-22T15:46:27.764Z level=INFO msg="Uploading 1 narinfos"7241lb # [ 157.465093] niks3-server[1122]: 2026/09/22 15:46:27 INFO Signed narinfos id=67 count=17242lb # [ 157.471565] niks3-server[1122]: 2026/09/22 15:46:27 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7243lb # [ 157.480057] niks3-server[1122]: 2026/09/22 15:46:27 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7244lb # [ 157.484155] niks3-server[1122]: 2026/09/22 15:46:27 INFO Completed upload id=677245worker1 # [ 157.493228] nix-grpc-daemon[1982]: time=2026-09-22T15:46:27.783Z level=INFO msg="Upload complete. (195ms)"7246lb # [ 157.485744] niks3-server[1122]: 2026/09/22 15:46:27 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7247worker1 # [ 157.496349] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=17248worker1 # [ 157.501166] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:27Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17249worker1 # [ 157.505058] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:27Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07250worker1 # [ 157.511578] nix-daemon[846]: reaped child process 2079, status = succeeded7251worker1 # [ 157.544434] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127252worker1 # [ 157.553299] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127253worker1 # [ 157.563241] nix-grpc-daemon[1982]: time=2026-09-22T15:46:27.853Z level=INFO msg="All 1 paths already cached"7254worker1 # [ 157.569152] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399127255worker1 # [ 157.577132] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7256worker1 # [ 157.671184] nix-daemon[846]: accepted connection from pid 2106, user nix-grpc-daemon (trusted)7257worker1 # [ 157.686766] nix-daemon[846]: reaped child process 2109, status = succeeded7258worker1 # [ 157.774409] nix-daemon[846]: accepted connection from pid 2111, user nix-grpc-daemon (trusted)7259worker1 # [ 157.789643] nix-daemon[846]: reaped child process 2114, status = succeeded7260lb # [ 157.787119] niks3-server[1122]: 2026/09/22 15:46:28 INFO Received uploads request method=POST path=/api/pending_closures7261worker1 # [ 157.805373] nix-grpc-daemon[1982]: time=2026-09-22T15:46:28.095Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7262worker1 # [ 157.806823] nix-grpc-daemon[1982]: time=2026-09-22T15:46:28.095Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7263lb # [ 157.817986] niks3-server[1122]: 2026/09/22 15:46:28 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7264lb # [ 157.829784] niks3-server[1122]: 2026/09/22 15:46:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7265worker1 # [ 157.845555] nix-grpc-daemon[1982]: time=2026-09-22T15:46:28.135Z level=INFO msg="Uploading 1 narinfos"7266lb # [ 157.837104] niks3-server[1122]: 2026/09/22 15:46:28 INFO Signed narinfos id=68 count=17267lb # [ 157.842192] niks3-server[1122]: 2026/09/22 15:46:28 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7268lb # [ 157.856331] niks3-server[1122]: 2026/09/22 15:46:28 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7269worker1 # [ 157.872064] nix-grpc-daemon[1982]: time=2026-09-22T15:46:28.162Z level=INFO msg="Upload complete. (286ms)"7270lb # [ 157.863337] niks3-server[1122]: 2026/09/22 15:46:28 INFO Completed upload id=687271lb # [ 157.865531] niks3-server[1122]: 2026/09/22 15:46:28 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7272worker1 # [ 157.875063] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 paths=1 nar_bytes_in=6247273lb # [ 157.872820] envoy[926]: conn peer=[2001:db8:1::3]:33334 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=958 tx=344 ms=181787274worker1 # [ 157.880158] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127275worker1 # [ 157.890284] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7276worker1 # [ 157.896724] nix-daemon[846]: reaped child process 2104, status = succeeded7277worker1 # [ 157.901896] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7278worker1 # [ 158.972416] nix-daemon[846]: reaped child process 2118, status = succeeded7279worker1 # [ 159.025767] nix-daemon[846]: accepted connection from pid 2134, user nix-grpc-daemon (trusted)7280worker1 # [ 159.040616] nix-daemon[846]: reaped child process 2137, status = succeeded7281lb # [ 159.036975] niks3-server[1122]: 2026/09/22 15:46:29 INFO Received uploads request method=POST path=/api/pending_closures7282worker1 # [ 159.056334] nix-grpc-daemon[1982]: time=2026-09-22T15:46:29.346Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7283worker1 # [ 159.057951] nix-grpc-daemon[1982]: time=2026-09-22T15:46:29.348Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7284lb # [ 159.076416] niks3-server[1122]: 2026/09/22 15:46:29 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7285lb # [ 159.091160] niks3-server[1122]: 2026/09/22 15:46:29 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7286lb # [ 159.100431] niks3-server[1122]: 2026/09/22 15:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7287worker1 # [ 159.113953] nix-grpc-daemon[1982]: time=2026-09-22T15:46:29.404Z level=INFO msg="Uploading 1 narinfos"7288lb # [ 159.105504] niks3-server[1122]: 2026/09/22 15:46:29 INFO Signed narinfos id=69 count=17289lb # [ 159.113430] niks3-server[1122]: 2026/09/22 15:46:29 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7290lb # [ 159.122359] niks3-server[1122]: 2026/09/22 15:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7291lb # [ 159.126448] niks3-server[1122]: 2026/09/22 15:46:29 INFO Completed upload id=697292worker1 # [ 159.135333] nix-grpc-daemon[1982]: time=2026-09-22T15:46:29.425Z level=INFO msg="Upload complete. (225ms)"7293lb # [ 159.128319] niks3-server[1122]: 2026/09/22 15:46:29 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7294worker1 # [ 159.142149] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=17295worker1 # [ 159.154743] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:29Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17296worker1 # [ 159.159234] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:29Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07297worker1 # [ 159.169730] nix-daemon[846]: reaped child process 2116, status = succeeded7298worker1 # [ 159.194088] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127299worker1 # [ 159.206920] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127300worker1 # [ 159.220778] nix-grpc-daemon[1982]: time=2026-09-22T15:46:29.510Z level=INFO msg="All 1 paths already cached"7301worker1 # [ 159.229673] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399127302worker1 # [ 159.237344] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7303worker1 # [ 159.340545] nix-daemon[846]: accepted connection from pid 2143, user nix-grpc-daemon (trusted)7304worker1 # [ 159.357972] nix-daemon[846]: reaped child process 2146, status = succeeded7305worker1 # [ 159.442941] nix-daemon[846]: accepted connection from pid 2148, user nix-grpc-daemon (trusted)7306worker1 # [ 159.458510] nix-daemon[846]: reaped child process 2151, status = succeeded7307lb # [ 159.455467] niks3-server[1122]: 2026/09/22 15:46:29 INFO Received uploads request method=POST path=/api/pending_closures7308worker1 # [ 159.472722] nix-grpc-daemon[1982]: time=2026-09-22T15:46:29.762Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7309worker1 # [ 159.474167] nix-grpc-daemon[1982]: time=2026-09-22T15:46:29.762Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7310lb # [ 159.483576] niks3-server[1122]: 2026/09/22 15:46:29 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7311lb # [ 159.493989] niks3-server[1122]: 2026/09/22 15:46:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7312lb # [ 159.501432] niks3-server[1122]: 2026/09/22 15:46:29 INFO Signed narinfos id=70 count=17313worker1 # [ 159.513860] nix-grpc-daemon[1982]: time=2026-09-22T15:46:29.803Z level=INFO msg="Uploading 1 narinfos"7314lb # [ 159.506126] niks3-server[1122]: 2026/09/22 15:46:29 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7315lb # [ 159.521585] niks3-server[1122]: 2026/09/22 15:46:29 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7316lb # [ 159.528143] niks3-server[1122]: 2026/09/22 15:46:29 INFO Completed upload id=707317worker1 # [ 159.537235] nix-grpc-daemon[1982]: time=2026-09-22T15:46:29.827Z level=INFO msg="Upload complete. (287ms)"7318lb # [ 159.532163] niks3-server[1122]: 2026/09/22 15:46:29 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7319worker1 # [ 159.541233] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 paths=1 nar_bytes_in=6247320worker1 # [ 159.546975] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127321worker1 # [ 159.557567] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7322worker1 # [ 159.565054] nix-daemon[846]: reaped child process 2141, status = succeeded7323worker1 # [ 159.569713] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7324worker1 # [ 160.991112] nix-daemon[846]: reaped child process 2155, status = succeeded7325worker1 # [ 161.069101] nix-daemon[846]: accepted connection from pid 2171, user nix-grpc-daemon (trusted)7326worker1 # [ 161.082848] nix-daemon[846]: reaped child process 2174, status = succeeded7327lb # [ 161.080936] niks3-server[1122]: 2026/09/22 15:46:31 INFO Received uploads request method=POST path=/api/pending_closures7328worker1 # [ 161.095611] nix-grpc-daemon[1982]: time=2026-09-22T15:46:31.385Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7329worker1 # [ 161.097201] nix-grpc-daemon[1982]: time=2026-09-22T15:46:31.387Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7330lb # [ 161.105854] niks3-server[1122]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7331lb # [ 161.112941] niks3-server[1122]: 2026/09/22 15:46:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7332lb # [ 161.115108] niks3-server[1122]: 2026/09/22 15:46:31 INFO Signed narinfos id=71 count=17333worker1 # [ 161.123899] nix-grpc-daemon[1982]: time=2026-09-22T15:46:31.413Z level=INFO msg="Uploading 1 narinfos"7334lb # [ 161.116135] niks3-server[1122]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7335lb # [ 161.119705] niks3-server[1122]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7336lb # [ 161.125092] niks3-server[1122]: 2026/09/22 15:46:31 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7337lb # [ 161.129225] niks3-server[1122]: 2026/09/22 15:46:31 INFO Completed upload id=717338worker1 # [ 161.138069] nix-grpc-daemon[1982]: time=2026-09-22T15:46:31.428Z level=INFO msg="Upload complete. (160ms)"7339lb # [ 161.131260] niks3-server[1122]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7340worker1 # [ 161.141294] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=17341worker1 # [ 161.146528] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:31Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17342worker1 # [ 161.149336] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127343worker1 # [ 161.151426] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:31Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07344worker1 # [ 161.159294] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127345worker1 # [ 161.161865] nix-daemon[846]: reaped child process 2153, status = succeeded7346worker1 # [ 161.165654] nix-grpc-daemon[1982]: time=2026-09-22T15:46:31.455Z level=INFO msg="All 1 paths already cached"7347worker1 # [ 161.169626] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399127348worker1 # [ 161.173605] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7349worker1 # [ 161.266591] nix-daemon[846]: accepted connection from pid 2180, user nix-grpc-daemon (trusted)7350worker1 # [ 161.281631] nix-daemon[846]: reaped child process 2183, status = succeeded7351worker1 # [ 161.368764] nix-daemon[846]: accepted connection from pid 2185, user nix-grpc-daemon (trusted)7352worker1 # [ 161.384543] nix-daemon[846]: reaped child process 2188, status = succeeded7353lb # [ 161.380959] niks3-server[1122]: 2026/09/22 15:46:31 INFO Received uploads request method=POST path=/api/pending_closures7354worker1 # [ 161.399244] nix-grpc-daemon[1982]: time=2026-09-22T15:46:31.689Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7355worker1 # [ 161.400920] nix-grpc-daemon[1982]: time=2026-09-22T15:46:31.691Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7356lb # [ 161.410955] niks3-server[1122]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7357lb # [ 161.423528] niks3-server[1122]: 2026/09/22 15:46:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7358lb # [ 161.428872] niks3-server[1122]: 2026/09/22 15:46:31 INFO Signed narinfos id=72 count=17359worker1 # [ 161.440629] nix-grpc-daemon[1982]: time=2026-09-22T15:46:31.730Z level=INFO msg="Uploading 1 narinfos"7360lb # [ 161.443405] niks3-server[1122]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7361lb # [ 161.450380] niks3-server[1122]: 2026/09/22 15:46:31 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7362worker1 # [ 161.465927] nix-grpc-daemon[1982]: time=2026-09-22T15:46:31.756Z level=INFO msg="Upload complete. (286ms)"7363lb # [ 161.457300] niks3-server[1122]: 2026/09/22 15:46:31 INFO Completed upload id=727364lb # [ 161.460328] niks3-server[1122]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7365worker1 # [ 161.469480] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 paths=1 nar_bytes_in=6247366worker1 # [ 161.474609] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127367worker1 # [ 161.483264] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7368worker1 # [ 161.494112] nix-daemon[846]: reaped child process 2178, status = succeeded7369worker1 # [ 161.497842] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7370worker1 # [ 162.538956] nix-daemon[846]: reaped child process 2192, status = succeeded7371worker1 # [ 162.625089] nix-daemon[846]: accepted connection from pid 2208, user nix-grpc-daemon (trusted)7372worker1 # [ 162.638704] nix-daemon[846]: reaped child process 2211, status = succeeded7373lb # [ 162.638288] niks3-server[1122]: 2026/09/22 15:46:33 INFO Received uploads request method=POST path=/api/pending_closures7374worker1 # [ 162.652045] nix-grpc-daemon[1982]: time=2026-09-22T15:46:32.942Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7375worker1 # [ 162.654922] nix-grpc-daemon[1982]: time=2026-09-22T15:46:32.943Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7376lb # [ 162.657661] niks3-server[1122]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7377lb # [ 162.660329] niks3-server[1122]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7378lb # [ 162.665145] niks3-server[1122]: 2026/09/22 15:46:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7379lb # [ 162.667955] niks3-server[1122]: 2026/09/22 15:46:33 INFO Signed narinfos id=73 count=17380worker1 # [ 162.677158] nix-grpc-daemon[1982]: time=2026-09-22T15:46:32.967Z level=INFO msg="Uploading 1 narinfos"7381lb # [ 162.672470] niks3-server[1122]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7382lb # [ 162.676268] niks3-server[1122]: 2026/09/22 15:46:33 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7383lb # [ 162.679710] niks3-server[1122]: 2026/09/22 15:46:33 INFO Completed upload id=737384worker1 # [ 162.689991] nix-grpc-daemon[1982]: time=2026-09-22T15:46:32.980Z level=INFO msg="Upload complete. (182ms)"7385lb # [ 162.683751] niks3-server[1122]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7386worker1 # [ 162.692812] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=17387worker1 # [ 162.697772] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:32Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17388worker1 # [ 162.700733] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127389worker1 # [ 162.704411] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:32Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07390worker1 # [ 162.711309] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127391worker1 # [ 162.713693] nix-daemon[846]: reaped child process 2190, status = succeeded7392worker1 # [ 162.717612] nix-grpc-daemon[1982]: time=2026-09-22T15:46:33.007Z level=INFO msg="All 2 paths already cached"7393worker1 # [ 162.722052] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399127394worker1 # [ 162.726147] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7395worker1 # [ 162.816954] nix-daemon[846]: accepted connection from pid 2217, user nix-grpc-daemon (trusted)7396worker1 # [ 162.832430] nix-daemon[846]: reaped child process 2220, status = succeeded7397worker1 # [ 162.918179] nix-daemon[846]: accepted connection from pid 2222, user nix-grpc-daemon (trusted)7398worker1 # [ 162.934078] nix-daemon[846]: reaped child process 2225, status = succeeded7399lb # [ 162.929545] niks3-server[1122]: 2026/09/22 15:46:33 INFO Received uploads request method=POST path=/api/pending_closures7400worker1 # [ 162.942979] nix-grpc-daemon[1982]: time=2026-09-22T15:46:33.232Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7401worker1 # [ 162.944387] nix-grpc-daemon[1982]: time=2026-09-22T15:46:33.233Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7402lb # [ 162.944612] niks3-server[1122]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7403lb # [ 162.946915] niks3-server[1122]: 2026/09/22 15:46:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7404lb # [ 162.948767] niks3-server[1122]: 2026/09/22 15:46:33 INFO Signed narinfos id=74 count=17405worker1 # [ 162.958961] nix-grpc-daemon[1982]: time=2026-09-22T15:46:33.249Z level=INFO msg="Uploading 1 narinfos"7406lb # [ 162.952779] niks3-server[1122]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7407lb # [ 162.960571] niks3-server[1122]: 2026/09/22 15:46:33 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7408lb # [ 162.963407] niks3-server[1122]: 2026/09/22 15:46:33 INFO Completed upload id=747409worker1 # [ 162.972369] nix-grpc-daemon[1982]: time=2026-09-22T15:46:33.262Z level=INFO msg="Upload complete. (240ms)"7410lb # [ 162.965061] niks3-server[1122]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7411worker1 # [ 162.974722] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 paths=1 nar_bytes_in=7527412worker1 # [ 162.979469] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127413worker1 # [ 162.986199] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7414worker1 # [ 162.988589] nix-daemon[846]: reaped child process 2215, status = succeeded7415worker1 # [ 162.994175] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7416worker1 # [ 163.864156] nix-daemon[846]: reaped child process 2229, status = succeeded7417worker1 # [ 163.907476] nix-daemon[846]: accepted connection from pid 2245, user nix-grpc-daemon (trusted)7418worker1 # [ 163.922603] nix-daemon[846]: reaped child process 2248, status = succeeded7419lb # [ 163.921544] niks3-server[1122]: 2026/09/22 15:46:34 INFO Received uploads request method=POST path=/api/pending_closures7420worker1 # [ 163.938322] nix-grpc-daemon[1982]: time=2026-09-22T15:46:34.227Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7421worker1 # [ 163.939822] nix-grpc-daemon[1982]: time=2026-09-22T15:46:34.227Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7422lb # [ 163.961775] niks3-server[1122]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7423lb # [ 163.974967] niks3-server[1122]: 2026/09/22 15:46:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7424lb # [ 163.980359] niks3-server[1122]: 2026/09/22 15:46:34 INFO Signed narinfos id=75 count=17425worker1 # [ 163.992377] nix-grpc-daemon[1982]: time=2026-09-22T15:46:34.282Z level=INFO msg="Uploading 1 narinfos"7426lb # [ 163.985624] niks3-server[1122]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7427lb # [ 163.989968] niks3-server[1122]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7428lb # [ 164.001955] niks3-server[1122]: 2026/09/22 15:46:34 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7429lb # [ 164.005726] niks3-server[1122]: 2026/09/22 15:46:34 INFO Completed upload id=757430worker1 # [ 164.014590] nix-grpc-daemon[1982]: time=2026-09-22T15:46:34.304Z level=INFO msg="Upload complete. (212ms)"7431lb # [ 164.007588] niks3-server[1122]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7432worker1 # [ 164.017827] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=17433worker1 # [ 164.021654] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:34Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17434worker1 # [ 164.029248] nix-daemon[846]: reaped child process 2227, status = succeeded7435worker1 # [ 164.065884] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127436worker1 # [ 164.074927] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127437worker1 # [ 164.084397] nix-grpc-daemon[1982]: time=2026-09-22T15:46:34.374Z level=INFO msg="All 2 paths already cached"7438worker1 # [ 164.089894] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399127439worker1 # [ 164.093978] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7440worker1 # [ 164.185870] nix-daemon[846]: accepted connection from pid 2252, user nix-grpc-daemon (trusted)7441worker1 # [ 164.201386] nix-daemon[846]: reaped child process 2255, status = succeeded7442worker1 # [ 164.287185] nix-daemon[846]: accepted connection from pid 2257, user nix-grpc-daemon (trusted)7443worker1 # [ 164.301650] nix-daemon[846]: reaped child process 2260, status = succeeded7444lb # [ 164.299867] niks3-server[1122]: 2026/09/22 15:46:34 INFO Received uploads request method=POST path=/api/pending_closures7445worker1 # [ 164.317072] nix-grpc-daemon[1982]: time=2026-09-22T15:46:34.607Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7446worker1 # [ 164.319068] nix-grpc-daemon[1982]: time=2026-09-22T15:46:34.608Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7447lb # [ 164.334186] niks3-server[1122]: 2026/09/22 15:46:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7448lb # [ 164.341203] niks3-server[1122]: 2026/09/22 15:46:34 INFO Signed narinfos id=76 count=17449worker1 # [ 164.354139] nix-grpc-daemon[1982]: time=2026-09-22T15:46:34.644Z level=INFO msg="Uploading 1 narinfos"7450lb # [ 164.348312] niks3-server[1122]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7451lb # [ 164.353855] niks3-server[1122]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7452lb # [ 164.365051] niks3-server[1122]: 2026/09/22 15:46:34 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7453worker1 # [ 164.380411] nix-grpc-daemon[1982]: time=2026-09-22T15:46:34.670Z level=INFO msg="Upload complete. (279ms)"7454lb # [ 164.371562] niks3-server[1122]: 2026/09/22 15:46:34 INFO Completed upload id=767455worker1 # [ 164.382853] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 paths=1 nar_bytes_in=7527456worker1 # [ 164.386963] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127457lb # [ 164.378410] niks3-server[1122]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7458worker1 # [ 164.394297] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7459worker1 # [ 164.398235] nix-daemon[846]: reaped child process 2250, status = succeeded7460worker1 # [ 164.403229] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7461worker1 # [ 165.621263] nix-daemon[846]: reaped child process 2264, status = succeeded7462worker1 # [ 165.699700] nix-daemon[846]: accepted connection from pid 2280, user nix-grpc-daemon (trusted)7463worker1 # [ 165.716092] nix-daemon[846]: reaped child process 2283, status = succeeded7464lb # [ 165.709103] niks3-server[1122]: 2026/09/22 15:46:36 INFO Received uploads request method=POST path=/api/pending_closures7465worker1 # [ 165.724887] nix-grpc-daemon[1982]: time=2026-09-22T15:46:36.014Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7466worker1 # [ 165.726469] nix-grpc-daemon[1982]: time=2026-09-22T15:46:36.016Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7467lb # [ 165.752327] niks3-server[1122]: 2026/09/22 15:46:36 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7468lb # [ 165.757378] niks3-server[1122]: 2026/09/22 15:46:36 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7469lb # [ 165.764649] niks3-server[1122]: 2026/09/22 15:46:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7470worker1 # [ 165.778715] nix-grpc-daemon[1982]: time=2026-09-22T15:46:36.068Z level=INFO msg="Uploading 1 narinfos"7471lb # [ 165.769580] niks3-server[1122]: 2026/09/22 15:46:36 INFO Signed narinfos id=77 count=17472lb # [ 165.776284] niks3-server[1122]: 2026/09/22 15:46:36 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7473lb # [ 165.785067] niks3-server[1122]: 2026/09/22 15:46:36 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7474lb # [ 165.787786] niks3-server[1122]: 2026/09/22 15:46:36 INFO Completed upload id=777475worker1 # [ 165.798238] nix-grpc-daemon[1982]: time=2026-09-22T15:46:36.088Z level=INFO msg="Upload complete. (188ms)"7476lb # [ 165.792848] niks3-server[1122]: 2026/09/22 15:46:36 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7477worker1 # [ 165.801199] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=17478worker1 # [ 165.805359] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:36Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17479worker1 # [ 165.812632] nix-daemon[846]: reaped child process 2262, status = succeeded7480worker1 # [ 165.848935] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127481worker1 # [ 165.857990] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127482worker1 # [ 165.867379] nix-grpc-daemon[1982]: time=2026-09-22T15:46:36.157Z level=INFO msg="All 2 paths already cached"7483worker1 # [ 165.872761] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:399127484worker1 # [ 165.878177] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7485worker1 # [ 165.964265] nix-daemon[846]: accepted connection from pid 2287, user nix-grpc-daemon (trusted)7486worker1 # [ 165.980560] nix-daemon[846]: reaped child process 2290, status = succeeded7487worker1 # [ 166.058865] nix-daemon[846]: accepted connection from pid 2292, user nix-grpc-daemon (trusted)7488worker1 # [ 166.074258] nix-daemon[846]: reaped child process 2295, status = succeeded7489lb # [ 166.069506] niks3-server[1122]: 2026/09/22 15:46:36 INFO Received uploads request method=POST path=/api/pending_closures7490worker1 # [ 166.082810] nix-grpc-daemon[1982]: time=2026-09-22T15:46:36.373Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7491worker1 # [ 166.084686] nix-grpc-daemon[1982]: time=2026-09-22T15:46:36.374Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7492lb # [ 166.085514] niks3-server[1122]: 2026/09/22 15:46:36 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7493lb # [ 166.088565] niks3-server[1122]: 2026/09/22 15:46:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7494lb # [ 166.090304] niks3-server[1122]: 2026/09/22 15:46:36 INFO Signed narinfos id=78 count=17495worker1 # [ 166.100568] nix-grpc-daemon[1982]: time=2026-09-22T15:46:36.390Z level=INFO msg="Uploading 1 narinfos"7496lb # [ 166.097381] niks3-server[1122]: 2026/09/22 15:46:36 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7497lb # [ 166.098804] niks3-server[1122]: 2026/09/22 15:46:36 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7498lb # [ 166.101470] niks3-server[1122]: 2026/09/22 15:46:36 INFO Completed upload id=787499worker1 # [ 166.111613] nix-grpc-daemon[1982]: time=2026-09-22T15:46:36.401Z level=INFO msg="Upload complete. (228ms)"7500worker1 # [ 166.113868] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 paths=1 nar_bytes_in=7527501lb # [ 166.105941] niks3-server[1122]: 2026/09/22 15:46:36 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7502worker1 # [ 166.118068] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:399127503worker1 # [ 166.124606] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7504worker1 # [ 166.127502] nix-daemon[846]: reaped child process 2285, status = succeeded7505worker1 # [ 166.132633] nix-daemon[846]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7506worker1 # [ 166.752945] nix-daemon[846]: reaped child process 2299, status = succeeded7507worker1 # [ 166.824842] nix-daemon[846]: accepted connection from pid 2315, user nix-grpc-daemon (trusted)7508worker1 # [ 166.840871] nix-daemon[846]: reaped child process 2318, status = succeeded7509lb # [ 166.836990] niks3-server[1122]: 2026/09/22 15:46:37 INFO Received uploads request method=POST path=/api/pending_closures7510worker1 # [ 166.855039] nix-grpc-daemon[1982]: time=2026-09-22T15:46:37.145Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7511worker1 # [ 166.856568] nix-grpc-daemon[1982]: time=2026-09-22T15:46:37.146Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7512lb # [ 166.873166] niks3-server[1122]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7513lb # [ 166.885519] niks3-server[1122]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7514lb # [ 166.900710] niks3-server[1122]: 2026/09/22 15:46:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7515lb # [ 166.906780] niks3-server[1122]: 2026/09/22 15:46:37 INFO Signed narinfos id=79 count=17516worker1 # [ 166.918150] nix-grpc-daemon[1982]: time=2026-09-22T15:46:37.208Z level=INFO msg="Uploading 1 narinfos"7517lb # [ 166.910100] niks3-server[1122]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7518lb # [ 166.919574] niks3-server[1122]: 2026/09/22 15:46:37 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7519lb # [ 166.923651] niks3-server[1122]: 2026/09/22 15:46:37 INFO Completed upload id=797520worker1 # [ 166.932446] nix-grpc-daemon[1982]: time=2026-09-22T15:46:37.222Z level=INFO msg="Upload complete. (193ms)"7521lb # [ 166.925257] niks3-server[1122]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7522worker1 # [ 166.935451] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39912 duration_s=0 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=17523worker1 # [ 166.944979] nix-daemon[846]: reaped child process 2297, status = succeeded7524client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.05 seconds)7525worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7526worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)7527client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7528client # [ 167.064328] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7529client # Running as unit: gpu.service; invocation ID: 693d205b7d9e44ea90307e25b30bcba27530client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7531lb: must succeed: curl -sf localhost:9901/clusters7532lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7533lb: must succeed: curl -sf localhost:9901/clusters7534lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7535worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7536worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7537worker1 # [ 167.210591] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:399127538client # [ 167.215756] nix[1533]: don't know how to build these paths:7539client # [ 167.217095] nix[1533]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7540worker2 # [ 167.231223] nix-grpc-daemon[1558]: ts=2026-09-22T15:46:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:548687541worker1 # [ 167.235226] nix-grpc-daemon[1917]: ts=2026-09-22T15:46:37Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07542client # [ 167.239224] nix[1533]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7543lb # [ 167.872760] envoy[926]: conn peer=[2001:db8:1::1]:44650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27038 tx=12318 ms=148657544lb: must succeed: curl -sf localhost:9901/clusters7545lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7546lb: must succeed: curl -sf localhost:9901/clusters7547lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7548worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7549worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7550client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7551client # [ 197.240458] nix[1533]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7552client # [ 197.250344] nix[1533]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7553client # [ 197.281661] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7554client # [ 197.285488] systemd[1]: gpu.service: Failed with result 'exit-code'.7555client # [ 197.289427] systemd[1]: gpu.service: Consumed 159ms CPU time over 30.222s wall clock time, 37.6M memory peak, 5.2K incoming IP traffic, 5.1K outgoing IP traffic.7556client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.27 seconds)7557client: must succeed: journalctl -u gpu -o cat7558client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7559lb: must succeed: curl -sf localhost:9901/clusters7560lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7561lb: must succeed: curl -sf localhost:9901/clusters7562lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7563worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7564worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7565(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.76 seconds)7566subtest: low disk drains a worker and builds go to the other7567worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7568lb # [ 197.874971] envoy[926]: conn peer=[2001:db8:1::1]:49266 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=562 ms=300537569worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.97 seconds)7570worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7571worker1 # [ 198.748333] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:09Z level=info event=unhealthy reason=min_free7572worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.05 seconds)7573client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27574worker1 # [ 199.871443] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:399127575client # don't know how to build these paths:7576client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7577worker1 # [ 199.876736] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:399127578worker1 # [ 199.888597] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:10Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07579worker1 # [ 199.890262] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:10Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17580worker2 # [ 199.946272] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548687581worker2 # [ 199.959911] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:548687582client # copying 1 paths to 'lb:50051'...7583client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7584worker2 # [ 199.972950] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:548687585worker2 # [ 199.985462] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7586worker2 # [ 200.097885] nix-daemon[866]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)7587worker2 # [ 200.112607] nix-daemon[866]: reaped child process 1830, status = succeeded7588worker2 # [ 200.204725] nix-daemon[866]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)7589worker2 # [ 200.218950] nix-daemon[866]: reaped child process 1835, status = succeeded7590lb # [ 200.205644] niks3-server[1122]: 2026/09/22 15:47:10 INFO Received uploads request method=POST path=/api/pending_closures7591worker2 # [ 200.238552] nix-grpc-daemon[1650]: time=2026-09-22T15:47:11.344Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7592worker2 # [ 200.240364] nix-grpc-daemon[1650]: time=2026-09-22T15:47:11.346Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7593lb # [ 200.248215] niks3-server[1122]: 2026/09/22 15:47:10 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7594lb # [ 200.266834] niks3-server[1122]: 2026/09/22 15:47:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7595lb # [ 200.271933] niks3-server[1122]: 2026/09/22 15:47:10 INFO Signed narinfos id=80 count=17596worker2 # [ 200.296268] nix-grpc-daemon[1650]: time=2026-09-22T15:47:11.402Z level=INFO msg="Uploading 1 narinfos"7597lb # [ 200.280659] niks3-server[1122]: 2026/09/22 15:47:10 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7598lb # [ 200.290412] niks3-server[1122]: 2026/09/22 15:47:10 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7599lb # [ 200.293436] niks3-server[1122]: 2026/09/22 15:47:10 INFO Completed upload id=807600worker2 # [ 200.315474] nix-grpc-daemon[1650]: time=2026-09-22T15:47:11.421Z level=INFO msg="Upload complete. (317ms)"7601lb # [ 200.295732] niks3-server[1122]: 2026/09/22 15:47:10 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7602worker2 # [ 200.320735] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=0 paths=1 nar_bytes_in=5127603worker2 # [ 200.327284] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548687604worker2 # [ 200.338132] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7605worker2 # [ 200.348751] nix-daemon[866]: reaped child process 1825, status = succeeded7606worker2 # [ 200.356811] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7607client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7608client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7609client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7610client # farm-a> LOG-farm-a-drain7611worker2 # [ 200.972236] nix-daemon[866]: reaped child process 1839, status = succeeded7612worker2 # [ 201.052643] nix-daemon[866]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)7613worker2 # [ 201.067806] nix-daemon[866]: reaped child process 1858, status = succeeded7614lb # [ 201.050866] niks3-server[1122]: 2026/09/22 15:47:11 INFO Received uploads request method=POST path=/api/pending_closures7615worker2 # [ 201.081616] nix-grpc-daemon[1650]: time=2026-09-22T15:47:12.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7616worker2 # [ 201.083280] nix-grpc-daemon[1650]: time=2026-09-22T15:47:12.189Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7617lb # [ 201.090871] niks3-server[1122]: 2026/09/22 15:47:11 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7618lb # [ 201.099581] niks3-server[1122]: 2026/09/22 15:47:11 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7619lb # [ 201.105282] niks3-server[1122]: 2026/09/22 15:47:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7620lb # [ 201.109512] niks3-server[1122]: 2026/09/22 15:47:11 INFO Signed narinfos id=81 count=17621worker2 # [ 201.133729] nix-grpc-daemon[1650]: time=2026-09-22T15:47:12.239Z level=INFO msg="Uploading 1 narinfos"7622lb # [ 201.117061] niks3-server[1122]: 2026/09/22 15:47:11 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7623lb # [ 201.122973] niks3-server[1122]: 2026/09/22 15:47:11 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7624lb # [ 201.126273] niks3-server[1122]: 2026/09/22 15:47:11 INFO Completed upload id=817625worker2 # [ 201.148190] nix-grpc-daemon[1650]: time=2026-09-22T15:47:12.254Z level=INFO msg="Upload complete. (185ms)"7626lb # [ 201.129057] niks3-server[1122]: 2026/09/22 15:47:11 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7627worker2 # [ 201.153266] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=0 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17628worker2 # [ 201.166152] nix-daemon[866]: reaped child process 1837, status = succeeded7629worker1 # [ 201.187785] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:11Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07630worker1 # [ 201.189788] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:11Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17631worker2 # [ 201.245143] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548687632worker2 # [ 201.258451] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:548687633worker2 # [ 201.272972] nix-grpc-daemon[1650]: time=2026-09-22T15:47:12.378Z level=INFO msg="All 1 paths already cached"7634client # copying 1 paths to 'lb:50051'...7635client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7636worker2 # [ 201.281394] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:548687637worker2 # [ 201.291356] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7638worker2 # [ 201.395674] nix-daemon[866]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)7639worker2 # [ 201.411425] nix-daemon[866]: reaped child process 1865, status = succeeded7640worker2 # [ 201.496576] nix-daemon[866]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)7641worker2 # [ 201.511945] nix-daemon[866]: reaped child process 1870, status = succeeded7642lb # [ 201.496623] niks3-server[1122]: 2026/09/22 15:47:11 INFO Received uploads request method=POST path=/api/pending_closures7643worker2 # [ 201.527187] nix-grpc-daemon[1650]: time=2026-09-22T15:47:12.632Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7644worker2 # [ 201.528581] nix-grpc-daemon[1650]: time=2026-09-22T15:47:12.632Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7645lb # [ 201.527283] niks3-server[1122]: 2026/09/22 15:47:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7646lb # [ 201.535232] niks3-server[1122]: 2026/09/22 15:47:11 INFO Signed narinfos id=82 count=17647worker2 # [ 201.560284] nix-grpc-daemon[1650]: time=2026-09-22T15:47:12.666Z level=INFO msg="Uploading 1 narinfos"7648lb # [ 201.542551] niks3-server[1122]: 2026/09/22 15:47:11 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7649lb # [ 201.547986] niks3-server[1122]: 2026/09/22 15:47:11 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7650lb # [ 201.558869] niks3-server[1122]: 2026/09/22 15:47:11 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7651lb # [ 201.562442] niks3-server[1122]: 2026/09/22 15:47:11 INFO Completed upload id=827652worker2 # [ 201.585400] nix-grpc-daemon[1650]: time=2026-09-22T15:47:12.691Z level=INFO msg="Upload complete. (281ms)"7653lb # [ 201.564507] niks3-server[1122]: 2026/09/22 15:47:11 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7654worker2 # [ 201.589070] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=0 paths=1 nar_bytes_in=6247655worker2 # [ 201.595080] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548687656worker2 # [ 201.602433] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7657worker2 # [ 201.608609] nix-daemon[866]: reaped child process 1860, status = succeeded7658worker2 # [ 201.613627] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7659client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7660client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7661client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7662client # farm-b> LOG-farm-b-drain7663worker2 # [ 202.634284] nix-daemon[866]: reaped child process 1874, status = succeeded7664worker2 # [ 202.708600] nix-daemon[866]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)7665worker2 # [ 202.722118] nix-daemon[866]: reaped child process 1899, status = succeeded7666lb # [ 202.706865] niks3-server[1122]: 2026/09/22 15:47:13 INFO Received uploads request method=POST path=/api/pending_closures7667worker2 # [ 202.737544] nix-grpc-daemon[1650]: time=2026-09-22T15:47:13.843Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7668worker2 # [ 202.739093] nix-grpc-daemon[1650]: time=2026-09-22T15:47:13.845Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7669lb # [ 202.750570] niks3-server[1122]: 2026/09/22 15:47:13 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7670lb # [ 202.758229] niks3-server[1122]: 2026/09/22 15:47:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7671worker2 # [ 202.789626] nix-grpc-daemon[1650]: time=2026-09-22T15:47:13.895Z level=INFO msg="Uploading 1 narinfos"7672lb # [ 202.768226] niks3-server[1122]: 2026/09/22 15:47:13 INFO Signed narinfos id=83 count=17673lb # [ 202.773088] niks3-server[1122]: 2026/09/22 15:47:13 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7674lb # [ 202.777057] niks3-server[1122]: 2026/09/22 15:47:13 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7675lb # [ 202.788907] niks3-server[1122]: 2026/09/22 15:47:13 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7676lb # [ 202.791732] niks3-server[1122]: 2026/09/22 15:47:13 INFO Completed upload id=837677lb # [ 202.793165] niks3-server[1122]: 2026/09/22 15:47:13 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7678worker2 # [ 202.813612] nix-grpc-daemon[1650]: time=2026-09-22T15:47:13.919Z level=INFO msg="Upload complete. (193ms)"7679worker2 # [ 202.819777] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17680worker1 # [ 202.813862] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:13Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07681worker1 # [ 202.818304] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:13Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17682worker2 # [ 202.835264] nix-daemon[866]: reaped child process 1872, status = succeeded7683worker2 # [ 202.870178] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548687684worker2 # [ 202.882833] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:548687685worker2 # [ 202.895374] nix-grpc-daemon[1650]: time=2026-09-22T15:47:14.001Z level=INFO msg="All 2 paths already cached"7686client # copying 1 paths to 'lb:50051'...7687client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7688worker2 # [ 202.904286] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:548687689worker2 # [ 202.914169] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7690worker2 # [ 203.015853] nix-daemon[866]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)7691worker2 # [ 203.031465] nix-daemon[866]: reaped child process 1906, status = succeeded7692worker2 # [ 203.117093] nix-daemon[866]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)7693worker2 # [ 203.131448] nix-daemon[866]: reaped child process 1911, status = succeeded7694lb # [ 203.117404] niks3-server[1122]: 2026/09/22 15:47:13 INFO Received uploads request method=POST path=/api/pending_closures7695worker2 # [ 203.146931] nix-grpc-daemon[1650]: time=2026-09-22T15:47:14.252Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7696worker2 # [ 203.148341] nix-grpc-daemon[1650]: time=2026-09-22T15:47:14.252Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7697lb # [ 203.142950] niks3-server[1122]: 2026/09/22 15:47:13 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7698lb # [ 203.152153] niks3-server[1122]: 2026/09/22 15:47:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7699lb # [ 203.158504] niks3-server[1122]: 2026/09/22 15:47:13 INFO Signed narinfos id=84 count=17700worker2 # [ 203.184093] nix-grpc-daemon[1650]: time=2026-09-22T15:47:14.289Z level=INFO msg="Uploading 1 narinfos"7701lb # [ 203.165255] niks3-server[1122]: 2026/09/22 15:47:13 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7702lb # [ 203.180180] niks3-server[1122]: 2026/09/22 15:47:13 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7703worker2 # [ 203.206238] nix-grpc-daemon[1650]: time=2026-09-22T15:47:14.312Z level=INFO msg="Upload complete. (279ms)"7704lb # [ 203.184675] niks3-server[1122]: 2026/09/22 15:47:13 INFO Completed upload id=847705worker2 # [ 203.208437] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=0 paths=1 nar_bytes_in=7527706lb # [ 203.190862] niks3-server[1122]: 2026/09/22 15:47:13 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7707worker2 # [ 203.212637] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:548687708worker2 # [ 203.223456] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7709worker2 # [ 203.225612] nix-daemon[866]: reaped child process 1901, status = succeeded7710worker2 # [ 203.231051] nix-daemon[866]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)7711client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7712client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7713client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7714client # farm-top> LOG-farm-top-drain7715worker2 # [ 204.061803] nix-daemon[866]: reaped child process 1915, status = succeeded7716worker2 # [ 204.141427] nix-daemon[866]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)7717worker2 # [ 204.156657] nix-daemon[866]: reaped child process 1934, status = succeeded7718lb # [ 204.142066] niks3-server[1122]: 2026/09/22 15:47:14 INFO Received uploads request method=POST path=/api/pending_closures7719worker2 # [ 204.172826] nix-grpc-daemon[1650]: time=2026-09-22T15:47:15.278Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7720worker2 # [ 204.174300] nix-grpc-daemon[1650]: time=2026-09-22T15:47:15.280Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7721lb # [ 204.176394] niks3-server[1122]: 2026/09/22 15:47:14 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7722lb # [ 204.190818] niks3-server[1122]: 2026/09/22 15:47:14 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7723lb # [ 204.202659] niks3-server[1122]: 2026/09/22 15:47:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7724lb # [ 204.206925] niks3-server[1122]: 2026/09/22 15:47:14 INFO Signed narinfos id=85 count=17725worker2 # [ 204.231506] nix-grpc-daemon[1650]: time=2026-09-22T15:47:15.337Z level=INFO msg="Uploading 1 narinfos"7726lb # [ 204.212087] niks3-server[1122]: 2026/09/22 15:47:14 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7727lb # [ 204.220767] niks3-server[1122]: 2026/09/22 15:47:14 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7728lb # [ 204.224419] niks3-server[1122]: 2026/09/22 15:47:14 INFO Completed upload id=857729worker2 # [ 204.246299] nix-grpc-daemon[1650]: time=2026-09-22T15:47:15.352Z level=INFO msg="Upload complete. (206ms)"7730lb # [ 204.227094] niks3-server[1122]: 2026/09/22 15:47:14 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7731worker2 # [ 204.252914] nix-grpc-daemon[1558]: ts=2026-09-22T15:47:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54868 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17732worker2 # [ 204.267534] nix-daemon[866]: reaped child process 1913, status = succeeded7733client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.54 seconds)7734client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7735client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7736worker1: must succeed: rm /nix/.rw-store/fill7737worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.09 seconds)7738worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7739worker1 # [ 204.751835] nix-grpc-daemon[1917]: ts=2026-09-22T15:47:15Z level=info event=healthy reason=min_free7740worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7741(finished: subtest: low disk drains a worker and builds go to the other, in 7.89 seconds)7742(finished: run the VM test script, in 206.60 seconds)7743test script finished in 206.77s7744cleanup7745kill QemuMachine (pid 50)7746client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7747client # [2026-09-22T15:47:16Z INFO virtiofsd] Client disconnected, shutting down7748client # [2026-09-22T15:47:16Z INFO virtiofsd] Client disconnected, shutting down7749client # [2026-09-22T15:47:16Z INFO virtiofsd] Client disconnected, shutting down7750kill QemuMachine (pid 49)7751lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7752lb # [2026-09-22T15:47:16Z INFO virtiofsd] Client disconnected, shutting down7753lb # [2026-09-22T15:47:16Z INFO virtiofsd] Client disconnected, shutting down7754lb # [2026-09-22T15:47:16Z INFO virtiofsd] Client disconnected, shutting down7755kill QemuMachine (pid 52)7756worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7757worker1 # [2026-09-22T15:47:17Z INFO virtiofsd] Client disconnected, shutting down7758worker1 # [2026-09-22T15:47:17Z INFO virtiofsd] Client disconnected, shutting down7759worker1 # [2026-09-22T15:47:17Z INFO virtiofsd] Client disconnected, shutting down7760kill QemuMachine (pid 51)7761worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7762worker2 # [2026-09-22T15:47:17Z INFO virtiofsd] Client disconnected, shutting down7763worker2 # [2026-09-22T15:47:17Z INFO virtiofsd] Client disconnected, shutting down7764worker2 # [2026-09-22T15:47:17Z INFO virtiofsd] Client disconnected, shutting down7765(finished: cleanup, in 1.09 seconds)7766additionally exposed symbols:7767 client, lb, worker1, worker2,7768 vlan1,7769 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh