vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #555
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Disk image does not exist, creating the virtualisation disk image...14worker2: QEMU running (pid 51)15worker1 # Formatting '/build/vm-state-worker1/tmp.NShA0jcJos', fmt=raw size=107374182416worker1: QEMU running (pid 50)17worker1 # mke2fs 1.47.4 (6-Mar-2025)18worker1 # Discarding device blocks: 0/262144 done19worker2 # Formatting '/build/vm-state-worker2/tmp.fmHsLHSsoe', fmt=raw size=107374182420lb # Disk image does not exist, creating the virtualisation disk image...21worker2 # mke2fs 1.47.4 (6-Mar-2025)22lb: QEMU running (pid 52)23worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes24lb # Formatting '/build/vm-state-lb/tmp.MwrQ0IDwMl', fmt=raw size=107374182425worker1 # Filesystem UUID: cc077db4-20b5-4e1c-a158-0a8190c218a926lb # mke2fs 1.47.4 (6-Mar-2025)27worker1 # Superblock backups stored on blocks:28lb # Discarding device blocks: 0/262144 done29worker1 # 32768, 98304, 163840, 22937630lb # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # 32lb # Filesystem UUID: 2d696889-416f-466d-9fa9-26c489cca00e33worker1 # Allocating group tables: 0/8 done34lb # Superblock backups stored on blocks:35worker1 # Writing inode tables: 0/8 done36lb # 32768, 98304, 163840, 22937637worker1 # Creating journal (8192 blocks): done38lb # 39client # Disk image does not exist, creating the virtualisation disk image...40worker1 # Writing superblocks and filesystem accounting information: 0/8 done41lb # Allocating group tables: 0/8 done42client # Formatting '/build/vm-state-client/tmp.F90Pq4Ay1T', fmt=raw size=107374182443worker1 # 44lb # Writing inode tables: 0/8 done45client # mke2fs 1.47.4 (6-Mar-2025)46worker1 # Virtualisation disk image created.47lb # Creating journal (8192 blocks): done48client # Discarding device blocks: 0/262144 done49worker1 # Starting virtiofs daemons...50lb # Writing superblocks and filesystem accounting information: 0/8 done51client # Creating filesystem with 262144 4k blocks and 65536 inodes52worker1 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)53lb # 54client # Filesystem UUID: bc4bd039-9c84-4628-b353-463c8839111055worker1 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether56lb # Virtualisation disk image created.57client # Superblock backups stored on blocks:58worker1 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)59lb # Starting virtiofs daemons...60client # 32768, 98304, 163840, 22937661worker1 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether62lb # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)63client # 64worker1 # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...65lb # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether66client # Allocating group tables: 0/8 done67worker1 # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...68lb # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...69client # Writing inode tables: 0/8 done70worker1 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)71lb # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)72client # Creating journal (8192 blocks): done73worker1 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether74lb # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether75client # Writing superblocks and filesystem accounting information: 0/8 done76worker1 # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...77lb # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...78client # 79worker1 # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests80lb # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)81client # Virtualisation disk image created.82worker1 # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests83lb # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84client # Starting virtiofs daemons...85worker1 # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests86lb # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...87client # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88worker2 # Discarding device blocks: 0/262144 done89client: QEMU running (pid 49)90lb # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests91client # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92lb # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests93client # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...94lb # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests95worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes96(finished: start all VMs, in 0.23 seconds)97worker2 # Filesystem UUID: 65ece40a-d9ae-4768-8dab-87f2d51bad4898worker2 # Superblock backups stored on blocks:99worker2 # 32768, 98304, 163840, 229376100client: waiting for unit network-addresses-eth1.service101client # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)102client: waiting for the VM to finish booting103client # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104client # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)105worker2 # 106client # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether107worker2 # Allocating group tables: 0/8 done108client # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...109worker2 # Writing inode tables: 0/8 done110client # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...111worker2 # Creating journal (8192 blocks): done112client # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests113worker2 # Writing superblocks and filesystem accounting information: 0/8 done114client # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests115worker2 # 116client # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests117worker2 # Virtualisation disk image created.118worker2 # Starting virtiofs daemons...119worker2 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker2 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker2 # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker2 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker2 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker2 # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker2 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker2 # [2026-09-25T15:12:38Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker2 # [2026-09-25T15:12:38Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker2 # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests129worker2 # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests130worker2 # [2026-09-25T15:12:38Z INFO virtiofsd] Client connected, servicing requests131worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker1 # 136worker1 # 137worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138lb # 139lb # 140lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...142worker1 # 143worker1 # 144worker1 # 145worker1 # 146worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00147lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...148lb # 149lb # 150lb # 151lb # 152lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00153worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...154worker1 # 155worker1 # 156lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...157lb # 158lb # 159client # 160client # 161client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00162worker2 # 163worker2 # 164worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00165client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...166client # 167client # 168client # 169client # 170client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00171worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...172worker2 # 173worker2 # 174worker2 # 175worker2 # 176worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00177client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178client # 179client # 180worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker2 # 182worker2 # 183worker1 # Booting from ROM...184lb # Booting from ROM...185client # Booting from ROM...186worker2 # Booting from ROM...187worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026189worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/l43lgd9fw6x3f87dwymd1s3rqn6a6sln-nixos-system-worker1-test/init regInfo=/nix/store/3rmyn12yf8b55cpvcp28gsadc21gxx56-closure-info/registration console=ttyS0,115200n8 console=tty0190worker1 # [ 0.000000] BIOS-provided physical RAM map:191worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable192worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved193worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved194lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0195lb # [ 0.000000] BIOS-provided physical RAM map:196worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable197lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable198lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved199worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved200lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved201worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved202lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable203worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved204lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved205worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved206lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved207worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved208lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved209worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved210lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved211worker1 # [ 0.000000] NX (Execute Disable) protection: active212lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved213worker1 # [ 0.000000] APIC: Static calls initialized214worker1 # [ 0.000000] SMBIOS 2.8 present.215lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved216lb # [ 0.000000] NX (Execute Disable) protection: active217worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014218lb # [ 0.000000] APIC: Static calls initialized219worker1 # [ 0.000000] DMI: Memory slots populated: 1/1220lb # [ 0.000000] SMBIOS 2.8 present.221worker1 # [ 0.000000] Hypervisor detected: KVM222client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026223worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000224lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014225worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00226lb # [ 0.000000] DMI: Memory slots populated: 1/1227lb # [ 0.000000] Hypervisor detected: KVM228worker1 # [ 0.000000] kvm-clock: using sched offset of 475872841 cycles229lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000230lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00231worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns232client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/06ljfqp30yw0k96bramnjwwmx4zj0rxb-nixos-system-client-test/init regInfo=/nix/store/qphv9d6khr6g6mdrc69p4bvgdiyanqss-closure-info/registration console=ttyS0,115200n8 console=tty0233worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor234lb # [ 0.000000] kvm-clock: using sched offset of 461943710 cycles235client # [ 0.000000] BIOS-provided physical RAM map:236worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026237worker1 # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000238client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable239lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns240worker1 # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs241client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved242lb # [ 0.000005] tsc: Detected 2400.012 MHz processor243worker1 # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT244lb # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000245client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved246worker1 # [ 0.002736] found SMP MP-table at [mem 0x000f5450-0x000f545f]247client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable248lb # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs249worker1 # [ 0.002749] Using GB pages for direct mapping250worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/21kicvrb3q0il9r6n7cv4xjicnnqfdk9-nixos-system-worker2-test/init regInfo=/nix/store/wb05yk17njbxf589bx4way1c2prk6511-closure-info/registration console=ttyS0,115200n8 console=tty0251client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved252worker1 # [ 0.002864] RAMDISK: [mem 0x5e370000-0x5ffcffff]253lb # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT254worker2 # [ 0.000000] BIOS-provided physical RAM map:255worker1 # [ 0.002872] ACPI: Early table checksum verification disabled256lb # [ 0.002810] found SMP MP-table at [mem 0x000f5450-0x000f545f]257worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable258client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved259lb # [ 0.002845] Using GB pages for direct mapping260worker1 # [ 0.002876] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )261client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved262worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved263lb # [ 0.002994] RAMDISK: [mem 0x5e36e000-0x5ffcffff]264worker1 # [ 0.002880] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)265client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved266worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved267lb # [ 0.003016] ACPI: Early table checksum verification disabled268client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved269lb # [ 0.003021] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )270worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable271worker1 # [ 0.002885] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)272client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved273worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved274lb # [ 0.003027] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)275worker1 # [ 0.002893] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)276client # [ 0.000000] NX (Execute Disable) protection: active277worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved278worker1 # [ 0.002895] ACPI: FACS 0x000000005FFE0000 000040279client # [ 0.000000] APIC: Static calls initialized280lb # [ 0.003032] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)281client # [ 0.000000] SMBIOS 2.8 present.282worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved283worker1 # [ 0.002896] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)284lb # [ 0.003041] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)285worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved286client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014287lb # [ 0.003044] ACPI: FACS 0x000000005FFE0000 000040288worker1 # [ 0.002898] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)289client # [ 0.000000] DMI: Memory slots populated: 1/1290worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved291client # [ 0.000000] Hypervisor detected: KVM292lb # [ 0.003045] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)293worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved294worker1 # [ 0.002900] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)295client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000296worker2 # [ 0.000000] NX (Execute Disable) protection: active297client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00298lb # [ 0.003047] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)299worker2 # [ 0.000000] APIC: Static calls initialized300worker1 # [ 0.002901] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)301client # [ 0.000000] kvm-clock: using sched offset of 490860337 cycles302worker2 # [ 0.000000] SMBIOS 2.8 present.303worker1 # [ 0.002902] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]304lb # [ 0.003049] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)305client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns306worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014307worker1 # [ 0.002903] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]308lb # [ 0.003050] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)309worker2 # [ 0.000000] DMI: Memory slots populated: 1/1310client # [ 0.000005] tsc: Detected 2400.012 MHz processor311worker2 # [ 0.000000] Hypervisor detected: KVM312worker1 # [ 0.002904] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]313client # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000314lb # [ 0.003051] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]315worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000316worker1 # [ 0.002904] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]317lb # [ 0.003053] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]318client # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs319worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00320worker1 # [ 0.002905] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]321worker2 # [ 0.000000] kvm-clock: using sched offset of 480423717 cycles322lb # [ 0.003053] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]323client # [ 0.000833] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT324client # [ 0.002723] found SMP MP-table at [mem 0x000f5450-0x000f545f]325worker1 # [ 0.002905] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]326lb # [ 0.003054] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]327worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns328client # [ 0.002734] Using GB pages for direct mapping329worker1 # [ 0.002906] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]330client # [ 0.002834] RAMDISK: [mem 0x5e370000-0x5ffcffff]331lb # [ 0.003054] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]332worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor333worker1 # [ 0.003130] No NUMA configuration found334worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000335client # [ 0.002841] ACPI: Early table checksum verification disabled336lb # [ 0.003055] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]337worker1 # [ 0.003131] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]338client # [ 0.002844] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )339worker2 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs340lb # [ 0.003055] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]341worker1 # [ 0.003134] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]342worker1 # [ 0.005337] Zone ranges:343lb # [ 0.003372] No NUMA configuration found344client # [ 0.002848] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)345worker2 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT346worker1 # [ 0.005337] DMA [mem 0x0000000000001000-0x0000000000ffffff]347lb # [ 0.003373] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]348worker2 # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f]349client # [ 0.002851] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)350worker2 # [ 0.002741] Using GB pages for direct mapping351worker1 # [ 0.005339] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]352lb # [ 0.003376] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]353worker1 # [ 0.005340] Normal empty354lb # [ 0.005675] Zone ranges:355worker2 # [ 0.002851] RAMDISK: [mem 0x5e36f000-0x5ffcffff]356worker1 # [ 0.005341] Device empty357client # [ 0.002859] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)358lb # [ 0.005677] DMA [mem 0x0000000000001000-0x0000000000ffffff]359client # [ 0.002861] ACPI: FACS 0x000000005FFE0000 000040360worker2 # [ 0.002859] ACPI: Early table checksum verification disabled361worker1 # [ 0.005342] Movable zone start for each node362worker1 # [ 0.005342] Early memory node ranges363worker2 # [ 0.002861] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )364lb # [ 0.005678] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]365client # [ 0.002862] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)366lb # [ 0.005679] Normal empty367worker1 # [ 0.005343] node 0: [mem 0x0000000000001000-0x000000000009efff]368lb # [ 0.005680] Device empty369worker2 # [ 0.002865] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)370client # [ 0.002864] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)371lb # [ 0.005681] Movable zone start for each node372worker1 # [ 0.005344] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]373lb # [ 0.005681] Early memory node ranges374worker2 # [ 0.002869] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)375worker1 # [ 0.005345] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]376client # [ 0.002865] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)377lb # [ 0.005682] node 0: [mem 0x0000000000001000-0x000000000009efff]378worker1 # [ 0.005374] On node 0, zone DMA: 1 pages in unavailable ranges379worker2 # [ 0.002876] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)380lb # [ 0.005683] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]381client # [ 0.002867] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)382worker2 # [ 0.002878] ACPI: FACS 0x000000005FFE0000 000040383worker1 # [ 0.005631] On node 0, zone DMA: 97 pages in unavailable ranges384lb # [ 0.005684] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]385client # [ 0.002868] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]386worker1 # [ 0.031771] On node 0, zone DMA32: 40 pages in unavailable ranges387worker2 # [ 0.002879] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)388lb # [ 0.005704] On node 0, zone DMA: 1 pages in unavailable ranges389worker1 # [ 0.032281] ACPI: PM-Timer IO Port: 0x608390client # [ 0.002869] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]391worker2 # [ 0.002881] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)392lb # [ 0.005958] On node 0, zone DMA: 97 pages in unavailable ranges393worker1 # [ 0.032292] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])394client # [ 0.002870] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]395lb # [ 0.031889] On node 0, zone DMA32: 40 pages in unavailable ranges396worker1 # [ 0.032319] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23397worker2 # [ 0.002882] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)398client # [ 0.002870] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]399lb # [ 0.032347] ACPI: PM-Timer IO Port: 0x608400worker1 # [ 0.032322] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)401client # [ 0.002871] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]402worker2 # [ 0.002884] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)403lb # [ 0.032359] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])404worker1 # [ 0.032323] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)405client # [ 0.002871] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]406worker2 # [ 0.002885] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]407lb # [ 0.032386] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23408worker1 # [ 0.032325] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)409worker2 # [ 0.002886] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]410lb # [ 0.032388] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)411client # [ 0.002872] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]412worker1 # [ 0.032326] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)413worker2 # [ 0.002887] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]414lb # [ 0.032390] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)415client # [ 0.003099] No NUMA configuration found416worker1 # [ 0.032326] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)417lb # [ 0.032391] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)418worker2 # [ 0.002887] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]419worker1 # [ 0.032329] ACPI: Using ACPI (MADT) for SMP configuration information420lb # [ 0.032392] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)421worker2 # [ 0.002888] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]422client # [ 0.003100] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]423worker1 # [ 0.032330] ACPI: HPET id: 0x8086a201 base: 0xfed00000424worker1 # [ 0.032333] TSC deadline timer available425client # [ 0.003102] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]426lb # [ 0.032393] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)427worker2 # [ 0.002888] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]428client # [ 0.005325] Zone ranges:429worker1 # [ 0.032337] CPU topo: Max. logical packages: 1430lb # [ 0.032395] ACPI: Using ACPI (MADT) for SMP configuration information431worker2 # [ 0.002889] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]432client # [ 0.005325] DMA [mem 0x0000000000001000-0x0000000000ffffff]433worker1 # [ 0.032338] CPU topo: Max. logical dies: 1434worker2 # [ 0.003110] No NUMA configuration found435lb # [ 0.032396] ACPI: HPET id: 0x8086a201 base: 0xfed00000436worker1 # [ 0.032338] CPU topo: Max. dies per package: 1437client # [ 0.005327] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]438lb # [ 0.032400] TSC deadline timer available439worker2 # [ 0.003111] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]440client # [ 0.005328] Normal empty441worker1 # [ 0.032342] CPU topo: Max. threads per core: 1442client # [ 0.005328] Device empty443lb # [ 0.032404] CPU topo: Max. logical packages: 1444worker2 # [ 0.003113] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]445worker1 # [ 0.032342] CPU topo: Num. cores per package: 1446client # [ 0.005329] Movable zone start for each node447lb # [ 0.032404] CPU topo: Max. logical dies: 1448worker2 # [ 0.005341] Zone ranges:449client # [ 0.005330] Early memory node ranges450worker1 # [ 0.032343] CPU topo: Num. threads per package: 1451lb # [ 0.032405] CPU topo: Max. dies per package: 1452worker2 # [ 0.005342] DMA [mem 0x0000000000001000-0x0000000000ffffff]453worker1 # [ 0.032343] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs454client # [ 0.005330] node 0: [mem 0x0000000000001000-0x000000000009efff]455lb # [ 0.032408] CPU topo: Max. threads per core: 1456worker2 # [ 0.005344] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]457lb # [ 0.032408] CPU topo: Num. cores per package: 1458worker2 # [ 0.005345] Normal empty459client # [ 0.005331] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]460worker1 # [ 0.032371] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()461worker2 # [ 0.005345] Device empty462lb # [ 0.032409] CPU topo: Num. threads per package: 1463client # [ 0.005332] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]464worker2 # [ 0.005346] Movable zone start for each node465worker1 # [ 0.032400] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]466lb # [ 0.032409] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs467worker2 # [ 0.005347] Early memory node ranges468client # [ 0.005351] On node 0, zone DMA: 1 pages in unavailable ranges469worker2 # [ 0.005347] node 0: [mem 0x0000000000001000-0x000000000009efff]470worker1 # [ 0.032401] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]471client # [ 0.005607] On node 0, zone DMA: 97 pages in unavailable ranges472lb # [ 0.032434] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()473worker2 # [ 0.005348] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]474client # [ 0.031404] On node 0, zone DMA32: 40 pages in unavailable ranges475worker1 # [ 0.032403] [mem 0x60000000-0xafffffff] available for PCI devices476lb # [ 0.032465] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]477worker1 # [ 0.032404] Booting paravirtualized kernel on KVM478client # [ 0.031859] ACPI: PM-Timer IO Port: 0x608479worker2 # [ 0.005349] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]480client # [ 0.031870] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])481lb # [ 0.032467] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]482worker2 # [ 0.005368] On node 0, zone DMA: 1 pages in unavailable ranges483worker1 # [ 0.032407] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns484lb # [ 0.032469] [mem 0x60000000-0xafffffff] available for PCI devices485client # [ 0.031895] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23486worker2 # [ 0.005625] On node 0, zone DMA: 97 pages in unavailable ranges487lb # [ 0.032470] Booting paravirtualized kernel on KVM488client # [ 0.031898] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)489worker1 # [ 0.036900] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1490worker2 # [ 0.031549] On node 0, zone DMA32: 40 pages in unavailable ranges491worker2 # [ 0.032014] ACPI: PM-Timer IO Port: 0x608492client # [ 0.031899] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)493worker1 # [ 0.039013] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152494lb # [ 0.032473] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns495worker2 # [ 0.032025] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])496worker1 # [ 0.039054] kvm-guest: PV spinlocks disabled, single CPU497client # [ 0.031900] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)498lb # [ 0.036955] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1499worker2 # [ 0.032050] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23500client # [ 0.031901] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)501worker2 # [ 0.032053] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)502lb # [ 0.039070] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152503client # [ 0.031901] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)504lb # [ 0.039114] kvm-guest: PV spinlocks disabled, single CPU505worker2 # [ 0.032054] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)506client # [ 0.031904] ACPI: Using ACPI (MADT) for SMP configuration information507worker2 # [ 0.032055] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)508client # [ 0.031905] ACPI: HPET id: 0x8086a201 base: 0xfed00000509client # [ 0.031908] TSC deadline timer available510worker2 # [ 0.032056] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)511worker1 # [ 0.039056] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/l43lgd9fw6x3f87dwymd1s3rqn6a6sln-nixos-system-worker1-test/init regInfo=/nix/store/3rmyn12yf8b55cpvcp28gsadc21gxx56-closure-info/registration console=ttyS0,115200n8 console=tty0512client # [ 0.031912] CPU topo: Max. logical packages: 1513worker2 # [ 0.032057] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)514client # [ 0.031913] CPU topo: Max. logical dies: 1515client # [ 0.031913] CPU topo: Max. dies per package: 1516worker2 # [ 0.032059] ACPI: Using ACPI (MADT) for SMP configuration information517client # [ 0.031916] CPU topo: Max. threads per core: 1518worker1 # [ 0.039148] Unknown kernel command line parameters "regInfo=/nix/store/3rmyn12yf8b55cpvcp28gsadc21gxx56-closure-info/registration", will be passed to user space.519lb # [ 0.039116] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0520worker2 # [ 0.032060] ACPI: HPET id: 0x8086a201 base: 0xfed00000521worker1 # [ 0.039161] random: crng init done522client # [ 0.031917] CPU topo: Num. cores per package: 1523worker2 # [ 0.032063] TSC deadline timer available524client # [ 0.031917] CPU topo: Num. threads per package: 1525worker2 # [ 0.032067] CPU topo: Max. logical packages: 1526worker1 # [ 0.039162] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes527worker2 # [ 0.032068] CPU topo: Max. logical dies: 1528client # [ 0.031918] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs529lb # [ 0.039210] Unknown kernel command line parameters "regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration", will be passed to user space.530worker2 # [ 0.032068] CPU topo: Max. dies per package: 1531worker1 # [ 0.040925] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)532lb # [ 0.039224] random: crng init done533client # [ 0.031937] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()534worker2 # [ 0.032071] CPU topo: Max. threads per core: 1535worker1 # [ 0.040947] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)536lb # [ 0.039225] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes537worker2 # [ 0.032072] CPU topo: Num. cores per package: 1538client # [ 0.031973] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]539worker1 # [ 0.040975] Fallback order for Node 0: 0540worker2 # [ 0.032072] CPU topo: Num. threads per package: 1541lb # [ 0.041287] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)542client # [ 0.031974] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]543worker1 # [ 0.040978] Built 1 zonelists, mobility grouping on. Total pages: 393078544worker2 # [ 0.032072] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs545worker1 # [ 0.040978] Policy zone: DMA32546client # [ 0.031975] [mem 0x60000000-0xafffffff] available for PCI devices547lb # [ 0.042336] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)548worker2 # [ 0.032089] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()549client # [ 0.031977] Booting paravirtualized kernel on KVM550lb # [ 0.042366] Fallback order for Node 0: 0551worker1 # [ 0.043451] mem auto-init: stack:all(zero), heap alloc:on, heap free:off552worker2 # [ 0.032117] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]553lb # [ 0.042368] Built 1 zonelists, mobility grouping on. Total pages: 393078554worker1 # [ 0.046458] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1555client # [ 0.031980] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns556lb # [ 0.042369] Policy zone: DMA32557worker1 # [ 0.049723] allocated 3145728 bytes of page_ext558worker2 # [ 0.032118] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]559client # [ 0.036457] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1560lb # [ 0.044964] mem auto-init: stack:all(zero), heap alloc:on, heap free:off561worker1 # [ 0.059670] ftrace: allocating 48787 entries in 192 pages562worker2 # [ 0.032120] [mem 0x60000000-0xafffffff] available for PCI devices563client # [ 0.038554] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152564worker1 # [ 0.059671] ftrace: allocated 192 pages with 2 groups565worker2 # [ 0.032121] Booting paravirtualized kernel on KVM566lb # [ 0.047921] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1567worker1 # [ 0.060703] Dynamic Preempt: lazy568lb # [ 0.051122] allocated 3145728 bytes of page_ext569client # [ 0.038597] kvm-guest: PV spinlocks disabled, single CPU570worker1 # [ 0.060867] rcu: Preemptible hierarchical RCU implementation.571worker2 # [ 0.032124] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns572lb # [ 0.060876] ftrace: allocating 48787 entries in 192 pages573worker1 # [ 0.060868] rcu: RCU event tracing is enabled.574lb # [ 0.060879] ftrace: allocated 192 pages with 2 groups575worker2 # [ 0.036585] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1576lb # [ 0.061727] Dynamic Preempt: lazy577worker1 # [ 0.060869] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.578lb # [ 0.061860] rcu: Preemptible hierarchical RCU implementation.579worker1 # [ 0.060870] Trampoline variant of Tasks RCU enabled.580worker2 # [ 0.038872] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152581lb # [ 0.061860] rcu: RCU event tracing is enabled.582worker1 # [ 0.060871] Rude variant of Tasks RCU enabled.583worker2 # [ 0.038915] kvm-guest: PV spinlocks disabled, single CPU584client # [ 0.038598] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/06ljfqp30yw0k96bramnjwwmx4zj0rxb-nixos-system-client-test/init regInfo=/nix/store/qphv9d6khr6g6mdrc69p4bvgdiyanqss-closure-info/registration console=ttyS0,115200n8 console=tty0585worker1 # [ 0.060871] Tracing variant of Tasks RCU enabled.586lb # [ 0.061861] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.587lb # [ 0.061862] Trampoline variant of Tasks RCU enabled.588worker1 # [ 0.060872] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.589lb # [ 0.061863] Rude variant of Tasks RCU enabled.590client # [ 0.038688] Unknown kernel command line parameters "regInfo=/nix/store/qphv9d6khr6g6mdrc69p4bvgdiyanqss-closure-info/registration", will be passed to user space.591worker1 # [ 0.060872] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1592lb # [ 0.061863] Tracing variant of Tasks RCU enabled.593client # [ 0.038700] random: crng init done594lb # [ 0.061864] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.595client # [ 0.038701] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes596worker1 # [ 0.060881] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.597worker2 # [ 0.038917] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/21kicvrb3q0il9r6n7cv4xjicnnqfdk9-nixos-system-worker2-test/init regInfo=/nix/store/wb05yk17njbxf589bx4way1c2prk6511-closure-info/registration console=ttyS0,115200n8 console=tty0598lb # [ 0.061865] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1599client # [ 0.040773] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)600worker1 # [ 0.060883] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.601client # [ 0.041991] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)602lb # [ 0.061919] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.603worker2 # [ 0.039007] Unknown kernel command line parameters "regInfo=/nix/store/wb05yk17njbxf589bx4way1c2prk6511-closure-info/registration", will be passed to user space.604worker1 # [ 0.060883] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.605worker2 # [ 0.039020] random: crng init done606client # [ 0.042026] Fallback order for Node 0: 0607worker1 # [ 0.065232] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16608lb # [ 0.061921] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.609client # [ 0.042029] Built 1 zonelists, mobility grouping on. Total pages: 393078610worker2 # [ 0.039021] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes611client # [ 0.042030] Policy zone: DMA32612worker1 # [ 0.065519] rcu: srcu_init: Setting srcu_struct sizes based on contention.613worker2 # [ 0.041082] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)614lb # [ 0.061921] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.615client # [ 0.044454] mem auto-init: stack:all(zero), heap alloc:on, heap free:off616lb # [ 0.066317] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16617worker2 # [ 0.042118] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)618client # [ 0.047446] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1619worker1 # [ 0.065525] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns620worker2 # [ 0.042150] Fallback order for Node 0: 0621client # [ 0.050655] allocated 3145728 bytes of page_ext622lb # [ 0.066597] rcu: srcu_init: Setting srcu_struct sizes based on contention.623client # [ 0.060465] ftrace: allocating 48787 entries in 192 pages624worker1 # [ 0.065629] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)625worker2 # [ 0.042152] Built 1 zonelists, mobility grouping on. Total pages: 393078626worker2 # [ 0.042153] Policy zone: DMA32627worker1 # [ 0.069204] Console: colour VGA+ 80x25628client # [ 0.060467] ftrace: allocated 192 pages with 2 groups629lb # [ 0.066603] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns630client # [ 0.061254] Dynamic Preempt: lazy631worker1 # [ 0.069207] printk: legacy console [tty0] enabled632worker2 # [ 0.044561] mem auto-init: stack:all(zero), heap alloc:on, heap free:off633client # [ 0.061404] rcu: Preemptible hierarchical RCU implementation.634worker1 # [ 0.100269] printk: legacy console [ttyS0] enabled635lb # [ 0.066700] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)636worker2 # [ 0.047539] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1637client # [ 0.061405] rcu: RCU event tracing is enabled.638lb # [ 0.070452] Console: colour VGA+ 80x25639worker1 # [ 0.203614] ACPI: Core revision 20250807640worker2 # [ 0.050924] allocated 3145728 bytes of page_ext641client # [ 0.061405] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.642lb # [ 0.070455] printk: legacy console [tty0] enabled643worker2 # [ 0.060579] ftrace: allocating 48787 entries in 192 pages644client # [ 0.061407] Trampoline variant of Tasks RCU enabled.645lb # [ 0.101433] printk: legacy console [ttyS0] enabled646worker2 # [ 0.060581] ftrace: allocated 192 pages with 2 groups647client # [ 0.061407] Rude variant of Tasks RCU enabled.648worker1 # [ 0.204503] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns649worker2 # [ 0.061396] Dynamic Preempt: lazy650lb # [ 0.202494] ACPI: Core revision 20250807651client # [ 0.061407] Tracing variant of Tasks RCU enabled.652worker2 # [ 0.061538] rcu: Preemptible hierarchical RCU implementation.653worker1 # [ 0.206079] APIC: Switch to symmetric I/O mode setup654worker2 # [ 0.061538] rcu: RCU event tracing is enabled.655client # [ 0.061408] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.656worker1 # [ 0.207097] x2apic enabled657lb # [ 0.203410] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns658worker2 # [ 0.061539] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.659client # [ 0.061409] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1660worker2 # [ 0.061540] Trampoline variant of Tasks RCU enabled.661worker1 # [ 0.207847] APIC: Switched APIC routing to: physical x2apic662lb # [ 0.205017] APIC: Switch to symmetric I/O mode setup663client # [ 0.061424] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.664worker2 # [ 0.061541] Rude variant of Tasks RCU enabled.665lb # [ 0.206045] x2apic enabled666worker2 # [ 0.061541] Tracing variant of Tasks RCU enabled.667client # [ 0.061425] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.668worker1 # [ 0.209723] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1669worker2 # [ 0.061542] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.670lb # [ 0.206852] APIC: Switched APIC routing to: physical x2apic671client # [ 0.061426] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.672worker2 # [ 0.061543] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1673client # [ 0.065782] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16674worker1 # [ 0.210728] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns675worker2 # [ 0.061551] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.676lb # [ 0.208771] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1677client # [ 0.066062] rcu: srcu_init: Setting srcu_struct sizes based on contention.678worker2 # [ 0.061553] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.679worker1 # [ 0.212465] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)680client # [ 0.066068] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns681lb # [ 0.209805] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns682worker2 # [ 0.061554] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.683worker1 # [ 0.213552] x86/cpu: User Mode Instruction Prevention (UMIP) activated684client # [ 0.066170] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)685worker2 # [ 0.065945] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16686client # [ 0.069896] Console: colour VGA+ 80x25687lb # [ 0.211547] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)688worker1 # [ 0.215494] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127689client # [ 0.069899] printk: legacy console [tty0] enabled690worker2 # [ 0.066206] rcu: srcu_init: Setting srcu_struct sizes based on contention.691client # [ 0.099336] printk: legacy console [ttyS0] enabled692worker1 # [ 0.216427] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0693lb # [ 0.213560] x86/cpu: User Mode Instruction Prevention (UMIP) activated694worker2 # [ 0.066212] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns695client # [ 0.196356] ACPI: Core revision 20250807696lb # [ 0.214676] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127697worker2 # [ 0.066310] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)698worker1 # [ 0.217465] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto699worker2 # [ 0.069828] Console: colour VGA+ 80x25700client # [ 0.197296] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns701lb # [ 0.215544] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0702worker2 # [ 0.069831] printk: legacy console [tty0] enabled703client # [ 0.198794] APIC: Switch to symmetric I/O mode setup704worker2 # [ 0.099315] printk: legacy console [ttyS0] enabled705worker1 # [ 0.218462] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl706client # [ 0.199724] x2apic enabled707worker2 # [ 0.195589] ACPI: Core revision 20250807708lb # [ 0.216548] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto709worker1 # [ 0.220463] Transient Scheduler Attacks: Vulnerable: No microcode710client # [ 0.200460] APIC: Switched APIC routing to: physical x2apic711worker1 # [ 0.221462] Spectre V2 : Mitigation: Enhanced / Automatic IBRS712worker2 # [ 0.196432] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns713lb # [ 0.217543] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl714worker1 # [ 0.222462] Speculative Return Stack Overflow: Mitigation: Safe RET715worker2 # [ 0.197943] APIC: Switch to symmetric I/O mode setup716client # [ 0.202272] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1717lb # [ 0.218544] Transient Scheduler Attacks: Vulnerable: No microcode718worker2 # [ 0.198920] x2apic enabled719lb # [ 0.219543] Spectre V2 : Mitigation: Enhanced / Automatic IBRS720worker2 # [ 0.199707] APIC: Switched APIC routing to: physical x2apic721client # [ 0.203235] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns722lb # [ 0.220544] Speculative Return Stack Overflow: Mitigation: Safe RET723client # [ 0.204874] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)724worker2 # [ 0.201551] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1725worker1 # [ 0.223462] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization726client # [ 0.205957] x86/cpu: User Mode Instruction Prevention (UMIP) activated727worker1 # [ 0.225470] Spectre V2 : Enabling IBPB for BPF728client # [ 0.206997] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127729worker2 # [ 0.202550] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns730worker1 # [ 0.226463] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier731client # [ 0.207872] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0732lb # [ 0.221543] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization733worker2 # [ 0.204254] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)734worker1 # [ 0.227463] active return thunk: srso_alias_return_thunk735lb # [ 0.223551] Spectre V2 : Enabling IBPB for BPF736worker2 # [ 0.205337] x86/cpu: User Mode Instruction Prevention (UMIP) activated737client # [ 0.208874] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto738worker1 # [ 0.228485] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'739lb # [ 0.225104] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier740worker2 # [ 0.207148] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127741worker1 # [ 0.229462] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'742client # [ 0.209871] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl743lb # [ 0.226542] active return thunk: srso_alias_return_thunk744worker2 # [ 0.207921] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0745worker1 # [ 0.230462] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'746client # [ 0.210871] Transient Scheduler Attacks: Vulnerable: No microcode747lb # [ 0.227557] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'748client # [ 0.211872] Spectre V2 : Mitigation: Enhanced / Automatic IBRS749worker1 # [ 0.231463] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750worker2 # [ 0.208993] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto751lb # [ 0.228543] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'752client # [ 0.212871] Speculative Return Stack Overflow: Mitigation: Safe RET753worker1 # [ 0.232462] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'754worker2 # [ 0.210251] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl755lb # [ 0.229543] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'756worker1 # [ 0.233462] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'757worker2 # [ 0.211251] Transient Scheduler Attacks: Vulnerable: No microcode758lb # [ 0.230543] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'759worker2 # [ 0.212972] Spectre V2 : Mitigation: Enhanced / Automatic IBRS760worker1 # [ 0.235309] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'761lb # [ 0.231543] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'762worker2 # [ 0.213941] Speculative Return Stack Overflow: Mitigation: Safe RET763worker1 # [ 0.236462] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256764client # [ 0.213871] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization765lb # [ 0.232543] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'766worker1 # [ 0.237462] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64767client # [ 0.215878] Spectre V2 : Enabling IBPB for BPF768lb # [ 0.233543] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'769worker1 # [ 0.238462] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512770client # [ 0.216872] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier771lb # [ 0.235544] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256772worker1 # [ 0.239462] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024773client # [ 0.217872] active return thunk: srso_alias_return_thunk774worker2 # [ 0.214251] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization775lb # [ 0.236543] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64776worker1 # [ 0.240462] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8777client # [ 0.218891] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'778worker2 # [ 0.216258] Spectre V2 : Enabling IBPB for BPF779lb # [ 0.237543] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512780client # [ 0.219871] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'781worker1 # [ 0.241462] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.782worker2 # [ 0.217252] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier783lb # [ 0.238543] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024784client # [ 0.220871] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'785worker2 # [ 0.218252] active return thunk: srso_alias_return_thunk786lb # [ 0.239543] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8787client # [ 0.221871] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'788worker2 # [ 0.219896] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'789client # [ 0.222871] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'790lb # [ 0.240544] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.791worker2 # [ 0.221155] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'792client # [ 0.223871] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'793worker2 # [ 0.221998] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'794client # [ 0.224871] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'795worker2 # [ 0.222993] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'796client # [ 0.226765] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256797worker2 # [ 0.224010] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'798client # [ 0.227575] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64799worker2 # [ 0.224994] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'800client # [ 0.228580] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512801worker2 # [ 0.226043] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'802client # [ 0.229582] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024803worker2 # [ 0.227251] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256804client # [ 0.229871] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8805worker2 # [ 0.228227] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64806client # [ 0.230871] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.807worker2 # [ 0.229015] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808worker2 # [ 0.230251] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809worker2 # [ 0.231251] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810worker2 # [ 0.232251] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811worker1 # [ 0.275677] Freeing SMP alternatives memory: 44K812worker1 # [ 0.276465] pid_max: default: 32768 minimum: 301813worker1 # [ 0.277331] LSM: initializing lsm=capability,landlock,yama,bpf,ima814worker1 # [ 0.278485] landlock: Up and running.815lb # [ 0.274717] Freeing SMP alternatives memory: 44K816worker1 # [ 0.279150] Yama: becoming mindful.817lb # [ 0.275546] pid_max: default: 32768 minimum: 301818worker1 # [ 0.279675] LSM support for eBPF active819lb # [ 0.276414] LSM: initializing lsm=capability,landlock,yama,bpf,ima820worker1 # [ 0.280617] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)821lb # [ 0.277616] landlock: Up and running.822lb # [ 0.278276] Yama: becoming mindful.823worker1 # [ 0.281501] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)824lb # [ 0.278755] LSM support for eBPF active825lb # [ 0.279648] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)826worker1 # [ 0.284573] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)827lb # [ 0.280584] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)828worker1 # [ 0.286018] Performance Events: Fam17h+ core perfctr, AMD PMU driver.829client # [ 0.265730] Freeing SMP alternatives memory: 44K830worker1 # [ 0.286471] ... version: 2831client # [ 0.266422] pid_max: default: 32768 minimum: 301832lb # [ 0.283791] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)833worker1 # [ 0.287206] ... bit width: 48834client # [ 0.266955] LSM: initializing lsm=capability,landlock,yama,bpf,ima835worker1 # [ 0.287465] ... generic counters: 6836client # [ 0.267974] landlock: Up and running.837worker1 # [ 0.288207] ... generic bitmap: 000000000000003f838lb # [ 0.285345] Performance Events: Fam17h+ core perfctr, AMD PMU driver.839client # [ 0.268872] Yama: becoming mindful.840worker2 # [ 0.266057] Freeing SMP alternatives memory: 44K841worker1 # [ 0.288465] ... fixed-purpose counters: 0842lb # [ 0.286389] ... version: 2843client # [ 0.269645] LSM support for eBPF active844worker2 # [ 0.266796] pid_max: default: 32768 minimum: 301845worker1 # [ 0.289208] ... fixed-purpose bitmap: 0000000000000000846lb # [ 0.287110] ... bit width: 48847worker2 # [ 0.267339] LSM: initializing lsm=capability,landlock,yama,bpf,ima848client # [ 0.270433] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)849lb # [ 0.287597] ... generic counters: 6850worker1 # [ 0.289465] ... value mask: 0000ffffffffffff851worker2 # [ 0.268350] landlock: Up and running.852lb # [ 0.288342] ... generic bitmap: 000000000000003f853worker1 # [ 0.290375] ... max period: 00007fffffffffff854worker2 # [ 0.269251] Yama: becoming mindful.855client # [ 0.271744] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)856lb # [ 0.289219] ... fixed-purpose counters: 0857worker1 # [ 0.291141] ... global_ctrl mask: 000000000000003f858worker2 # [ 0.270042] LSM support for eBPF active859lb # [ 0.289546] ... fixed-purpose bitmap: 0000000000000000860worker1 # [ 0.291559] signal: max sigframe size: 3376861worker2 # [ 0.270809] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)862lb # [ 0.290502] ... value mask: 0000ffffffffffff863worker1 # [ 0.292428] rcu: Hierarchical SRCU implementation.864client # [ 0.274083] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)865lb # [ 0.291264] ... max period: 00007fffffffffff866worker1 # [ 0.293076] rcu: Max phase no-delay instances is 400.867worker2 # [ 0.272146] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)868client # [ 0.275436] Performance Events: Fam17h+ core perfctr, AMD PMU driver.869lb # [ 0.291546] ... global_ctrl mask: 000000000000003f870client # [ 0.275880] ... version: 2871lb # [ 0.292639] signal: max sigframe size: 3376872client # [ 0.276550] ... bit width: 48873worker2 # [ 0.274489] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)874lb # [ 0.293485] rcu: Hierarchical SRCU implementation.875client # [ 0.276873] ... generic counters: 6876lb # [ 0.294184] rcu: Max phase no-delay instances is 400.877client # [ 0.277585] ... generic bitmap: 000000000000003f878worker2 # [ 0.275817] Performance Events: Fam17h+ core perfctr, AMD PMU driver.879client # [ 0.277873] ... fixed-purpose counters: 0880worker2 # [ 0.276260] ... version: 2881worker1 # [ 0.298007] smp: Bringing up secondary CPUs ...882client # [ 0.278542] ... fixed-purpose bitmap: 0000000000000000883worker2 # [ 0.276968] ... bit width: 48884worker1 # [ 0.298474] smp: Brought up 1 node, 1 CPU885client # [ 0.278874] ... value mask: 0000ffffffffffff886worker2 # [ 0.277253] ... generic counters: 6887client # [ 0.279747] ... max period: 00007fffffffffff888worker1 # [ 0.299155] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)889worker2 # [ 0.277976] ... generic bitmap: 000000000000003f890client # [ 0.280542] ... global_ctrl mask: 000000000000003f891worker2 # [ 0.278253] ... fixed-purpose counters: 0892client # [ 0.280982] signal: max sigframe size: 3376893worker2 # [ 0.278972] ... fixed-purpose bitmap: 0000000000000000894lb # [ 0.299191] smp: Bringing up secondary CPUs ...895worker1 # [ 0.299660] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)896client # [ 0.281760] rcu: Hierarchical SRCU implementation.897worker2 # [ 0.279253] ... value mask: 0000ffffffffffff898lb # [ 0.299561] smp: Brought up 1 node, 1 CPU899worker1 # [ 0.300686] devtmpfs: initialized900client # [ 0.282418] rcu: Max phase no-delay instances is 400.901worker2 # [ 0.280106] ... max period: 00007fffffffffff902lb # [ 0.300297] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)903worker1 # [ 0.301537] x86/mm: Memory block size: 128MB904worker2 # [ 0.280897] ... global_ctrl mask: 000000000000003f905worker2 # [ 0.281348] signal: max sigframe size: 3376906worker2 # [ 0.282157] rcu: Hierarchical SRCU implementation.907lb # [ 0.300716] Memory: 1454632K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)908worker1 # [ 0.303426] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)909worker2 # [ 0.282901] rcu: Max phase no-delay instances is 400.910lb # [ 0.301816] devtmpfs: initialized911worker1 # [ 0.304381] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).912lb # [ 0.302648] x86/mm: Memory block size: 128MB913client # [ 0.287480] smp: Bringing up secondary CPUs ...914worker1 # [ 0.305557] pinctrl core: initialized pinctrl subsystem915client # [ 0.287886] smp: Brought up 1 node, 1 CPU916worker1 # [ 0.306797] PM: RTC time: 15:12:38, date: 2026-09-25917client # [ 0.288563] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)918lb # [ 0.304476] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)919lb # [ 0.305513] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).920worker2 # [ 0.287752] smp: Bringing up secondary CPUs ...921client # [ 0.289016] Memory: 1454608K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)922worker2 # [ 0.288266] smp: Brought up 1 node, 1 CPU923lb # [ 0.306623] pinctrl core: initialized pinctrl subsystem924client # [ 0.290130] devtmpfs: initialized925worker1 # [ 0.310340] NET: Registered PF_NETLINK/PF_ROUTE protocol family926lb # [ 0.307790] PM: RTC time: 15:12:38, date: 2026-09-25927worker2 # [ 0.288953] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)928client # [ 0.291324] x86/mm: Memory block size: 128MB929worker1 # [ 0.311709] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations930worker2 # [ 0.289426] Memory: 1454608K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)931worker1 # [ 0.312486] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations932worker2 # [ 0.290491] devtmpfs: initialized933client # [ 0.292946] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)934lb # [ 0.311341] NET: Registered PF_NETLINK/PF_ROUTE protocol family935worker2 # [ 0.291346] x86/mm: Memory block size: 128MB936client # [ 0.293899] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).937worker1 # [ 0.313750] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations938lb # [ 0.312815] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations939worker1 # [ 0.314475] audit: initializing netlink subsys (disabled)940client # [ 0.294975] pinctrl core: initialized pinctrl subsystem941worker2 # [ 0.293087] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)942client # [ 0.296188] PM: RTC time: 15:12:38, date: 2026-09-25943worker1 # [ 0.315636] thermal_sys: Registered thermal governor 'fair_share'944lb # [ 0.313568] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations945worker2 # [ 0.294136] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).946worker1 # [ 0.315638] thermal_sys: Registered thermal governor 'bang_bang'947lb # [ 0.314819] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations948worker2 # [ 0.295340] pinctrl core: initialized pinctrl subsystem949worker1 # [ 0.316466] thermal_sys: Registered thermal governor 'step_wise'950lb # [ 0.315557] audit: initializing netlink subsys (disabled)951worker2 # [ 0.296505] PM: RTC time: 15:12:38, date: 2026-09-25952worker1 # [ 0.317468] audit: type=2000 audit(1790349159.228:1): state=initialized audit_enabled=0 res=1953client # [ 0.299755] NET: Registered PF_NETLINK/PF_ROUTE protocol family954lb # [ 0.316792] thermal_sys: Registered thermal governor 'fair_share'955worker1 # [ 0.319468] thermal_sys: Registered thermal governor 'user_space'956lb # [ 0.316794] thermal_sys: Registered thermal governor 'bang_bang'957client # [ 0.301253] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations958worker1 # [ 0.319469] thermal_sys: Registered thermal governor 'power_allocator'959worker1 # [ 0.320477] cpuidle: using governor menu960lb # [ 0.317551] audit: type=2000 audit(1790349159.234:1): state=initialized audit_enabled=0 res=1961worker2 # [ 0.300084] NET: Registered PF_NETLINK/PF_ROUTE protocol family962client # [ 0.301902] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations963lb # [ 0.319548] thermal_sys: Registered thermal governor 'step_wise'964worker1 # [ 0.322763] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5965worker2 # [ 0.301488] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations966client # [ 0.303150] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations967lb # [ 0.319549] thermal_sys: Registered thermal governor 'user_space'968client # [ 0.303884] audit: initializing netlink subsys (disabled)969worker1 # [ 0.323696] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]970lb # [ 0.320546] thermal_sys: Registered thermal governor 'power_allocator'971worker2 # [ 0.302275] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations972lb # [ 0.321562] cpuidle: using governor menu973client # [ 0.304978] thermal_sys: Registered thermal governor 'fair_share'974worker1 # [ 0.324468] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry975worker2 # [ 0.303526] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations976client # [ 0.304980] thermal_sys: Registered thermal governor 'bang_bang'977worker1 # [ 0.325476] PCI: Using configuration type 1 for base access978lb # [ 0.323832] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5979worker2 # [ 0.304263] audit: initializing netlink subsys (disabled)980client # [ 0.305875] thermal_sys: Registered thermal governor 'step_wise'981worker2 # [ 0.305415] thermal_sys: Registered thermal governor 'fair_share'982worker1 # [ 0.326621] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.983client # [ 0.306831] thermal_sys: Registered thermal governor 'user_space'984lb # [ 0.324850] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]985worker2 # [ 0.305418] thermal_sys: Registered thermal governor 'bang_bang'986client # [ 0.307595] audit: type=2000 audit(1790349159.244:1): state=initialized audit_enabled=0 res=1987lb # [ 0.325549] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry988worker2 # [ 0.306254] thermal_sys: Registered thermal governor 'step_wise'989lb # [ 0.326558] PCI: Using configuration type 1 for base access990client # [ 0.308878] thermal_sys: Registered thermal governor 'power_allocator'991worker2 # [ 0.307239] thermal_sys: Registered thermal governor 'user_space'992client # [ 0.308907] cpuidle: using governor menu993lb # [ 0.327673] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.994worker2 # [ 0.307993] audit: type=2000 audit(1790349159.249:1): state=initialized audit_enabled=0 res=1995client # [ 0.311130] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5996worker2 # [ 0.309256] thermal_sys: Registered thermal governor 'power_allocator'997worker2 # [ 0.309270] cpuidle: using governor menu998worker1 # [ 0.332706] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages999client # [ 0.312105] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1000worker1 # [ 0.333466] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1001worker2 # [ 0.311446] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51002client # [ 0.312877] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1003client # [ 0.313886] PCI: Using configuration type 1 for base access1004worker2 # [ 0.312480] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1005lb # [ 0.332804] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1006client # [ 0.315066] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1007worker2 # [ 0.313256] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1008lb # [ 0.333547] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1009worker2 # [ 0.314265] PCI: Using configuration type 1 for base access1010worker1 # [ 0.338466] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1011worker2 # [ 0.315341] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1012worker1 # [ 0.339465] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1013lb # [ 0.338548] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1014client # [ 0.320151] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1015lb # [ 0.339546] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1016client # [ 0.320875] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1017worker2 # [ 0.320327] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1018worker2 # [ 0.321254] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1019worker1 # [ 0.347706] ACPI: Added _OSI(Module Device)1020client # [ 0.325875] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1021worker1 # [ 0.348466] ACPI: Added _OSI(Processor Device)1022client # [ 0.326874] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1023worker1 # [ 0.349198] ACPI: Added _OSI(Processor Aggregator Device)1024worker2 # [ 0.326254] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1025worker2 # [ 0.327254] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1026lb # [ 0.349971] ACPI: Added _OSI(Module Device)1027lb # [ 0.350548] ACPI: Added _OSI(Processor Device)1028lb # [ 0.351299] ACPI: Added _OSI(Processor Aggregator Device)1029worker1 # [ 0.358055] ACPI: 1 ACPI AML tables successfully acquired and loaded1030client # [ 0.337313] ACPI: Added _OSI(Module Device)1031client # [ 0.337876] ACPI: Added _OSI(Processor Device)1032client # [ 0.338665] ACPI: Added _OSI(Processor Aggregator Device)1033worker2 # [ 0.337598] ACPI: Added _OSI(Module Device)1034worker2 # [ 0.338255] ACPI: Added _OSI(Processor Device)1035worker2 # [ 0.339014] ACPI: Added _OSI(Processor Aggregator Device)1036worker1 # [ 0.361897] ACPI: Interpreter enabled1037lb # [ 0.359092] ACPI: 1 ACPI AML tables successfully acquired and loaded1038worker1 # [ 0.362480] ACPI: PM: (supports S0 S3 S4 S5)1039worker1 # [ 0.363225] ACPI: Using IOAPIC for interrupt routing1040lb # [ 0.362954] ACPI: Interpreter enabled1041lb # [ 0.363562] ACPI: PM: (supports S0 S3 S4 S5)1042worker1 # [ 0.365520] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1043lb # [ 0.364327] ACPI: Using IOAPIC for interrupt routing1044worker1 # [ 0.366466] PCI: Using E820 reservations for host bridge windows1045client # [ 0.348445] ACPI: 1 ACPI AML tables successfully acquired and loaded1046worker2 # [ 0.347275] ACPI: 1 ACPI AML tables successfully acquired and loaded1047worker1 # [ 0.369611] ACPI: Enabled 2 GPEs in block 00 to 3F1048lb # [ 0.367632] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1049lb # [ 0.368547] PCI: Using E820 reservations for host bridge windows1050client # [ 0.352285] ACPI: Interpreter enabled1051client # [ 0.352890] ACPI: PM: (supports S0 S3 S4 S5)1052worker2 # [ 0.351704] ACPI: Interpreter enabled1053client # [ 0.353599] ACPI: Using IOAPIC for interrupt routing1054lb # [ 0.371667] ACPI: Enabled 2 GPEs in block 00 to 3F1055worker2 # [ 0.352269] ACPI: PM: (supports S0 S3 S4 S5)1056worker2 # [ 0.353007] ACPI: Using IOAPIC for interrupt routing1057client # [ 0.355912] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1058worker1 # [ 0.377937] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1059worker2 # [ 0.355336] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1060client # [ 0.356875] PCI: Using E820 reservations for host bridge windows1061worker2 # [ 0.356254] PCI: Using E820 reservations for host bridge windows1062worker1 # [ 0.378471] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1063worker1 # [ 0.379544] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1064client # [ 0.359969] ACPI: Enabled 2 GPEs in block 00 to 3F1065lb # [ 0.378977] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1066worker1 # [ 0.380588] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1067worker2 # [ 0.359365] ACPI: Enabled 2 GPEs in block 00 to 3F1068lb # [ 0.379553] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1069worker1 # [ 0.381814] PCI host bridge to bus 0000:001070worker1 # [ 0.382470] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1071lb # [ 0.380625] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1072worker1 # [ 0.383466] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1073lb # [ 0.381676] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1074worker1 # [ 0.384465] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1075lb # [ 0.382926] PCI host bridge to bus 0000:001076lb # [ 0.383551] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1077worker1 # [ 0.385465] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1078client # [ 0.366611] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1079lb # [ 0.384547] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1080worker1 # [ 0.386465] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1081client # [ 0.366881] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1082worker2 # [ 0.366804] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1083lb # [ 0.385547] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1084worker1 # [ 0.387465] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1085client # [ 0.367957] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1086worker2 # [ 0.367260] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1087worker1 # [ 0.388466] pci_bus 0000:00: root bus resource [bus 00-ff]1088lb # [ 0.386546] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1089client # [ 0.368998] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1090worker2 # [ 0.368332] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1091lb # [ 0.387546] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1092worker1 # [ 0.389509] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1093client # [ 0.370266] PCI host bridge to bus 0000:001094worker2 # [ 0.369376] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1095lb # [ 0.388546] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1096client # [ 0.370879] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1097worker2 # [ 0.370604] PCI host bridge to bus 0000:001098worker1 # [ 0.390938] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1099lb # [ 0.389546] pci_bus 0000:00: root bus resource [bus 00-ff]1100client # [ 0.371874] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1101worker2 # [ 0.371258] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1102lb # [ 0.390571] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1103client # [ 0.372874] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1104worker2 # [ 0.372254] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1105client # [ 0.373875] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1106worker2 # [ 0.373253] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1107lb # [ 0.391980] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1108client # [ 0.374875] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1109worker2 # [ 0.374253] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1110client # [ 0.375874] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1111worker2 # [ 0.375254] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1112client # [ 0.376880] pci_bus 0000:00: root bus resource [bus 00-ff]1113worker2 # [ 0.376259] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1114worker1 # [ 0.393647] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1115worker2 # [ 0.377254] pci_bus 0000:00: root bus resource [bus 00-ff]1116client # [ 0.377900] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1117worker1 # [ 0.394477] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1118worker2 # [ 0.378263] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1119worker1 # [ 0.395470] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1120client # [ 0.379392] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1121worker2 # [ 0.379763] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1122lb # [ 0.394654] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1123worker1 # [ 0.396769] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1124lb # [ 0.395560] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1125lb # [ 0.396573] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1126worker1 # [ 0.398214] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1127lb # [ 0.398169] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1128worker1 # [ 0.401484] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1129lb # [ 0.399269] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1130worker1 # [ 0.402391] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1131client # [ 0.381966] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1132worker1 # [ 0.403245] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1133client # [ 0.382887] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1134worker2 # [ 0.382304] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1135worker1 # [ 0.404312] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1136client # [ 0.383873] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1137worker2 # [ 0.383266] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1138lb # [ 0.402585] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1139worker2 # [ 0.384233] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1140lb # [ 0.403517] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1141client # [ 0.385290] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1142worker1 # [ 0.405835] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1143lb # [ 0.404334] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1144worker2 # [ 0.385191] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1145client # [ 0.386656] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1146lb # [ 0.405445] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1147worker2 # [ 0.386954] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1148worker1 # [ 0.408482] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1149worker1 # [ 0.409465] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1150lb # [ 0.407180] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1151worker1 # [ 0.410488] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1152client # [ 0.389915] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1153worker2 # [ 0.389290] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1154client # [ 0.390808] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1155worker2 # [ 0.390180] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1156lb # [ 0.408585] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1157worker1 # [ 0.412022] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1158client # [ 0.391659] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1159worker2 # [ 0.390996] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1160lb # [ 0.409472] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1161client # [ 0.392703] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1162worker2 # [ 0.391259] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1163lb # [ 0.410310] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1164client # [ 0.394819] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1165lb # [ 0.411142] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1166worker2 # [ 0.392817] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1167worker1 # [ 0.414474] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1168worker1 # [ 0.416115] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1169worker1 # [ 0.416487] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1170worker2 # [ 0.395293] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1171client # [ 0.397912] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1172worker2 # [ 0.396260] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1173lb # [ 0.413589] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1174client # [ 0.398874] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1175worker1 # [ 0.418029] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1176lb # [ 0.414554] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1177worker2 # [ 0.397253] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1178client # [ 0.399856] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1179lb # [ 0.415528] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1180worker2 # [ 0.398809] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1181client # [ 0.401307] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1182worker1 # [ 0.419474] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1183worker1 # [ 0.420386] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1184lb # [ 0.417077] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1185worker1 # [ 0.421271] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1186lb # [ 0.418592] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1187worker1 # [ 0.422361] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1188client # [ 0.404856] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1189worker2 # [ 0.401527] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1190lb # [ 0.419511] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1191client # [ 0.405494] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1192worker2 # [ 0.402860] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1193worker1 # [ 0.424071] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1194client # [ 0.405896] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1195worker2 # [ 0.403276] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1196lb # [ 0.420367] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1197lb # [ 0.421430] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1198worker1 # [ 0.425479] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1199client # [ 0.407466] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1200worker2 # [ 0.404831] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1201worker1 # [ 0.426438] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1202lb # [ 0.422935] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1203worker2 # [ 0.406290] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1204client # [ 0.409914] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1205worker1 # [ 0.427983] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1206worker2 # [ 0.407175] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1207lb # [ 0.424560] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1208client # [ 0.410749] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1209worker2 # [ 0.408011] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1210lb # [ 0.425562] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1211worker1 # [ 0.429480] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1212client # [ 0.411636] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1213worker2 # [ 0.408259] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1214client # [ 0.411881] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1215lb # [ 0.427187] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1216worker1 # [ 0.430490] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1217worker2 # [ 0.409788] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1218client # [ 0.413412] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1219worker1 # [ 0.432049] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1220lb # [ 0.428896] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1221worker2 # [ 0.411298] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1222client # [ 0.414887] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1223lb # [ 0.429568] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1224worker1 # [ 0.433489] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1225worker2 # [ 0.412300] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1226client # [ 0.415810] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1227worker1 # [ 0.434489] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1228lb # [ 0.431203] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1229worker2 # [ 0.413801] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1230client # [ 0.417347] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1231worker1 # [ 0.436034] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1232lb # [ 0.432591] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1233worker2 # [ 0.415299] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1234lb # [ 0.433568] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1235client # [ 0.419587] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1236worker1 # [ 0.437493] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1237worker2 # [ 0.416275] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1238client # [ 0.419896] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1239worker1 # [ 0.438491] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1240lb # [ 0.435086] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1241worker2 # [ 0.417842] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1242client # [ 0.421498] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1243worker1 # [ 0.440015] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1244lb # [ 0.437064] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1245worker2 # [ 0.419297] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1246lb # [ 0.437570] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1247client # [ 0.422920] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1248worker1 # [ 0.441494] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1249worker2 # [ 0.420276] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1250worker1 # [ 0.442364] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1251client # [ 0.423890] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1252lb # [ 0.439130] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1253worker2 # [ 0.421785] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1254worker1 # [ 0.443247] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1255client # [ 0.425421] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1256worker1 # [ 0.444883] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1257worker2 # [ 0.423297] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1258lb # [ 0.440567] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1259lb # [ 0.441468] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1260client # [ 0.427559] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1261worker2 # [ 0.424275] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1262lb # [ 0.442393] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1263client # [ 0.427898] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1264worker1 # [ 0.447211] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1265worker2 # [ 0.425782] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1266worker1 # [ 0.447472] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1267client # [ 0.429422] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1268lb # [ 0.444606] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1269worker1 # [ 0.448488] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1270worker2 # [ 0.427306] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1271worker2 # [ 0.428226] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1272lb # [ 0.446555] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1273worker1 # [ 0.450053] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1274client # [ 0.431832] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1275lb # [ 0.447447] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1276worker2 # [ 0.429005] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1277client # [ 0.432526] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1278worker1 # [ 0.451110] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1279lb # [ 0.448298] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1280client # [ 0.432926] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1281worker2 # [ 0.429828] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1282worker1 # [ 0.451671] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1283lb # [ 0.449175] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1284client # [ 0.434955] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1285worker1 # [ 0.453123] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1286lb # [ 0.450270] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1287worker2 # [ 0.431261] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1288worker1 # [ 0.453674] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1289worker2 # [ 0.432948] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1290client # [ 0.437671] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1291lb # [ 0.450754] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1292worker1 # [ 0.455213] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1293worker2 # [ 0.433275] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1294client # [ 0.438563] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1295lb # [ 0.453270] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1296client # [ 0.438896] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1297worker2 # [ 0.434833] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1298lb # [ 0.453757] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1299client # [ 0.440468] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1300worker2 # [ 0.435945] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1301lb # [ 0.455156] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1302client # [ 0.441618] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1303worker2 # [ 0.436457] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1304worker2 # [ 0.437861] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1305client # [ 0.442875] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1306worker2 # [ 0.438448] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1307client # [ 0.444736] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1308worker2 # [ 0.439913] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1309client # [ 0.445752] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1310client # [ 0.447615] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1311worker1 # [ 0.455704] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1312worker1 # [ 0.458175] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1313worker1 # [ 0.458763] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1314lb # [ 0.455781] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1315worker1 # [ 0.459986] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1316worker1 # [ 0.460719] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1317lb # [ 0.458243] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1318lb # [ 0.458820] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1319worker1 # [ 0.462495] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1320worker1 # [ 0.463421] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1321worker2 # [ 0.440468] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1322lb # [ 0.459855] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1323worker1 # [ 0.464579] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1324worker2 # [ 0.442866] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1325lb # [ 0.460805] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1326client # [ 0.448122] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1327worker1 # [ 0.466107] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1328worker2 # [ 0.443582] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1329lb # [ 0.462564] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1330client # [ 0.450639] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1331worker2 # [ 0.444545] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1332lb # [ 0.463524] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1333client # [ 0.451903] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1334worker2 # [ 0.445638] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1335lb # [ 0.464670] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1336worker1 # [ 0.469944] ACPI: PCI: Interrupt link LNKA configured for IRQ 101337client # [ 0.453205] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1338worker2 # [ 0.447307] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1339lb # [ 0.466243] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1340worker1 # [ 0.470575] ACPI: PCI: Interrupt link LNKB configured for IRQ 101341worker2 # [ 0.448147] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1342client # [ 0.454135] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1343worker1 # [ 0.471567] ACPI: PCI: Interrupt link LNKC configured for IRQ 111344worker2 # [ 0.449317] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1345worker1 # [ 0.472565] ACPI: PCI: Interrupt link LNKD configured for IRQ 111346client # [ 0.455934] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1347worker1 # [ 0.473565] ACPI: PCI: Interrupt link LNKE configured for IRQ 101348client # [ 0.456834] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1349worker2 # [ 0.451140] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1350lb # [ 0.469849] ACPI: PCI: Interrupt link LNKA configured for IRQ 101351worker1 # [ 0.474571] ACPI: PCI: Interrupt link LNKF configured for IRQ 101352lb # [ 0.470655] ACPI: PCI: Interrupt link LNKB configured for IRQ 101353worker1 # [ 0.475568] ACPI: PCI: Interrupt link LNKG configured for IRQ 111354client # [ 0.457986] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1355lb # [ 0.471652] ACPI: PCI: Interrupt link LNKC configured for IRQ 111356worker1 # [ 0.476569] ACPI: PCI: Interrupt link LNKH configured for IRQ 111357lb # [ 0.472656] ACPI: PCI: Interrupt link LNKD configured for IRQ 111358client # [ 0.459695] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1359worker1 # [ 0.477507] ACPI: PCI: Interrupt link GSIA configured for IRQ 161360worker2 # [ 0.455561] ACPI: PCI: Interrupt link LNKA configured for IRQ 101361lb # [ 0.473811] ACPI: PCI: Interrupt link LNKE configured for IRQ 101362worker1 # [ 0.478480] ACPI: PCI: Interrupt link GSIB configured for IRQ 171363worker2 # [ 0.456364] ACPI: PCI: Interrupt link LNKB configured for IRQ 101364lb # [ 0.474645] ACPI: PCI: Interrupt link LNKF configured for IRQ 101365worker1 # [ 0.479482] ACPI: PCI: Interrupt link GSIC configured for IRQ 181366worker2 # [ 0.457353] ACPI: PCI: Interrupt link LNKC configured for IRQ 111367lb # [ 0.475650] ACPI: PCI: Interrupt link LNKG configured for IRQ 111368worker1 # [ 0.480480] ACPI: PCI: Interrupt link GSID configured for IRQ 191369client # [ 0.463645] ACPI: PCI: Interrupt link LNKA configured for IRQ 101370worker2 # [ 0.458351] ACPI: PCI: Interrupt link LNKD configured for IRQ 111371worker1 # [ 0.481477] ACPI: PCI: Interrupt link GSIE configured for IRQ 201372lb # [ 0.476673] ACPI: PCI: Interrupt link LNKH configured for IRQ 111373client # [ 0.464751] ACPI: PCI: Interrupt link LNKB configured for IRQ 101374worker2 # [ 0.459352] ACPI: PCI: Interrupt link LNKE configured for IRQ 101375worker1 # [ 0.482437] ACPI: PCI: Interrupt link GSIF configured for IRQ 211376lb # [ 0.477600] ACPI: PCI: Interrupt link GSIA configured for IRQ 161377client # [ 0.465722] ACPI: PCI: Interrupt link LNKC configured for IRQ 111378worker1 # [ 0.483212] ACPI: PCI: Interrupt link GSIG configured for IRQ 221379worker2 # [ 0.460352] ACPI: PCI: Interrupt link LNKF configured for IRQ 101380lb # [ 0.478561] ACPI: PCI: Interrupt link GSIB configured for IRQ 171381client # [ 0.466727] ACPI: PCI: Interrupt link LNKD configured for IRQ 111382worker1 # [ 0.483478] ACPI: PCI: Interrupt link GSIH configured for IRQ 231383lb # [ 0.479553] ACPI: PCI: Interrupt link GSIC configured for IRQ 181384worker2 # [ 0.461355] ACPI: PCI: Interrupt link LNKG configured for IRQ 111385worker2 # [ 0.462351] ACPI: PCI: Interrupt link LNKH configured for IRQ 111386client # [ 0.467704] ACPI: PCI: Interrupt link LNKE configured for IRQ 101387lb # [ 0.480558] ACPI: PCI: Interrupt link GSID configured for IRQ 191388worker1 # [ 0.485501] iommu: Default domain type: Translated1389lb # [ 0.481561] ACPI: PCI: Interrupt link GSIE configured for IRQ 201390client # [ 0.468933] ACPI: PCI: Interrupt link LNKF configured for IRQ 101391worker2 # [ 0.463292] ACPI: PCI: Interrupt link GSIA configured for IRQ 161392worker1 # [ 0.486322] iommu: DMA domain TLB invalidation policy: lazy mode1393worker2 # [ 0.464269] ACPI: PCI: Interrupt link GSIB configured for IRQ 171394client # [ 0.469987] ACPI: PCI: Interrupt link LNKG configured for IRQ 111395lb # [ 0.482558] ACPI: PCI: Interrupt link GSIF configured for IRQ 211396worker1 # [ 0.487497] ACPI: bus type USB registered1397worker2 # [ 0.465265] ACPI: PCI: Interrupt link GSIC configured for IRQ 181398client # [ 0.470983] ACPI: PCI: Interrupt link LNKH configured for IRQ 111399lb # [ 0.483563] ACPI: PCI: Interrupt link GSIG configured for IRQ 221400worker1 # [ 0.488262] usbcore: registered new interface driver usbfs1401worker2 # [ 0.466257] ACPI: PCI: Interrupt link GSID configured for IRQ 191402client # [ 0.471916] ACPI: PCI: Interrupt link GSIA configured for IRQ 161403lb # [ 0.484558] ACPI: PCI: Interrupt link GSIH configured for IRQ 231404worker1 # [ 0.489153] usbcore: registered new interface driver hub1405worker2 # [ 0.467230] ACPI: PCI: Interrupt link GSIE configured for IRQ 201406client # [ 0.472875] ACPI: PCI: Interrupt link GSIB configured for IRQ 171407worker1 # [ 0.489474] usbcore: registered new device driver usb1408worker2 # [ 0.467975] ACPI: PCI: Interrupt link GSIF configured for IRQ 211409client # [ 0.473856] ACPI: PCI: Interrupt link GSIC configured for IRQ 181410lb # [ 0.486693] iommu: Default domain type: Translated1411worker1 # [ 0.491220] NetLabel: Initializing1412worker2 # [ 0.468265] ACPI: PCI: Interrupt link GSIG configured for IRQ 221413client # [ 0.474665] ACPI: PCI: Interrupt link GSID configured for IRQ 191414lb # [ 0.487557] iommu: DMA domain TLB invalidation policy: lazy mode1415worker1 # [ 0.491471] NetLabel: domain hash size = 1281416worker2 # [ 0.469213] ACPI: PCI: Interrupt link GSIH configured for IRQ 231417client # [ 0.475625] ACPI: PCI: Interrupt link GSIE configured for IRQ 201418lb # [ 0.488834] ACPI: bus type USB registered1419worker1 # [ 0.492233] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1420client # [ 0.475888] ACPI: PCI: Interrupt link GSIF configured for IRQ 211421lb # [ 0.489609] usbcore: registered new interface driver usbfs1422worker1 # [ 0.493225] NetLabel: unlabeled traffic allowed by default1423worker2 # [ 0.471075] iommu: Default domain type: Translated1424lb # [ 0.490558] usbcore: registered new interface driver hub1425client # [ 0.476863] ACPI: PCI: Interrupt link GSIG configured for IRQ 221426worker1 # [ 0.493478] PCI: Using ACPI for IRQ routing1427worker2 # [ 0.471854] iommu: DMA domain TLB invalidation policy: lazy mode1428lb # [ 0.491490] usbcore: registered new device driver usb1429client # [ 0.477745] ACPI: PCI: Interrupt link GSIH configured for IRQ 231430worker2 # [ 0.472541] ACPI: bus type USB registered1431lb # [ 0.493144] NetLabel: Initializing1432worker2 # [ 0.473315] usbcore: registered new interface driver usbfs1433client # [ 0.479740] iommu: Default domain type: Translated1434lb # [ 0.493547] NetLabel: domain hash size = 1281435worker2 # [ 0.474233] usbcore: registered new interface driver hub1436client # [ 0.480502] iommu: DMA domain TLB invalidation policy: lazy mode1437worker2 # [ 0.474998] usbcore: registered new device driver usb1438lb # [ 0.494411] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1439client # [ 0.481271] ACPI: bus type USB registered1440lb # [ 0.495382] NetLabel: unlabeled traffic allowed by default1441worker2 # [ 0.476085] NetLabel: Initializing1442client # [ 0.481933] usbcore: registered new interface driver usbfs1443lb # [ 0.496367] PCI: Using ACPI for IRQ routing1444worker2 # [ 0.476681] NetLabel: domain hash size = 1281445client # [ 0.482861] usbcore: registered new interface driver hub1446worker2 # [ 0.477253] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1447client # [ 0.483569] usbcore: registered new device driver usb1448worker2 # [ 0.478234] NetLabel: unlabeled traffic allowed by default1449worker2 # [ 0.479020] PCI: Using ACPI for IRQ routing1450client # [ 0.484738] NetLabel: Initializing1451client # [ 0.485536] NetLabel: domain hash size = 1281452client # [ 0.485874] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453client # [ 0.486918] NetLabel: unlabeled traffic allowed by default1454client # [ 0.487888] PCI: Using ACPI for IRQ routing1455worker1 # [ 0.542903] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456worker1 # [ 0.543461] pci 0000:00:01.0: vgaarb: bridge control possible1457worker1 # [ 0.543461] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458worker1 # [ 0.543470] vgaarb: loaded1459worker1 # [ 0.544262] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460worker2 # [ 0.522884] pci 0000:00:01.0: vgaarb: setting as boot VGA device1461worker1 # [ 0.545114] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1462worker2 # [ 0.523250] pci 0000:00:01.0: vgaarb: bridge control possible1463worker2 # [ 0.523250] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1464worker2 # [ 0.523258] vgaarb: loaded1465lb # [ 0.546192] pci 0000:00:01.0: vgaarb: setting as boot VGA device1466worker2 # [ 0.524034] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01467lb # [ 0.546542] pci 0000:00:01.0: vgaarb: bridge control possible1468worker2 # [ 0.524255] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1469client # [ 0.532952] pci 0000:00:01.0: vgaarb: setting as boot VGA device1470lb # [ 0.546542] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1471worker1 # [ 0.549550] clocksource: Switched to clocksource kvm-clock1472client # [ 0.533870] pci 0000:00:01.0: vgaarb: bridge control possible1473lb # [ 0.546551] vgaarb: loaded1474lb # [ 0.547322] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01475client # [ 0.533870] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1476client # [ 0.533878] vgaarb: loaded1477lb # [ 0.547546] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1478worker2 # [ 0.527353] clocksource: Switched to clocksource kvm-clock1479client # [ 0.534651] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01480worker1 # [ 0.553045] VFS: Disk quotas dquot_6.6.01481client # [ 0.534877] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1482worker1 # [ 0.553754] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1483worker2 # [ 0.529006] VFS: Disk quotas dquot_6.6.01484worker1 # [ 0.555216] pnp: PnP ACPI init1485worker1 # [ 0.556069] ACPI: IRQ 4 override to edge(!), high(!)1486lb # [ 0.551631] clocksource: Switched to clocksource kvm-clock1487worker2 # [ 0.531493] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1488worker1 # [ 0.557081] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1489client # [ 0.538960] clocksource: Switched to clocksource kvm-clock1490worker2 # [ 0.532874] pnp: PnP ACPI init1491worker1 # [ 0.558689] pnp: PnP ACPI: found 5 devices1492worker2 # [ 0.533681] ACPI: IRQ 4 override to edge(!), high(!)1493client # [ 0.540676] VFS: Disk quotas dquot_6.6.01494lb # [ 0.555342] VFS: Disk quotas dquot_6.6.01495worker2 # [ 0.534680] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1496lb # [ 0.556106] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1497worker2 # [ 0.536255] pnp: PnP ACPI: found 5 devices1498lb # [ 0.557511] pnp: PnP ACPI init1499client # [ 0.543207] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1500lb # [ 0.558316] ACPI: IRQ 4 override to edge(!), high(!)1501client # [ 0.544719] pnp: PnP ACPI init1502lb # [ 0.559314] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1503client # [ 0.545552] ACPI: IRQ 4 override to edge(!), high(!)1504lb # [ 0.560885] pnp: PnP ACPI: found 5 devices1505client # [ 0.546551] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1506worker1 # [ 0.566344] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1507client # [ 0.548222] pnp: PnP ACPI: found 5 devices1508worker1 # [ 0.567914] clocksource: Switched to clocksource acpi_pm1509worker1 # [ 0.568932] NET: Registered PF_INET protocol family1510worker2 # [ 0.544195] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1511worker1 # [ 0.570176] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1512worker2 # [ 0.545719] clocksource: Switched to clocksource acpi_pm1513worker2 # [ 0.546834] NET: Registered PF_INET protocol family1514lb # [ 0.568614] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1515worker2 # [ 0.548180] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1516lb # [ 0.570139] clocksource: Switched to clocksource acpi_pm1517client # [ 0.555824] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1518lb # [ 0.571166] NET: Registered PF_INET protocol family1519client # [ 0.557305] clocksource: Switched to clocksource acpi_pm1520client # [ 0.558335] NET: Registered PF_INET protocol family1521lb # [ 0.572396] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1522client # [ 0.559498] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1523worker1 # [ 0.586671] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1524worker1 # [ 0.588323] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1525worker1 # [ 0.589694] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1526worker2 # [ 0.565509] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1527worker1 # [ 0.591141] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1528worker1 # [ 0.592449] TCP: Hash tables configured (established 16384 bind 16384)1529worker2 # [ 0.567121] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1530worker1 # [ 0.593603] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1531lb # [ 0.589088] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1532worker2 # [ 0.568493] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1533worker1 # [ 0.594889] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1534client # [ 0.575991] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1535lb # [ 0.590601] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1536worker2 # [ 0.569818] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1537worker1 # [ 0.596005] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1538worker2 # [ 0.571021] TCP: Hash tables configured (established 16384 bind 16384)1539client # [ 0.577479] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1540lb # [ 0.591966] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1541worker1 # [ 0.597233] NET: Registered PF_UNIX/PF_LOCAL protocol family1542worker2 # [ 0.572145] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1543client # [ 0.578739] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1544worker1 # [ 0.598217] NET: Registered PF_XDP protocol family1545lb # [ 0.593339] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1546worker2 # [ 0.573388] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1547worker1 # [ 0.599082] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1548lb # [ 0.594624] TCP: Hash tables configured (established 16384 bind 16384)1549client # [ 0.580059] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1550worker2 # [ 0.574462] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1551worker1 # [ 0.600094] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1552client # [ 0.581264] TCP: Hash tables configured (established 16384 bind 16384)1553lb # [ 0.595752] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1554worker2 # [ 0.575652] NET: Registered PF_UNIX/PF_LOCAL protocol family1555worker1 # [ 0.601158] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1556client # [ 0.582372] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1557lb # [ 0.597140] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1558worker2 # [ 0.576650] NET: Registered PF_XDP protocol family1559worker1 # [ 0.602297] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1560client # [ 0.583601] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1561worker2 # [ 0.577513] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1562lb # [ 0.598335] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1563worker1 # [ 0.603452] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1564client # [ 0.584659] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1565worker2 # [ 0.578490] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1566lb # [ 0.599571] NET: Registered PF_UNIX/PF_LOCAL protocol family1567worker1 # [ 0.604554] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1568client # [ 0.585835] NET: Registered PF_UNIX/PF_LOCAL protocol family1569lb # [ 0.600631] NET: Registered PF_XDP protocol family1570worker2 # [ 0.579479] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1571client # [ 0.586759] NET: Registered PF_XDP protocol family1572lb # [ 0.601485] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1573worker2 # [ 0.580555] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1574worker1 # [ 0.606388] ACPI: \_SB_.GSIA: Enabled at IRQ 161575client # [ 0.587662] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1576lb # [ 0.602477] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1577worker2 # [ 0.581617] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1578client # [ 0.588626] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1579lb # [ 0.603511] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1580worker2 # [ 0.582698] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1581worker1 # [ 0.608762] ACPI: \_SB_.GSIB: Enabled at IRQ 171582client # [ 0.589594] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1583lb # [ 0.604620] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1584client # [ 0.590645] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1585worker2 # [ 0.584558] ACPI: \_SB_.GSIA: Enabled at IRQ 161586lb # [ 0.605790] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1587client # [ 0.591728] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1588worker1 # [ 0.611363] ACPI: \_SB_.GSIC: Enabled at IRQ 181589lb # [ 0.606915] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1590client # [ 0.592852] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1591lb # [ 0.608663] ACPI: \_SB_.GSIA: Enabled at IRQ 161592worker2 # [ 0.587659] ACPI: \_SB_.GSIB: Enabled at IRQ 171593worker1 # [ 0.613420] ACPI: \_SB_.GSID: Enabled at IRQ 191594client # [ 0.594571] ACPI: \_SB_.GSIA: Enabled at IRQ 161595worker1 # [ 0.615143] PCI: CLS 0 bytes, default 641596worker2 # [ 0.590079] ACPI: \_SB_.GSIC: Enabled at IRQ 181597lb # [ 0.611085] ACPI: \_SB_.GSIB: Enabled at IRQ 171598client # [ 0.597106] ACPI: \_SB_.GSIB: Enabled at IRQ 171599worker1 # [ 0.616115] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1600worker2 # [ 0.592199] ACPI: \_SB_.GSID: Enabled at IRQ 191601client # [ 0.599002] ACPI: \_SB_.GSIC: Enabled at IRQ 181602worker1 # [ 0.617882] Trying to unpack rootfs image as initramfs...1603lb # [ 0.613626] ACPI: \_SB_.GSIC: Enabled at IRQ 181604worker2 # [ 0.594196] PCI: CLS 0 bytes, default 641605client # [ 0.600877] ACPI: \_SB_.GSID: Enabled at IRQ 191606lb # [ 0.615727] ACPI: \_SB_.GSID: Enabled at IRQ 191607client # [ 0.602532] PCI: CLS 0 bytes, default 641608worker2 # [ 0.595250] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1609lb # [ 0.617524] PCI: CLS 0 bytes, default 641610worker2 # [ 0.597017] Trying to unpack rootfs image as initramfs...1611client # [ 0.603432] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1612lb # [ 0.618608] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1613client # [ 0.605142] Trying to unpack rootfs image as initramfs...1614lb # [ 0.620463] Trying to unpack rootfs image as initramfs...1615worker1 # [ 0.661841] Initialise system trusted keyrings1616worker1 # [ 0.663003] workingset: timestamp_bits=40 max_order=19 bucket_order=01617worker2 # [ 0.641350] Initialise system trusted keyrings1618lb # [ 0.663744] Initialise system trusted keyrings1619client # [ 0.649558] Initialise system trusted keyrings1620worker2 # [ 0.644246] workingset: timestamp_bits=40 max_order=19 bucket_order=01621lb # [ 0.667008] workingset: timestamp_bits=40 max_order=19 bucket_order=01622client # [ 0.652879] workingset: timestamp_bits=40 max_order=19 bucket_order=01623worker1 # [ 0.683913] Key type asymmetric registered1624worker1 # [ 0.686973] Asymmetric key parser 'x509' registered1625worker1 # [ 0.687863] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1626worker1 # [ 0.691104] io scheduler mq-deadline registered1627worker1 # [ 0.691901] io scheduler kyber registered1628client # [ 0.673764] Key type asymmetric registered1629lb # [ 0.688256] Key type asymmetric registered1630worker2 # [ 0.668171] Key type asymmetric registered1631worker2 # [ 0.669166] Asymmetric key parser 'x509' registered1632client # [ 0.675838] Asymmetric key parser 'x509' registered1633lb # [ 0.691019] Asymmetric key parser 'x509' registered1634worker2 # [ 0.670029] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1635client # [ 0.676697] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1636worker1 # [ 0.695979] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1637lb # [ 0.692042] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1638worker1 # [ 0.697309] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1639worker2 # [ 0.673239] io scheduler mq-deadline registered1640client # [ 0.679969] io scheduler mq-deadline registered1641client # [ 0.680782] io scheduler kyber registered1642lb # [ 0.696168] io scheduler mq-deadline registered1643worker2 # [ 0.676072] io scheduler kyber registered1644lb # [ 0.697059] io scheduler kyber registered1645client # [ 0.683435] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1646worker1 # [ 0.702730] Linux agpgart interface v0.1031647worker2 # [ 0.677477] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1648worker1 # [ 0.703545] ACPI: bus type drm_connector registered1649client # [ 0.684726] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1650worker2 # [ 0.678795] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1651lb # [ 0.701673] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1652worker1 # [ 0.706408] usbcore: registered new interface driver usbserial_generic1653worker1 # [ 0.707744] usbserial: USB Serial support registered for generic1654lb # [ 0.703249] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1655client # [ 0.690437] Linux agpgart interface v0.1031656client # [ 0.691262] ACPI: bus type drm_connector registered1657worker2 # [ 0.684962] Linux agpgart interface v0.1031658worker2 # [ 0.685711] ACPI: bus type drm_connector registered1659worker1 # [ 0.710991] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1660worker1 # [ 0.710991] Please enable it if your BIOS has the CPPC option.1661client # [ 0.693979] usbcore: registered new interface driver usbserial_generic1662lb # [ 0.708881] Linux agpgart interface v0.1031663worker1 # [ 0.713323] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1664client # [ 0.695057] usbserial: USB Serial support registered for generic1665lb # [ 0.709707] ACPI: bus type drm_connector registered1666worker2 # [ 0.690628] usbcore: registered new interface driver usbserial_generic1667client # [ 0.697841] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1668worker2 # [ 0.691838] usbserial: USB Serial support registered for generic1669client # [ 0.697841] Please enable it if your BIOS has the CPPC option.1670worker1 # [ 0.718171] drop_monitor: Initializing network drop monitor service1671lb # [ 0.714134] usbcore: registered new interface driver usbserial_generic1672client # [ 0.700058] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1673worker1 # [ 0.719413] NET: Registered PF_INET6 protocol family1674lb # [ 0.715316] usbserial: USB Serial support registered for generic1675worker2 # [ 0.695072] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1676worker2 # [ 0.695072] Please enable it if your BIOS has the CPPC option.1677client # [ 0.702947] drop_monitor: Initializing network drop monitor service1678client # [ 0.704122] NET: Registered PF_INET6 protocol family1679lb # [ 0.717982] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1680worker2 # [ 0.697423] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1681worker1 # [ 0.723552] Segment Routing with IPv61682lb # [ 0.717982] Please enable it if your BIOS has the CPPC option.1683worker1 # [ 0.724266] In-situ OAM (IOAM) with IPv61684lb # [ 0.720309] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1685worker2 # [ 0.700228] drop_monitor: Initializing network drop monitor service1686worker1 # [ 0.726278] IPI shorthand broadcast: enabled1687worker2 # [ 0.701490] NET: Registered PF_INET6 protocol family1688client # [ 0.709255] Segment Routing with IPv61689lb # [ 0.724154] drop_monitor: Initializing network drop monitor service1690lb # [ 0.725441] NET: Registered PF_INET6 protocol family1691client # [ 0.711842] In-situ OAM (IOAM) with IPv61692worker2 # [ 0.706187] Segment Routing with IPv61693client # [ 0.712924] IPI shorthand broadcast: enabled1694worker2 # [ 0.706851] In-situ OAM (IOAM) with IPv61695lb # [ 0.730565] Segment Routing with IPv61696worker1 # [ 0.734644] sched_clock: Marking stable (592016738, 141936515)->(807998680, -74045427)1697worker2 # [ 0.710369] IPI shorthand broadcast: enabled1698worker1 # [ 0.738317] registered taskstats version 11699lb # [ 0.734003] In-situ OAM (IOAM) with IPv61700worker1 # [ 0.739327] Loading compiled-in X.509 certificates1701lb # [ 0.735220] IPI shorthand broadcast: enabled1702client # [ 0.721237] sched_clock: Marking stable (586014765, 134802456)->(793448052, -72630831)1703worker2 # [ 0.716909] sched_clock: Marking stable (582017700, 134037860)->(791047755, -74992195)1704client # [ 0.725921] registered taskstats version 11705worker2 # [ 0.720278] registered taskstats version 11706client # [ 0.726871] Loading compiled-in X.509 certificates1707worker2 # [ 0.721284] Loading compiled-in X.509 certificates1708lb # [ 0.743718] sched_clock: Marking stable (603014231, 139939856)->(818510598, -75556511)1709lb # [ 0.747221] registered taskstats version 11710lb # [ 0.748305] Loading compiled-in X.509 certificates1711worker1 # [ 0.758277] Demotion targets for Node 0: null1712worker1 # [ 0.761008] Key type .fscrypt registered1713worker1 # [ 0.761688] Key type fscrypt-provisioning registered1714worker1 # [ 0.762651] ima: No TPM chip found, activating TPM-bypass!1715client # [ 0.744830] Demotion targets for Node 0: null1716client # [ 0.745691] Key type .fscrypt registered1717worker2 # [ 0.740081] Demotion targets for Node 0: null1718worker1 # [ 0.766966] ima: Allocated hash algorithm: sha11719worker1 # [ 0.767741] ima: No architecture policies found1720client # [ 0.748822] Key type fscrypt-provisioning registered1721worker2 # [ 0.743110] Key type .fscrypt registered1722client # [ 0.749757] ima: No TPM chip found, activating TPM-bypass!1723worker2 # [ 0.743776] Key type fscrypt-provisioning registered1724client # [ 0.750687] ima: Allocated hash algorithm: sha11725worker1 # [ 0.769964] PM: Magic number: 10:656:2351726worker2 # [ 0.744739] ima: No TPM chip found, activating TPM-bypass!1727client # [ 0.751497] ima: No architecture policies found1728worker1 # [ 0.771560] RAS: Correctable Errors collector initialized.1729worker2 # [ 0.747074] ima: Allocated hash algorithm: sha11730client # [ 0.753828] PM: Magic number: 10:656:2351731lb # [ 0.768184] Demotion targets for Node 0: null1732worker2 # [ 0.747860] ima: No architecture policies found1733client # [ 0.755373] RAS: Correctable Errors collector initialized.1734lb # [ 0.771022] Key type .fscrypt registered1735lb # [ 0.771763] Key type fscrypt-provisioning registered1736worker2 # [ 0.752064] PM: Magic number: 10:656:2351737lb # [ 0.772802] ima: No TPM chip found, activating TPM-bypass!1738worker2 # [ 0.753785] RAS: Correctable Errors collector initialized.1739worker1 # [ 0.779599] clk: Disabling unused clocks1740lb # [ 0.774974] ima: Allocated hash algorithm: sha11741lb # [ 0.775839] ima: No architecture policies found1742worker1 # [ 0.782970] PM: genpd: Disabling unused power domains1743client # [ 0.764490] clk: Disabling unused clocks1744lb # [ 0.779971] PM: Magic number: 10:656:2351745lb # [ 0.781811] RAS: Correctable Errors collector initialized.1746client # [ 0.768850] PM: genpd: Disabling unused power domains1747worker2 # [ 0.762774] clk: Disabling unused clocks1748worker2 # [ 0.765073] PM: genpd: Disabling unused power domains1749lb # [ 0.790660] clk: Disabling unused clocks1750lb # [ 0.793980] PM: genpd: Disabling unused power domains1751client # [ 0.929850] Freeing initrd memory: 29056K1752client # [ 0.933001] Freeing unused decrypted memory: 2028K1753client # [ 0.935581] Freeing unused kernel image (initmem) memory: 3652K1754client # [ 0.936687] Write protecting the kernel read-only data: 32768k1755client # [ 0.938564] Freeing unused kernel image (text/rodata gap) memory: 1184K1756client # [ 0.940080] Freeing unused kernel image (rodata/data gap) memory: 720K1757worker1 # [ 0.961161] Freeing initrd memory: 29056K1758worker1 # [ 0.964315] Freeing unused decrypted memory: 2028K1759worker2 # [ 0.941172] Freeing initrd memory: 29060K1760worker1 # [ 0.966974] Freeing unused kernel image (initmem) memory: 3652K1761worker1 # [ 0.968075] Write protecting the kernel read-only data: 32768k1762worker2 # [ 0.944636] Freeing unused decrypted memory: 2028K1763worker1 # [ 0.969953] Freeing unused kernel image (text/rodata gap) memory: 1184K1764worker1 # [ 0.971436] Freeing unused kernel image (rodata/data gap) memory: 720K1765worker2 # [ 0.947569] Freeing unused kernel image (initmem) memory: 3652K1766worker2 # [ 0.948708] Write protecting the kernel read-only data: 32768k1767worker2 # [ 0.950718] Freeing unused kernel image (text/rodata gap) memory: 1184K1768worker2 # [ 0.952333] Freeing unused kernel image (rodata/data gap) memory: 720K1769lb # [ 0.981996] Freeing initrd memory: 29064K1770lb # [ 0.985474] Freeing unused decrypted memory: 2028K1771lb # [ 0.988397] Freeing unused kernel image (initmem) memory: 3652K1772lb # [ 0.989490] Write protecting the kernel read-only data: 32768k1773lb # [ 0.991444] Freeing unused kernel image (text/rodata gap) memory: 1184K1774lb # [ 0.993081] Freeing unused kernel image (rodata/data gap) memory: 720K1775client # [ 0.990998] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776client # [ 0.992091] Run /init as init process1777client # [ 1.002903] systemd[1]: Inserted module 'autofs4'1778worker1 # [ 1.022343] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779worker1 # [ 1.023426] Run /init as init process1780worker2 # [ 1.003372] x86/mm: Checked W+X mappings: passed, no W+X pages found.1781worker2 # [ 1.004508] Run /init as init process1782worker1 # [ 1.034323] systemd[1]: Inserted module 'autofs4'1783client # [ 1.022556] fuse: init (API version 7.45)1784worker2 # [ 1.016500] systemd[1]: Inserted module 'autofs4'1785lb # [ 1.043911] x86/mm: Checked W+X mappings: passed, no W+X pages found.1786client # [ 1.029910] ACPI: \_SB_.GSIG: Enabled at IRQ 221787lb # [ 1.045118] Run /init as init process1788client # [ 1.032321] ACPI: \_SB_.GSIH: Enabled at IRQ 231789worker1 # [ 1.053973] fuse: init (API version 7.45)1790client # [ 1.035332] ACPI: \_SB_.GSIE: Enabled at IRQ 201791client # [ 1.037273] ACPI: \_SB_.GSIF: Enabled at IRQ 211792worker1 # [ 1.061413] ACPI: \_SB_.GSIG: Enabled at IRQ 221793client # [ 1.042354] virtiofs virtio5: discovered new tag: nix-store1794lb # [ 1.057226] systemd[1]: Inserted module 'autofs4'1795worker2 # [ 1.036737] fuse: init (API version 7.45)1796client # [ 1.043902] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1797worker1 # [ 1.063873] ACPI: \_SB_.GSIH: Enabled at IRQ 231798worker1 # [ 1.066701] ACPI: \_SB_.GSIE: Enabled at IRQ 201799worker1 # [ 1.068679] ACPI: \_SB_.GSIF: Enabled at IRQ 211800worker2 # [ 1.044389] ACPI: \_SB_.GSIG: Enabled at IRQ 221801client # [ 1.051234] virtiofs virtio6: discovered new tag: shared1802client # [ 1.052977] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1803worker2 # [ 1.047152] ACPI: \_SB_.GSIH: Enabled at IRQ 231804worker1 # [ 1.073771] virtiofs virtio5: discovered new tag: nix-store1805client # [ 1.056315] virtiofs virtio7: discovered new tag: xchg1806worker1 # [ 1.075342] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1807worker2 # [ 1.050869] ACPI: \_SB_.GSIE: Enabled at IRQ 201808client # [ 1.057846] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1809worker2 # [ 1.053149] ACPI: \_SB_.GSIF: Enabled at IRQ 211810lb # [ 1.077623] fuse: init (API version 7.45)1811worker1 # [ 1.082152] virtiofs virtio6: discovered new tag: shared1812worker1 # [ 1.083632] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1813worker2 # [ 1.059066] virtiofs virtio5: discovered new tag: nix-store1814worker2 # [ 1.060686] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1815worker1 # [ 1.086930] virtiofs virtio7: discovered new tag: xchg1816worker1 # [ 1.088442] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1817lb # [ 1.085305] ACPI: \_SB_.GSIG: Enabled at IRQ 221818lb # [ 1.088220] ACPI: \_SB_.GSIH: Enabled at IRQ 231819worker2 # [ 1.067843] virtiofs virtio6: discovered new tag: shared1820worker2 # [ 1.069646] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1821lb # [ 1.091878] ACPI: \_SB_.GSIE: Enabled at IRQ 201822client # [ 1.077573] systemd[1]: Successfully made /usr/ read-only.1823lb # [ 1.094264] ACPI: \_SB_.GSIF: Enabled at IRQ 211824worker2 # [ 1.074193] virtiofs virtio7: discovered new tag: xchg1825worker2 # [ 1.076120] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1826lb # [ 1.100274] virtiofs virtio5: discovered new tag: nix-store1827lb # [ 1.101911] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1828worker1 # [ 1.111119] systemd[1]: Successfully made /usr/ read-only.1829lb # [ 1.110065] virtiofs virtio6: discovered new tag: shared1830lb # [ 1.111720] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1831lb # [ 1.115025] virtiofs virtio7: discovered new tag: xchg1832lb # [ 1.116542] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833worker2 # [ 1.099084] systemd[1]: Successfully made /usr/ read-only.1834lb # [ 1.136828] systemd[1]: Successfully made /usr/ read-only.1835client # [ 1.414048] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836client # [ 1.425909] systemd[1]: Detected virtualization kvm.1837client # [ 1.427999] systemd[1]: Detected architecture x86-64.1838client # [ 1.430082] systemd[1]: Running in initrd.1839client # [ 1.432436] systemd[1]: Initializing machine ID from random generator.1840client # [ 1.435267] systemd[1]: Hostname set to <client>.1841worker1 # [ 1.448039] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842worker1 # [ 1.460153] systemd[1]: Detected virtualization kvm.1843worker1 # [ 1.462322] systemd[1]: Detected architecture x86-64.1844worker2 # [ 1.434729] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1845worker1 # [ 1.464466] systemd[1]: Running in initrd.1846worker2 # [ 1.440282] systemd[1]: Detected virtualization kvm.1847worker2 # [ 1.441182] systemd[1]: Detected architecture x86-64.1848worker2 # [ 1.442137] systemd[1]: Running in initrd.1849worker1 # [ 1.467076] systemd[1]: Initializing machine ID from random generator.1850worker2 # [ 1.443293] systemd[1]: Initializing machine ID from random generator.1851worker2 # [ 1.444522] systemd[1]: Hostname set to <worker2>.1852worker1 # [ 1.469920] systemd[1]: Hostname set to <worker1>.1853lb # [ 1.472314] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854lb # [ 1.477208] systemd[1]: Detected virtualization kvm.1855lb # [ 1.478090] systemd[1]: Detected architecture x86-64.1856lb # [ 1.478967] systemd[1]: Running in initrd.1857lb # [ 1.480041] systemd[1]: Initializing machine ID from random generator.1858lb # [ 1.481252] systemd[1]: Hostname set to <lb>.1859worker2 # [ 1.649618] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860lb # [ 1.673328] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861client # [ 1.658136] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862worker1 # [ 1.696549] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863lb # [ 1.711224] systemd[1]: Queued start job for default target Initrd Default Target.1864lb # [ 1.715216] systemd[1]: Created slice Slice /system/modprobe.1865lb # [ 1.716377] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866lb # [ 1.717817] systemd[1]: Expecting device /dev/disk/by-label/nixos...1867lb # [ 1.718904] systemd[1]: Reached target Path Units.1868lb # [ 1.719770] systemd[1]: Reached target Slice Units.1869client # [ 1.705455] systemd[1]: Queued start job for default target Initrd Default Target.1870lb # [ 1.720639] systemd[1]: Reached target Swaps.1871lb # [ 1.721402] systemd[1]: Reached target Timer Units.1872lb # [ 1.722330] systemd[1]: Listening on D-Bus System Message Bus Socket.1873lb # [ 1.723470] systemd[1]: Listening on Journal Socket (/dev/log).1874lb # [ 1.724569] systemd[1]: Listening on Journal Sockets.1875client # [ 1.710115] systemd[1]: Created slice Slice /system/modprobe.1876worker2 # [ 1.703848] systemd[1]: Queued start job for default target Initrd Default Target.1877lb # [ 1.725526] systemd[1]: Listening on udev Control Socket.1878client # [ 1.711271] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1879lb # [ 1.726497] systemd[1]: Listening on udev Kernel Socket.1880client # [ 1.712662] systemd[1]: Expecting device /dev/disk/by-label/nixos...1881lb # [ 1.727419] systemd[1]: Reached target Socket Units.1882client # [ 1.713847] systemd[1]: Reached target Path Units.1883client # [ 1.714759] systemd[1]: Reached target Slice Units.1884lb # [ 1.729230] systemd[1]: Starting Create List of Static Device Nodes...1885worker2 # [ 1.708449] systemd[1]: Created slice Slice /system/modprobe.1886client # [ 1.715636] systemd[1]: Reached target Swaps.1887client # [ 1.716471] systemd[1]: Reached target Timer Units.1888worker2 # [ 1.709594] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1889client # [ 1.717451] systemd[1]: Listening on D-Bus System Message Bus Socket.1890worker2 # [ 1.711054] systemd[1]: Expecting device /dev/disk/by-label/nixos...1891lb # [ 1.732817] systemd[1]: Starting Load Kernel Module configfs...1892worker2 # [ 1.712145] systemd[1]: Reached target Path Units.1893client # [ 1.718752] systemd[1]: Listening on Journal Socket (/dev/log).1894worker2 # [ 1.713010] systemd[1]: Reached target Slice Units.1895client # [ 1.719953] systemd[1]: Listening on Journal Sockets.1896worker2 # [ 1.713888] systemd[1]: Reached target Swaps.1897client # [ 1.720952] systemd[1]: Listening on udev Control Socket.1898worker2 # [ 1.714689] systemd[1]: Reached target Timer Units.1899client # [ 1.721981] systemd[1]: Listening on udev Kernel Socket.1900worker2 # [ 1.715692] systemd[1]: Listening on D-Bus System Message Bus Socket.1901client # [ 1.722946] systemd[1]: Reached target Socket Units.1902worker2 # [ 1.716909] systemd[1]: Listening on Journal Socket (/dev/log).1903worker2 # [ 1.718122] systemd[1]: Listening on Journal Sockets.1904client # [ 1.724704] systemd[1]: Starting Create List of Static Device Nodes...1905worker2 # [ 1.719146] systemd[1]: Listening on udev Control Socket.1906worker2 # [ 1.720202] systemd[1]: Listening on udev Kernel Socket.1907worker2 # [ 1.721206] systemd[1]: Reached target Socket Units.1908client # [ 1.728620] systemd[1]: Starting Load Kernel Module configfs...1909worker2 # [ 1.723121] systemd[1]: Starting Create List of Static Device Nodes...1910worker1 # [ 1.751631] systemd[1]: Queued start job for default target Initrd Default Target.1911worker2 # [ 1.726935] systemd[1]: Starting Load Kernel Module configfs...1912worker1 # [ 1.756357] systemd[1]: Created slice Slice /system/modprobe.1913worker1 # [ 1.757635] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1914lb # [ 1.742376] systemd[1]: Starting Journal Service...1915worker1 # [ 1.759131] systemd[1]: Expecting device /dev/disk/by-label/nixos...1916worker1 # [ 1.760265] systemd[1]: Reached target Path Units.1917worker1 # [ 1.761201] systemd[1]: Reached target Slice Units.1918worker1 # [ 1.762109] systemd[1]: Reached target Swaps.1919worker1 # [ 1.762893] systemd[1]: Reached target Timer Units.1920worker1 # [ 1.763876] systemd[1]: Listening on D-Bus System Message Bus Socket.1921worker1 # [ 1.765109] systemd[1]: Listening on Journal Socket (/dev/log).1922client # [ 1.735467] systemd[1]: Starting Journal Service...1923worker1 # [ 1.766292] systemd[1]: Listening on Journal Sockets.1924worker1 # [ 1.767306] systemd[1]: Listening on udev Control Socket.1925worker1 # [ 1.768338] systemd[1]: Listening on udev Kernel Socket.1926worker1 # [ 1.769325] systemd[1]: Reached target Socket Units.1927lb # [ 1.765143] systemd[1]: Starting Load Kernel Modules...1928worker1 # [ 1.771245] systemd[1]: Starting Create List of Static Device Nodes...1929worker2 # [ 1.736128] systemd[1]: Starting Journal Service...1930worker1 # [ 1.775022] systemd[1]: Starting Load Kernel Module configfs...1931lb # [ 1.771071] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1932lb # [ 1.776142] systemd[1]: Starting Coldplug All udev Devices...1933client # [ 1.762019] systemd[1]: Starting Load Kernel Modules...1934client # [ 1.765911] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1935worker2 # [ 1.761687] systemd[1]: Starting Load Kernel Modules...1936worker2 # [ 1.766189] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1937client # [ 1.773892] systemd[1]: Starting Coldplug All udev Devices...1938lb # [ 1.790297] systemd[1]: Finished Create List of Static Device Nodes.1939worker1 # [ 1.786049] systemd[1]: Starting Journal Service...1940lb # [ 1.793106] systemd-journald[66]: Collecting audit messages is disabled.1941lb # [ 1.794853] systemd[1]: modprobe@configfs.service: Deactivated successfully.1942worker2 # [ 1.776266] systemd[1]: Starting Coldplug All udev Devices...1943client # [ 1.785334] systemd-journald[66]: Collecting audit messages is disabled.1944client # [ 1.787138] systemd[1]: Finished Create List of Static Device Nodes.1945lb # [ 1.803446] systemd[1]: Finished Load Kernel Module configfs.1946worker2 # [ 1.785741] systemd-journald[66]: Collecting audit messages is disabled.1947worker1 # [ 1.812143] systemd[1]: Starting Load Kernel Modules...1948client # [ 1.793303] systemd[1]: modprobe@configfs.service: Deactivated successfully.1949worker2 # [ 1.789186] systemd[1]: Finished Create List of Static Device Nodes.1950lb # [ 1.809388] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1951worker1 # [ 1.816062] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1952client # [ 1.800209] systemd[1]: Finished Load Kernel Module configfs.1953worker2 # [ 1.796722] systemd[1]: modprobe@configfs.service: Deactivated successfully.1954lb # [ 1.819141] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1955client # [ 1.805238] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1956worker1 # [ 1.826201] systemd[1]: Starting Coldplug All udev Devices...1957worker2 # [ 1.802482] systemd[1]: Finished Load Kernel Module configfs.1958worker2 # [ 1.808497] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1959lb # [ 1.829977] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1960worker1 # [ 1.836601] systemd-journald[66]: Collecting audit messages is disabled.1961client # [ 1.817891] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1962worker1 # [ 1.841067] systemd[1]: Finished Create List of Static Device Nodes.1963client # [ 1.822638] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1964lb # [ 1.839099] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1965worker2 # [ 1.820129] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1966worker1 # [ 1.847089] systemd[1]: modprobe@configfs.service: Deactivated successfully.1967worker1 # [ 1.853378] systemd[1]: Finished Load Kernel Module configfs.1968client # [ 1.835000] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1969worker2 # [ 1.832476] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1970worker1 # [ 1.860405] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1971lb # [ 1.861499] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1972worker2 # [ 1.841282] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1973worker1 # [ 1.872026] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1974lb # [ 1.870140] systemd[1]: Starting Create Static Device Nodes in /dev...1975client # [ 1.856889] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1976worker1 # [ 1.882434] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1977client # [ 1.867031] systemd[1]: Starting Create Static Device Nodes in /dev...1978lb # [ 1.882044] systemd[1]: Finished Load Kernel Modules.1979worker2 # [ 1.863652] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1980worker1 # [ 1.892994] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1981lb # [ 1.891186] systemd[1]: Starting Apply Kernel Variables...1982client # [ 1.878349] systemd[1]: Finished Load Kernel Modules.1983worker2 # [ 1.872261] systemd[1]: Starting Create Static Device Nodes in /dev...1984client # [ 1.887033] systemd[1]: Starting Apply Kernel Variables...1985worker2 # [ 1.884122] systemd[1]: Finished Load Kernel Modules.1986lb # [ 1.911500] systemd[1]: Finished Create Static Device Nodes in /dev.1987worker1 # [ 1.916592] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1988worker2 # [ 1.893766] systemd[1]: Starting Apply Kernel Variables...1989lb # [ 1.916142] systemd[1]: Started Journal Service.1990lb # [ 1.778056] systemd-modules-load[67]: Inserted module 'dm_mod'1991lb # [ 1.779329] systemd-modules-load[67]: Inserted module 'virtio_balloon'1992client # [ 1.905367] systemd[1]: Finished Create Static Device Nodes in /dev.1993lb # [ 1.780428] systemd-modules-load[67]: Inserted module 'virtio_gpu'1994worker1 # [ 1.928205] systemd[1]: Starting Create Static Device Nodes in /dev...1995lb # [ 1.784288] systemd[1]: Reached target Preparation for Local File Systems.1996client # [ 1.773515] systemd-modules-load[67]: Inserted module 'dm_mod'1997lb # [ 1.785454] systemd[1]: Reached target Local File Systems.1998client # [ 1.776359] systemd-modules-load[67]: Inserted module 'virtio_balloon'1999client # [ 1.777419] systemd-modules-load[67]: Inserted module 'virtio_gpu'2000client # [ 1.913181] systemd[1]: Started Journal Service.
loading 2,522 hidden lines…
4523client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.18 seconds)4524client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4525client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.09 seconds)4526lb: must succeed: curl -sf localhost:9901/clusters4527lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4528lb: must succeed: curl -sf localhost:9901/clusters4529lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4530worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4531worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4532(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.67 seconds)4533subtest: inputs already in the cache are not sent to the scheduler4534lb: must succeed: curl -sf localhost:9901/clusters4535lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4536lb: must succeed: curl -sf localhost:9901/clusters4537lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4538worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4539worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.02 seconds)4540client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24541client # don't know how to build these paths:4542client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4543worker1 # [ 48.669683] nix-grpc-daemon[511]: ts=2026-09-25T15:13:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550584544worker2 # [ 48.654277] nix-grpc-daemon[786]: ts=2026-09-25T15:13:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380664545worker2 # [ 48.664681] nix-grpc-daemon[786]: ts=2026-09-25T15:13:26Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04546worker2 # [ 48.666639] nix-grpc-daemon[786]: ts=2026-09-25T15:13:26Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14547worker1 # [ 48.690692] nix-grpc-daemon[511]: ts=2026-09-25T15:13:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550584548worker1 # [ 48.700137] nix-grpc-daemon[511]: ts=2026-09-25T15:13:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550584549worker1 # [ 48.710649] nix-grpc-daemon[861]: time=2026-09-25T15:13:26.962Z level=INFO msg="All 2 paths already cached"4550client # copying 1 paths to 'lb:50051'...4551client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4552worker1 # [ 48.717761] nix-grpc-daemon[511]: ts=2026-09-25T15:13:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550584553worker1 # [ 48.728905] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4554worker1 # [ 48.831756] nix-daemon[848]: accepted connection from pid 977, user nix-grpc-daemon (trusted)4555worker1 # [ 48.846790] nix-daemon[848]: reaped child process 980, status = succeeded4556lb # [ 48.849176] niks3-server[955]: 2026/09/25 15:13:28 INFO Received push request method=POST path=/api/pushes4557worker1 # [ 48.856378] nix-grpc-daemon[861]: time=2026-09-25T15:13:27.108Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4558worker1 # [ 48.857812] nix-grpc-daemon[861]: time=2026-09-25T15:13:27.108Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4559lb # [ 48.865324] niks3-server[955]: 2026/09/25 15:13:28 INFO Received sign narinfos request method=POST path=/api/pushes/7/sign4560lb # [ 48.868274] niks3-server[955]: 2026/09/25 15:13:28 INFO Signed narinfos id=7 count=14561worker1 # [ 48.871111] nix-grpc-daemon[861]: time=2026-09-25T15:13:27.122Z level=INFO msg="Uploading 1 narinfos"4562lb # [ 48.874445] niks3-server[955]: 2026/09/25 15:13:28 INFO Received complete push request method=POST path=/api/pushes/7/complete4563lb # [ 48.883159] niks3-server[955]: 2026/09/25 15:13:28 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4564lb # [ 48.887084] niks3-server[955]: 2026/09/25 15:13:28 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4565worker1 # [ 48.893839] nix-grpc-daemon[861]: time=2026-09-25T15:13:27.145Z level=INFO msg="Upload complete. (153ms)"4566lb # [ 48.891803] niks3-server[955]: 2026/09/25 15:13:28 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4567worker1 # [ 48.895966] nix-grpc-daemon[511]: ts=2026-09-25T15:13:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=1 nar_bytes_in=7524568worker1 # [ 48.899067] nix-grpc-daemon[511]: ts=2026-09-25T15:13:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550584569worker1 # [ 48.907578] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4570worker1 # [ 48.909266] nix-daemon[848]: reaped child process 975, status = succeeded4571worker1 # [ 48.914425] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4572client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4573client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4574client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4575client # farm-top> LOG-farm-top-t1b4576worker1 # [ 49.941589] nix-daemon[848]: reaped child process 984, status = succeeded4577worker1 # [ 50.027310] nix-daemon[848]: accepted connection from pid 1000, user nix-grpc-daemon (trusted)4578worker1 # [ 50.038817] nix-daemon[848]: reaped child process 1003, status = succeeded4579lb # [ 50.044216] niks3-server[955]: 2026/09/25 15:13:29 INFO Received push request method=POST path=/api/pushes4580worker1 # [ 50.054956] nix-grpc-daemon[861]: time=2026-09-25T15:13:28.306Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4581worker1 # [ 50.057421] nix-grpc-daemon[861]: time=2026-09-25T15:13:28.308Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4582lb # [ 50.088146] niks3-server[955]: 2026/09/25 15:13:29 INFO Received sign narinfos request method=POST path=/api/pushes/8/sign4583worker1 # [ 50.097533] nix-grpc-daemon[861]: time=2026-09-25T15:13:28.349Z level=INFO msg="Uploading 1 narinfos"4584lb # [ 50.095120] niks3-server[955]: 2026/09/25 15:13:29 INFO Signed narinfos id=8 count=14585lb # [ 50.101046] niks3-server[955]: 2026/09/25 15:13:29 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4586lb # [ 50.104934] niks3-server[955]: 2026/09/25 15:13:29 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4587lb # [ 50.109139] niks3-server[955]: 2026/09/25 15:13:29 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4588lb # [ 50.119255] niks3-server[955]: 2026/09/25 15:13:29 INFO Received complete push request method=POST path=/api/pushes/8/complete4589worker1 # [ 50.126407] nix-grpc-daemon[861]: time=2026-09-25T15:13:28.378Z level=INFO msg="Upload complete. (204ms)"4590lb # [ 50.124181] niks3-server[955]: 2026/09/25 15:13:29 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4591worker1 # [ 50.129115] nix-grpc-daemon[511]: ts=2026-09-25T15:13:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14592worker1 # [ 50.135961] nix-daemon[848]: reaped child process 982, status = succeeded4593client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.63 seconds)4594client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4595client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds)4596lb: must succeed: curl -sf localhost:9901/clusters4597lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4598lb: must succeed: curl -sf localhost:9901/clusters4599lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4600worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4601worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4602(finished: subtest: inputs already in the cache are not sent to the scheduler, in 1.91 seconds)4603subtest: outputs read back through any worker carry the cache signature4604client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24605client # don't know how to build these paths:4606client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4607worker2 # [ 50.483943] nix-grpc-daemon[786]: ts=2026-09-25T15:13:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380664608worker2 # [ 50.489733] nix-grpc-daemon[786]: ts=2026-09-25T15:13:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380664609worker2 # [ 50.507625] nix-grpc-daemon[786]: ts=2026-09-25T15:13:28Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04610worker2 # [ 50.512974] nix-grpc-daemon[786]: ts=2026-09-25T15:13:28Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.4:50051 clients=14611worker2 # [ 50.518311] nix-grpc-daemon[786]: ts=2026-09-25T15:13:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664612worker2 # [ 50.529980] nix-grpc-daemon[786]: ts=2026-09-25T15:13:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380664613client # copying 1 paths to 'lb:50051'...4614client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4615worker2 # [ 50.540935] nix-grpc-daemon[786]: ts=2026-09-25T15:13:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380664616worker2 # [ 50.549983] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4617worker2 # [ 50.570852] nix-grpc-daemon[786]: ts=2026-09-25T15:13:28Z level=info event=niks3_push_started pid=9244618worker2 # [ 50.698700] nix-grpc-daemon[924]: time=2026-09-25T15:13:29.008Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4619worker2 # [ 50.791168] nix-daemon[873]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4620worker2 # [ 50.805429] nix-daemon[873]: reaped child process 940, status = succeeded4621lb # [ 50.824888] niks3-server[955]: 2026/09/25 15:13:30 INFO Received push request method=POST path=/api/pushes4622worker2 # [ 50.818479] nix-grpc-daemon[924]: time=2026-09-25T15:13:29.129Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4623worker2 # [ 50.819998] nix-grpc-daemon[924]: time=2026-09-25T15:13:29.130Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4624lb # [ 50.859505] niks3-server[955]: 2026/09/25 15:13:30 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4625lb # [ 50.866491] niks3-server[955]: 2026/09/25 15:13:30 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4626worker2 # [ 50.858097] nix-grpc-daemon[924]: time=2026-09-25T15:13:29.168Z level=INFO msg="Uploading 1 narinfos"4627lb # [ 50.872576] niks3-server[955]: 2026/09/25 15:13:30 INFO Signed narinfos id=9 count=14628lb # [ 50.888562] niks3-server[955]: 2026/09/25 15:13:30 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4629lb # [ 50.892934] niks3-server[955]: 2026/09/25 15:13:30 INFO Received complete push request method=POST path=/api/pushes/9/complete4630worker2 # [ 50.886245] nix-grpc-daemon[924]: time=2026-09-25T15:13:29.196Z level=INFO msg="Upload complete. (182ms)"4631lb # [ 50.902499] niks3-server[955]: 2026/09/25 15:13:30 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4632worker2 # [ 50.888504] nix-grpc-daemon[786]: ts=2026-09-25T15:13:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_in=5044633worker2 # [ 50.891877] nix-grpc-daemon[786]: ts=2026-09-25T15:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664634worker2 # [ 50.898756] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4635worker2 # [ 50.904156] nix-daemon[873]: reaped child process 922, status = succeeded4636client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4637client # farm-a> worker2: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4638worker2 # [ 50.907320] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4639client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4640client # farm-a> LOG-farm-a-sig4641worker2 # [ 52.556360] nix-daemon[873]: reaped child process 944, status = succeeded4642worker2 # [ 52.632823] nix-daemon[873]: accepted connection from pid 960, user nix-grpc-daemon (trusted)4643worker2 # [ 52.647413] nix-daemon[873]: reaped child process 963, status = succeeded4644lb # [ 52.667459] niks3-server[955]: 2026/09/25 15:13:31 INFO Received push request method=POST path=/api/pushes4645worker2 # [ 52.659457] nix-grpc-daemon[924]: time=2026-09-25T15:13:30.969Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4646worker2 # [ 52.660919] nix-grpc-daemon[924]: time=2026-09-25T15:13:30.971Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4647lb # [ 52.713638] niks3-server[955]: 2026/09/25 15:13:31 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4648lb # [ 52.718300] niks3-server[955]: 2026/09/25 15:13:31 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4649lb # [ 52.723694] niks3-server[955]: 2026/09/25 15:13:31 INFO Signed narinfos id=10 count=14650worker2 # [ 52.711061] nix-grpc-daemon[924]: time=2026-09-25T15:13:31.021Z level=INFO msg="Uploading 1 narinfos"4651lb # [ 52.726341] niks3-server[955]: 2026/09/25 15:13:31 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4652lb # [ 52.733359] niks3-server[955]: 2026/09/25 15:13:31 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4653lb # [ 52.736068] niks3-server[955]: 2026/09/25 15:13:31 INFO Received complete push request method=POST path=/api/pushes/10/complete4654lb # [ 52.740942] niks3-server[955]: 2026/09/25 15:13:31 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4655worker2 # [ 52.726349] nix-grpc-daemon[924]: time=2026-09-25T15:13:31.037Z level=INFO msg="Upload complete. (210ms)"4656worker2 # [ 52.731504] nix-grpc-daemon[786]: ts=2026-09-25T15:13:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14657worker2 # [ 52.741628] nix-grpc-daemon[786]: ts=2026-09-25T15:13:31Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04658worker2 # [ 52.746319] nix-grpc-daemon[786]: ts=2026-09-25T15:13:31Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=14659worker2 # [ 52.749704] nix-grpc-daemon[786]: ts=2026-09-25T15:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664660worker2 # [ 52.756286] nix-daemon[873]: reaped child process 942, status = succeeded4661worker2 # [ 52.758103] nix-grpc-daemon[786]: ts=2026-09-25T15:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380664662worker2 # [ 52.763895] nix-grpc-daemon[924]: time=2026-09-25T15:13:31.074Z level=INFO msg="All 1 paths already cached"4663client # copying 1 paths to 'lb:50051'...4664worker2 # [ 52.768797] nix-grpc-daemon[786]: ts=2026-09-25T15:13:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380664665client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4666worker2 # [ 52.773939] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4667worker2 # [ 52.860257] nix-daemon[873]: accepted connection from pid 969, user nix-grpc-daemon (trusted)4668worker2 # [ 52.875418] nix-daemon[873]: reaped child process 972, status = succeeded4669lb # [ 52.895599] niks3-server[955]: 2026/09/25 15:13:32 INFO Received push request method=POST path=/api/pushes4670worker2 # [ 52.887818] nix-grpc-daemon[924]: time=2026-09-25T15:13:31.198Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4671worker2 # [ 52.889453] nix-grpc-daemon[924]: time=2026-09-25T15:13:31.200Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4672lb # [ 52.924243] niks3-server[955]: 2026/09/25 15:13:32 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4673worker2 # [ 52.916599] nix-grpc-daemon[924]: time=2026-09-25T15:13:31.227Z level=INFO msg="Uploading 1 narinfos"4674lb # [ 52.932326] niks3-server[955]: 2026/09/25 15:13:32 INFO Signed narinfos id=11 count=14675lb # [ 52.939541] niks3-server[955]: 2026/09/25 15:13:32 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4676lb # [ 52.943520] niks3-server[955]: 2026/09/25 15:13:32 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4677lb # [ 52.958443] niks3-server[955]: 2026/09/25 15:13:32 INFO Received complete push request method=POST path=/api/pushes/11/complete4678worker2 # [ 52.953712] nix-grpc-daemon[924]: time=2026-09-25T15:13:31.264Z level=INFO msg="Upload complete. (173ms)"4679lb # [ 52.969291] niks3-server[955]: 2026/09/25 15:13:32 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4680worker2 # [ 52.957642] nix-grpc-daemon[786]: ts=2026-09-25T15:13:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_in=6244681worker2 # [ 52.962789] nix-grpc-daemon[786]: ts=2026-09-25T15:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664682worker2 # [ 52.972808] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4683worker2 # [ 52.980574] nix-daemon[873]: reaped child process 967, status = succeeded4684client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4685worker2 # [ 52.985655] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4686client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4687client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4688client # farm-b> LOG-farm-b-sig4689worker2 # [ 54.203353] nix-daemon[873]: reaped child process 976, status = succeeded4690worker2 # [ 54.275416] nix-daemon[873]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4691worker2 # [ 54.286805] nix-daemon[873]: reaped child process 995, status = succeeded4692lb # [ 54.308557] niks3-server[955]: 2026/09/25 15:13:33 INFO Received push request method=POST path=/api/pushes4693worker2 # [ 54.300583] nix-grpc-daemon[924]: time=2026-09-25T15:13:32.611Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4694worker2 # [ 54.302347] nix-grpc-daemon[924]: time=2026-09-25T15:13:32.612Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4695lb # [ 54.344865] niks3-server[955]: 2026/09/25 15:13:33 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4696lb # [ 54.355505] niks3-server[955]: 2026/09/25 15:13:33 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4697lb # [ 54.363505] niks3-server[955]: 2026/09/25 15:13:33 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4698lb # [ 54.365890] niks3-server[955]: 2026/09/25 15:13:33 INFO Signed narinfos id=12 count=14699worker2 # [ 54.351289] nix-grpc-daemon[924]: time=2026-09-25T15:13:32.661Z level=INFO msg="Uploading 1 narinfos"4700lb # [ 54.369473] niks3-server[955]: 2026/09/25 15:13:33 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4701lb # [ 54.376080] niks3-server[955]: 2026/09/25 15:13:33 INFO Received complete push request method=POST path=/api/pushes/12/complete4702lb # [ 54.384305] niks3-server[955]: 2026/09/25 15:13:33 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4703worker2 # [ 54.368913] nix-grpc-daemon[924]: time=2026-09-25T15:13:32.679Z level=INFO msg="Upload complete. (176ms)"4704worker2 # [ 54.379534] nix-grpc-daemon[786]: ts=2026-09-25T15:13:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14705worker2 # [ 54.385980] nix-grpc-daemon[786]: ts=2026-09-25T15:13:32Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04706worker2 # [ 54.389942] nix-grpc-daemon[786]: ts=2026-09-25T15:13:32Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=14707worker2 # [ 54.393925] nix-grpc-daemon[786]: ts=2026-09-25T15:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664708worker2 # [ 54.403874] nix-grpc-daemon[786]: ts=2026-09-25T15:13:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380664709worker2 # [ 54.406103] nix-daemon[873]: reaped child process 974, status = succeeded4710worker2 # [ 54.580800] nix-grpc-daemon[924]: time=2026-09-25T15:13:32.891Z level=INFO msg="All 2 paths already cached"4711client # copying 1 paths to 'lb:50051'...4712client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4713worker2 # [ 54.591407] nix-grpc-daemon[786]: ts=2026-09-25T15:13:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380664714worker2 # [ 54.599566] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4715worker2 # [ 54.708815] nix-daemon[873]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4716worker2 # [ 54.721713] nix-daemon[873]: reaped child process 1004, status = succeeded4717lb # [ 54.742487] niks3-server[955]: 2026/09/25 15:13:33 INFO Received push request method=POST path=/api/pushes4718worker2 # [ 54.734400] nix-grpc-daemon[924]: time=2026-09-25T15:13:33.045Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4719worker2 # [ 54.735897] nix-grpc-daemon[924]: time=2026-09-25T15:13:33.046Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4720lb # [ 54.771919] niks3-server[955]: 2026/09/25 15:13:33 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4721lb # [ 54.774786] niks3-server[955]: 2026/09/25 15:13:33 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4722lb # [ 54.778528] niks3-server[955]: 2026/09/25 15:13:33 INFO Signed narinfos id=13 count=14723worker2 # [ 54.765884] nix-grpc-daemon[924]: time=2026-09-25T15:13:33.076Z level=INFO msg="Uploading 1 narinfos"4724lb # [ 54.785729] niks3-server[955]: 2026/09/25 15:13:33 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4725lb # [ 54.791036] niks3-server[955]: 2026/09/25 15:13:33 INFO Received complete push request method=POST path=/api/pushes/13/complete4726worker2 # [ 54.780796] nix-grpc-daemon[924]: time=2026-09-25T15:13:33.091Z level=INFO msg="Upload complete. (167ms)"4727lb # [ 54.797493] niks3-server[955]: 2026/09/25 15:13:33 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4728worker2 # [ 54.783153] nix-grpc-daemon[786]: ts=2026-09-25T15:13:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_in=7524729worker2 # [ 54.786690] nix-grpc-daemon[786]: ts=2026-09-25T15:13:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664730worker2 # [ 54.793419] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4731worker2 # [ 54.795703] nix-daemon[873]: reaped child process 999, status = succeeded4732worker2 # [ 54.800865] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4733client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4734client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4735client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4736client # farm-top> LOG-farm-top-sig4737worker2 # [ 55.618314] nix-daemon[873]: reaped child process 1008, status = succeeded4738worker2 # [ 55.687851] nix-daemon[873]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4739worker2 # [ 55.700077] nix-daemon[873]: reaped child process 1027, status = succeeded4740lb # [ 55.721393] niks3-server[955]: 2026/09/25 15:13:34 INFO Received push request method=POST path=/api/pushes4741worker2 # [ 55.714336] nix-grpc-daemon[924]: time=2026-09-25T15:13:34.024Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4742worker2 # [ 55.717057] nix-grpc-daemon[924]: time=2026-09-25T15:13:34.026Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4743lb # [ 55.760166] niks3-server[955]: 2026/09/25 15:13:34 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4744lb # [ 55.768059] niks3-server[955]: 2026/09/25 15:13:34 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4745lb # [ 55.774469] niks3-server[955]: 2026/09/25 15:13:34 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4746lb # [ 55.780635] niks3-server[955]: 2026/09/25 15:13:34 INFO Signed narinfos id=14 count=14747worker2 # [ 55.769375] nix-grpc-daemon[924]: time=2026-09-25T15:13:34.079Z level=INFO msg="Uploading 1 narinfos"4748lb # [ 55.790160] niks3-server[955]: 2026/09/25 15:13:34 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4749lb # [ 55.797699] niks3-server[955]: 2026/09/25 15:13:34 INFO Received complete push request method=POST path=/api/pushes/14/complete4750worker2 # [ 55.798568] nix-grpc-daemon[924]: time=2026-09-25T15:13:34.109Z level=INFO msg="Upload complete. (191ms)"4751lb # [ 55.816397] niks3-server[955]: 2026/09/25 15:13:35 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo4752worker2 # [ 55.805874] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=14753worker2 # [ 55.821674] nix-daemon[873]: reaped child process 1006, status = succeeded4754client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.47 seconds)4755client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath4756client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds)4757client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4758worker1 # [ 56.092180] nix-grpc-daemon[511]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:426044759client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4760client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a4761client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b4762client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4763worker1 # [ 56.108541] nix-grpc-daemon[511]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:426044764worker1 # [ 56.115065] nix-grpc-daemon[511]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...4765worker1 # [ 56.129506] nix-grpc-daemon[511]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...4766worker1 # [ 56.140976] nix-grpc-daemon[511]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...4767client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.19 seconds)4768client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4769worker2 # [ 56.262878] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:447224770worker2 # [ 56.269098] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:447224771client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.12 seconds)4772client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4773client # finding garbage collector roots...4774client # removing stale temporary roots file "/nix/var/nix/temproots/860"4775client # removing stale temporary roots file "/nix/var/nix/temproots/827"4776client # removing stale temporary roots file "/nix/var/nix/temproots/868"4777client # removing stale temporary roots file "/nix/var/nix/temproots/889"4778client # removing stale temporary roots file "/nix/var/nix/temproots/904"4779client # removing stale temporary roots file "/nix/var/nix/temproots/848"4780client # removing stale temporary roots file "/nix/var/nix/temproots/881"4781client # removing stale temporary roots file "/nix/var/nix/temproots/812"4782client # deleting specified paths...4783client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'4784client # deleting unused links...4785client # note: hard linking is currently saving 0.0 KiB4786worker2 # [ 56.519627] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380664787worker2 # [ 56.525994] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380664788worker2 # [ 56.531336] nix-grpc-daemon[924]: time=2026-09-25T15:13:34.841Z level=INFO msg="All 1 paths already cached"4789worker2 # [ 56.538498] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:380664790worker2 # [ 56.546238] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:380664791client # copying 3 paths...4792worker1 # [ 56.575180] nix-grpc-daemon[511]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550584793client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...4794worker2 # [ 56.574325] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:380664795worker2 # [ 56.580588] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_out=2324796worker1 # [ 56.599730] nix-grpc-daemon[511]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550584797worker1 # [ 56.605077] nix-grpc-daemon[511]: ts=2026-09-25T15:13:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=1 nar_bytes_out=1764798worker2 # [ 56.588755] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:380664799worker2 # [ 56.594127] nix-grpc-daemon[786]: ts=2026-09-25T15:13:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_out=1284800client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...4801client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...4802client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.34 seconds)4803(finished: subtest: outputs read back through any worker carry the cache signature, in 6.26 seconds)4804subtest: two clients wanting the same drv build it once4805client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4806client # [ 56.655434] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4807client # Running as unit: dedup1.service; invocation ID: eaa7299556774d9f9849e2ba26dcdbff4808client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)4809client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4810client # [ 56.783972] nix[957]: don't know how to build these paths:4811client # [ 56.784926] nix[957]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4812worker2 # [ 56.776736] nix-grpc-daemon[786]: ts=2026-09-25T15:13:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380664813worker2 # [ 56.788147] nix-grpc-daemon[786]: ts=2026-09-25T15:13:35Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04814worker2 # [ 56.793660] nix-grpc-daemon[786]: ts=2026-09-25T15:13:35Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14815worker1 # [ 56.814507] nix-grpc-daemon[511]: ts=2026-09-25T15:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550584816worker1 # [ 56.826896] nix-grpc-daemon[511]: ts=2026-09-25T15:13:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550584817client # [ 56.825565] nix[957]: copying 1 paths to 'lb:50051'...4818client # [ 56.826396] nix[957]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4819worker1 # [ 56.838441] nix-grpc-daemon[511]: ts=2026-09-25T15:13:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550584820worker1 # [ 56.849278] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4821worker1 # [ 56.955767] nix-daemon[848]: accepted connection from pid 1010, user nix-grpc-daemon (trusted)4822worker1 # [ 56.969316] nix-daemon[848]: reaped child process 1013, status = succeeded4823lb # [ 56.973281] niks3-server[955]: 2026/09/25 15:13:36 INFO Received push request method=POST path=/api/pushes4824worker1 # [ 56.983942] nix-grpc-daemon[861]: time=2026-09-25T15:13:35.235Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4825worker1 # [ 56.985878] nix-grpc-daemon[861]: time=2026-09-25T15:13:35.237Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4826lb # [ 57.004599] niks3-server[955]: 2026/09/25 15:13:36 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4827lb # [ 57.014798] niks3-server[955]: 2026/09/25 15:13:36 INFO Signed narinfos id=15 count=14828worker1 # [ 57.020998] nix-grpc-daemon[861]: time=2026-09-25T15:13:35.272Z level=INFO msg="Uploading 1 narinfos"4829lb # [ 57.019794] niks3-server[955]: 2026/09/25 15:13:36 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4830lb # [ 57.026154] niks3-server[955]: 2026/09/25 15:13:36 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4831lb # [ 57.040416] niks3-server[955]: 2026/09/25 15:13:36 INFO Received complete push request method=POST path=/api/pushes/15/complete4832lb # [ 57.049361] niks3-server[955]: 2026/09/25 15:13:36 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4833worker1 # [ 57.055136] nix-grpc-daemon[861]: time=2026-09-25T15:13:35.307Z level=INFO msg="Upload complete. (194ms)"4834worker1 # [ 57.059887] nix-grpc-daemon[511]: ts=2026-09-25T15:13:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=1 nar_bytes_in=4324835worker1 # [ 57.066336] nix-grpc-daemon[511]: ts=2026-09-25T15:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550584836lb # [ 57.067136] envoy[924]: conn peer=[2001:db8:1::1]:48426 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9490 tx=4323 ms=53094837lb # [ 57.068617] envoy[924]: conn peer=[2001:db8:1::1]:38938 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=304838lb # [ 57.070031] envoy[924]: conn peer=[2001:db8:1::1]:38946 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4165 tx=1876 ms=14694839lb # [ 57.071507] envoy[924]: conn peer=[2001:db8:1::1]:38958 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4328 ms=53424840lb # [ 57.073142] envoy[924]: conn peer=[2001:db8:1::1]:52266 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=444841lb # [ 57.074608] envoy[924]: conn peer=[2001:db8:1::1]:52260 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2874 ms=1014842worker1 # [ 57.076770] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4843lb # [ 57.076114] envoy[924]: conn peer=[2001:db8:1::1]:52278 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=304844lb # [ 57.077522] envoy[924]: conn peer=[2001:db8:1::1]:52272 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=384845worker1 # [ 57.088322] nix-daemon[848]: reaped child process 1008, status = succeeded4846worker1 # [ 57.092164] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4847client # [ 57.080102] nix[957]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4848client # [ 57.708581] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4849client # Running as unit: dedup2.service; invocation ID: 6f532a9821a749b6bdd1e28e4da350dd4850client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4851worker2 # [ 57.874676] nix-grpc-daemon[786]: ts=2026-09-25T15:13:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380664852client # [ 57.886272] nix[973]: don't know how to build these paths:4853client # [ 57.887525] nix[973]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4854worker2 # [ 57.889138] nix-grpc-daemon[786]: ts=2026-09-25T15:13:36Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04855worker1 # [ 57.916227] nix-grpc-daemon[511]: ts=2026-09-25T15:13:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550584856client # [ 57.907258] nix[973]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4857client: must succeed: sleep 24858client: (finished: must succeed: sleep 2, in 2.04 seconds)4859client: waiting for success: ! systemctl is-active dedup1 dedup24860worker1 # [ 61.110758] nix-daemon[848]: reaped child process 1017, status = succeeded4861worker1 # [ 61.187127] nix-daemon[848]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)4862worker1 # [ 61.199221] nix-daemon[848]: reaped child process 1050, status = succeeded4863lb # [ 61.202922] niks3-server[955]: 2026/09/25 15:13:40 INFO Received push request method=POST path=/api/pushes4864worker1 # [ 61.213605] nix-grpc-daemon[861]: time=2026-09-25T15:13:39.465Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4865worker1 # [ 61.215161] nix-grpc-daemon[861]: time=2026-09-25T15:13:39.467Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4866lb # [ 61.250572] niks3-server[955]: 2026/09/25 15:13:40 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4867lb # [ 61.256888] niks3-server[955]: 2026/09/25 15:13:40 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4868lb # [ 61.265859] niks3-server[955]: 2026/09/25 15:13:40 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4869lb # [ 61.271519] niks3-server[955]: 2026/09/25 15:13:40 INFO Signed narinfos id=16 count=14870worker1 # [ 61.276591] nix-grpc-daemon[861]: time=2026-09-25T15:13:39.528Z level=INFO msg="Uploading 1 narinfos"4871lb # [ 61.278123] niks3-server[955]: 2026/09/25 15:13:40 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4872lb # [ 61.288331] niks3-server[955]: 2026/09/25 15:13:40 INFO Received complete push request method=POST path=/api/pushes/16/complete4873lb # [ 61.293465] niks3-server[955]: 2026/09/25 15:13:40 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4874worker1 # [ 61.296245] nix-grpc-daemon[861]: time=2026-09-25T15:13:39.548Z level=INFO msg="Upload complete. (194ms)"4875worker1 # [ 61.302996] nix-grpc-daemon[511]: ts=2026-09-25T15:13:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14876worker1 # [ 61.310253] nix-grpc-daemon[511]: ts=2026-09-25T15:13:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14877worker1 # [ 61.322901] nix-daemon[848]: reaped child process 1015, status = succeeded4878client # [ 61.314268] systemd[1]: dedup2.service: Deactivated successfully.4879client # [ 61.316109] systemd[1]: dedup2.service: Consumed 137ms CPU time over 3.606s wall clock time, 35.4M memory peak, 5.9K incoming IP traffic, 5.5K outgoing IP traffic.4880client # [ 61.318833] systemd[1]: dedup1.service: Deactivated successfully.4881client # [ 61.319909] systemd[1]: dedup1.service: Consumed 105ms CPU time over 4.663s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.4882client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4883client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4884client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4885worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4886worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4887worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4888worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4889worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4890worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4891worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4892worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4893(finished: subtest: two clients wanting the same drv build it once, in 5.73 seconds)4894subtest: --timeout and --max-silent-time reach the builder4895worker2 # [ 62.495472] nix-grpc-daemon[786]: ts=2026-09-25T15:13:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380664896worker2 # [ 62.507879] nix-grpc-daemon[786]: ts=2026-09-25T15:13:40Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=04897worker2 # [ 62.513207] nix-grpc-daemon[786]: ts=2026-09-25T15:13:40Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=14898worker2 # [ 62.519399] nix-grpc-daemon[786]: ts=2026-09-25T15:13:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664899worker2 # [ 62.527167] nix-grpc-daemon[786]: ts=2026-09-25T15:13:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380664900worker2 # [ 62.537745] nix-grpc-daemon[786]: ts=2026-09-25T15:13:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380664901worker2 # [ 62.549099] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4902worker2 # [ 62.646098] nix-daemon[873]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)4903worker2 # [ 62.661052] nix-daemon[873]: reaped child process 1061, status = succeeded4904lb # [ 62.683702] niks3-server[955]: 2026/09/25 15:13:41 INFO Received push request method=POST path=/api/pushes4905worker2 # [ 62.676580] nix-grpc-daemon[924]: time=2026-09-25T15:13:40.987Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4906worker2 # [ 62.678719] nix-grpc-daemon[924]: time=2026-09-25T15:13:40.988Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"4907lb # [ 62.715228] niks3-server[955]: 2026/09/25 15:13:41 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4908lb # [ 62.722065] niks3-server[955]: 2026/09/25 15:13:41 INFO Signed narinfos id=17 count=14909worker2 # [ 62.711428] nix-grpc-daemon[924]: time=2026-09-25T15:13:41.021Z level=INFO msg="Uploading 1 narinfos"4910lb # [ 62.735969] niks3-server[955]: 2026/09/25 15:13:41 INFO Received complete push request method=POST path=/api/pushes/17/complete4911lb # [ 62.768742] niks3-server[955]: 2026/09/25 15:13:41 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls4912lb # [ 62.775822] niks3-server[955]: 2026/09/25 15:13:41 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst4913worker2 # [ 62.768141] nix-grpc-daemon[924]: time=2026-09-25T15:13:41.078Z level=INFO msg="Upload complete. (207ms)"4914lb # [ 62.784331] niks3-server[955]: 2026/09/25 15:13:41 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo4915worker2 # [ 62.773338] nix-grpc-daemon[786]: ts=2026-09-25T15:13:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_in=4244916worker2 # [ 62.779165] nix-grpc-daemon[786]: ts=2026-09-25T15:13:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664917worker2 # [ 62.793208] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4918worker2 # [ 62.802272] nix-daemon[873]: reaped child process 1056, status = succeeded4919worker2 # [ 62.810479] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4920lb # [ 67.067731] envoy[924]: conn peer=[2001:db8:1::1]:52294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2994 tx=1519 ms=45244921lb # [ 67.071913] envoy[924]: conn peer=[2001:db8:1::1]:52298 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=34284922worker2 # [ 69.030692] nix-grpc-daemon[786]: ts=2026-09-25T15:13:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=04923worker2 # [ 69.055224] nix-daemon[873]: reaped child process 1063, status = succeeded4924worker2 # [ 69.057365] nix-daemon[873]: reaped child process 1065, status = succeeded4925worker2 # [ 69.153837] nix-grpc-daemon[786]: ts=2026-09-25T15:13:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380664926worker2 # [ 69.165322] nix-grpc-daemon[786]: ts=2026-09-25T15:13:47Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=04927worker2 # [ 69.169902] nix-grpc-daemon[786]: ts=2026-09-25T15:13:47Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=14928worker1 # [ 69.190222] nix-grpc-daemon[511]: ts=2026-09-25T15:13:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550584929worker1 # [ 69.201174] nix-grpc-daemon[511]: ts=2026-09-25T15:13:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550584930worker1 # [ 69.211670] nix-grpc-daemon[511]: ts=2026-09-25T15:13:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550584931worker1 # [ 69.222628] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4932worker1 # [ 69.324585] nix-daemon[848]: accepted connection from pid 1063, user nix-grpc-daemon (trusted)4933worker1 # [ 69.341023] nix-daemon[848]: reaped child process 1066, status = succeeded4934lb # [ 69.339336] niks3-server[955]: 2026/09/25 15:13:48 INFO Received push request method=POST path=/api/pushes4935worker1 # [ 69.346356] nix-grpc-daemon[861]: time=2026-09-25T15:13:47.597Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4936worker1 # [ 69.347882] nix-grpc-daemon[861]: time=2026-09-25T15:13:47.597Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"4937lb # [ 69.354711] niks3-server[955]: 2026/09/25 15:13:48 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4938lb # [ 69.357259] niks3-server[955]: 2026/09/25 15:13:48 INFO Signed narinfos id=18 count=14939worker1 # [ 69.360186] nix-grpc-daemon[861]: time=2026-09-25T15:13:47.612Z level=INFO msg="Uploading 1 narinfos"4940lb # [ 69.361250] niks3-server[955]: 2026/09/25 15:13:48 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst4941lb # [ 69.364481] niks3-server[955]: 2026/09/25 15:13:48 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls4942lb # [ 69.368730] niks3-server[955]: 2026/09/25 15:13:48 INFO Received complete push request method=POST path=/api/pushes/18/complete4943worker1 # [ 69.375947] nix-grpc-daemon[861]: time=2026-09-25T15:13:47.627Z level=INFO msg="Upload complete. (141ms)"4944lb # [ 69.376303] niks3-server[955]: 2026/09/25 15:13:48 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo4945worker1 # [ 69.378350] nix-grpc-daemon[511]: ts=2026-09-25T15:13:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=1 nar_bytes_in=4244946worker1 # [ 69.381501] nix-grpc-daemon[511]: ts=2026-09-25T15:13:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550584947worker1 # [ 69.388295] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4948worker1 # [ 69.391485] nix-daemon[848]: reaped child process 1061, status = succeeded4949worker1 # [ 69.396518] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)4950worker1 # [ 75.809581] nix-grpc-daemon[511]: ts=2026-09-25T15:13:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=04951worker1 # [ 75.820384] nix-daemon[848]: reaped child process 1068, status = succeeded4952worker1 # [ 75.821433] nix-daemon[848]: reaped child process 1070, status = succeeded4953worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4954worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4955worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4956worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4957(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.55 seconds)4958subtest: a failing build is counted with its own reason4959worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4960worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4961worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4962worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4963worker2 # [ 76.089327] nix-grpc-daemon[786]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380664964worker2 # [ 76.095623] nix-grpc-daemon[786]: ts=2026-09-25T15:13:54Z level=debug event=want client=9 drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv cached=04965worker2 # [ 76.098679] nix-grpc-daemon[786]: ts=2026-09-25T15:13:54Z level=debug event=assigned drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv worker=192.168.1.4:50051 clients=14966worker2 # [ 76.100809] nix-grpc-daemon[786]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664967worker2 # [ 76.104992] nix-grpc-daemon[786]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380664968worker2 # [ 76.112338] nix-grpc-daemon[786]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380664969worker2 # [ 76.117470] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4970worker2 # [ 76.199529] nix-daemon[873]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)4971worker2 # [ 76.213668] nix-daemon[873]: reaped child process 1101, status = succeeded4972lb # [ 76.234065] niks3-server[955]: 2026/09/25 15:13:55 INFO Received push request method=POST path=/api/pushes4973worker2 # [ 76.227298] nix-grpc-daemon[924]: time=2026-09-25T15:13:54.537Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4974worker2 # [ 76.229327] nix-grpc-daemon[924]: time=2026-09-25T15:13:54.539Z level=INFO msg="Uploading j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv (400B)"4975lb # [ 76.271175] niks3-server[955]: 2026/09/25 15:13:55 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4976lb # [ 76.277758] niks3-server[955]: 2026/09/25 15:13:55 INFO Signed narinfos id=19 count=14977worker2 # [ 76.266222] nix-grpc-daemon[924]: time=2026-09-25T15:13:54.576Z level=INFO msg="Uploading 1 narinfos"4978lb # [ 76.284627] niks3-server[955]: 2026/09/25 15:13:55 INFO Registered completed upload object_key=nar/1awa0khyixxmb53bhxv3vk1qy0fd3xzskbp3b8g2fzdmcz3cvv97.nar.zst4979lb # [ 76.295363] niks3-server[955]: 2026/09/25 15:13:55 INFO Received complete push request method=POST path=/api/pushes/19/complete4980lb # [ 76.298708] niks3-server[955]: 2026/09/25 15:13:55 INFO Registered completed upload object_key=j5sh943zaa0qjagii0xkivjfb4c2c5p0.ls4981worker2 # [ 76.286119] nix-grpc-daemon[924]: time=2026-09-25T15:13:54.596Z level=INFO msg="Upload complete. (163ms)"4982lb # [ 76.301531] niks3-server[955]: 2026/09/25 15:13:55 INFO Registered completed upload object_key=j5sh943zaa0qjagii0xkivjfb4c2c5p0.narinfo4983worker2 # [ 76.288385] nix-grpc-daemon[786]: ts=2026-09-25T15:13:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_in=4004984worker2 # [ 76.292187] nix-grpc-daemon[786]: ts=2026-09-25T15:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380664985worker2 # [ 76.299088] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4986worker2 # [ 76.301339] nix-daemon[873]: reaped child process 1096, status = succeeded4987worker2 # [ 76.306634] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4988lb # [ 77.067445] envoy[924]: conn peer=[2001:db8:1::1]:52306 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=936 ms=65434989lb # [ 77.071349] envoy[924]: conn peer=[2001:db8:1::1]:60312 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=934 ms=66494990worker2 # [ 77.514259] nix-grpc-daemon[786]: ts=2026-09-25T15:13:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=1 drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv assign_id=11 outputs=04991worker2 # [ 77.525612] nix-daemon[873]: reaped child process 1105, status = succeeded4992worker2 # [ 77.526859] nix-daemon[873]: reaped child process 1103, status = succeeded4993worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4994worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4995worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4996worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4997worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4998worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4999worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5000worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5001worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5002worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5003worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5004worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5005worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5006worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5007worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5008worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5009(finished: subtest: a failing build is counted with its own reason, in 1.97 seconds)5010subtest: the first client leaving does not fail the second5011client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5012client # [ 77.928955] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5013client # Running as unit: share1.service; invocation ID: cbc1050e836b4baca156c3f9bdfd5f515014client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5015client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5016worker2 # [ 78.067090] nix-grpc-daemon[786]: ts=2026-09-25T15:13:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380665017client # [ 78.079114] nix[1064]: don't know how to build these paths:5018client # [ 78.079953] nix[1064]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5019worker2 # [ 78.080157] nix-grpc-daemon[786]: ts=2026-09-25T15:13:56Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05020worker2 # [ 78.084355] nix-grpc-daemon[786]: ts=2026-09-25T15:13:56Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=15021worker1 # [ 78.110275] nix-grpc-daemon[511]: ts=2026-09-25T15:13:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550585022worker1 # [ 78.121994] nix-grpc-daemon[511]: ts=2026-09-25T15:13:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550585023client # [ 78.120114] nix[1064]: copying 1 paths to 'lb:50051'...5024client # [ 78.120893] nix[1064]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5025worker1 # [ 78.133259] nix-grpc-daemon[511]: ts=2026-09-25T15:13:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550585026worker1 # [ 78.143864] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5027worker1 # [ 78.246170] nix-daemon[848]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5028worker1 # [ 78.258735] nix-daemon[848]: reaped child process 1114, status = succeeded5029lb # [ 78.264330] niks3-server[955]: 2026/09/25 15:13:57 INFO Received push request method=POST path=/api/pushes5030worker1 # [ 78.275436] nix-grpc-daemon[861]: time=2026-09-25T15:13:56.527Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5031worker1 # [ 78.277317] nix-grpc-daemon[861]: time=2026-09-25T15:13:56.528Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5032lb # [ 78.296169] niks3-server[955]: 2026/09/25 15:13:57 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign5033worker1 # [ 78.304579] nix-grpc-daemon[861]: time=2026-09-25T15:13:56.556Z level=INFO msg="Uploading 1 narinfos"5034lb # [ 78.301608] niks3-server[955]: 2026/09/25 15:13:57 INFO Signed narinfos id=20 count=15035lb # [ 78.308782] niks3-server[955]: 2026/09/25 15:13:57 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5036lb # [ 78.314464] niks3-server[955]: 2026/09/25 15:13:57 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5037lb # [ 78.326501] niks3-server[955]: 2026/09/25 15:13:57 INFO Received complete push request method=POST path=/api/pushes/20/complete5038worker1 # [ 78.336796] nix-grpc-daemon[861]: time=2026-09-25T15:13:56.588Z level=INFO msg="Upload complete. (181ms)"5039lb # [ 78.335428] niks3-server[955]: 2026/09/25 15:13:57 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5040worker1 # [ 78.341461] nix-grpc-daemon[511]: ts=2026-09-25T15:13:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=1 nar_bytes_in=4325041worker1 # [ 78.347251] nix-grpc-daemon[511]: ts=2026-09-25T15:13:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550585042worker1 # [ 78.360178] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5043worker1 # [ 78.364919] nix-daemon[848]: reaped child process 1109, status = succeeded5044worker1 # [ 78.371771] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5045client # [ 78.359665] nix[1064]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5046client # [ 79.002947] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5047client # Running as unit: share2.service; invocation ID: c2f4ab5dca5742a18df86f63f05e9bbf5048client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5049worker2 # [ 79.139609] nix-grpc-daemon[786]: ts=2026-09-25T15:13:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380665050client # [ 79.151277] nix[1080]: don't know how to build these paths:5051client # [ 79.152177] nix[1080]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5052worker2 # [ 79.151719] nix-grpc-daemon[786]: ts=2026-09-25T15:13:57Z level=debug event=want client=11 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05053worker1 # [ 79.172798] nix-grpc-daemon[511]: ts=2026-09-25T15:13:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550585054client # [ 79.165802] nix[1080]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5055client: must succeed: sleep 2; systemctl kill -s INT share15056client # [ 82.257935] systemd[1]: share1.service: Sent signal SIGINT to main process 1064 (nix) on client request.5057client # [ 82.263738] nix[1064]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5058client # [ 82.269959] nix[1064]: error: interrupted by the user5059client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5060client: waiting for success: ! systemctl is-active share15061client # [ 82.311694] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5062client # [ 82.313163] systemd[1]: share1.service: Failed with result 'exit-code'.5063client # [ 82.315090] systemd[1]: share1.service: Consumed 115ms CPU time over 4.386s wall clock time, 41.5M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.5064client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5065client: waiting for success: ! systemctl is-active share25066worker1 # [ 83.600341] nix-daemon[848]: reaped child process 1118, status = succeeded5067worker1 # [ 83.680716] nix-daemon[848]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)5068worker1 # [ 83.693412] nix-daemon[848]: reaped child process 1155, status = succeeded5069lb # [ 83.698818] niks3-server[955]: 2026/09/25 15:14:02 INFO Received push request method=POST path=/api/pushes5070worker1 # [ 83.709347] nix-grpc-daemon[861]: time=2026-09-25T15:14:01.961Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5071worker1 # [ 83.710986] nix-grpc-daemon[861]: time=2026-09-25T15:14:01.962Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5072lb # [ 83.744298] niks3-server[955]: 2026/09/25 15:14:02 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5073lb # [ 83.750348] niks3-server[955]: 2026/09/25 15:14:02 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign5074worker1 # [ 83.759866] nix-grpc-daemon[861]: time=2026-09-25T15:14:02.011Z level=INFO msg="Uploading 1 narinfos"5075lb # [ 83.757049] niks3-server[955]: 2026/09/25 15:14:02 INFO Signed narinfos id=21 count=15076lb # [ 83.765516] niks3-server[955]: 2026/09/25 15:14:02 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5077lb # [ 83.771781] niks3-server[955]: 2026/09/25 15:14:02 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5078lb # [ 83.780159] niks3-server[955]: 2026/09/25 15:14:02 INFO Received complete push request method=POST path=/api/pushes/21/complete5079worker1 # [ 83.785813] nix-grpc-daemon[861]: time=2026-09-25T15:14:02.037Z level=INFO msg="Upload complete. (202ms)"5080lb # [ 83.786299] niks3-server[955]: 2026/09/25 15:14:02 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5081worker1 # [ 83.792756] nix-grpc-daemon[511]: ts=2026-09-25T15:14:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=12 outputs=15082worker1 # [ 83.800354] nix-grpc-daemon[511]: ts=2026-09-25T15:14:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=12 outputs=15083worker1 # [ 83.812742] nix-daemon[848]: reaped child process 1116, status = succeeded5084client # [ 83.813179] systemd[1]: share2.service: Deactivated successfully.5085client # [ 83.815741] systemd[1]: share2.service: Consumed 119ms CPU time over 4.811s wall clock time, 33.4M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.5086client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5087client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5088client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5089(finished: subtest: the first client leaving does not fail the second, in 6.75 seconds)5090subtest: build hook: nix-daemon with builders = grpc://lb5091client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5092client # [ 84.728167] systemd[1]: Started Nix Daemon.5093client # [ 84.813764] nix-daemon[1131]: accepted connection from pid 1128, user root (trusted)5094worker1 # [ 84.939318] nix-grpc-daemon[511]: ts=2026-09-25T15:14:03Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:550585095worker1 # [ 84.944564] nix-grpc-daemon[511]: ts=2026-09-25T15:14:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550585096worker2 # [ 84.941929] nix-grpc-daemon[786]: ts=2026-09-25T15:14:03Z level=debug event=want client=12 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05097worker2 # [ 84.946389] nix-grpc-daemon[786]: ts=2026-09-25T15:14:03Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15098worker2 # [ 84.950661] nix-grpc-daemon[786]: ts=2026-09-25T15:14:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380665099worker2 # [ 84.961200] nix-grpc-daemon[786]: ts=2026-09-25T15:14:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380665100worker2 # [ 84.971901] nix-grpc-daemon[786]: ts=2026-09-25T15:14:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380665101worker2 # [ 84.982386] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5102worker2 # [ 85.087164] nix-daemon[873]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)5103worker2 # [ 85.101252] nix-daemon[873]: reaped child process 1160, status = succeeded5104lb # [ 85.124457] niks3-server[955]: 2026/09/25 15:14:04 INFO Received push request method=POST path=/api/pushes5105worker2 # [ 85.117330] nix-grpc-daemon[924]: time=2026-09-25T15:14:03.427Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5106worker2 # [ 85.119371] nix-grpc-daemon[924]: time=2026-09-25T15:14:03.429Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5107lb # [ 85.153458] niks3-server[955]: 2026/09/25 15:14:04 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign5108worker2 # [ 85.145449] nix-grpc-daemon[924]: time=2026-09-25T15:14:03.456Z level=INFO msg="Uploading 1 narinfos"5109lb # [ 85.161130] niks3-server[955]: 2026/09/25 15:14:04 INFO Signed narinfos id=22 count=15110lb # [ 85.165908] niks3-server[955]: 2026/09/25 15:14:04 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5111lb # [ 85.173332] niks3-server[955]: 2026/09/25 15:14:04 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5112lb # [ 85.185383] niks3-server[955]: 2026/09/25 15:14:04 INFO Received complete push request method=POST path=/api/pushes/22/complete5113worker2 # [ 85.175415] nix-grpc-daemon[924]: time=2026-09-25T15:14:03.486Z level=INFO msg="Upload complete. (181ms)"5114lb # [ 85.193061] niks3-server[955]: 2026/09/25 15:14:04 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5115worker2 # [ 85.178880] nix-grpc-daemon[786]: ts=2026-09-25T15:14:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_in=5125116worker2 # [ 85.185156] nix-grpc-daemon[786]: ts=2026-09-25T15:14:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380665117worker2 # [ 85.193602] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5118worker2 # [ 85.198793] nix-daemon[873]: reaped child process 1155, status = succeeded5119worker2 # [ 85.203862] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5120worker2 # [ 86.640994] nix-daemon[873]: reaped child process 1164, status = succeeded5121worker2 # [ 86.734974] nix-daemon[873]: accepted connection from pid 1180, user nix-grpc-daemon (trusted)5122worker2 # [ 86.747376] nix-daemon[873]: reaped child process 1183, status = succeeded5123lb # [ 86.770981] niks3-server[955]: 2026/09/25 15:14:05 INFO Received push request method=POST path=/api/pushes5124worker2 # [ 86.765908] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.076Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5125worker2 # [ 86.767349] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.078Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5126lb # [ 86.819157] niks3-server[955]: 2026/09/25 15:14:06 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign5127lb # [ 86.822149] niks3-server[955]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5128worker2 # [ 86.812921] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.123Z level=INFO msg="Uploading 1 narinfos"5129lb # [ 86.827776] niks3-server[955]: 2026/09/25 15:14:06 INFO Signed narinfos id=23 count=15130lb # [ 86.833444] niks3-server[955]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5131lb # [ 86.843524] niks3-server[955]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5132lb # [ 86.849325] niks3-server[955]: 2026/09/25 15:14:06 INFO Received complete push request method=POST path=/api/pushes/23/complete5133lb # [ 86.862161] niks3-server[955]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5134worker2 # [ 86.846333] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.156Z level=INFO msg="Upload complete. (234ms)"5135worker2 # [ 86.854422] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=13 outputs=15136worker2 # [ 86.861127] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:380665137worker2 # [ 86.873573] nix-daemon[873]: reaped child process 1162, status = succeeded5138worker1 # [ 86.896784] nix-grpc-daemon[511]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550585139worker1 # [ 86.906924] nix-grpc-daemon[511]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5140worker1 # [ 86.918931] nix-grpc-daemon[511]: ts=2026-09-25T15:14:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=1 nar_bytes_out=1285141lb # [ 87.068657] envoy[924]: conn peer=[2001:db8:1::1]:33720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2866 tx=1272 ms=14405142lb # [ 87.074251] envoy[924]: conn peer=[2001:db8:1::1]:33726 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=794 ms=42085143lb # [ 87.078936] envoy[924]: conn peer=[2001:db8:1::1]:33740 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=46455144lb # [ 87.082729] envoy[924]: conn peer=[2001:db8:1::1]:45650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3407 tx=1948 ms=19925145lb # [ 87.086316] envoy[924]: conn peer=[2001:db8:1::1]:45666 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=334 ms=345146worker1 # [ 87.095988] nix-grpc-daemon[511]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:550585147worker2 # [ 87.086133] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380665148worker2 # [ 87.095179] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.405Z level=INFO msg="All 1 paths already cached"5149worker2 # [ 87.109249] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=debug event=want client=13 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05150worker2 # [ 87.114488] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15151worker2 # [ 87.120471] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380665152worker2 # [ 87.131465] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380665153worker2 # [ 87.143427] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.454Z level=INFO msg="All 1 paths already cached"5154worker2 # [ 87.151233] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380665155worker2 # [ 87.159561] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5156worker2 # [ 87.266345] nix-daemon[873]: accepted connection from pid 1189, user nix-grpc-daemon (trusted)5157worker2 # [ 87.281206] nix-daemon[873]: reaped child process 1192, status = succeeded5158lb # [ 87.304921] niks3-server[955]: 2026/09/25 15:14:06 INFO Received push request method=POST path=/api/pushes5159worker2 # [ 87.297988] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.608Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5160worker2 # [ 87.299510] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.610Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5161lb # [ 87.335229] niks3-server[955]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5162lb # [ 87.345960] niks3-server[955]: 2026/09/25 15:14:06 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign5163worker2 # [ 87.335479] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.646Z level=INFO msg="Uploading 1 narinfos"5164lb # [ 87.350760] niks3-server[955]: 2026/09/25 15:14:06 INFO Signed narinfos id=24 count=15165lb # [ 87.358109] niks3-server[955]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5166lb # [ 87.370124] niks3-server[955]: 2026/09/25 15:14:06 INFO Received complete push request method=POST path=/api/pushes/24/complete5167worker2 # [ 87.364324] nix-grpc-daemon[924]: time=2026-09-25T15:14:05.674Z level=INFO msg="Upload complete. (192ms)"5168lb # [ 87.380083] niks3-server[955]: 2026/09/25 15:14:06 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5169worker2 # [ 87.369267] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_in=6245170worker2 # [ 87.375200] nix-grpc-daemon[786]: ts=2026-09-25T15:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380665171worker2 # [ 87.385649] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5172worker2 # [ 87.393488] nix-daemon[873]: reaped child process 1187, status = succeeded5173worker2 # [ 87.399470] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5174worker2 # [ 88.617172] nix-daemon[873]: reaped child process 1196, status = succeeded5175worker2 # [ 88.692825] nix-daemon[873]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)5176worker2 # [ 88.706582] nix-daemon[873]: reaped child process 1215, status = succeeded5177lb # [ 88.728982] niks3-server[955]: 2026/09/25 15:14:07 INFO Received push request method=POST path=/api/pushes5178worker2 # [ 88.722460] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.032Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5179worker2 # [ 88.725056] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.034Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5180lb # [ 88.762235] niks3-server[955]: 2026/09/25 15:14:07 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5181lb # [ 88.776291] niks3-server[955]: 2026/09/25 15:14:07 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign5182lb # [ 88.785606] niks3-server[955]: 2026/09/25 15:14:07 INFO Signed narinfos id=25 count=15183worker2 # [ 88.773081] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.083Z level=INFO msg="Uploading 1 narinfos"5184lb # [ 88.789711] niks3-server[955]: 2026/09/25 15:14:07 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5185lb # [ 88.794156] niks3-server[955]: 2026/09/25 15:14:07 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5186lb # [ 88.805264] niks3-server[955]: 2026/09/25 15:14:07 INFO Received complete push request method=POST path=/api/pushes/25/complete5187worker2 # [ 88.796287] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.106Z level=INFO msg="Upload complete. (190ms)"5188lb # [ 88.813555] niks3-server[955]: 2026/09/25 15:14:08 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5189worker2 # [ 88.803076] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=14 outputs=15190worker2 # [ 88.811611] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:380665191worker2 # [ 88.822624] nix-daemon[873]: reaped child process 1194, status = succeeded5192worker2 # [ 88.828328] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:380665193worker2 # [ 88.832406] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_out=1765194worker2 # [ 88.999118] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:380665195worker2 # [ 89.005183] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380665196worker2 # [ 89.011073] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.320Z level=INFO msg="All 2 paths already cached"5197worker2 # [ 89.020621] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=want client=14 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05198worker2 # [ 89.025727] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15199worker2 # [ 89.031378] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380665200worker2 # [ 89.043284] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380665201worker2 # [ 89.056693] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.367Z level=INFO msg="All 2 paths already cached"5202worker2 # [ 89.064503] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380665203worker2 # [ 89.074173] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5204worker2 # [ 89.172462] nix-daemon[873]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5205worker2 # [ 89.186350] nix-daemon[873]: reaped child process 1224, status = succeeded5206lb # [ 89.208482] niks3-server[955]: 2026/09/25 15:14:08 INFO Received push request method=POST path=/api/pushes5207worker2 # [ 89.202765] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.513Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5208worker2 # [ 89.204876] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.515Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5209lb # [ 89.242165] niks3-server[955]: 2026/09/25 15:14:08 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign5210lb # [ 89.247606] niks3-server[955]: 2026/09/25 15:14:08 INFO Signed narinfos id=26 count=15211worker2 # [ 89.235842] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.546Z level=INFO msg="Uploading 1 narinfos"5212lb # [ 89.256219] niks3-server[955]: 2026/09/25 15:14:08 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5213lb # [ 89.263205] niks3-server[955]: 2026/09/25 15:14:08 INFO Received complete push request method=POST path=/api/pushes/26/complete5214worker2 # [ 89.256536] nix-grpc-daemon[924]: time=2026-09-25T15:14:07.567Z level=INFO msg="Upload complete. (170ms)"5215lb # [ 89.274540] niks3-server[955]: 2026/09/25 15:14:08 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5216worker2 # [ 89.260070] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_in=7525217lb # [ 89.277640] niks3-server[955]: 2026/09/25 15:14:08 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5218worker2 # [ 89.265620] nix-grpc-daemon[786]: ts=2026-09-25T15:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380665219worker2 # [ 89.274933] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5220worker2 # [ 89.278844] nix-daemon[873]: reaped child process 1219, status = succeeded5221worker2 # [ 89.283817] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5222worker2 # [ 89.901538] nix-daemon[873]: reaped child process 1228, status = succeeded5223worker2 # [ 89.975932] nix-daemon[873]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5224worker2 # [ 89.991859] nix-daemon[873]: reaped child process 1247, status = succeeded5225lb # [ 90.013456] niks3-server[955]: 2026/09/25 15:14:09 INFO Received push request method=POST path=/api/pushes5226worker2 # [ 90.007232] nix-grpc-daemon[924]: time=2026-09-25T15:14:08.317Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5227worker2 # [ 90.008674] nix-grpc-daemon[924]: time=2026-09-25T15:14:08.319Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5228lb # [ 90.057487] niks3-server[955]: 2026/09/25 15:14:09 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign5229worker2 # [ 90.050828] nix-grpc-daemon[924]: time=2026-09-25T15:14:08.361Z level=INFO msg="Uploading 1 narinfos"5230lb # [ 90.066606] niks3-server[955]: 2026/09/25 15:14:09 INFO Signed narinfos id=27 count=15231lb # [ 90.070985] niks3-server[955]: 2026/09/25 15:14:09 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5232lb # [ 90.078638] niks3-server[955]: 2026/09/25 15:14:09 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5233lb # [ 90.082936] niks3-server[955]: 2026/09/25 15:14:09 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5234lb # [ 90.096688] niks3-server[955]: 2026/09/25 15:14:09 INFO Received complete push request method=POST path=/api/pushes/27/complete5235worker2 # [ 90.085385] nix-grpc-daemon[924]: time=2026-09-25T15:14:08.396Z level=INFO msg="Upload complete. (195ms)"5236lb # [ 90.102631] niks3-server[955]: 2026/09/25 15:14:09 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5237worker2 # [ 90.092989] nix-grpc-daemon[786]: ts=2026-09-25T15:14:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=15 outputs=15238worker1 # [ 90.115699] nix-grpc-daemon[511]: ts=2026-09-25T15:14:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550585239worker2 # [ 90.108263] nix-daemon[873]: reaped child process 1226, status = succeeded5240worker1 # [ 90.127721] nix-grpc-daemon[511]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5241worker1 # [ 90.137275] nix-grpc-daemon[511]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5242worker1 # [ 90.160825] nix-grpc-daemon[511]: ts=2026-09-25T15:14:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550585243worker1 # [ 90.166185] nix-grpc-daemon[511]: ts=2026-09-25T15:14:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=1 nar_bytes_out=2325244client # [ 90.223658] nix-daemon[1131]: reaped child process 1133, status = succeeded5245client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.60 seconds)5246client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5247client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5248client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5249client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5250client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5251client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5252(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.65 seconds)5253subtest: an input only one worker has still reaches the builder5254client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5255client # this derivation will be built:5256client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5257client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5258client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.24 seconds)5259client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5260client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5261worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5262worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.12 seconds)5263worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5264worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5265client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25266client # [ 90.877797] nix-daemon[1131]: accepted connection from pid 1214, user root (trusted)5267client # this derivation will be built:5268client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5269worker1 # [ 91.003601] nix-grpc-daemon[511]: ts=2026-09-25T15:14:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:550585270client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5271worker2 # [ 90.989868] nix-grpc-daemon[786]: ts=2026-09-25T15:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380665272client # copying 1 paths...5273client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5274worker2 # [ 91.002505] nix-grpc-daemon[786]: ts=2026-09-25T15:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380665275worker2 # [ 91.014187] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5276worker2 # [ 91.121438] nix-daemon[873]: accepted connection from pid 1254, user nix-grpc-daemon (trusted)5277worker2 # [ 91.137130] nix-daemon[873]: reaped child process 1257, status = succeeded5278lb # [ 91.157341] niks3-server[955]: 2026/09/25 15:14:10 INFO Received push request method=POST path=/api/pushes5279worker2 # [ 91.152194] nix-grpc-daemon[924]: time=2026-09-25T15:14:09.462Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5280worker2 # [ 91.154183] nix-grpc-daemon[924]: time=2026-09-25T15:14:09.464Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5281lb # [ 91.190348] niks3-server[955]: 2026/09/25 15:14:10 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5282lb # [ 91.197384] niks3-server[955]: 2026/09/25 15:14:10 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign5283lb # [ 91.201914] niks3-server[955]: 2026/09/25 15:14:10 INFO Signed narinfos id=28 count=15284worker2 # [ 91.190088] nix-grpc-daemon[924]: time=2026-09-25T15:14:09.500Z level=INFO msg="Uploading 1 narinfos"5285lb # [ 91.213155] niks3-server[955]: 2026/09/25 15:14:10 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5286lb # [ 91.222887] niks3-server[955]: 2026/09/25 15:14:10 INFO Received complete push request method=POST path=/api/pushes/28/complete5287lb # [ 91.234038] niks3-server[955]: 2026/09/25 15:14:10 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5288worker2 # [ 91.217935] nix-grpc-daemon[924]: time=2026-09-25T15:14:09.528Z level=INFO msg="Upload complete. (191ms)"5289worker2 # [ 91.224613] nix-grpc-daemon[786]: ts=2026-09-25T15:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_in=1205290worker2 # [ 91.236124] nix-grpc-daemon[786]: ts=2026-09-25T15:14:09Z level=debug event=want client=15 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05291worker2 # [ 91.240238] nix-grpc-daemon[786]: ts=2026-09-25T15:14:09Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15292worker1 # [ 91.261968] nix-grpc-daemon[511]: ts=2026-09-25T15:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550585293worker2 # [ 91.249681] nix-daemon[873]: reaped child process 1252, status = succeeded5294worker1 # [ 91.272511] nix-grpc-daemon[511]: ts=2026-09-25T15:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550585295client # copying 2 paths to 'lb:50051'...5296client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5297worker1 # [ 91.287568] nix-grpc-daemon[511]: ts=2026-09-25T15:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550585298client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5299worker1 # [ 91.302257] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5300worker1 # [ 91.405860] nix-daemon[848]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)5301worker1 # [ 91.418357] nix-daemon[848]: reaped child process 1170, status = succeeded5302lb # [ 91.424067] niks3-server[955]: 2026/09/25 15:14:10 INFO Received push request method=POST path=/api/pushes5303worker1 # [ 91.435329] nix-grpc-daemon[861]: time=2026-09-25T15:14:09.687Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5304worker1 # [ 91.439156] nix-grpc-daemon[861]: time=2026-09-25T15:14:09.688Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5305worker1 # [ 91.440876] nix-grpc-daemon[861]: time=2026-09-25T15:14:09.688Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5306lb # [ 91.467995] niks3-server[955]: 2026/09/25 15:14:10 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5307lb # [ 91.475554] niks3-server[955]: 2026/09/25 15:14:10 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5308lb # [ 91.483101] niks3-server[955]: 2026/09/25 15:14:10 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign5309lb # [ 91.487542] niks3-server[955]: 2026/09/25 15:14:10 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5310lb # [ 91.493863] niks3-server[955]: 2026/09/25 15:14:10 INFO Signed narinfos id=29 count=25311worker1 # [ 91.498777] nix-grpc-daemon[861]: time=2026-09-25T15:14:09.750Z level=INFO msg="Uploading 2 narinfos"5312lb # [ 91.498515] niks3-server[955]: 2026/09/25 15:14:10 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5313lb # [ 91.505562] niks3-server[955]: 2026/09/25 15:14:10 INFO Received complete push request method=POST path=/api/pushes/29/complete5314lb # [ 91.510519] niks3-server[955]: 2026/09/25 15:14:10 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5315lb # [ 91.513678] niks3-server[955]: 2026/09/25 15:14:10 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5316worker1 # [ 91.518534] nix-grpc-daemon[861]: time=2026-09-25T15:14:09.770Z level=INFO msg="Upload complete. (203ms)"5317worker1 # [ 91.524447] nix-grpc-daemon[511]: ts=2026-09-25T15:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=2 nar_bytes_in=9765318worker1 # [ 91.531305] nix-grpc-daemon[511]: ts=2026-09-25T15:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550585319worker1 # [ 91.543992] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5320worker1 # [ 91.550356] nix-daemon[848]: reaped child process 1165, status = succeeded5321worker1 # [ 91.556215] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5322client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5323client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5324client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5325worker1 # [ 92.787799] nix-daemon[848]: reaped child process 1174, status = succeeded5326worker1 # [ 92.876242] nix-daemon[848]: accepted connection from pid 1190, user nix-grpc-daemon (trusted)5327worker1 # [ 92.888805] nix-daemon[848]: reaped child process 1193, status = succeeded5328lb # [ 92.894435] niks3-server[955]: 2026/09/25 15:14:12 INFO Received push request method=POST path=/api/pushes5329worker1 # [ 92.906798] nix-grpc-daemon[861]: time=2026-09-25T15:14:11.158Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5330lb # [ 92.926916] niks3-server[955]: 2026/09/25 15:14:12 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign5331lb # [ 92.934682] niks3-server[955]: 2026/09/25 15:14:12 INFO Signed narinfos id=30 count=15332worker1 # [ 92.941185] nix-grpc-daemon[861]: time=2026-09-25T15:14:11.193Z level=INFO msg="Uploading 1 narinfos"5333lb # [ 92.944717] niks3-server[955]: 2026/09/25 15:14:12 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5334lb # [ 92.950161] niks3-server[955]: 2026/09/25 15:14:12 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5335lb # [ 92.959201] niks3-server[955]: 2026/09/25 15:14:12 INFO Received complete push request method=POST path=/api/pushes/30/complete5336worker1 # [ 92.972397] nix-grpc-daemon[861]: time=2026-09-25T15:14:11.224Z level=INFO msg="Upload complete. (208ms)"5337lb # [ 92.972531] niks3-server[955]: 2026/09/25 15:14:12 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5338client # copying 1 paths...5339worker1 # [ 92.976763] nix-grpc-daemon[511]: ts=2026-09-25T15:14:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=16 outputs=15340worker1 # [ 92.984429] nix-grpc-daemon[511]: ts=2026-09-25T15:14:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550585341client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5342worker1 # [ 92.995215] nix-daemon[848]: reaped child process 1172, status = succeeded5343worker1 # [ 92.997699] nix-grpc-daemon[511]: ts=2026-09-25T15:14:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550585344worker1 # [ 93.001789] nix-grpc-daemon[511]: ts=2026-09-25T15:14:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=1 nar_bytes_out=1205345client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.27 seconds)5346client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25347client # [ 93.042991] nix-daemon[1131]: reaped child process 1217, status = succeeded5348client # [ 93.127593] nix-daemon[1131]: accepted connection from pid 1236, user root (trusted)5349client # this derivation will be built:5350client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5351worker1 # [ 93.257976] nix-grpc-daemon[511]: ts=2026-09-25T15:14:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:550585352client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5353worker1 # [ 93.263566] nix-grpc-daemon[511]: ts=2026-09-25T15:14:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550585354worker1 # [ 93.273463] nix-grpc-daemon[861]: time=2026-09-25T15:14:11.525Z level=INFO msg="All 1 paths already cached"5355worker2 # [ 93.272831] nix-grpc-daemon[786]: ts=2026-09-25T15:14:11Z level=debug event=want client=16 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05356worker2 # [ 93.278369] nix-grpc-daemon[786]: ts=2026-09-25T15:14:11Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15357worker2 # [ 93.282858] nix-grpc-daemon[786]: ts=2026-09-25T15:14:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380665358worker2 # [ 93.294285] nix-grpc-daemon[786]: ts=2026-09-25T15:14:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380665359client # copying 1 paths to 'lb:50051'...5360client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5361worker2 # [ 93.311071] nix-grpc-daemon[786]: ts=2026-09-25T15:14:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380665362worker2 # [ 93.322240] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5363worker2 # [ 93.338072] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5)5364worker2 # [ 93.356169] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)5365worker2 # [ 93.366717] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 395 ms (attempt 3/5)5366worker2 # [ 93.769416] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 574 ms (attempt 4/5)5367worker2 # [ 94.346935] nix-grpc-daemon[786]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5368worker2 # [ 94.350521] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5369worker2 # [ 94.355742] nix-grpc-daemon[786]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5370worker2 # [ 94.474415] nix-daemon[873]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5371worker2 # [ 94.493237] nix-daemon[873]: reaped child process 1273, status = succeeded5372lb # [ 94.512756] niks3-server[955]: 2026/09/25 15:14:13 INFO Received push request method=POST path=/api/pushes5373worker2 # [ 94.507899] nix-grpc-daemon[924]: time=2026-09-25T15:14:12.818Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5374worker2 # [ 94.509485] nix-grpc-daemon[924]: time=2026-09-25T15:14:12.820Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5375lb # [ 94.546931] niks3-server[955]: 2026/09/25 15:14:13 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign5376lb # [ 94.557270] niks3-server[955]: 2026/09/25 15:14:13 INFO Signed narinfos id=31 count=15377worker2 # [ 94.545749] nix-grpc-daemon[924]: time=2026-09-25T15:14:12.856Z level=INFO msg="Uploading 1 narinfos"5378lb # [ 94.562442] niks3-server[955]: 2026/09/25 15:14:13 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5379lb # [ 94.568287] niks3-server[955]: 2026/09/25 15:14:13 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5380lb # [ 94.580934] niks3-server[955]: 2026/09/25 15:14:13 INFO Received complete push request method=POST path=/api/pushes/31/complete5381worker2 # [ 94.574931] nix-grpc-daemon[924]: time=2026-09-25T15:14:12.885Z level=INFO msg="Upload complete. (198ms)"5382lb # [ 94.590623] niks3-server[955]: 2026/09/25 15:14:13 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5383worker2 # [ 94.580166] nix-grpc-daemon[786]: ts=2026-09-25T15:14:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=1 paths=1 nar_bytes_in=5605384worker2 # [ 94.586333] nix-grpc-daemon[786]: ts=2026-09-25T15:14:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380665385worker2 # [ 94.601177] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5386worker2 # [ 94.609597] nix-daemon[873]: reaped child process 1263, status = succeeded5387worker2 # [ 94.616197] nix-daemon[873]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5388client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5389client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5390client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5391worker2 # [ 95.432247] nix-daemon[873]: reaped child process 1277, status = succeeded5392worker2 # [ 95.510962] nix-daemon[873]: accepted connection from pid 1293, user nix-grpc-daemon (trusted)5393worker2 # [ 95.525376] nix-daemon[873]: reaped child process 1296, status = succeeded5394lb # [ 95.547402] niks3-server[955]: 2026/09/25 15:14:14 INFO Received push request method=POST path=/api/pushes5395worker2 # [ 95.541860] nix-grpc-daemon[924]: time=2026-09-25T15:14:13.852Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5396lb # [ 95.585434] niks3-server[955]: 2026/09/25 15:14:14 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign5397lb # [ 95.591750] niks3-server[955]: 2026/09/25 15:14:14 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5398worker2 # [ 95.585938] nix-grpc-daemon[924]: time=2026-09-25T15:14:13.896Z level=INFO msg="Uploading 1 narinfos"5399lb # [ 95.600871] niks3-server[955]: 2026/09/25 15:14:14 INFO Signed narinfos id=32 count=15400lb # [ 95.605568] niks3-server[955]: 2026/09/25 15:14:14 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5401lb # [ 95.621052] niks3-server[955]: 2026/09/25 15:14:14 INFO Received complete push request method=POST path=/api/pushes/32/complete5402lb # [ 95.629779] niks3-server[955]: 2026/09/25 15:14:14 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5403worker2 # [ 95.618634] nix-grpc-daemon[924]: time=2026-09-25T15:14:13.929Z level=INFO msg="Upload complete. (195ms)"5404worker2 # [ 95.625896] nix-grpc-daemon[786]: ts=2026-09-25T15:14:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=17 outputs=15405client # copying 1 paths...5406worker2 # [ 95.635428] nix-grpc-daemon[786]: ts=2026-09-25T15:14:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:380665407client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5408worker2 # [ 95.646410] nix-daemon[873]: reaped child process 1275, status = succeeded5409worker2 # [ 95.655595] nix-grpc-daemon[786]: ts=2026-09-25T15:14:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:380665410worker2 # [ 95.659537] nix-grpc-daemon[786]: ts=2026-09-25T15:14:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_out=1205411client # [ 95.733697] nix-daemon[1131]: reaped child process 1239, status = succeeded5412client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.70 seconds)5413client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25414client # [ 95.833082] nix-daemon[1131]: accepted connection from pid 1258, user root (trusted)5415client # this derivation will be built:5416client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5417worker2 # [ 95.945249] nix-grpc-daemon[786]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:380665418client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5419worker1 # [ 95.967097] nix-grpc-daemon[511]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550585420worker1 # [ 95.980796] nix-grpc-daemon[861]: time=2026-09-25T15:14:14.232Z level=INFO msg="All 1 paths already cached"5421worker2 # [ 95.978156] nix-grpc-daemon[786]: ts=2026-09-25T15:14:14Z level=debug event=want client=17 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05422worker2 # [ 95.982380] nix-grpc-daemon[786]: ts=2026-09-25T15:14:14Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15423worker1 # [ 96.007841] nix-grpc-daemon[511]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550585424worker1 # [ 96.021444] nix-grpc-daemon[511]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550585425worker1 # [ 96.035629] nix-grpc-daemon[861]: time=2026-09-25T15:14:14.287Z level=INFO msg="All 1 paths already cached"5426client # copying 1 paths to 'lb:50051'...5427client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5428worker1 # [ 96.045094] nix-grpc-daemon[511]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550585429worker1 # [ 96.056896] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5430worker1 # [ 96.161188] nix-daemon[848]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5431worker1 # [ 96.175620] nix-daemon[848]: reaped child process 1200, status = succeeded5432lb # [ 96.180087] niks3-server[955]: 2026/09/25 15:14:15 INFO Received push request method=POST path=/api/pushes5433worker1 # [ 96.191949] nix-grpc-daemon[861]: time=2026-09-25T15:14:14.443Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5434worker1 # [ 96.193465] nix-grpc-daemon[861]: time=2026-09-25T15:14:14.445Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5435lb # [ 96.215389] niks3-server[955]: 2026/09/25 15:14:15 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign5436lb # [ 96.224289] niks3-server[955]: 2026/09/25 15:14:15 INFO Signed narinfos id=33 count=15437worker1 # [ 96.230058] nix-grpc-daemon[861]: time=2026-09-25T15:14:14.481Z level=INFO msg="Uploading 1 narinfos"5438lb # [ 96.236392] niks3-server[955]: 2026/09/25 15:14:15 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5439lb # [ 96.241728] niks3-server[955]: 2026/09/25 15:14:15 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5440lb # [ 96.252114] niks3-server[955]: 2026/09/25 15:14:15 INFO Received complete push request method=POST path=/api/pushes/33/complete5441lb # [ 96.258706] niks3-server[955]: 2026/09/25 15:14:15 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5442worker1 # [ 96.265361] nix-grpc-daemon[861]: time=2026-09-25T15:14:14.517Z level=INFO msg="Upload complete. (196ms)"5443worker1 # [ 96.270611] nix-grpc-daemon[511]: ts=2026-09-25T15:14:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=0 paths=1 nar_bytes_in=5605444worker1 # [ 96.277320] nix-grpc-daemon[511]: ts=2026-09-25T15:14:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550585445worker1 # [ 96.287613] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5446worker1 # [ 96.299941] nix-daemon[848]: reaped child process 1195, status = succeeded5447worker1 # [ 96.302972] nix-daemon[848]: accepted connection from pid 511, user nix-grpc-daemon (trusted)5448client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5449client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5450lb # [ 97.070646] envoy[924]: conn peer=[2001:db8:1::1]:45672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=215451lb # [ 97.074521] envoy[924]: conn peer=[2001:db8:1::1]:45670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3863 tx=2164 ms=17855452lb # [ 97.078226] envoy[924]: conn peer=[2001:db8:1::1]:45676 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4281 tx=2380 ms=11655453lb # [ 97.081782] envoy[924]: conn peer=[2001:db8:1::1]:45678 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=185454lb # [ 97.085258] envoy[924]: conn peer=[2001:db8:1::1]:45684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4215 tx=2050 ms=20065455lb # [ 97.088771] envoy[924]: conn peer=[2001:db8:1::1]:41910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=460 tx=328 ms=115456lb # [ 97.093184] envoy[924]: conn peer=[2001:db8:1::1]:41918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3742 tx=2123 ms=24325457lb # [ 97.096888] envoy[924]: conn peer=[2001:db8:1::1]:41922 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=460 tx=307 ms=155458client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5459worker1 # [ 97.718856] nix-daemon[848]: reaped child process 1204, status = succeeded5460worker1 # [ 97.795469] nix-daemon[848]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)5461worker1 # [ 97.806840] nix-daemon[848]: reaped child process 1223, status = succeeded5462lb # [ 97.813623] niks3-server[955]: 2026/09/25 15:14:17 INFO Received push request method=POST path=/api/pushes5463worker1 # [ 97.825319] nix-grpc-daemon[861]: time=2026-09-25T15:14:16.077Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5464lb # [ 97.848211] niks3-server[955]: 2026/09/25 15:14:17 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5465lb # [ 97.854645] niks3-server[955]: 2026/09/25 15:14:17 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5466lb # [ 97.859737] niks3-server[955]: 2026/09/25 15:14:17 INFO Signed narinfos id=34 count=15467worker1 # [ 97.865818] nix-grpc-daemon[861]: time=2026-09-25T15:14:16.117Z level=INFO msg="Uploading 1 narinfos"5468lb # [ 97.869762] niks3-server[955]: 2026/09/25 15:14:17 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5469lb # [ 97.883130] niks3-server[955]: 2026/09/25 15:14:17 INFO Received complete push request method=POST path=/api/pushes/34/complete5470lb # [ 97.892485] niks3-server[955]: 2026/09/25 15:14:17 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5471worker1 # [ 97.893784] nix-grpc-daemon[861]: time=2026-09-25T15:14:16.145Z level=INFO msg="Upload complete. (183ms)"5472client # copying 1 paths...5473worker1 # [ 97.899944] nix-grpc-daemon[511]: ts=2026-09-25T15:14:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55058 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=18 outputs=15474worker1 # [ 97.908526] nix-grpc-daemon[511]: ts=2026-09-25T15:14:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550585475client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5476worker1 # [ 97.919853] nix-daemon[848]: reaped child process 1202, status = succeeded5477worker2 # [ 97.915484] nix-grpc-daemon[786]: ts=2026-09-25T15:14:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:380665478worker2 # [ 97.925974] nix-grpc-daemon[786]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5479worker2 # [ 97.939702] nix-grpc-daemon[786]: ts=2026-09-25T15:14:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38066 duration_s=0 paths=1 nar_bytes_out=1205480client # [ 98.018596] nix-daemon[1131]: reaped child process 1261, status = succeeded5481client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.28 seconds)5482(finished: subtest: an input only one worker has still reaches the builder, in 7.73 seconds)5483subtest: upload whose reference the worker lacks is completed from the cache5484client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5485client # this derivation will be built:5486client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5487client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5488client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5489client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5490client # this derivation will be built:5491client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5492client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5493client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.24 seconds)5494client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5495client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5496worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5497worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5498worker2 # [ 98.757763] systemd[1]: Stopping Nix worker-protocol over gRPC...5499worker2 # [ 99.673855] nix-grpc-daemon[786]: ts=2026-09-25T15:14:17Z level=info event=signal_exit5500lb # [ 100.257402] niks3-server[955]: 2026/09/25 15:14:19 INFO lead: released remote=192.168.1.4:514125501worker2 # [ 101.240963] nix-grpc-daemon[786]: ts=2026-09-25T15:14:19Z level=info event=scheduler_restarting5502lb # [ 101.256414] envoy[924]: [2026-09-25 15:14:20.449][924][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5503worker2 # [ 101.248962] nix-grpc-daemon[786]: ts=2026-09-25T15:14:19Z level=info event=scheduler_restarting addr=lb:500515504worker1 # [ 101.267626] nix-grpc-daemon[511]: ts=2026-09-25T15:14:19Z level=info event=scheduler_restarting addr=lb:500515505worker2 # [ 101.255902] nix-grpc-daemon[786]: ts=2026-09-25T15:14:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:380465506worker2 # [ 101.260199] nix-grpc-daemon[786]: ts=2026-09-25T15:14:19Z level=info event=worker_gone addr=192.168.1.4:500515507worker2 # [ 101.263312] nix-grpc-daemon[786]: ts=2026-09-25T15:14:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:380465508worker2 # [ 101.267240] nix-grpc-daemon[786]: ts=2026-09-25T15:14:19Z level=info event=worker_gone addr=192.168.1.3:500515509worker2 # [ 101.291980] nix-daemon[873]: reaped child process 875, status = succeeded5510worker1 # [ 101.467877] nix-grpc-daemon[511]: ts=2026-09-25T15:14:19Z level=info event=scheduler_connected addr=lb:500515511worker1 # [ 101.472512] nix-grpc-daemon[511]: ts=2026-09-25T15:14:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5512worker2 # [ 101.465265] nix-grpc-daemon[786]: ts=2026-09-25T15:14:19Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5513worker1 # [ 101.673314] nix-grpc-daemon[511]: ts=2026-09-25T15:14:19Z level=info event=scheduler_connected addr=lb:500515514worker1 # [ 101.677507] nix-grpc-daemon[511]: ts=2026-09-25T15:14:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5515worker1 # [ 101.878200] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_connected addr=lb:500515516worker1 # [ 101.882138] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5517worker2 # [ 101.931181] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5518worker2 # [ 101.934133] systemd[1]: Stopped Nix worker-protocol over gRPC.5519worker2 # [ 101.937504] systemd[1]: nix-grpc-daemon.service: Consumed 2.340s CPU time over 1min 25.426s wall clock time, 63.2M memory peak, 439.2K incoming IP traffic, 382.1K outgoing IP traffic.5520lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux5521lb: must succeed: curl -sf localhost:9901/clusters5522lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5523lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds)5524client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25525worker1 # [ 102.083145] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_connected addr=lb:500515526worker1 # [ 102.087158] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5527worker1 # [ 102.165496] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550585528client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5529worker1 # [ 102.170720] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550585530client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5531worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5532worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5533worker2 # [ 102.230596] systemd[1]: Starting scheduler-token.service...5534worker2 # [ 102.264727] systemd[1]: scheduler-token.service: Deactivated successfully.5535worker2 # [ 102.266283] systemd[1]: Finished scheduler-token.service.5536worker2 # [ 102.267736] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 35ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5537worker2 # [ 102.272098] systemd[1]: Starting Nix worker-protocol over gRPC...5538worker1 # [ 102.288929] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_connected addr=lb:500515539worker1 # [ 102.292102] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5540worker2 # [ 102.350712] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5541worker2 # [ 102.355209] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5542worker2 # [ 102.359180] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15543worker2 # [ 102.361059] nix-grpc-daemon[1323]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5544worker2 # [ 102.362352] nix-grpc-daemon[1323]: E0000 00:00:1790349260.668548 1323 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5545worker2 # [ 102.365324] nix-grpc-daemon[1323]: E0000 00:00:1790349260.675800 1323 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5546worker2 # [ 102.371974] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:20Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5547worker2 # [ 102.375170] systemd[1]: Started Nix worker-protocol over gRPC.5548worker2 # [ 102.378084] nix-grpc-daemon[1323]: E0000 00:00:1790349260.687584 1365 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5549lb # [ 102.393367] niks3-server[955]: 2026/09/25 15:14:21 INFO lead: acquired remote=192.168.1.4:589845550worker2 # [ 102.381141] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:20Z level=info event=scheduler_take_over5551worker2 # [ 102.392820] nix-grpc-daemon[1323]: E0000 00:00:1790349260.702175 1365 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5552worker2 # [ 102.397074] nix-grpc-daemon[1323]: E0000 00:00:1790349260.702457 1365 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5553worker2 # [ 102.406644] nix-grpc-daemon[1323]: E0000 00:00:1790349260.717318 1365 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5554worker2 # [ 102.416223] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:20Z level=info event=scheduler_connected addr=lb:500515555worker2 # [ 102.417924] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5556worker1 # [ 102.459785] systemd[1]: Stopping Nix worker-protocol over gRPC...5557worker1 # [ 102.489889] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_connected addr=lb:500515558worker1 # [ 102.494079] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5559worker2 # [ 102.619614] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:20Z level=info event=scheduler_connected addr=lb:500515560worker2 # [ 102.623670] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5561worker1 # [ 102.694984] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_connected addr=lb:500515562worker1 # [ 102.698950] nix-grpc-daemon[511]: ts=2026-09-25T15:14:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5563worker1 # [ 102.840547] nix-grpc-daemon[511]: ts=2026-09-25T15:14:21Z level=info event=signal_exit5564worker2 # [ 102.824998] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=scheduler_connected addr=lb:500515565worker2 # [ 102.829179] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5566worker1 # [ 102.899662] nix-grpc-daemon[511]: ts=2026-09-25T15:14:21Z level=info event=scheduler_connected addr=lb:500515567worker1 # [ 102.904174] nix-grpc-daemon[511]: ts=2026-09-25T15:14:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5568worker2 # [ 103.030614] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=scheduler_connected addr=lb:500515569worker2 # [ 103.034617] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5570worker1 # [ 103.105173] nix-grpc-daemon[511]: ts=2026-09-25T15:14:21Z level=info event=scheduler_connected addr=lb:500515571worker1 # [ 103.109627] nix-grpc-daemon[511]: ts=2026-09-25T15:14:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5572worker2 # [ 103.235964] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=scheduler_connected addr=lb:500515573worker2 # [ 103.240247] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5574worker1 # [ 103.311109] nix-grpc-daemon[511]: ts=2026-09-25T15:14:21Z level=info event=scheduler_connected addr=lb:500515575worker2 # [ 103.296637] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:609965576worker2 # [ 103.305811] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:327745577worker2 # [ 103.311827] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05578worker2 # [ 103.442304] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=scheduler_connected addr=lb:500515579worker2 # [ 103.453152] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=oidc_jwks provider=mock keys=15580worker2 # [ 103.457215] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:327745581worker2 # [ 103.465288] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05582worker1 # [ 104.257181] nix-grpc-daemon[511]: ts=2026-09-25T15:14:22Z level=info event=scheduler_restarting5583worker2 # [ 104.248076] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:327745584worker2 # [ 104.254524] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:22Z level=info event=worker_gone addr=192.168.1.3:500515585worker1 # [ 104.279831] nix-grpc-daemon[511]: ts=2026-09-25T15:14:22Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5586worker1 # [ 104.300690] nix-daemon[848]: reaped child process 850, status = succeeded5587worker1 # [ 105.907667] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5588worker1 # [ 105.911592] systemd[1]: Stopped Nix worker-protocol over gRPC.5589worker1 # [ 105.915637] systemd[1]: nix-grpc-daemon.service: Consumed 2.197s CPU time over 1min 37.372s wall clock time, 68.2M memory peak, 181.6K incoming IP traffic, 304.2K outgoing IP traffic.5590lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux5591lb: must succeed: curl -sf localhost:9901/clusters5592lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5593lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.05 seconds)5594client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25595worker2 # [ 106.117857] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:328025596worker2 # [ 106.129749] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5597worker2 # [ 106.145991] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 1/5)5598worker2 # [ 106.184381] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 2/5)5599worker2 # [ 106.292462] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 320 ms (attempt 3/5)5600worker2 # [ 106.614899] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 752 ms (attempt 4/5)5601lb # [ 107.070689] envoy[924]: conn peer=[2001:db8:1::1]:41932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=2106 ms=20125602lb # [ 107.074718] envoy[924]: conn peer=[2001:db8:1::1]:41934 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=329 ms=445603lb # [ 107.078245] envoy[924]: conn peer=192.168.1.4:58224 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2996 tx=1122 ms=682905604lb # [ 107.081820] envoy[924]: conn peer=[2001:db8:1::1]:41950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=155605lb # [ 107.085284] envoy[924]: conn peer=192.168.1.3:42950 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4394 tx=1391 ms=699255606worker2 # [ 107.371222] nix-grpc-daemon[1323]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5607worker2 # [ 107.374593] nix-grpc-daemon[1323]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5608client # copying 2 paths...5609client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5610worker2 # [ 107.396975] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:328025611client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5612worker2 # [ 107.410378] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5613worker2 # [ 107.433150] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:25Z level=info event=niks3_push_started pid=13835614worker2 # [ 107.538699] nix-grpc-daemon[1383]: time=2026-09-25T15:14:25.849Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5615worker2 # [ 107.629958] nix-daemon[873]: accepted connection from pid 1396, user nix-grpc-daemon (trusted)5616worker2 # [ 107.643488] nix-daemon[873]: reaped child process 1399, status = succeeded5617lb # [ 107.665886] niks3-server[955]: 2026/09/25 15:14:26 INFO Received push request method=POST path=/api/pushes5618worker2 # [ 107.661806] nix-grpc-daemon[1383]: time=2026-09-25T15:14:25.972Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5619worker2 # [ 107.663377] nix-grpc-daemon[1383]: time=2026-09-25T15:14:25.974Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5620worker2 # [ 107.665310] nix-grpc-daemon[1383]: time=2026-09-25T15:14:25.976Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5621lb # [ 107.728356] niks3-server[955]: 2026/09/25 15:14:26 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5622lb # [ 107.730860] niks3-server[955]: 2026/09/25 15:14:26 INFO Signed narinfos id=35 count=25623worker2 # [ 107.718858] nix-grpc-daemon[1383]: time=2026-09-25T15:14:26.028Z level=INFO msg="Uploading 2 narinfos"5624lb # [ 107.735687] niks3-server[955]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5625lb # [ 107.738512] niks3-server[955]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5626lb # [ 107.744446] niks3-server[955]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5627lb # [ 107.757178] niks3-server[955]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5628lb # [ 107.762449] niks3-server[955]: 2026/09/25 15:14:26 INFO Received complete push request method=POST path=/api/pushes/35/complete5629worker2 # [ 107.753415] nix-grpc-daemon[1383]: time=2026-09-25T15:14:26.064Z level=INFO msg="Upload complete. (211ms)"5630lb # [ 107.769990] niks3-server[955]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5631lb # [ 107.772368] niks3-server[955]: 2026/09/25 15:14:26 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5632worker2 # [ 107.759284] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32802 duration_s=0 paths=2 nar_bytes_in=3125633worker2 # [ 107.772714] nix-daemon[873]: reaped child process 1381, status = succeeded5634client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.83 seconds)5635worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5636worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5637worker1 # [ 107.876334] systemd[1]: Starting Nix worker-protocol over gRPC...5638worker1 # [ 107.967338] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5639worker1 # [ 107.969661] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5640worker1 # [ 107.973159] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15641worker1 # [ 107.978057] nix-grpc-daemon[1246]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5642worker1 # [ 107.979650] nix-grpc-daemon[1246]: E0000 00:00:1790349266.229532 1246 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5643worker1 # [ 107.983362] nix-grpc-daemon[1246]: E0000 00:00:1790349266.235331 1246 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5644worker1 # [ 107.988617] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:26Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5645worker1 # [ 107.992287] systemd[1]: Started Nix worker-protocol over gRPC.5646worker1 # [ 107.996113] nix-grpc-daemon[1246]: E0000 00:00:1790349266.247844 1288 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5647lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux5648lb: must succeed: curl -sf localhost:9901/clusters5649worker1 # [ 108.009212] nix-grpc-daemon[1246]: E0000 00:00:1790349266.261217 1288 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5650worker1 # [ 108.013135] nix-grpc-daemon[1246]: E0000 00:00:1790349266.263880 1288 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5651worker1 # [ 108.024052] nix-grpc-daemon[1246]: E0000 00:00:1790349266.276062 1288 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5652worker1 # [ 108.037444] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:26Z level=info event=scheduler_connected addr=lb:500515653worker2 # [ 108.022848] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:327745654worker2 # [ 108.026628] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05655lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5656lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.06 seconds)5657(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.02 seconds)5658subtest: a niks3 restart does not cost the leader its role5659lb: must succeed: curl -sf localhost:9901/clusters5660lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5661lb: must succeed: curl -sf localhost:9901/clusters5662lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5663lb # [ 108.127754] systemd[1]: Stopping niks3 server...5664lb # [ 108.129343] niks3-server[955]: 2026/09/25 15:14:27 INFO Shutdown signal received, draining in-flight requests timeout=10s5665lb # [ 108.134537] niks3-server[955]: 2026/09/25 15:14:27 INFO lead: released remote=192.168.1.4:589845666worker1 # [ 108.136156] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:26Z level=info event=scheduler_lock_lost status=2005667worker2 # [ 108.120084] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:26Z level=info event=scheduler_lock_lost status=2005668lb # [ 108.150393] systemd[1]: niks3.service: Deactivated successfully.5669lb # [ 108.151847] systemd[1]: Stopped niks3 server.5670lb # [ 108.153214] systemd[1]: niks3.service: Consumed 685ms CPU time over 1min 30.135s wall clock time, 29M memory peak, 94.1K incoming IP traffic, 137.5K outgoing IP traffic.5671lb # [ 108.161164] systemd[1]: Starting niks3 server...5672lb # [ 108.217381] niks3-server[1136]: 2026/09/25 15:14:27 goose: no migrations to run. current version: 202609231200005673lb # [ 108.224425] niks3-server[1136]: 2026/09/25 15:14:27 OK 1_commit_pending_closure.sql (4.53ms)5674lb # [ 108.228214] niks3-server[1136]: 2026/09/25 15:14:27 OK 2_object_stats_trigger.sql (2.38ms)5675lb # [ 108.231925] niks3-server[1136]: 2026/09/25 15:14:27 OK 3_commit_push.sql (3.31ms)5676lb # [ 108.232932] niks3-server[1136]: 2026/09/25 15:14:27 goose: up to current file version: 35677lb # [ 108.234995] niks3-server[1136]: 2026/09/25 15:14:27 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5678lb # [ 108.246800] niks3-server[1136]: 2026/09/25 15:14:27 INFO Read proxy enabled — serving cache objects from S35679lb # [ 108.248122] niks3-server[1136]: 2026/09/25 15:14:27 INFO Using socket-activated listener address=0.0.0.0:57515680lb # [ 108.250157] systemd[1]: Started niks3 server.5681lb # [ 108.251455] niks3-server[1136]: 2026/09/25 15:14:27 INFO systemd watchdog enabled interval=15s5682lb # [ 108.253093] niks3-server[1136]: 2026/09/25 15:14:27 INFO Starting HTTP server address=0.0.0.0:57515683lb: waiting for unit niks3.service5684lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5685lb: must succeed: curl -sf localhost:9901/clusters5686lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5687worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5688worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5689lb: must succeed: curl -sf localhost:9901/clusters5690lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5691lb: must succeed: curl -sf localhost:9901/clusters5692lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5693worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5694worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5695client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25696worker1 # [ 108.599719] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467805697worker1 # [ 108.611320] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5698client # don't know how to build these paths:5699client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5700worker1 # [ 108.625652] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467805701worker1 # [ 108.635722] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5)5702worker1 # [ 108.672119] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 148 ms (attempt 2/5)5703worker1 # [ 108.822238] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 381 ms (attempt 3/5)5704lb # [ 109.141308] niks3-server[1136]: 2026/09/25 15:14:28 INFO lead: acquired remote=192.168.1.4:403605705worker1 # [ 109.211076] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 532 ms (attempt 4/5)5706worker1 # [ 109.744872] nix-grpc-daemon[1246]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5707worker1 # [ 109.748480] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5708worker2 # [ 109.772705] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:28Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05709worker2 # [ 109.778894] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:28Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15710worker2 # [ 109.783342] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328025711worker2 # [ 109.797685] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:328025712client # copying 1 paths to 'lb:50051'...5713client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5714worker2 # [ 109.811091] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:328025715worker2 # [ 109.822166] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5716worker2 # [ 109.926644] nix-daemon[873]: accepted connection from pid 1418, user nix-grpc-daemon (trusted)5717worker2 # [ 109.940387] nix-daemon[873]: reaped child process 1421, status = succeeded5718lb # [ 109.962926] niks3-server[1136]: 2026/09/25 15:14:29 INFO Received push request method=POST path=/api/pushes5719worker2 # [ 109.961310] nix-grpc-daemon[1383]: time=2026-09-25T15:14:28.271Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5720worker2 # [ 109.963603] nix-grpc-daemon[1383]: time=2026-09-25T15:14:28.273Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5721lb # [ 110.008766] niks3-server[1136]: 2026/09/25 15:14:29 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5722lb # [ 110.018598] niks3-server[1136]: 2026/09/25 15:14:29 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5723lb # [ 110.035743] niks3-server[1136]: 2026/09/25 15:14:29 INFO Signed narinfos id=36 count=15724worker2 # [ 110.023158] nix-grpc-daemon[1383]: time=2026-09-25T15:14:28.333Z level=INFO msg="Uploading 1 narinfos"5725lb # [ 110.042765] niks3-server[1136]: 2026/09/25 15:14:29 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5726lb # [ 110.047475] niks3-server[1136]: 2026/09/25 15:14:29 INFO Received complete push request method=POST path=/api/pushes/36/complete5727lb # [ 110.055166] niks3-server[1136]: 2026/09/25 15:14:29 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5728worker2 # [ 110.044570] nix-grpc-daemon[1383]: time=2026-09-25T15:14:28.355Z level=INFO msg="Upload complete. (210ms)"5729worker2 # [ 110.049499] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32802 duration_s=0 paths=1 nar_bytes_in=5365730worker2 # [ 110.055656] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328025731worker2 # [ 110.066408] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5732worker2 # [ 110.080234] nix-daemon[873]: reaped child process 1416, status = succeeded5733worker2 # [ 110.083668] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5734client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5735client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5736client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5737client # farm-a> LOG-farm-a-after-niks3-restart5738worker2 # [ 111.319855] nix-daemon[873]: reaped child process 1425, status = succeeded5739worker2 # [ 111.407463] nix-daemon[873]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5740worker2 # [ 111.421402] nix-daemon[873]: reaped child process 1444, status = succeeded5741lb # [ 111.444362] niks3-server[1136]: 2026/09/25 15:14:30 INFO Received push request method=POST path=/api/pushes5742worker2 # [ 111.440763] nix-grpc-daemon[1383]: time=2026-09-25T15:14:29.751Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5743worker2 # [ 111.442353] nix-grpc-daemon[1383]: time=2026-09-25T15:14:29.753Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5744lb # [ 111.493440] niks3-server[1136]: 2026/09/25 15:14:30 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5745lb # [ 111.500145] niks3-server[1136]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5746lb # [ 111.504669] niks3-server[1136]: 2026/09/25 15:14:30 INFO Signed narinfos id=37 count=15747worker2 # [ 111.493786] nix-grpc-daemon[1383]: time=2026-09-25T15:14:29.804Z level=INFO msg="Uploading 1 narinfos"5748lb # [ 111.514123] niks3-server[1136]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5749lb # [ 111.517234] niks3-server[1136]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5750lb # [ 111.522658] niks3-server[1136]: 2026/09/25 15:14:30 INFO Received complete push request method=POST path=/api/pushes/37/complete5751lb # [ 111.529729] niks3-server[1136]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5752worker2 # [ 111.518534] nix-grpc-daemon[1383]: time=2026-09-25T15:14:29.829Z level=INFO msg="Upload complete. (226ms)"5753worker2 # [ 111.525671] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32802 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15754worker2 # [ 111.538864] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:29Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05755worker2 # [ 111.543350] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:29Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15756worker2 # [ 111.546799] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328025757worker2 # [ 111.557148] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:328025758worker2 # [ 111.558983] nix-daemon[873]: reaped child process 1423, status = succeeded5759worker2 # [ 111.564564] nix-grpc-daemon[1383]: time=2026-09-25T15:14:29.875Z level=INFO msg="All 1 paths already cached"5760client # copying 1 paths to 'lb:50051'...5761worker2 # [ 111.569746] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:328025762client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5763worker2 # [ 111.580561] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5764worker2 # [ 111.668938] nix-daemon[873]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)5765worker2 # [ 111.683177] nix-daemon[873]: reaped child process 1453, status = succeeded5766lb # [ 111.704727] niks3-server[1136]: 2026/09/25 15:14:30 INFO Received push request method=POST path=/api/pushes5767worker2 # [ 111.699902] nix-grpc-daemon[1383]: time=2026-09-25T15:14:30.010Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5768worker2 # [ 111.701370] nix-grpc-daemon[1383]: time=2026-09-25T15:14:30.012Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5769lb # [ 111.727200] niks3-server[1136]: 2026/09/25 15:14:30 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5770lb # [ 111.729157] niks3-server[1136]: 2026/09/25 15:14:30 INFO Signed narinfos id=38 count=15771worker2 # [ 111.715689] nix-grpc-daemon[1383]: time=2026-09-25T15:14:30.026Z level=INFO msg="Uploading 1 narinfos"5772lb # [ 111.732841] niks3-server[1136]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5773lb # [ 111.739468] niks3-server[1136]: 2026/09/25 15:14:30 INFO Received complete push request method=POST path=/api/pushes/38/complete5774lb # [ 111.741315] niks3-server[1136]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5775lb # [ 111.748688] niks3-server[1136]: 2026/09/25 15:14:30 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5776worker2 # [ 111.737141] nix-grpc-daemon[1383]: time=2026-09-25T15:14:30.047Z level=INFO msg="Upload complete. (150ms)"5777worker2 # [ 111.739435] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32802 duration_s=0 paths=1 nar_bytes_in=6565778worker2 # [ 111.742492] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328025779worker2 # [ 111.749641] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5780worker2 # [ 111.752970] nix-daemon[873]: reaped child process 1448, status = succeeded5781worker2 # [ 111.758049] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5782client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5783client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5784client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5785client # farm-b> LOG-farm-b-after-niks3-restart5786worker2 # [ 112.597562] nix-daemon[873]: reaped child process 1457, status = succeeded5787worker2 # [ 112.678614] nix-daemon[873]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5788worker2 # [ 112.692758] nix-daemon[873]: reaped child process 1476, status = succeeded5789lb # [ 112.716478] niks3-server[1136]: 2026/09/25 15:14:31 INFO Received push request method=POST path=/api/pushes5790worker2 # [ 112.709438] nix-grpc-daemon[1383]: time=2026-09-25T15:14:31.019Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5791worker2 # [ 112.710902] nix-grpc-daemon[1383]: time=2026-09-25T15:14:31.021Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5792lb # [ 112.758341] niks3-server[1136]: 2026/09/25 15:14:31 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5793lb # [ 112.768447] niks3-server[1136]: 2026/09/25 15:14:31 INFO Signed narinfos id=39 count=15794worker2 # [ 112.756679] nix-grpc-daemon[1383]: time=2026-09-25T15:14:31.067Z level=INFO msg="Uploading 1 narinfos"5795lb # [ 112.775376] niks3-server[1136]: 2026/09/25 15:14:31 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5796lb # [ 112.779914] niks3-server[1136]: 2026/09/25 15:14:31 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5797lb # [ 112.789745] niks3-server[1136]: 2026/09/25 15:14:31 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5798lb # [ 112.794210] niks3-server[1136]: 2026/09/25 15:14:31 INFO Received complete push request method=POST path=/api/pushes/39/complete5799lb # [ 112.798864] niks3-server[1136]: 2026/09/25 15:14:31 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5800worker2 # [ 112.782352] nix-grpc-daemon[1383]: time=2026-09-25T15:14:31.092Z level=INFO msg="Upload complete. (217ms)"5801worker2 # [ 112.789461] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32802 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15802worker2 # [ 112.801213] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:31Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05803worker2 # [ 112.806277] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:31Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15804worker2 # [ 112.809765] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328025805worker2 # [ 112.820310] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:328025806worker2 # [ 112.822638] nix-daemon[873]: reaped child process 1455, status = succeeded5807worker2 # [ 112.826874] nix-grpc-daemon[1383]: time=2026-09-25T15:14:31.137Z level=INFO msg="All 2 paths already cached"5808client # copying 1 paths to 'lb:50051'...5809worker2 # [ 112.832156] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:328025810client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5811worker2 # [ 112.837093] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5812worker2 # [ 112.926514] nix-daemon[873]: accepted connection from pid 1482, user nix-grpc-daemon (trusted)5813worker2 # [ 112.939177] nix-daemon[873]: reaped child process 1485, status = succeeded5814lb # [ 112.960687] niks3-server[1136]: 2026/09/25 15:14:32 INFO Received push request method=POST path=/api/pushes5815worker2 # [ 112.954992] nix-grpc-daemon[1383]: time=2026-09-25T15:14:31.265Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5816worker2 # [ 112.956533] nix-grpc-daemon[1383]: time=2026-09-25T15:14:31.267Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5817lb # [ 112.994806] niks3-server[1136]: 2026/09/25 15:14:32 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5818lb # [ 113.002709] niks3-server[1136]: 2026/09/25 15:14:32 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5819lb # [ 113.008222] niks3-server[1136]: 2026/09/25 15:14:32 INFO Signed narinfos id=40 count=15820worker2 # [ 112.998862] nix-grpc-daemon[1383]: time=2026-09-25T15:14:31.309Z level=INFO msg="Uploading 1 narinfos"5821lb # [ 113.018263] niks3-server[1136]: 2026/09/25 15:14:32 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5822lb # [ 113.029121] niks3-server[1136]: 2026/09/25 15:14:32 INFO Received complete push request method=POST path=/api/pushes/40/complete5823lb # [ 113.035604] niks3-server[1136]: 2026/09/25 15:14:32 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5824worker2 # [ 113.019704] nix-grpc-daemon[1383]: time=2026-09-25T15:14:31.330Z level=INFO msg="Upload complete. (176ms)"5825worker2 # [ 113.023995] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32802 duration_s=0 paths=1 nar_bytes_in=7845826worker2 # [ 113.028886] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328025827worker2 # [ 113.040063] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5828worker2 # [ 113.045203] nix-daemon[873]: reaped child process 1480, status = succeeded5829worker2 # [ 113.049630] nix-daemon[873]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5830client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5831client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5832client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5833client # farm-top> LOG-farm-top-after-niks3-restart5834worker2 # [ 114.291499] nix-daemon[873]: reaped child process 1489, status = succeeded5835worker2 # [ 114.373935] nix-daemon[873]: accepted connection from pid 1505, user nix-grpc-daemon (trusted)5836worker2 # [ 114.386045] nix-daemon[873]: reaped child process 1508, status = succeeded5837lb # [ 114.410571] niks3-server[1136]: 2026/09/25 15:14:33 INFO Received push request method=POST path=/api/pushes5838worker2 # [ 114.404459] nix-grpc-daemon[1383]: time=2026-09-25T15:14:32.714Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5839worker2 # [ 114.405926] nix-grpc-daemon[1383]: time=2026-09-25T15:14:32.716Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5840lb # [ 114.450402] niks3-server[1136]: 2026/09/25 15:14:33 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5841worker2 # [ 114.445216] nix-grpc-daemon[1383]: time=2026-09-25T15:14:32.755Z level=INFO msg="Uploading 1 narinfos"5842lb # [ 114.459947] niks3-server[1136]: 2026/09/25 15:14:33 INFO Signed narinfos id=41 count=15843lb # [ 114.463234] niks3-server[1136]: 2026/09/25 15:14:33 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5844lb # [ 114.470470] niks3-server[1136]: 2026/09/25 15:14:33 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5845lb # [ 114.473558] niks3-server[1136]: 2026/09/25 15:14:33 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5846lb # [ 114.478845] niks3-server[1136]: 2026/09/25 15:14:33 INFO Received complete push request method=POST path=/api/pushes/41/complete5847lb # [ 114.484484] niks3-server[1136]: 2026/09/25 15:14:33 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5848worker2 # [ 114.468872] nix-grpc-daemon[1383]: time=2026-09-25T15:14:32.779Z level=INFO msg="Upload complete. (210ms)"5849worker2 # [ 114.477508] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32802 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15850worker2 # [ 114.489903] nix-daemon[873]: reaped child process 1487, status = succeeded5851client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 6.07 seconds)5852client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5853client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.11 seconds)5854(finished: subtest: a niks3 restart does not cost the leader its role, in 6.57 seconds)5855subtest: scheduler down: the other node takes the lock and keeps it5856lb: must succeed: curl -sf localhost:9901/clusters5857lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5858lb: must succeed: curl -sf localhost:9901/clusters5859lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5860lb: must succeed: curl -sf localhost:9901/clusters5861lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5862lb: must succeed: curl -sf localhost:9901/clusters5863lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5864worker2 # [ 114.771936] systemd[1]: Stopping Nix worker-protocol over gRPC...5865worker2 # [ 115.370913] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:33Z level=info event=signal_exit5866lb # [ 116.144923] niks3-server[1136]: 2026/09/25 15:14:35 INFO lead: released remote=192.168.1.4:403605867lb # [ 117.070262] envoy[924]: conn peer=[2001:db8:1::1]:57990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=282 ms=16565868lb # [ 117.077109] envoy[924]: conn peer=[2001:db8:1::1]:57996 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9705 tx=4407 ms=59115869worker2 # [ 117.130996] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:35Z level=info event=scheduler_restarting5870lb # [ 117.145457] envoy[924]: [2026-09-25 15:14:36.338][924][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5871worker2 # [ 117.138357] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:35Z level=info event=scheduler_restarting addr=lb:500515872worker2 # [ 117.141768] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:327745873worker1 # [ 117.157728] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:35Z level=info event=scheduler_restarting addr=lb:500515874worker2 # [ 117.143297] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:35Z level=info event=worker_gone addr=192.168.1.4:500515875worker2 # [ 117.144740] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:327745876worker2 # [ 117.146553] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:35Z level=info event=worker_gone addr=192.168.1.3:500515877worker2 # [ 117.160305] nix-daemon[873]: reaped child process 1375, status = succeeded5878worker1 # [ 117.357740] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:35Z level=info event=scheduler_connected addr=lb:500515879worker2 # [ 117.345138] nix-grpc-daemon[1323]: ts=2026-09-25T15:14:35Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5880worker1 # [ 117.361720] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5881worker1 # [ 117.562549] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:35Z level=info event=scheduler_connected addr=lb:500515882worker1 # [ 117.566502] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5883worker1 # [ 117.767282] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_connected addr=lb:500515884worker1 # [ 117.771397] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5885worker1 # [ 117.971608] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_connected addr=lb:500515886worker1 # [ 117.973429] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5887worker1 # [ 118.174437] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_connected addr=lb:500515888worker1 # [ 118.176037] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5889worker1 # [ 118.376681] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_connected addr=lb:500515890worker1 # [ 118.378456] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5891worker2 # [ 118.369495] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5892worker2 # [ 118.373385] systemd[1]: Stopped Nix worker-protocol over gRPC.5893worker2 # [ 118.377367] systemd[1]: nix-grpc-daemon.service: Consumed 961ms CPU time over 16.100s wall clock time, 37.9M memory peak, 431.1K incoming IP traffic, 121.7K outgoing IP traffic.5894worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5895worker1 # [ 118.579277] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_connected addr=lb:500515896worker1 # [ 118.583255] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5897worker1 # [ 118.784073] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=scheduler_connected addr=lb:500515898worker1 # [ 118.787923] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5899worker1 # [ 118.988495] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=scheduler_connected addr=lb:500515900worker1 # [ 118.992708] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5901worker1 # [ 119.143158] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=scheduler_take_over5902lb # [ 119.140715] niks3-server[1136]: 2026/09/25 15:14:38 INFO lead: acquired remote=192.168.1.3:425405903worker1 # [ 119.194149] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=scheduler_connected addr=lb:500515904worker1 # [ 119.197940] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5905worker1 # [ 119.399073] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=scheduler_connected addr=lb:500515906worker1 # [ 119.406138] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:449745907worker1 # [ 119.409659] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05908worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.08 seconds)5909worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5910worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5911client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25912worker1 # [ 119.644199] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467805913client # don't know how to build these paths:5914client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5915worker1 # [ 119.652863] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467805916worker1 # [ 119.674241] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05917worker1 # [ 119.679532] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15918worker1 # [ 119.684845] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467805919worker1 # [ 119.695874] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467805920client # copying 1 paths to 'lb:50051'...5921client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5922worker1 # [ 119.707551] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467805923worker1 # [ 119.718164] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5924worker1 # [ 119.734831] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:37Z level=info event=niks3_push_started pid=13255925worker1 # [ 119.828578] nix-grpc-daemon[1325]: time=2026-09-25T15:14:38.080Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5926worker1 # [ 119.914048] nix-daemon[848]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5927worker1 # [ 119.924923] nix-daemon[848]: reaped child process 1340, status = succeeded5928lb # [ 119.930576] niks3-server[1136]: 2026/09/25 15:14:39 INFO Received push request method=POST path=/api/pushes5929worker1 # [ 119.940612] nix-grpc-daemon[1325]: time=2026-09-25T15:14:38.192Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5930worker1 # [ 119.942308] nix-grpc-daemon[1325]: time=2026-09-25T15:14:38.194Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5931lb # [ 119.970674] niks3-server[1136]: 2026/09/25 15:14:39 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5932lb # [ 119.975710] niks3-server[1136]: 2026/09/25 15:14:39 INFO Signed narinfos id=42 count=15933worker1 # [ 119.981408] nix-grpc-daemon[1325]: time=2026-09-25T15:14:38.233Z level=INFO msg="Uploading 1 narinfos"5934lb # [ 119.980256] niks3-server[1136]: 2026/09/25 15:14:39 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5935lb # [ 119.987346] niks3-server[1136]: 2026/09/25 15:14:39 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5936lb # [ 120.001493] niks3-server[1136]: 2026/09/25 15:14:39 INFO Received complete push request method=POST path=/api/pushes/42/complete5937worker1 # [ 120.011692] nix-grpc-daemon[1325]: time=2026-09-25T15:14:38.263Z level=INFO msg="Upload complete. (180ms)"5938lb # [ 120.010769] niks3-server[1136]: 2026/09/25 15:14:39 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5939worker1 # [ 120.014526] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46780 duration_s=0 paths=1 nar_bytes_in=5205940worker1 # [ 120.021201] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467805941worker1 # [ 120.030541] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5942worker1 # [ 120.034987] nix-daemon[848]: reaped child process 1323, status = succeeded5943worker1 # [ 120.037915] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5944client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5945client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5946client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5947client # farm-a> LOG-farm-a-on-standby5948worker1 # [ 120.852978] nix-daemon[848]: reaped child process 1344, status = succeeded5949worker1 # [ 120.922649] nix-daemon[848]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5950worker1 # [ 120.936401] nix-daemon[848]: reaped child process 1363, status = succeeded5951lb # [ 120.938862] niks3-server[1136]: 2026/09/25 15:14:40 INFO Received push request method=POST path=/api/pushes5952worker1 # [ 120.950746] nix-grpc-daemon[1325]: time=2026-09-25T15:14:39.202Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5953worker1 # [ 120.952391] nix-grpc-daemon[1325]: time=2026-09-25T15:14:39.204Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5954lb # [ 120.971779] niks3-server[1136]: 2026/09/25 15:14:40 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5955lb # [ 120.988895] niks3-server[1136]: 2026/09/25 15:14:40 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5956worker1 # [ 120.997832] nix-grpc-daemon[1325]: time=2026-09-25T15:14:39.249Z level=INFO msg="Uploading 1 narinfos"5957lb # [ 120.994759] niks3-server[1136]: 2026/09/25 15:14:40 INFO Signed narinfos id=43 count=15958lb # [ 121.002501] niks3-server[1136]: 2026/09/25 15:14:40 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5959lb # [ 121.007396] niks3-server[1136]: 2026/09/25 15:14:40 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5960lb # [ 121.018067] niks3-server[1136]: 2026/09/25 15:14:40 INFO Received complete push request method=POST path=/api/pushes/43/complete5961worker1 # [ 121.028170] nix-grpc-daemon[1325]: time=2026-09-25T15:14:39.280Z level=INFO msg="Upload complete. (184ms)"5962lb # [ 121.025922] niks3-server[1136]: 2026/09/25 15:14:40 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5963worker1 # [ 121.032966] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46780 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15964worker1 # [ 121.037387] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:39Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05965worker1 # [ 121.040480] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:39Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15966worker1 # [ 121.043167] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467805967worker1 # [ 121.049162] nix-daemon[848]: reaped child process 1342, status = succeeded5968worker1 # [ 121.050439] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467805969worker1 # [ 121.055978] nix-grpc-daemon[1325]: time=2026-09-25T15:14:39.308Z level=INFO msg="All 1 paths already cached"5970client # copying 1 paths to 'lb:50051'...5971client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5972worker1 # [ 121.058774] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467805973worker1 # [ 121.063476] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5974worker1 # [ 121.145319] nix-daemon[848]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5975worker1 # [ 121.157859] nix-daemon[848]: reaped child process 1372, status = succeeded5976lb # [ 121.162656] niks3-server[1136]: 2026/09/25 15:14:40 INFO Received push request method=POST path=/api/pushes5977worker1 # [ 121.172612] nix-grpc-daemon[1325]: time=2026-09-25T15:14:39.424Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5978worker1 # [ 121.174084] nix-grpc-daemon[1325]: time=2026-09-25T15:14:39.426Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5979lb # [ 121.193262] niks3-server[1136]: 2026/09/25 15:14:40 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5980lb # [ 121.199334] niks3-server[1136]: 2026/09/25 15:14:40 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5981lb # [ 121.204060] niks3-server[1136]: 2026/09/25 15:14:40 INFO Signed narinfos id=44 count=15982worker1 # [ 121.209784] nix-grpc-daemon[1325]: time=2026-09-25T15:14:39.461Z level=INFO msg="Uploading 1 narinfos"5983lb # [ 121.211290] niks3-server[1136]: 2026/09/25 15:14:40 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5984lb # [ 121.221343] niks3-server[1136]: 2026/09/25 15:14:40 INFO Received complete push request method=POST path=/api/pushes/44/complete5985worker1 # [ 121.233204] nix-grpc-daemon[1325]: time=2026-09-25T15:14:39.485Z level=INFO msg="Upload complete. (165ms)"5986lb # [ 121.232368] niks3-server[1136]: 2026/09/25 15:14:40 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5987worker1 # [ 121.235398] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46780 duration_s=0 paths=1 nar_bytes_in=6405988worker1 # [ 121.239177] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467805989worker1 # [ 121.245561] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5990worker1 # [ 121.247956] nix-daemon[848]: reaped child process 1367, status = succeeded5991client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5992worker1 # [ 121.253083] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5993client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5994client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5995client # farm-b> LOG-farm-b-on-standby5996worker1 # [ 122.286879] nix-daemon[848]: reaped child process 1376, status = succeeded5997worker1 # [ 122.350835] nix-daemon[848]: accepted connection from pid 1392, user nix-grpc-daemon (trusted)5998worker1 # [ 122.365126] nix-daemon[848]: reaped child process 1395, status = succeeded5999lb # [ 122.362685] niks3-server[1136]: 2026/09/25 15:14:41 INFO Received push request method=POST path=/api/pushes6000worker1 # [ 122.372933] nix-grpc-daemon[1325]: time=2026-09-25T15:14:40.624Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6001worker1 # [ 122.374436] nix-grpc-daemon[1325]: time=2026-09-25T15:14:40.626Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6002lb # [ 122.402839] niks3-server[1136]: 2026/09/25 15:14:41 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6003lb # [ 122.410070] niks3-server[1136]: 2026/09/25 15:14:41 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign6004worker1 # [ 122.418090] nix-grpc-daemon[1325]: time=2026-09-25T15:14:40.669Z level=INFO msg="Uploading 1 narinfos"6005lb # [ 122.415253] niks3-server[1136]: 2026/09/25 15:14:41 INFO Signed narinfos id=45 count=16006lb # [ 122.423422] niks3-server[1136]: 2026/09/25 15:14:41 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6007lb # [ 122.433343] niks3-server[1136]: 2026/09/25 15:14:41 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6008lb # [ 122.439586] niks3-server[1136]: 2026/09/25 15:14:41 INFO Received complete push request method=POST path=/api/pushes/45/complete6009lb # [ 122.447729] niks3-server[1136]: 2026/09/25 15:14:41 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6010worker1 # [ 122.452881] nix-grpc-daemon[1325]: time=2026-09-25T15:14:40.704Z level=INFO msg="Upload complete. (192ms)"6011worker1 # [ 122.460642] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46780 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16012worker1 # [ 122.470721] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:40Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06013worker1 # [ 122.477280] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:40Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16014worker1 # [ 122.481195] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467806015worker1 # [ 122.490193] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467806016worker1 # [ 122.495618] nix-daemon[848]: reaped child process 1374, status = succeeded6017worker1 # [ 122.498548] nix-grpc-daemon[1325]: time=2026-09-25T15:14:40.750Z level=INFO msg="All 2 paths already cached"6018client # copying 1 paths to 'lb:50051'...6019client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6020worker1 # [ 122.502908] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467806021worker1 # [ 122.506443] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6022worker1 # [ 122.591421] nix-daemon[848]: accepted connection from pid 1401, user nix-grpc-daemon (trusted)6023worker1 # [ 122.604311] nix-daemon[848]: reaped child process 1404, status = succeeded6024lb # [ 122.608894] niks3-server[1136]: 2026/09/25 15:14:41 INFO Received push request method=POST path=/api/pushes6025worker1 # [ 122.619238] nix-grpc-daemon[1325]: time=2026-09-25T15:14:40.871Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6026worker1 # [ 122.620984] nix-grpc-daemon[1325]: time=2026-09-25T15:14:40.872Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6027lb # [ 122.639628] niks3-server[1136]: 2026/09/25 15:14:41 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6028lb # [ 122.645719] niks3-server[1136]: 2026/09/25 15:14:41 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign6029worker1 # [ 122.655684] nix-grpc-daemon[1325]: time=2026-09-25T15:14:40.907Z level=INFO msg="Uploading 1 narinfos"6030lb # [ 122.652872] niks3-server[1136]: 2026/09/25 15:14:41 INFO Signed narinfos id=46 count=16031lb # [ 122.661889] niks3-server[1136]: 2026/09/25 15:14:41 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6032lb # [ 122.671189] niks3-server[1136]: 2026/09/25 15:14:41 INFO Received complete push request method=POST path=/api/pushes/46/complete6033worker1 # [ 122.681891] nix-grpc-daemon[1325]: time=2026-09-25T15:14:40.933Z level=INFO msg="Upload complete. (169ms)"6034lb # [ 122.680593] niks3-server[1136]: 2026/09/25 15:14:41 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6035worker1 # [ 122.684534] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46780 duration_s=0 paths=1 nar_bytes_in=7686036worker1 # [ 122.689394] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467806037worker1 # [ 122.697242] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6038worker1 # [ 122.705890] nix-daemon[848]: reaped child process 1399, status = succeeded6039client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6040worker1 # [ 122.709806] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6041client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6042client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6043client # farm-top> LOG-farm-top-on-standby6044worker1 # [ 123.322752] nix-daemon[848]: reaped child process 1408, status = succeeded6045worker1 # [ 123.397076] nix-daemon[848]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6046worker1 # [ 123.408261] nix-daemon[848]: reaped child process 1427, status = succeeded6047lb # [ 123.413357] niks3-server[1136]: 2026/09/25 15:14:42 INFO Received push request method=POST path=/api/pushes6048worker1 # [ 123.424512] nix-grpc-daemon[1325]: time=2026-09-25T15:14:41.676Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6049worker1 # [ 123.426256] nix-grpc-daemon[1325]: time=2026-09-25T15:14:41.678Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6050lb # [ 123.453607] niks3-server[1136]: 2026/09/25 15:14:42 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6051lb # [ 123.463080] niks3-server[1136]: 2026/09/25 15:14:42 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign6052lb # [ 123.468721] niks3-server[1136]: 2026/09/25 15:14:42 INFO Signed narinfos id=47 count=16053worker1 # [ 123.475256] nix-grpc-daemon[1325]: time=2026-09-25T15:14:41.727Z level=INFO msg="Uploading 1 narinfos"6054lb # [ 123.477102] niks3-server[1136]: 2026/09/25 15:14:42 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6055lb # [ 123.488123] niks3-server[1136]: 2026/09/25 15:14:42 INFO Received complete push request method=POST path=/api/pushes/47/complete6056worker1 # [ 123.496988] nix-grpc-daemon[1325]: time=2026-09-25T15:14:41.748Z level=INFO msg="Upload complete. (182ms)"6057lb # [ 123.495827] niks3-server[1136]: 2026/09/25 15:14:42 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6058worker1 # [ 123.503900] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46780 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16059lb # [ 123.505971] niks3-server[1136]: 2026/09/25 15:14:42 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6060worker1 # [ 123.519146] nix-daemon[848]: reaped child process 1406, status = succeeded6061worker1 # [ 123.910621] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:42Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:446246062client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.03 seconds)6063client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6064client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6065worker2 # [ 124.685592] systemd[1]: Starting scheduler-token.service...6066worker2 # [ 124.728685] systemd[1]: scheduler-token.service: Deactivated successfully.6067worker2 # [ 124.730336] systemd[1]: Finished scheduler-token.service.6068worker2 # [ 124.732216] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 45ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6069worker2 # [ 124.736277] systemd[1]: Starting Nix worker-protocol over gRPC...6070worker2 # [ 124.818401] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6071worker2 # [ 124.820597] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6072worker2 # [ 124.824256] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16073worker2 # [ 124.829052] nix-grpc-daemon[1529]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6074worker2 # [ 124.831163] nix-grpc-daemon[1529]: E0000 00:00:1790349283.139559 1529 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6075worker2 # [ 124.834604] nix-grpc-daemon[1529]: E0000 00:00:1790349283.145214 1529 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6076worker2 # [ 124.838451] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:43Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6077worker2 # [ 124.842120] systemd[1]: Started Nix worker-protocol over gRPC.6078worker2 # [ 124.848108] nix-grpc-daemon[1529]: E0000 00:00:1790349283.158271 1571 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6079lb: must succeed: curl -sf localhost:9901/clusters6080worker2 # [ 124.861062] nix-grpc-daemon[1529]: E0000 00:00:1790349283.171714 1571 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6081worker2 # [ 124.865141] nix-grpc-daemon[1529]: E0000 00:00:1790349283.174036 1571 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6082worker2 # [ 124.875359] nix-grpc-daemon[1529]: E0000 00:00:1790349283.186037 1571 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6083worker2 # [ 124.885102] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:43Z level=info event=scheduler_connected addr=lb:500516084worker1 # [ 124.907442] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=info event=oidc_jwks provider=mock keys=16085worker1 # [ 124.909429] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:449746086worker1 # [ 124.913547] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06087lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6088worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6089worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6090lb: must succeed: curl -sf localhost:9901/clusters6091lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6092lb: must succeed: curl -sf localhost:9901/clusters6093lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6094worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6095worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6096(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.41 seconds)6097subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6098client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6099client # [ 125.072231] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6100client # Running as unit: rs.service; invocation ID: e7e99d7ff6c848a492d1c7e15eeba9976101client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6102worker1 # [ 125.227141] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:467806103client # [ 125.221577] nix[1383]: don't know how to build these paths:6104client # [ 125.222542] nix[1383]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6105worker1 # [ 125.241979] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06106worker1 # [ 125.247318] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16107worker1 # [ 125.252913] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467806108worker1 # [ 125.264606] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:467806109client # [ 125.264074] nix[1383]: copying 1 paths to 'lb:50051'...6110client # [ 125.264951] nix[1383]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6111worker1 # [ 125.278135] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:467806112worker1 # [ 125.289119] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6113worker1 # [ 125.385420] nix-daemon[848]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)6114worker1 # [ 125.399044] nix-daemon[848]: reaped child process 1443, status = succeeded6115lb # [ 125.403140] niks3-server[1136]: 2026/09/25 15:14:44 INFO Received push request method=POST path=/api/pushes6116worker1 # [ 125.418627] nix-grpc-daemon[1325]: time=2026-09-25T15:14:43.670Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6117worker1 # [ 125.420279] nix-grpc-daemon[1325]: time=2026-09-25T15:14:43.672Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6118lb # [ 125.438516] niks3-server[1136]: 2026/09/25 15:14:44 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign6119worker1 # [ 125.451255] nix-grpc-daemon[1325]: time=2026-09-25T15:14:43.703Z level=INFO msg="Uploading 1 narinfos"6120lb # [ 125.447984] niks3-server[1136]: 2026/09/25 15:14:44 INFO Signed narinfos id=48 count=16121lb # [ 125.451318] niks3-server[1136]: 2026/09/25 15:14:44 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6122lb # [ 125.456406] niks3-server[1136]: 2026/09/25 15:14:44 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6123lb # [ 125.466621] niks3-server[1136]: 2026/09/25 15:14:44 INFO Received complete push request method=POST path=/api/pushes/48/complete6124worker1 # [ 125.474316] nix-grpc-daemon[1325]: time=2026-09-25T15:14:43.726Z level=INFO msg="Upload complete. (173ms)"6125lb # [ 125.472578] niks3-server[1136]: 2026/09/25 15:14:44 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6126worker1 # [ 125.477915] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46780 duration_s=0 paths=1 nar_bytes_in=4166127worker1 # [ 125.483552] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:467806128worker1 # [ 125.493681] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6129worker1 # [ 125.498239] nix-daemon[848]: reaped child process 1438, status = succeeded6130client # [ 125.490610] nix[1383]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6131worker1 # [ 125.502974] nix-daemon[848]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)6132lb # [ 127.071460] envoy[924]: conn peer=[2001:db8:1::4]:59086 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5897 tx=665 ms=149296133lb # [ 127.075304] envoy[924]: conn peer=[2001:db8:1::1]:42696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9517 tx=4351 ms=38766134lb: must succeed: curl -sf localhost:9901/clusters6135lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6136lb: must succeed: curl -sf localhost:9901/clusters6137lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6138lb: must succeed: curl -sf localhost:9901/clusters6139lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6140lb: must succeed: curl -sf localhost:9901/clusters6141lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6142worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6143worker2 # [ 127.743221] systemd-journald[285]: Received client request to rotate journal, rotating.6144worker2 # [ 127.763135] systemd-journald[285]: Vacuuming done, freed 0B of archived journals from /var/log/journal/37a81c223eaa48f08fcc2671e63225c1.6145worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6146worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6147worker1 # [ 127.687096] systemd[1]: Stopping Nix worker-protocol over gRPC...6148worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6149client: must succeed: sleep 5.06150worker1 # [ 127.994678] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:46Z level=info event=draining6151client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6152worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6153worker1 # [ 132.927847] nix-daemon[848]: reaped child process 1447, status = succeeded6154worker1 # [ 132.996911] nix-daemon[848]: accepted connection from pid 1488, user nix-grpc-daemon (trusted)6155worker1 # [ 133.012296] nix-daemon[848]: reaped child process 1491, status = succeeded6156lb # [ 133.014399] niks3-server[1136]: 2026/09/25 15:14:52 INFO Received push request method=POST path=/api/pushes6157worker1 # [ 133.025215] nix-grpc-daemon[1325]: time=2026-09-25T15:14:51.276Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6158worker1 # [ 133.026982] nix-grpc-daemon[1325]: time=2026-09-25T15:14:51.278Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6159lb # [ 133.058417] niks3-server[1136]: 2026/09/25 15:14:52 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign6160lb # [ 133.067560] niks3-server[1136]: 2026/09/25 15:14:52 INFO Signed narinfos id=49 count=16161worker1 # [ 133.073825] nix-grpc-daemon[1325]: time=2026-09-25T15:14:51.325Z level=INFO msg="Uploading 1 narinfos"6162lb # [ 133.078671] niks3-server[1136]: 2026/09/25 15:14:52 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6163lb # [ 133.083585] niks3-server[1136]: 2026/09/25 15:14:52 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6164lb # [ 133.087901] niks3-server[1136]: 2026/09/25 15:14:52 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6165lb # [ 133.097573] niks3-server[1136]: 2026/09/25 15:14:52 INFO Received complete push request method=POST path=/api/pushes/49/complete6166lb # [ 133.103476] niks3-server[1136]: 2026/09/25 15:14:52 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6167worker1 # [ 133.105231] nix-grpc-daemon[1325]: time=2026-09-25T15:14:51.357Z level=INFO msg="Upload complete. (189ms)"6168worker1 # [ 133.113849] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46780 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16169worker1 # [ 133.127271] nix-daemon[848]: reaped child process 1445, status = succeeded6170client # [ 133.131142] systemd[1]: rs.service: Deactivated successfully.6171client # [ 133.133889] systemd[1]: rs.service: Consumed 127ms CPU time over 8.058s wall clock time, 43.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6172worker1 # [ 133.998801] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:52Z level=info event=signal_exit6173lb # [ 134.141773] niks3-server[1136]: 2026/09/25 15:14:53 INFO lead: released remote=192.168.1.3:425406174lb # [ 134.865348] niks3-server[1136]: 2026/09/25 15:14:54 INFO lead: acquired remote=192.168.1.4:456886175worker2 # [ 134.851578] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:53Z level=info event=scheduler_take_over6176worker1 # [ 135.145105] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:53Z level=info event=scheduler_restarting6177lb # [ 135.142126] envoy[924]: [2026-09-25 15:14:54.334][924][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6178worker1 # [ 135.148202] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:53Z level=info event=scheduler_restarting addr=lb:500516179worker1 # [ 135.150798] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:449746180worker2 # [ 135.131993] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:53Z level=info event=scheduler_restarting addr=lb:500516181worker1 # [ 135.152688] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:53Z level=info event=worker_gone addr=192.168.1.3:500516182worker1 # [ 135.154255] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:449746183worker1 # [ 135.155639] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:53Z level=info event=worker_gone addr=192.168.1.4:500516184worker1 # [ 135.174955] nix-daemon[848]: reaped child process 1298, status = succeeded6185worker2 # [ 135.333413] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:53Z level=info event=scheduler_connected addr=lb:500516186worker2 # [ 135.339798] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:53Z level=info event=oidc_jwks provider=mock keys=16187worker2 # [ 135.341277] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:463686188worker2 # [ 135.342816] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06189worker1 # [ 135.363860] nix-grpc-daemon[1246]: ts=2026-09-25T15:14:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6190worker2 # [ 135.362076] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:53Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:463786191worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.16 seconds)6192client: waiting for success: ! systemctl is-active rs6193client: (finished: waiting for success: ! systemctl is-active rs, in 0.03 seconds)6194client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6195client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.02 seconds)6196client: must fail: journalctl -u rs | grep -q 'reconnecting'6197client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)6198lb: must succeed: curl -sf localhost:9901/clusters6199lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6200worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6201worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6202worker1 # [ 136.975579] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6203worker1 # [ 136.978993] systemd[1]: Stopped Nix worker-protocol over gRPC.6204worker1 # [ 136.982523] systemd[1]: nix-grpc-daemon.service: Consumed 988ms CPU time over 29.101s wall clock time, 38.1M memory peak, 328.6K incoming IP traffic, 139K outgoing IP traffic.6205worker1 # [ 136.997379] systemd[1]: Starting Nix worker-protocol over gRPC...6206lb # [ 137.072649] envoy[924]: conn peer=[2001:db8:1::1]:44010 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2943 tx=1498 ms=78966207lb # [ 137.074910] envoy[924]: conn peer=[2001:db8:1::3]:44092 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3620 tx=977 ms=273216208worker1 # [ 137.090389] nix-grpc-daemon[1500]: ts=2026-09-25T15:14:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6209worker1 # [ 137.092661] nix-grpc-daemon[1500]: ts=2026-09-25T15:14:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6210worker1 # [ 137.096618] nix-grpc-daemon[1500]: ts=2026-09-25T15:14:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16211worker1 # [ 137.101107] nix-grpc-daemon[1500]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6212worker1 # [ 137.102540] nix-grpc-daemon[1500]: E0000 00:00:1790349295.352448 1500 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6213worker1 # [ 137.106309] nix-grpc-daemon[1500]: E0000 00:00:1790349295.358245 1500 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6214worker1 # [ 137.110342] nix-grpc-daemon[1500]: ts=2026-09-25T15:14:55Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6215worker1 # [ 137.114644] systemd[1]: Started Nix worker-protocol over gRPC.6216worker1 # [ 137.118081] nix-grpc-daemon[1500]: E0000 00:00:1790349295.369377 1542 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6217worker1 # [ 137.130741] nix-grpc-daemon[1500]: E0000 00:00:1790349295.382767 1542 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6218worker1 # [ 137.136060] nix-grpc-daemon[1500]: E0000 00:00:1790349295.385381 1542 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6219worker1 # [ 137.145983] nix-grpc-daemon[1500]: E0000 00:00:1790349295.397934 1542 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6220worker1 # [ 137.154949] nix-grpc-daemon[1500]: ts=2026-09-25T15:14:55Z level=info event=scheduler_connected addr=lb:500516221worker2 # [ 137.138894] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:55Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:463686222worker2 # [ 137.142378] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:55Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06223lb: must succeed: curl -sf localhost:9901/clusters6224lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6225worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6226worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6227(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.20 seconds)6228subtest: scheduler restart mid-build: build finishes, no double build6229client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6230client # [ 137.275803] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6231client # Running as unit: mid.service; invocation ID: d87058416def4edaad246b4992a518dc6232client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6233worker1 # [ 137.436160] nix-grpc-daemon[1500]: ts=2026-09-25T15:14:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:360606234worker1 # [ 137.447889] nix-daemon[848]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)6235client # [ 137.449258] nix[1419]: don't know how to build these paths:6236client # [ 137.450187] nix[1419]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6237worker2 # [ 137.455228] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:55Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06238worker2 # [ 137.464654] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:55Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16239worker2 # [ 137.474546] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:463826240worker2 # [ 137.482883] nix-daemon[873]: accepted connection from pid 1529, user nix-grpc-daemon (trusted)6241worker2 # [ 137.501206] nix-grpc-daemon[1529]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5)6242worker2 # [ 137.601274] nix-grpc-daemon[1529]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 128 ms (attempt 2/5)6243worker2 # [ 137.731609] nix-grpc-daemon[1529]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 273 ms (attempt 3/5)6244worker2 # [ 138.006767] nix-grpc-daemon[1529]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 4/5)6245worker2 # [ 138.041242] nix-grpc-daemon[1529]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6246worker2 # [ 138.044197] nix-grpc-daemon[1529]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6247worker2 # [ 138.059853] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:463826248client # [ 138.075195] nix[1419]: copying 1 paths to 'lb:50051'...6249client # [ 138.076728] nix[1419]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6250worker2 # [ 138.070595] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:463826251worker2 # [ 138.078883] nix-daemon[873]: accepted connection from pid 1529, user nix-grpc-daemon (trusted)6252worker2 # [ 138.101683] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:56Z level=info event=niks3_push_started pid=16576253worker2 # [ 138.210490] nix-grpc-daemon[1657]: time=2026-09-25T15:14:56.520Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6254worker2 # [ 138.304695] nix-daemon[873]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6255worker2 # [ 138.319628] nix-daemon[873]: reaped child process 1673, status = succeeded6256lb # [ 138.341507] niks3-server[1136]: 2026/09/25 15:14:57 INFO Received push request method=POST path=/api/pushes6257worker2 # [ 138.333850] nix-grpc-daemon[1657]: time=2026-09-25T15:14:56.644Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6258worker2 # [ 138.335551] nix-grpc-daemon[1657]: time=2026-09-25T15:14:56.646Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6259lb # [ 138.371239] niks3-server[1136]: 2026/09/25 15:14:57 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign6260worker2 # [ 138.360398] nix-grpc-daemon[1657]: time=2026-09-25T15:14:56.671Z level=INFO msg="Uploading 1 narinfos"6261lb # [ 138.375496] niks3-server[1136]: 2026/09/25 15:14:57 INFO Signed narinfos id=50 count=16262lb # [ 138.380143] niks3-server[1136]: 2026/09/25 15:14:57 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6263lb # [ 138.384269] niks3-server[1136]: 2026/09/25 15:14:57 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6264lb # [ 138.391186] niks3-server[1136]: 2026/09/25 15:14:57 INFO Received complete push request method=POST path=/api/pushes/50/complete6265worker2 # [ 138.388130] nix-grpc-daemon[1657]: time=2026-09-25T15:14:56.698Z level=INFO msg="Upload complete. (174ms)"6266lb # [ 138.404395] niks3-server[1136]: 2026/09/25 15:14:57 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6267worker2 # [ 138.391164] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46382 duration_s=0 paths=1 nar_bytes_in=4246268worker2 # [ 138.394275] nix-grpc-daemon[1529]: ts=2026-09-25T15:14:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:463826269worker2 # [ 138.402028] nix-daemon[873]: accepted connection from pid 1529, user nix-grpc-daemon (trusted)6270worker2 # [ 138.408226] nix-daemon[873]: reaped child process 1655, status = succeeded6271worker2 # [ 138.411954] nix-daemon[873]: accepted connection from pid 1529, user nix-grpc-daemon (trusted)6272client # [ 138.417435] nix[1419]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6273lb: must succeed: curl -sf localhost:9901/clusters6274lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6275lb: must succeed: curl -sf localhost:9901/clusters6276lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6277worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6278worker2 # [ 139.649606] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1529 (nix-grpc-daemon) on client request.6279worker2 # [ 139.651757] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6280lb # [ 139.678952] niks3-server[1136]: 2026/09/25 15:14:58 INFO lead: released remote=192.168.1.4:456886281lb # [ 139.682884] envoy[924]: [2026-09-25 15:14:58.874][924][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}6282worker2 # [ 139.674134] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6283client # [ 139.680599] nix[1419]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6284worker2 # [ 139.675615] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6285worker1 # [ 139.692411] nix-grpc-daemon[1500]: ts=2026-09-25T15:14:57Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6286client # [ 139.682563] nix[1419]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6287worker2 # [ 139.678403] systemd[1]: nix-grpc-daemon.service: Consumed 403ms CPU time over 14.935s wall clock time, 35.8M memory peak, 214.8K incoming IP traffic, 53.9K outgoing IP traffic.6288worker2 # [ 139.688222] systemd[1]: Starting scheduler-token.service...6289worker2 # [ 139.714157] nix-daemon[873]: reaped child process 1675, status = succeeded6290worker2 # [ 139.716292] nix-daemon[873]: reaped child process 1649, status = succeeded6291worker2 # [ 139.725329] nix-daemon[1677]: unexpected Nix daemon error: error: interrupted by the user6292worker2 # [ 139.736327] nix-daemon[873]: reaped child process 1677, status = failed with exit code 16293worker2 # [ 139.748919] systemd[1]: scheduler-token.service: Deactivated successfully.6294worker2 # [ 139.750709] systemd[1]: Finished scheduler-token.service.6295worker2 # [ 139.752425] systemd[1]: scheduler-token.service: Consumed 19ms CPU time over 64ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6296worker2 # [ 139.755137] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6297worker2 # [ 139.758946] systemd[1]: Starting scheduler-token.service...6298worker2 # [ 139.796038] systemd[1]: scheduler-token.service: Deactivated successfully.6299worker2 # [ 139.797761] systemd[1]: Finished scheduler-token.service.6300worker2 # [ 139.800276] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 38ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6301worker2 # [ 139.804268] systemd[1]: Starting Nix worker-protocol over gRPC...6302worker1 # [ 139.893171] nix-grpc-daemon[1500]: ts=2026-09-25T15:14:58Z level=info event=scheduler_connected addr=lb:500516303worker2 # [ 139.889400] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6304worker2 # [ 139.891525] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6305worker2 # [ 139.895398] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16306worker2 # [ 139.900101] nix-grpc-daemon[1720]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6307worker2 # [ 139.903066] nix-grpc-daemon[1720]: E0000 00:00:1790349298.210487 1720 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6308worker2 # [ 139.905338] nix-grpc-daemon[1720]: E0000 00:00:1790349298.213475 1720 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6309worker2 # [ 139.911189] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6310worker2 # [ 139.912813] systemd[1]: Started Nix worker-protocol over gRPC.6311worker2 # [ 139.918109] nix-grpc-daemon[1720]: E0000 00:00:1790349298.228091 1761 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6312lb # [ 139.932989] niks3-server[1136]: 2026/09/25 15:14:59 INFO lead: acquired remote=192.168.1.4:450146313worker2 # [ 139.923218] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=scheduler_take_over6314worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.31 seconds)6315client: must succeed: sleep 5.06316worker2 # [ 139.938090] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:464026317worker2 # [ 139.942186] nix-grpc-daemon[1720]: E0000 00:00:1790349298.251495 1761 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6318worker2 # [ 139.944488] nix-grpc-daemon[1720]: E0000 00:00:1790349298.251623 1761 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6319worker2 # [ 139.950266] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06320worker2 # [ 139.952636] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:464146321worker2 # [ 139.964545] nix-grpc-daemon[1720]: E0000 00:00:1790349298.275167 1761 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6322worker2 # [ 139.974247] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=scheduler_connected addr=lb:500516323worker2 # [ 139.980066] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=oidc_jwks provider=mock keys=16324worker2 # [ 139.981720] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:464026325worker2 # [ 139.985227] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06326worker2 # [ 140.182361] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06327worker2 # [ 140.184445] nix-grpc-daemon[1720]: ts=2026-09-25T15:14:58Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16328worker1 # [ 140.204173] nix-grpc-daemon[1500]: ts=2026-09-25T15:14:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360606329worker1 # [ 140.210143] nix-grpc-daemon[1500]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5)6330worker1 # [ 140.306515] nix-grpc-daemon[1500]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5)6331worker1 # [ 140.340448] nix-grpc-daemon[1500]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 396 ms (attempt 3/5)6332worker1 # [ 140.739228] nix-grpc-daemon[1500]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 717 ms (attempt 4/5)6333worker1 # [ 141.458253] nix-grpc-daemon[1500]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6334worker1 # [ 141.461197] nix-grpc-daemon[1500]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6335worker1 # [ 141.476140] nix-grpc-daemon[1500]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6336worker1 # [ 141.500196] nix-daemon[848]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)6337client # [ 141.500337] nix[1419]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6338worker1 # [ 141.513662] nix-daemon[848]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)6339client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6340worker1 # [ 145.130484] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:03Z level=info event=niks3_push_started pid=16026341worker1 # [ 145.146772] nix-daemon[848]: reaped child process 1577, status = succeeded6342client: waiting for success: ! systemctl is-active mid6343worker1 # [ 145.223915] nix-grpc-daemon[1602]: time=2026-09-25T15:15:03.473Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6344worker1 # [ 145.301936] nix-daemon[848]: accepted connection from pid 1614, user nix-grpc-daemon (trusted)6345worker1 # [ 145.315382] nix-daemon[848]: reaped child process 1617, status = succeeded6346lb # [ 145.318260] niks3-server[1136]: 2026/09/25 15:15:04 INFO Received push request method=POST path=/api/pushes6347worker1 # [ 145.325702] nix-grpc-daemon[1602]: time=2026-09-25T15:15:03.577Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6348worker1 # [ 145.327198] nix-grpc-daemon[1602]: time=2026-09-25T15:15:03.579Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6349lb # [ 145.352966] niks3-server[1136]: 2026/09/25 15:15:04 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign6350lb # [ 145.358930] niks3-server[1136]: 2026/09/25 15:15:04 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6351lb # [ 145.363662] niks3-server[1136]: 2026/09/25 15:15:04 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6352lb # [ 145.374917] niks3-server[1136]: 2026/09/25 15:15:04 INFO Signed narinfos id=51 count=16353worker1 # [ 145.377508] nix-grpc-daemon[1602]: time=2026-09-25T15:15:03.629Z level=INFO msg="Uploading 1 narinfos"6354lb # [ 145.384283] niks3-server[1136]: 2026/09/25 15:15:04 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6355lb # [ 145.387415] niks3-server[1136]: 2026/09/25 15:15:04 INFO Received complete push request method=POST path=/api/pushes/51/complete6356worker1 # [ 145.398959] nix-grpc-daemon[1602]: time=2026-09-25T15:15:03.650Z level=INFO msg="Upload complete. (176ms)"6357lb # [ 145.399725] niks3-server[1136]: 2026/09/25 15:15:04 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6358worker1 # [ 145.401569] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36060 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16359worker1 # [ 145.408719] nix-daemon[848]: reaped child process 1575, status = succeeded6360client # [ 145.423199] systemd[1]: mid.service: Deactivated successfully.6361client # [ 145.426157] systemd[1]: mid.service: Consumed 141ms CPU time over 8.147s wall clock time, 43.5M memory peak, 8.6K incoming IP traffic, 10.3K outgoing IP traffic.6362client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6363client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6364client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6365(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.03 seconds)6366subtest: interrupting the client stops the build on the worker6367client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6368client # [ 146.277381] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6369client # Running as unit: intr.service; invocation ID: 7fa54d91678a42a68bd3104888306c346370client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6371worker2 # [ 146.396543] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:440006372worker2 # [ 146.408069] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6373client # [ 146.423914] nix[1455]: don't know how to build these paths:6374client # [ 146.424809] nix[1455]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6375worker2 # [ 146.427186] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:04Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06376worker2 # [ 146.431212] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:04Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16377worker2 # [ 146.436262] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006378worker2 # [ 146.444863] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)6379worker2 # [ 146.455943] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 2/5)6380worker2 # [ 146.566545] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 274 ms (attempt 3/5)6381worker2 # [ 146.842946] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 394 ms (attempt 4/5)6382lb # [ 147.073874] envoy[924]: conn peer=[2001:db8:1::4]:57672 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2089 tx=328 ms=147946383lb # [ 147.077726] envoy[924]: conn peer=[2001:db8:1::1]:40214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3934 tx=1965 ms=79846384worker2 # [ 147.239437] nix-grpc-daemon[1720]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6385worker2 # [ 147.242728] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6386worker2 # [ 147.259523] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440006387client # [ 147.275161] nix[1455]: copying 1 paths to 'lb:50051'...6388client # [ 147.279179] nix[1455]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6389worker2 # [ 147.274436] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440006390worker2 # [ 147.282163] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6391worker2 # [ 147.305483] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:05Z level=info event=niks3_push_started pid=17966392worker2 # [ 147.415354] nix-grpc-daemon[1796]: time=2026-09-25T15:15:05.725Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6393worker2 # [ 147.523645] nix-daemon[873]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6394worker2 # [ 147.544570] nix-daemon[873]: reaped child process 1816, status = succeeded6395lb # [ 147.558159] niks3-server[1136]: 2026/09/25 15:15:06 INFO Received push request method=POST path=/api/pushes6396worker2 # [ 147.554075] nix-grpc-daemon[1796]: time=2026-09-25T15:15:05.864Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6397worker2 # [ 147.555430] nix-grpc-daemon[1796]: time=2026-09-25T15:15:05.864Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6398lb # [ 147.598154] niks3-server[1136]: 2026/09/25 15:15:06 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6399lb # [ 147.607120] niks3-server[1136]: 2026/09/25 15:15:06 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign6400worker2 # [ 147.597949] nix-grpc-daemon[1796]: time=2026-09-25T15:15:05.907Z level=INFO msg="Uploading 1 narinfos"6401lb # [ 147.612562] niks3-server[1136]: 2026/09/25 15:15:06 INFO Signed narinfos id=52 count=16402lb # [ 147.619398] niks3-server[1136]: 2026/09/25 15:15:06 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6403lb # [ 147.630690] niks3-server[1136]: 2026/09/25 15:15:06 INFO Received complete push request method=POST path=/api/pushes/52/complete6404worker2 # [ 147.624160] nix-grpc-daemon[1796]: time=2026-09-25T15:15:05.934Z level=INFO msg="Upload complete. (205ms)"6405lb # [ 147.640200] niks3-server[1136]: 2026/09/25 15:15:06 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6406worker2 # [ 147.626852] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=0 paths=1 nar_bytes_in=4246407worker2 # [ 147.631739] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006408worker2 # [ 147.638394] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6409worker2 # [ 147.640935] nix-daemon[873]: reaped child process 1794, status = succeeded6410worker2 # [ 147.646329] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6411client # [ 147.652141] nix[1455]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6412client: must succeed: systemctl kill -s INT intr6413client # [ 148.660562] systemd[1]: intr.service: Sent signal SIGINT to main process 1455 (nix) on client request.6414client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6415worker2 # [ 148.661826] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:06Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6416client # [ 148.668946] nix[1455]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6417client # [ 148.674881] nix[1455]: error: interrupted by the user6418client # [ 148.703696] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6419client # [ 148.706564] systemd[1]: intr.service: Failed with result 'exit-code'.6420client # [ 148.709956] systemd[1]: intr.service: Consumed 119ms CPU time over 2.429s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6421worker2 # [ 148.866422] nix-daemon[1820]: unexpected Nix daemon error: error: interrupted by the user6422worker2 # [ 148.868734] nix-daemon[873]: reaped child process 1818, status = succeeded6423worker2 # [ 148.872824] nix-daemon[873]: reaped child process 1820, status = failed with exit code 16424(finished: subtest: interrupting the client stops the build on the worker, in 3.59 seconds)6425subtest: a deploy mid-build drains: the build finishes, the new generation takes over6426client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6427client # [ 149.898151] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6428client # Running as unit: sw.service; invocation ID: b4da6b7548114323a49902c42f1030a66429client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6430client # [ 150.037759] nix[1475]: don't know how to build these paths:6431client # [ 150.038722] nix[1475]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6432worker1 # [ 150.047071] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:360606433worker2 # [ 150.040627] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:08Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06434worker1 # [ 150.059961] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360606435worker2 # [ 150.044858] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:08Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16436worker1 # [ 150.071768] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360606437client # [ 150.069908] nix[1475]: copying 1 paths to 'lb:50051'...6438client # [ 150.070835] nix[1475]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6439worker1 # [ 150.083137] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:360606440worker1 # [ 150.093623] nix-daemon[848]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)6441worker1 # [ 150.197967] nix-daemon[848]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6442worker1 # [ 150.211133] nix-daemon[848]: reaped child process 1648, status = succeeded6443lb # [ 150.215539] niks3-server[1136]: 2026/09/25 15:15:09 INFO Received push request method=POST path=/api/pushes6444worker1 # [ 150.226888] nix-grpc-daemon[1602]: time=2026-09-25T15:15:08.478Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6445worker1 # [ 150.228349] nix-grpc-daemon[1602]: time=2026-09-25T15:15:08.478Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6446lb # [ 150.245764] niks3-server[1136]: 2026/09/25 15:15:09 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6447worker1 # [ 150.253220] nix-grpc-daemon[1602]: time=2026-09-25T15:15:08.504Z level=INFO msg="Uploading 1 narinfos"6448lb # [ 150.250131] niks3-server[1136]: 2026/09/25 15:15:09 INFO Signed narinfos id=53 count=16449lb # [ 150.258638] niks3-server[1136]: 2026/09/25 15:15:09 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6450lb # [ 150.264972] niks3-server[1136]: 2026/09/25 15:15:09 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6451lb # [ 150.276184] niks3-server[1136]: 2026/09/25 15:15:09 INFO Received complete push request method=POST path=/api/pushes/53/complete6452lb # [ 150.288515] niks3-server[1136]: 2026/09/25 15:15:09 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6453worker1 # [ 150.290217] nix-grpc-daemon[1602]: time=2026-09-25T15:15:08.542Z level=INFO msg="Upload complete. (184ms)"6454worker1 # [ 150.294807] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36060 duration_s=0 paths=1 nar_bytes_in=4166455worker1 # [ 150.300329] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360606456worker1 # [ 150.310058] nix-daemon[848]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)6457worker1 # [ 150.314861] nix-daemon[848]: reaped child process 1643, status = succeeded6458worker1 # [ 150.318987] nix-daemon[848]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)6459client # [ 150.307068] nix[1475]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6460worker1: must succeed: systemctl reload nix-grpc-daemon.service6461worker1 # [ 152.285236] systemd[1]: Reloading Nix worker-protocol over gRPC...6462worker1 # [ 152.312470] systemd[1]: Reloaded Nix worker-protocol over gRPC.6463worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6464worker1: must succeed: pgrep -f 'read -t [9]0 x'6465worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6466client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26467worker2 # [ 152.479511] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:440006468client # don't know how to build these paths:6469client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6470worker2 # [ 152.487518] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440006471worker2 # [ 152.505390] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:10Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06472worker2 # [ 152.511384] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:10Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16473worker2 # [ 152.516859] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006474worker2 # [ 152.526312] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440006475client # copying 1 paths to 'lb:50051'...6476client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6477worker2 # [ 152.535740] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440006478worker2 # [ 152.548186] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6479worker2 # [ 152.645079] nix-daemon[873]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6480worker2 # [ 152.659222] nix-daemon[873]: reaped child process 1874, status = succeeded6481lb # [ 152.679863] niks3-server[1136]: 2026/09/25 15:15:11 INFO Received push request method=POST path=/api/pushes6482worker2 # [ 152.670742] nix-grpc-daemon[1796]: time=2026-09-25T15:15:10.981Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6483worker2 # [ 152.673109] nix-grpc-daemon[1796]: time=2026-09-25T15:15:10.983Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6484lb # [ 152.700317] niks3-server[1136]: 2026/09/25 15:15:11 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6485lb # [ 152.704714] niks3-server[1136]: 2026/09/25 15:15:11 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6486lb # [ 152.708395] niks3-server[1136]: 2026/09/25 15:15:11 INFO Signed narinfos id=54 count=16487worker2 # [ 152.694096] nix-grpc-daemon[1796]: time=2026-09-25T15:15:11.004Z level=INFO msg="Uploading 1 narinfos"6488lb # [ 152.714457] niks3-server[1136]: 2026/09/25 15:15:11 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6489lb # [ 152.719536] niks3-server[1136]: 2026/09/25 15:15:11 INFO Received complete push request method=POST path=/api/pushes/54/complete6490lb # [ 152.723897] niks3-server[1136]: 2026/09/25 15:15:11 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6491worker2 # [ 152.712195] nix-grpc-daemon[1796]: time=2026-09-25T15:15:11.022Z level=INFO msg="Upload complete. (154ms)"6492worker2 # [ 152.714444] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=0 paths=1 nar_bytes_in=5286493worker2 # [ 152.717054] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006494worker2 # [ 152.724687] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6495worker2 # [ 152.730128] nix-daemon[873]: reaped child process 1869, status = succeeded6496worker2 # [ 152.732931] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6497client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6498client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6499worker1 # [ 153.113594] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:11Z level=info event=draining6500client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6501client # farm-a> LOG-farm-a-during-drain6502worker2 # [ 153.976509] nix-daemon[873]: reaped child process 1878, status = succeeded6503worker2 # [ 154.059994] nix-daemon[873]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6504worker2 # [ 154.072642] nix-daemon[873]: reaped child process 1897, status = succeeded6505lb # [ 154.093620] niks3-server[1136]: 2026/09/25 15:15:13 INFO Received push request method=POST path=/api/pushes6506worker2 # [ 154.083690] nix-grpc-daemon[1796]: time=2026-09-25T15:15:12.394Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6507worker2 # [ 154.087337] nix-grpc-daemon[1796]: time=2026-09-25T15:15:12.395Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6508lb # [ 154.113220] niks3-server[1136]: 2026/09/25 15:15:13 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6509lb # [ 154.119075] niks3-server[1136]: 2026/09/25 15:15:13 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6510lb # [ 154.120660] niks3-server[1136]: 2026/09/25 15:15:13 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6511lb # [ 154.123903] niks3-server[1136]: 2026/09/25 15:15:13 INFO Signed narinfos id=55 count=16512worker2 # [ 154.110556] nix-grpc-daemon[1796]: time=2026-09-25T15:15:12.421Z level=INFO msg="Uploading 1 narinfos"6513lb # [ 154.126027] niks3-server[1136]: 2026/09/25 15:15:13 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6514lb # [ 154.132233] niks3-server[1136]: 2026/09/25 15:15:13 INFO Received complete push request method=POST path=/api/pushes/55/complete6515lb # [ 154.136901] niks3-server[1136]: 2026/09/25 15:15:13 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6516worker2 # [ 154.124439] nix-grpc-daemon[1796]: time=2026-09-25T15:15:12.435Z level=INFO msg="Upload complete. (182ms)"6517worker2 # [ 154.127281] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16518worker2 # [ 154.134428] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:12Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06519worker2 # [ 154.137068] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:12Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16520worker2 # [ 154.138782] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006521worker2 # [ 154.143483] nix-daemon[873]: reaped child process 1876, status = succeeded6522worker2 # [ 154.146888] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440006523worker2 # [ 154.152958] nix-grpc-daemon[1796]: time=2026-09-25T15:15:12.463Z level=INFO msg="All 1 paths already cached"6524client # copying 1 paths to 'lb:50051'...6525worker2 # [ 154.157912] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440006526client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6527worker2 # [ 154.174577] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6528worker2 # [ 154.261961] nix-daemon[873]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)6529worker2 # [ 154.276675] nix-daemon[873]: reaped child process 1906, status = succeeded6530lb # [ 154.297123] niks3-server[1136]: 2026/09/25 15:15:13 INFO Received push request method=POST path=/api/pushes6531worker2 # [ 154.290318] nix-grpc-daemon[1796]: time=2026-09-25T15:15:12.601Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6532worker2 # [ 154.291707] nix-grpc-daemon[1796]: time=2026-09-25T15:15:12.602Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6533lb # [ 154.329590] niks3-server[1136]: 2026/09/25 15:15:13 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6534lb # [ 154.336873] niks3-server[1136]: 2026/09/25 15:15:13 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6535lb # [ 154.341922] niks3-server[1136]: 2026/09/25 15:15:13 INFO Signed narinfos id=56 count=16536worker2 # [ 154.330840] nix-grpc-daemon[1796]: time=2026-09-25T15:15:12.641Z level=INFO msg="Uploading 1 narinfos"6537lb # [ 154.355733] niks3-server[1136]: 2026/09/25 15:15:13 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6538lb # [ 154.364905] niks3-server[1136]: 2026/09/25 15:15:13 INFO Received complete push request method=POST path=/api/pushes/56/complete6539lb # [ 154.374473] niks3-server[1136]: 2026/09/25 15:15:13 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6540worker2 # [ 154.359062] nix-grpc-daemon[1796]: time=2026-09-25T15:15:12.669Z level=INFO msg="Upload complete. (179ms)"6541worker2 # [ 154.363793] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=0 paths=1 nar_bytes_in=6406542worker2 # [ 154.369496] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006543worker2 # [ 154.381107] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6544worker2 # [ 154.393839] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6545client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6546worker2 # [ 154.396057] nix-daemon[873]: reaped child process 1901, status = succeeded6547client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6548client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6549client # farm-b> LOG-farm-b-during-drain6550worker2 # [ 155.810917] nix-daemon[873]: reaped child process 1910, status = succeeded6551worker2 # [ 155.889531] nix-daemon[873]: accepted connection from pid 1926, user nix-grpc-daemon (trusted)6552worker2 # [ 155.903473] nix-daemon[873]: reaped child process 1929, status = succeeded6553lb # [ 155.927827] niks3-server[1136]: 2026/09/25 15:15:15 INFO Received push request method=POST path=/api/pushes6554worker2 # [ 155.919800] nix-grpc-daemon[1796]: time=2026-09-25T15:15:14.230Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6555worker2 # [ 155.921276] nix-grpc-daemon[1796]: time=2026-09-25T15:15:14.232Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6556lb # [ 155.968341] niks3-server[1136]: 2026/09/25 15:15:15 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6557lb # [ 155.973847] niks3-server[1136]: 2026/09/25 15:15:15 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6558worker2 # [ 155.962801] nix-grpc-daemon[1796]: time=2026-09-25T15:15:14.273Z level=INFO msg="Uploading 1 narinfos"6559lb # [ 155.977546] niks3-server[1136]: 2026/09/25 15:15:15 INFO Signed narinfos id=57 count=16560lb # [ 155.984751] niks3-server[1136]: 2026/09/25 15:15:15 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6561lb # [ 155.991982] niks3-server[1136]: 2026/09/25 15:15:15 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6562lb # [ 156.001392] niks3-server[1136]: 2026/09/25 15:15:15 INFO Received complete push request method=POST path=/api/pushes/57/complete6563lb # [ 156.007561] niks3-server[1136]: 2026/09/25 15:15:15 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6564worker2 # [ 155.991542] nix-grpc-daemon[1796]: time=2026-09-25T15:15:14.302Z level=INFO msg="Upload complete. (188ms)"6565worker2 # [ 156.000802] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16566worker2 # [ 156.010626] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:14Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06567worker2 # [ 156.016603] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:14Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16568worker2 # [ 156.020879] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006569worker2 # [ 156.030319] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440006570worker2 # [ 156.034394] nix-daemon[873]: reaped child process 1908, status = succeeded6571worker2 # [ 156.036512] nix-grpc-daemon[1796]: time=2026-09-25T15:15:14.347Z level=INFO msg="All 2 paths already cached"6572client # copying 1 paths to 'lb:50051'...6573worker2 # [ 156.041492] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440006574client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6575worker2 # [ 156.052344] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6576worker2 # [ 156.140794] nix-daemon[873]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)6577worker2 # [ 156.155144] nix-daemon[873]: reaped child process 1938, status = succeeded6578lb # [ 156.176741] niks3-server[1136]: 2026/09/25 15:15:15 INFO Received push request method=POST path=/api/pushes6579worker2 # [ 156.170818] nix-grpc-daemon[1796]: time=2026-09-25T15:15:14.481Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6580worker2 # [ 156.172444] nix-grpc-daemon[1796]: time=2026-09-25T15:15:14.483Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6581lb # [ 156.213539] niks3-server[1136]: 2026/09/25 15:15:15 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6582lb # [ 156.219497] niks3-server[1136]: 2026/09/25 15:15:15 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6583worker2 # [ 156.211978] nix-grpc-daemon[1796]: time=2026-09-25T15:15:14.522Z level=INFO msg="Uploading 1 narinfos"6584lb # [ 156.226596] niks3-server[1136]: 2026/09/25 15:15:15 INFO Signed narinfos id=58 count=16585lb # [ 156.235755] niks3-server[1136]: 2026/09/25 15:15:15 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6586lb # [ 156.247534] niks3-server[1136]: 2026/09/25 15:15:15 INFO Received complete push request method=POST path=/api/pushes/58/complete6587lb # [ 156.256475] niks3-server[1136]: 2026/09/25 15:15:15 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6588worker2 # [ 156.246818] nix-grpc-daemon[1796]: time=2026-09-25T15:15:14.557Z level=INFO msg="Upload complete. (188ms)"6589worker2 # [ 156.251675] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=0 paths=1 nar_bytes_in=7686590worker2 # [ 156.259124] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006591worker2 # [ 156.269662] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6592worker2 # [ 156.279403] nix-daemon[873]: reaped child process 1933, status = succeeded6593client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6594worker2 # [ 156.288552] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6595client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6596client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6597lb # [ 157.074136] envoy[924]: conn peer=[2001:db8:1::1]:42224 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=829 ms=22846598client # farm-top> LOG-farm-top-during-drain6599worker2 # [ 157.106615] nix-daemon[873]: reaped child process 1942, status = succeeded6600worker2 # [ 157.183888] nix-daemon[873]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6601worker2 # [ 157.197752] nix-daemon[873]: reaped child process 1961, status = succeeded6602lb # [ 157.221482] niks3-server[1136]: 2026/09/25 15:15:16 INFO Received push request method=POST path=/api/pushes6603worker2 # [ 157.217557] nix-grpc-daemon[1796]: time=2026-09-25T15:15:15.528Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6604worker2 # [ 157.219066] nix-grpc-daemon[1796]: time=2026-09-25T15:15:15.529Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6605lb # [ 157.265357] niks3-server[1136]: 2026/09/25 15:15:16 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6606lb # [ 157.274149] niks3-server[1136]: 2026/09/25 15:15:16 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6607lb # [ 157.279139] niks3-server[1136]: 2026/09/25 15:15:16 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6608lb # [ 157.285192] niks3-server[1136]: 2026/09/25 15:15:16 INFO Signed narinfos id=59 count=16609worker2 # [ 157.274410] nix-grpc-daemon[1796]: time=2026-09-25T15:15:15.584Z level=INFO msg="Uploading 1 narinfos"6610lb # [ 157.293057] niks3-server[1136]: 2026/09/25 15:15:16 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6611lb # [ 157.301539] niks3-server[1136]: 2026/09/25 15:15:16 INFO Received complete push request method=POST path=/api/pushes/59/complete6612lb # [ 157.310675] niks3-server[1136]: 2026/09/25 15:15:16 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6613worker2 # [ 157.294535] nix-grpc-daemon[1796]: time=2026-09-25T15:15:15.604Z level=INFO msg="Upload complete. (196ms)"6614worker2 # [ 157.304320] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16615worker2 # [ 157.317544] nix-daemon[873]: reaped child process 1940, status = succeeded6616client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.00 seconds)6617client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6618client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6619worker1: must succeed: pkill -f 'read -t [9]0 x'6620worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6621client: waiting for success: ! systemctl is-active sw6622worker1 # [ 157.539776] nix-daemon[848]: reaped child process 1652, status = succeeded6623worker1 # [ 157.612793] nix-daemon[848]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6624worker1 # [ 157.626324] nix-daemon[848]: reaped child process 1696, status = succeeded6625lb # [ 157.628434] niks3-server[1136]: 2026/09/25 15:15:16 INFO Received push request method=POST path=/api/pushes6626worker1 # [ 157.639040] nix-grpc-daemon[1602]: time=2026-09-25T15:15:15.890Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6627worker1 # [ 157.641343] nix-grpc-daemon[1602]: time=2026-09-25T15:15:15.892Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6628lb # [ 157.671604] niks3-server[1136]: 2026/09/25 15:15:16 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6629lb # [ 157.674308] niks3-server[1136]: 2026/09/25 15:15:16 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6630lb # [ 157.683135] niks3-server[1136]: 2026/09/25 15:15:16 INFO Signed narinfos id=60 count=16631worker1 # [ 157.688845] nix-grpc-daemon[1602]: time=2026-09-25T15:15:15.940Z level=INFO msg="Uploading 1 narinfos"6632lb # [ 157.686395] niks3-server[1136]: 2026/09/25 15:15:16 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6633lb # [ 157.693134] niks3-server[1136]: 2026/09/25 15:15:16 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6634lb # [ 157.705135] niks3-server[1136]: 2026/09/25 15:15:16 INFO Received complete push request method=POST path=/api/pushes/60/complete6635worker1 # [ 157.715576] nix-grpc-daemon[1602]: time=2026-09-25T15:15:15.967Z level=INFO msg="Upload complete. (184ms)"6636lb # [ 157.714392] niks3-server[1136]: 2026/09/25 15:15:16 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6637worker1 # [ 157.722636] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36060 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16638worker1 # [ 157.736467] nix-daemon[848]: reaped child process 1650, status = succeeded6639client # [ 157.739651] systemd[1]: sw.service: Deactivated successfully.6640client # [ 157.743579] systemd[1]: sw.service: Consumed 119ms CPU time over 7.845s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6641worker1 # [ 158.117081] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:16Z level=info event=signal_exit6642client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6643client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6644client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6645worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6646worker2 # [ 159.108309] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:464026647worker2 # [ 159.111546] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:17Z level=info event=worker_gone addr=192.168.1.3:500516648worker1 # [ 159.134159] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:17Z level=info event=scheduler_restarting6649worker1 # [ 159.141479] nix-grpc-daemon[1500]: ts=2026-09-25T15:15:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6650worker1 # [ 159.157848] nix-daemon[848]: reaped child process 1556, status = succeeded6651worker1 # [ 161.145165] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6652worker1 # [ 161.148390] systemd[1]: nix-grpc-daemon.service: Consumed 545ms CPU time over 24.147s wall clock time, 34.9M memory peak, 47.1K incoming IP traffic, 53.5K outgoing IP traffic.6653worker1 # [ 161.159810] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6654worker1 # [ 161.168521] systemd[1]: Starting Nix worker-protocol over gRPC...6655worker1 # [ 161.258374] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6656worker1 # [ 161.260433] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6657worker1 # [ 161.263624] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16658worker1 # [ 161.268049] nix-grpc-daemon[1711]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6659worker1 # [ 161.271223] nix-grpc-daemon[1711]: E0000 00:00:1790349319.519947 1711 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6660worker1 # [ 161.273415] nix-grpc-daemon[1711]: E0000 00:00:1790349319.523067 1711 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6661worker1 # [ 161.276665] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:19Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6662worker1 # [ 161.280705] systemd[1]: Started Nix worker-protocol over gRPC.6663worker1 # [ 161.286095] nix-grpc-daemon[1711]: E0000 00:00:1790349319.537098 1754 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6664worker1 # [ 161.300082] nix-grpc-daemon[1711]: E0000 00:00:1790349319.550665 1754 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6665worker1 # [ 161.302349] nix-grpc-daemon[1711]: E0000 00:00:1790349319.550966 1754 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6666worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.71 seconds)6667lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux6668lb: must succeed: curl -sf localhost:9901/clusters6669worker1 # [ 161.313278] nix-grpc-daemon[1711]: E0000 00:00:1790349319.565333 1754 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6670worker1 # [ 161.325541] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:19Z level=info event=scheduler_connected addr=lb:500516671worker2 # [ 161.311863] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:464026672worker2 # [ 161.316963] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06673lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6674lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.05 seconds)6675lb: must succeed: curl -sf localhost:9901/clusters6676lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6677worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6678worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6679(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 11.57 seconds)6680subtest: queue gauges show a burst while every build blocks6681lb: must succeed: curl -sf localhost:9901/clusters6682lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6683lb: must succeed: curl -sf localhost:9901/clusters6684lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6685worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6686worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6687client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6688client # [ 161.566878] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6689client # Running as unit: burst.service; invocation ID: d2f865d37d91447f8cfa3febd5953c0e6690client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6691lb: must succeed: curl -sf localhost:9901/clusters6692lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6693lb: must succeed: curl -sf localhost:9901/clusters6694lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6695worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6696worker1 # [ 161.716700] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366006697worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6698worker1 # [ 161.728310] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6699client # [ 161.728707] nix[1531]: don't know how to build these paths:6700client # [ 161.729581] nix[1531]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6701client # [ 161.730512] nix[1531]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6702client # [ 161.731993] nix[1531]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6703client # [ 161.733215] nix[1531]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6704worker2 # [ 161.731918] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=06705worker2 # [ 161.735438] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=16706worker2 # [ 161.737330] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=06707worker1 # [ 161.752962] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366006708worker2 # [ 161.739077] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=16709worker2 # [ 161.740969] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=06710worker2 # [ 161.742573] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=06711worker2 # [ 161.744369] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006712worker2 # [ 161.748593] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440006713worker1 # [ 161.763828] nix-grpc-daemon[1711]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)6714client # [ 161.761069] nix[1531]: copying 1 paths to 'lb:50051'...6715client # [ 161.761877] nix[1531]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...6716worker2 # [ 161.755804] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440006717worker2 # [ 161.761266] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6718worker1 # [ 161.786557] nix-grpc-daemon[1711]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 183 ms (attempt 2/5)6719worker2 # [ 161.853903] nix-daemon[873]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6720worker2 # [ 161.868863] nix-daemon[873]: reaped child process 1979, status = succeeded6721lb # [ 161.890312] niks3-server[1136]: 2026/09/25 15:15:21 INFO Received push request method=POST path=/api/pushes6722worker2 # [ 161.883383] nix-grpc-daemon[1796]: time=2026-09-25T15:15:20.193Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6723worker2 # [ 161.885429] nix-grpc-daemon[1796]: time=2026-09-25T15:15:20.195Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"6724lb # [ 161.923544] niks3-server[1136]: 2026/09/25 15:15:21 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6725worker2 # [ 161.917706] nix-grpc-daemon[1796]: time=2026-09-25T15:15:20.228Z level=INFO msg="Uploading 1 narinfos"6726lb # [ 161.932568] niks3-server[1136]: 2026/09/25 15:15:21 INFO Signed narinfos id=61 count=16727lb # [ 161.939266] niks3-server[1136]: 2026/09/25 15:15:21 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst6728lb # [ 161.948250] niks3-server[1136]: 2026/09/25 15:15:21 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls6729lb # [ 161.962657] niks3-server[1136]: 2026/09/25 15:15:21 INFO Received complete push request method=POST path=/api/pushes/61/complete6730worker2 # [ 161.952604] nix-grpc-daemon[1796]: time=2026-09-25T15:15:20.263Z level=INFO msg="Upload complete. (186ms)"6731lb # [ 161.968096] niks3-server[1136]: 2026/09/25 15:15:21 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo6732worker1 # [ 161.971587] nix-grpc-daemon[1711]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 3/5)6733worker2 # [ 161.956810] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=0 paths=1 nar_bytes_in=4166734worker2 # [ 161.961567] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006735worker2 # [ 161.973105] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6736worker2 # [ 161.978389] nix-daemon[873]: reaped child process 1974, status = succeeded6737worker2 # [ 161.983624] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6738client # [ 161.988870] nix[1531]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...6739worker1 # [ 162.001191] nix-grpc-daemon[1711]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 676 ms (attempt 4/5)6740worker1 # [ 162.703799] nix-grpc-daemon[1711]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6741worker1 # [ 162.707419] nix-grpc-daemon[1711]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6742lb: must succeed: curl -sf localhost:9901/clusters6743worker1 # [ 162.723285] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366006744client # [ 162.726207] nix[1531]: copying 1 paths to 'lb:50051'...6745client # [ 162.728106] nix[1531]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...6746worker1 # [ 162.739120] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366006747worker1 # [ 162.752448] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6748lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6749lb: must succeed: curl -sf localhost:9901/clusters6750worker1 # [ 162.773388] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:21Z level=info event=niks3_push_started pid=17716751lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6752worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6753worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6754worker1 # [ 162.884505] nix-grpc-daemon[1771]: time=2026-09-25T15:15:21.136Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6755worker1 # [ 162.973514] nix-daemon[848]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6756worker1 # [ 162.986573] nix-daemon[848]: reaped child process 1790, status = succeeded6757lb # [ 162.990454] niks3-server[1136]: 2026/09/25 15:15:22 INFO Received push request method=POST path=/api/pushes6758worker1 # [ 162.997821] nix-grpc-daemon[1771]: time=2026-09-25T15:15:21.249Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6759worker1 # [ 162.999336] nix-grpc-daemon[1771]: time=2026-09-25T15:15:21.251Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"6760lb # [ 163.013672] niks3-server[1136]: 2026/09/25 15:15:22 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6761lb # [ 163.018095] niks3-server[1136]: 2026/09/25 15:15:22 INFO Signed narinfos id=62 count=16762worker1 # [ 163.022285] nix-grpc-daemon[1771]: time=2026-09-25T15:15:21.274Z level=INFO msg="Uploading 1 narinfos"6763lb # [ 163.020896] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst6764lb # [ 163.023815] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls6765worker2 # [ 163.011245] nix-daemon[873]: reaped child process 1983, status = succeeded6766lb # [ 163.029951] niks3-server[1136]: 2026/09/25 15:15:22 INFO Received complete push request method=POST path=/api/pushes/62/complete6767worker1 # [ 163.038090] nix-grpc-daemon[1771]: time=2026-09-25T15:15:21.290Z level=INFO msg="Upload complete. (150ms)"6768lb # [ 163.036253] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo6769worker1 # [ 163.040346] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=4166770worker1 # [ 163.044053] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366006771worker1 # [ 163.050162] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6772worker1 # [ 163.052757] nix-daemon[848]: reaped child process 1769, status = succeeded6773client # [ 163.045381] nix[1531]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...6774worker1 # [ 163.057518] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6775worker2 # [ 163.081965] nix-daemon[873]: accepted connection from pid 2007, user nix-grpc-daemon (trusted)6776worker2 # [ 163.098254] nix-daemon[873]: reaped child process 2010, status = succeeded6777lb # [ 163.116738] niks3-server[1136]: 2026/09/25 15:15:22 INFO Received push request method=POST path=/api/pushes6778worker2 # [ 163.109562] nix-grpc-daemon[1796]: time=2026-09-25T15:15:21.420Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6779worker2 # [ 163.111110] nix-grpc-daemon[1796]: time=2026-09-25T15:15:21.421Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"6780lb # [ 163.156198] niks3-server[1136]: 2026/09/25 15:15:22 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6781worker2 # [ 163.147052] nix-grpc-daemon[1796]: time=2026-09-25T15:15:21.457Z level=INFO msg="Uploading 1 narinfos"6782lb # [ 163.162205] niks3-server[1136]: 2026/09/25 15:15:22 INFO Signed narinfos id=63 count=16783lb # [ 163.166773] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6784lb # [ 163.171191] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6785lb # [ 163.176185] niks3-server[1136]: 2026/09/25 15:15:22 INFO Received complete push request method=POST path=/api/pushes/63/complete6786worker2 # [ 163.164866] nix-grpc-daemon[1796]: time=2026-09-25T15:15:21.475Z level=INFO msg="Upload complete. (172ms)"6787lb # [ 163.182168] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls6788worker2 # [ 163.167765] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=7 outputs=16789lb # [ 163.185614] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo6790worker2 # [ 163.172171] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:21Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=16791worker2 # [ 163.175061] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006792worker2 # [ 163.182326] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440006793worker2 # [ 163.185694] nix-daemon[873]: reaped child process 1981, status = succeeded6794worker2 # [ 163.188783] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440006795client # [ 163.194755] nix[1531]: copying 1 paths to 'lb:50051'...6796client # [ 163.196695] nix[1531]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...6797worker2 # [ 163.194074] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6798worker2 # [ 163.288644] nix-daemon[873]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6799worker2 # [ 163.305216] nix-daemon[873]: reaped child process 2017, status = succeeded6800lb # [ 163.327905] niks3-server[1136]: 2026/09/25 15:15:22 INFO Received push request method=POST path=/api/pushes6801worker2 # [ 163.322671] nix-grpc-daemon[1796]: time=2026-09-25T15:15:21.633Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6802worker2 # [ 163.324341] nix-grpc-daemon[1796]: time=2026-09-25T15:15:21.635Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"6803lb # [ 163.359954] niks3-server[1136]: 2026/09/25 15:15:22 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6804worker2 # [ 163.350957] nix-grpc-daemon[1796]: time=2026-09-25T15:15:21.661Z level=INFO msg="Uploading 1 narinfos"6805lb # [ 163.366236] niks3-server[1136]: 2026/09/25 15:15:22 INFO Signed narinfos id=64 count=16806lb # [ 163.372922] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst6807lb # [ 163.376718] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls6808lb # [ 163.385207] niks3-server[1136]: 2026/09/25 15:15:22 INFO Received complete push request method=POST path=/api/pushes/64/complete6809worker2 # [ 163.375082] nix-grpc-daemon[1796]: time=2026-09-25T15:15:21.685Z level=INFO msg="Upload complete. (175ms)"6810lb # [ 163.391702] niks3-server[1136]: 2026/09/25 15:15:22 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo6811worker2 # [ 163.377604] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=0 paths=1 nar_bytes_in=4166812worker2 # [ 163.381056] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440006813worker2 # [ 163.388051] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6814worker2 # [ 163.392711] nix-daemon[873]: reaped child process 2012, status = succeeded6815worker2 # [ 163.396714] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6816client # [ 163.402199] nix[1531]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...6817worker1 # [ 164.073310] nix-daemon[848]: reaped child process 1794, status = succeeded6818worker1 # [ 164.144388] nix-daemon[848]: accepted connection from pid 1816, user nix-grpc-daemon (trusted)6819worker1 # [ 164.156492] nix-daemon[848]: reaped child process 1819, status = succeeded6820lb # [ 164.161211] niks3-server[1136]: 2026/09/25 15:15:23 INFO Received push request method=POST path=/api/pushes6821worker1 # [ 164.167949] nix-grpc-daemon[1771]: time=2026-09-25T15:15:22.419Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6822worker1 # [ 164.169495] nix-grpc-daemon[1771]: time=2026-09-25T15:15:22.421Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"6823lb # [ 164.183910] niks3-server[1136]: 2026/09/25 15:15:23 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls6824lb # [ 164.186827] niks3-server[1136]: 2026/09/25 15:15:23 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6825lb # [ 164.188397] niks3-server[1136]: 2026/09/25 15:15:23 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6826lb # [ 164.190159] niks3-server[1136]: 2026/09/25 15:15:23 INFO Signed narinfos id=65 count=16827worker1 # [ 164.193244] nix-grpc-daemon[1771]: time=2026-09-25T15:15:22.444Z level=INFO msg="Uploading 1 narinfos"6828lb # [ 164.196332] niks3-server[1136]: 2026/09/25 15:15:23 INFO Received complete push request method=POST path=/api/pushes/65/complete6829lb # [ 164.200573] niks3-server[1136]: 2026/09/25 15:15:23 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6830lb # [ 164.204278] niks3-server[1136]: 2026/09/25 15:15:23 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo6831worker1 # [ 164.210246] nix-grpc-daemon[1771]: time=2026-09-25T15:15:22.462Z level=INFO msg="Upload complete. (146ms)"6832worker1 # [ 164.212860] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=8 outputs=16833worker2 # [ 164.196382] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:22Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=16834worker1 # [ 164.217158] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366006835worker1 # [ 164.224087] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366006836worker1 # [ 164.225906] nix-daemon[848]: reaped child process 1792, status = succeeded6837worker1 # [ 164.230509] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366006838client # [ 164.218841] nix[1531]: copying 1 paths to 'lb:50051'...6839client # [ 164.221444] nix[1531]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...6840worker1 # [ 164.238969] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6841worker1 # [ 164.321475] nix-daemon[848]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)6842worker1 # [ 164.335097] nix-daemon[848]: reaped child process 1826, status = succeeded6843lb # [ 164.336823] niks3-server[1136]: 2026/09/25 15:15:23 INFO Received push request method=POST path=/api/pushes6844worker1 # [ 164.343427] nix-grpc-daemon[1771]: time=2026-09-25T15:15:22.595Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6845worker1 # [ 164.345375] nix-grpc-daemon[1771]: time=2026-09-25T15:15:22.596Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"6846lb # [ 164.351635] niks3-server[1136]: 2026/09/25 15:15:23 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6847lb # [ 164.353963] niks3-server[1136]: 2026/09/25 15:15:23 INFO Signed narinfos id=66 count=16848worker1 # [ 164.356977] nix-grpc-daemon[1771]: time=2026-09-25T15:15:22.608Z level=INFO msg="Uploading 1 narinfos"6849lb # [ 164.355588] niks3-server[1136]: 2026/09/25 15:15:23 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls6850lb # [ 164.359065] niks3-server[1136]: 2026/09/25 15:15:23 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst6851lb # [ 164.363973] niks3-server[1136]: 2026/09/25 15:15:23 INFO Received complete push request method=POST path=/api/pushes/66/complete6852worker1 # [ 164.371763] nix-grpc-daemon[1771]: time=2026-09-25T15:15:22.623Z level=INFO msg="Upload complete. (127ms)"6853lb # [ 164.370070] niks3-server[1136]: 2026/09/25 15:15:23 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo6854worker1 # [ 164.373935] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=4166855worker1 # [ 164.377141] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366006856worker1 # [ 164.384117] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6857worker1 # [ 164.386572] nix-daemon[848]: reaped child process 1821, status = succeeded6858worker1 # [ 164.391686] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6859client # [ 164.379240] nix[1531]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...6860worker1 # [ 165.205715] nix-daemon[848]: reaped child process 1830, status = succeeded6861worker2 # [ 165.215390] nix-daemon[873]: reaped child process 2021, status = succeeded6862worker1 # [ 165.280890] nix-daemon[848]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6863worker1 # [ 165.293890] nix-daemon[848]: reaped child process 1854, status = succeeded6864lb # [ 165.297507] niks3-server[1136]: 2026/09/25 15:15:24 INFO Received push request method=POST path=/api/pushes6865worker1 # [ 165.304119] nix-grpc-daemon[1771]: time=2026-09-25T15:15:23.555Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6866worker2 # [ 165.288813] nix-daemon[873]: accepted connection from pid 2046, user nix-grpc-daemon (trusted)6867worker1 # [ 165.305737] nix-grpc-daemon[1771]: time=2026-09-25T15:15:23.557Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"6868lb # [ 165.315977] niks3-server[1136]: 2026/09/25 15:15:24 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6869worker1 # [ 165.318949] nix-grpc-daemon[1771]: time=2026-09-25T15:15:23.570Z level=INFO msg="Uploading 1 narinfos"6870worker2 # [ 165.302969] nix-daemon[873]: reaped child process 2049, status = succeeded6871lb # [ 165.317936] niks3-server[1136]: 2026/09/25 15:15:24 INFO Signed narinfos id=67 count=16872lb # [ 165.322273] niks3-server[1136]: 2026/09/25 15:15:24 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6873lb # [ 165.324355] niks3-server[1136]: 2026/09/25 15:15:24 INFO Received push request method=POST path=/api/pushes6874lb # [ 165.326268] niks3-server[1136]: 2026/09/25 15:15:24 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls6875lb # [ 165.328558] niks3-server[1136]: 2026/09/25 15:15:24 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6876worker2 # [ 165.316850] nix-grpc-daemon[1796]: time=2026-09-25T15:15:23.627Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6877worker2 # [ 165.318479] nix-grpc-daemon[1796]: time=2026-09-25T15:15:23.629Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"6878lb # [ 165.333921] niks3-server[1136]: 2026/09/25 15:15:24 INFO Received complete push request method=POST path=/api/pushes/67/complete6879worker1 # [ 165.341201] nix-grpc-daemon[1771]: time=2026-09-25T15:15:23.593Z level=INFO msg="Upload complete. (143ms)"6880lb # [ 165.339430] niks3-server[1136]: 2026/09/25 15:15:24 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo6881worker1 # [ 165.343964] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=10 outputs=16882worker1 # [ 165.351352] nix-daemon[848]: reaped child process 1828, status = succeeded6883lb # [ 165.353441] niks3-server[1136]: 2026/09/25 15:15:24 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls6884lb # [ 165.356044] niks3-server[1136]: 2026/09/25 15:15:24 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6885lb # [ 165.359034] niks3-server[1136]: 2026/09/25 15:15:24 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6886lb # [ 165.361334] niks3-server[1136]: 2026/09/25 15:15:24 INFO Signed narinfos id=68 count=16887worker2 # [ 165.348061] nix-grpc-daemon[1796]: time=2026-09-25T15:15:23.658Z level=INFO msg="Uploading 1 narinfos"6888lb # [ 165.365981] niks3-server[1136]: 2026/09/25 15:15:24 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6889lb # [ 165.369076] niks3-server[1136]: 2026/09/25 15:15:24 INFO Received complete push request method=POST path=/api/pushes/68/complete6890worker2 # [ 165.359073] nix-grpc-daemon[1796]: time=2026-09-25T15:15:23.669Z level=INFO msg="Upload complete. (155ms)"6891lb # [ 165.376102] niks3-server[1136]: 2026/09/25 15:15:24 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo6892worker2 # [ 165.362045] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=9 outputs=16893worker2 # [ 165.370340] nix-daemon[873]: reaped child process 2019, status = succeeded6894client # [ 165.400536] systemd[1]: burst.service: Deactivated successfully.6895client # [ 165.403781] systemd[1]: burst.service: Consumed 142ms CPU time over 3.833s wall clock time, 57.9M memory peak, 13.6K incoming IP traffic, 18.1K outgoing IP traffic.6896client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6897client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.02 seconds)6898lb: must succeed: curl -sf localhost:9901/clusters6899lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6900lb: must succeed: curl -sf localhost:9901/clusters6901lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6902worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6903worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6904(finished: subtest: queue gauges show a burst while every build blocks, in 5.03 seconds)6905subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6906client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16907worker1 # [ 166.600461] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366006908worker1 # [ 166.603130] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366006909worker2 # [ 166.604853] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:24Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06910worker1 # [ 166.628527] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366006911worker2 # [ 166.609602] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:24Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16912worker2 # [ 166.614426] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:24Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06913worker1 # [ 166.635547] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366006914worker2 # [ 166.618752] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:24Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06915worker1 # [ 166.642249] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366006916worker1 # [ 166.647438] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6917worker1 # [ 166.729650] nix-daemon[848]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6918worker1 # [ 166.742494] nix-daemon[848]: reaped child process 1865, status = succeeded6919lb # [ 166.746344] niks3-server[1136]: 2026/09/25 15:15:25 INFO Received push request method=POST path=/api/pushes6920worker1 # [ 166.756559] nix-grpc-daemon[1771]: time=2026-09-25T15:15:25.008Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6921worker1 # [ 166.758578] nix-grpc-daemon[1771]: time=2026-09-25T15:15:25.010Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6922lb # [ 166.774385] niks3-server[1136]: 2026/09/25 15:15:25 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6923worker1 # [ 166.783022] nix-grpc-daemon[1771]: time=2026-09-25T15:15:25.034Z level=INFO msg="Uploading 1 narinfos"6924lb # [ 166.780884] niks3-server[1136]: 2026/09/25 15:15:25 INFO Signed narinfos id=69 count=16925lb # [ 166.787619] niks3-server[1136]: 2026/09/25 15:15:25 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6926lb # [ 166.795276] niks3-server[1136]: 2026/09/25 15:15:25 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6927lb # [ 166.804560] niks3-server[1136]: 2026/09/25 15:15:25 INFO Received complete push request method=POST path=/api/pushes/69/complete6928worker1 # [ 166.815487] nix-grpc-daemon[1771]: time=2026-09-25T15:15:25.067Z level=INFO msg="Upload complete. (163ms)"6929worker1 # [ 166.817659] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=5046930lb # [ 166.817080] niks3-server[1136]: 2026/09/25 15:15:26 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6931worker1 # [ 166.821981] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366006932worker1 # [ 166.828097] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6933worker1 # [ 166.830768] nix-daemon[848]: reaped child process 1860, status = succeeded6934worker1 # [ 166.835469] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6935lb # [ 167.075317] envoy[924]: conn peer=[2001:db8:1::1]:42240 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9551 tx=4388 ms=48346936lb # [ 167.078945] envoy[924]: conn peer=[2001:db8:1::1]:42234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2945 tx=1522 ms=76856937lb # [ 167.082473] envoy[924]: conn peer=[2001:db8:1::3]:49880 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1148 tx=482 ms=219876938lb # [ 167.086120] envoy[924]: conn peer=[2001:db8:1::1]:44116 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9583 tx=5132 ms=36786939worker1 # [ 167.870069] nix-daemon[848]: reaped child process 1869, status = succeeded6940worker1 # [ 167.932577] nix-daemon[848]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)6941worker1 # [ 167.944967] nix-daemon[848]: reaped child process 1888, status = succeeded6942lb # [ 167.947785] niks3-server[1136]: 2026/09/25 15:15:27 INFO Received push request method=POST path=/api/pushes6943worker1 # [ 167.954829] nix-grpc-daemon[1771]: time=2026-09-25T15:15:26.206Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6944worker1 # [ 167.956341] nix-grpc-daemon[1771]: time=2026-09-25T15:15:26.208Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6945lb # [ 167.967438] niks3-server[1136]: 2026/09/25 15:15:27 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6946lb # [ 167.970758] niks3-server[1136]: 2026/09/25 15:15:27 INFO Signed narinfos id=70 count=16947worker1 # [ 167.973606] nix-grpc-daemon[1771]: time=2026-09-25T15:15:26.225Z level=INFO msg="Uploading 1 narinfos"6948lb # [ 167.974124] niks3-server[1136]: 2026/09/25 15:15:27 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6949lb # [ 167.977399] niks3-server[1136]: 2026/09/25 15:15:27 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6950lb # [ 167.979849] niks3-server[1136]: 2026/09/25 15:15:27 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6951lb # [ 167.985055] niks3-server[1136]: 2026/09/25 15:15:27 INFO Received complete push request method=POST path=/api/pushes/70/complete6952worker1 # [ 167.992644] nix-grpc-daemon[1771]: time=2026-09-25T15:15:26.244Z level=INFO msg="Upload complete. (150ms)"6953lb # [ 167.990415] niks3-server[1136]: 2026/09/25 15:15:27 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6954worker1 # [ 167.995309] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=11 outputs=16955worker2 # [ 167.980375] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:26Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16956worker1 # [ 167.999439] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366006957worker2 # [ 167.984151] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:26Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06958worker1 # [ 168.006290] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366006959worker1 # [ 168.008828] nix-daemon[848]: reaped child process 1867, status = succeeded6960worker1 # [ 168.012757] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366006961worker1 # [ 168.016476] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6962worker1 # [ 168.099748] nix-daemon[848]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)6963worker1 # [ 168.113359] nix-daemon[848]: reaped child process 1895, status = succeeded6964lb # [ 168.114143] niks3-server[1136]: 2026/09/25 15:15:27 INFO Received push request method=POST path=/api/pushes6965worker1 # [ 168.120764] nix-grpc-daemon[1771]: time=2026-09-25T15:15:26.372Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6966worker1 # [ 168.122200] nix-grpc-daemon[1771]: time=2026-09-25T15:15:26.374Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6967lb # [ 168.128783] niks3-server[1136]: 2026/09/25 15:15:27 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6968worker1 # [ 168.134028] nix-grpc-daemon[1771]: time=2026-09-25T15:15:26.386Z level=INFO msg="Uploading 1 narinfos"6969lb # [ 168.131986] niks3-server[1136]: 2026/09/25 15:15:27 INFO Signed narinfos id=71 count=16970lb # [ 168.134725] niks3-server[1136]: 2026/09/25 15:15:27 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6971lb # [ 168.137725] niks3-server[1136]: 2026/09/25 15:15:27 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6972lb # [ 168.142461] niks3-server[1136]: 2026/09/25 15:15:27 INFO Received complete push request method=POST path=/api/pushes/71/complete6973worker1 # [ 168.149249] nix-grpc-daemon[1771]: time=2026-09-25T15:15:26.401Z level=INFO msg="Upload complete. (127ms)"6974lb # [ 168.147610] niks3-server[1136]: 2026/09/25 15:15:27 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6975worker1 # [ 168.151253] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=5046976worker1 # [ 168.154523] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366006977worker1 # [ 168.161661] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6978worker1 # [ 168.163265] nix-daemon[848]: reaped child process 1890, status = succeeded6979worker1 # [ 168.167728] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6980worker1 # [ 169.182277] nix-daemon[848]: reaped child process 1899, status = succeeded6981worker1 # [ 169.256036] nix-daemon[848]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6982worker1 # [ 169.268313] nix-daemon[848]: reaped child process 1918, status = succeeded6983lb # [ 169.273468] niks3-server[1136]: 2026/09/25 15:15:28 INFO Received push request method=POST path=/api/pushes6984worker1 # [ 169.284556] nix-grpc-daemon[1771]: time=2026-09-25T15:15:27.536Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6985worker1 # [ 169.286027] nix-grpc-daemon[1771]: time=2026-09-25T15:15:27.538Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6986lb # [ 169.317432] niks3-server[1136]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6987lb # [ 169.322103] niks3-server[1136]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6988lb # [ 169.329243] niks3-server[1136]: 2026/09/25 15:15:28 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6989worker1 # [ 169.337605] nix-grpc-daemon[1771]: time=2026-09-25T15:15:27.589Z level=INFO msg="Uploading 1 narinfos"6990lb # [ 169.335111] niks3-server[1136]: 2026/09/25 15:15:28 INFO Signed narinfos id=72 count=16991lb # [ 169.338426] niks3-server[1136]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6992lb # [ 169.346254] niks3-server[1136]: 2026/09/25 15:15:28 INFO Received complete push request method=POST path=/api/pushes/72/complete6993lb # [ 169.351244] niks3-server[1136]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6994worker1 # [ 169.352935] nix-grpc-daemon[1771]: time=2026-09-25T15:15:27.604Z level=INFO msg="Upload complete. (177ms)"6995worker1 # [ 169.359508] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=12 outputs=16996worker2 # [ 169.345176] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:27Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16997worker1 # [ 169.366207] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366006998worker2 # [ 169.352865] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:27Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06999worker1 # [ 169.375557] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366007000worker1 # [ 169.380167] nix-daemon[848]: reaped child process 1897, status = succeeded7001worker1 # [ 169.383696] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366007002worker1 # [ 169.387354] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7003worker1 # [ 169.470528] nix-daemon[848]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)7004worker1 # [ 169.483246] nix-daemon[848]: reaped child process 1925, status = succeeded7005lb # [ 169.487155] niks3-server[1136]: 2026/09/25 15:15:28 INFO Received push request method=POST path=/api/pushes7006worker1 # [ 169.498263] nix-grpc-daemon[1771]: time=2026-09-25T15:15:27.750Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7007worker1 # [ 169.500189] nix-grpc-daemon[1771]: time=2026-09-25T15:15:27.751Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7008lb # [ 169.517419] niks3-server[1136]: 2026/09/25 15:15:28 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign7009lb # [ 169.521665] niks3-server[1136]: 2026/09/25 15:15:28 INFO Signed narinfos id=73 count=17010worker1 # [ 169.527261] nix-grpc-daemon[1771]: time=2026-09-25T15:15:27.779Z level=INFO msg="Uploading 1 narinfos"7011lb # [ 169.528781] niks3-server[1136]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7012lb # [ 169.533755] niks3-server[1136]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7013lb # [ 169.542661] niks3-server[1136]: 2026/09/25 15:15:28 INFO Received complete push request method=POST path=/api/pushes/73/complete7014worker1 # [ 169.548306] nix-grpc-daemon[1771]: time=2026-09-25T15:15:27.800Z level=INFO msg="Upload complete. (155ms)"7015lb # [ 169.547042] niks3-server[1136]: 2026/09/25 15:15:28 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7016worker1 # [ 169.550583] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=5047017worker1 # [ 169.553673] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007018worker1 # [ 169.557411] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7019worker1 # [ 169.565117] nix-daemon[848]: reaped child process 1920, status = succeeded7020worker1 # [ 169.568131] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7021worker1 # [ 170.583604] nix-daemon[848]: reaped child process 1929, status = succeeded7022worker1 # [ 170.656209] nix-daemon[848]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)7023worker1 # [ 170.668623] nix-daemon[848]: reaped child process 1948, status = succeeded7024lb # [ 170.673757] niks3-server[1136]: 2026/09/25 15:15:29 INFO Received push request method=POST path=/api/pushes7025worker1 # [ 170.683640] nix-grpc-daemon[1771]: time=2026-09-25T15:15:28.935Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7026worker1 # [ 170.685179] nix-grpc-daemon[1771]: time=2026-09-25T15:15:28.937Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7027lb # [ 170.715805] niks3-server[1136]: 2026/09/25 15:15:29 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7028lb # [ 170.722793] niks3-server[1136]: 2026/09/25 15:15:29 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign7029worker1 # [ 170.731389] nix-grpc-daemon[1771]: time=2026-09-25T15:15:28.983Z level=INFO msg="Uploading 1 narinfos"7030lb # [ 170.728628] niks3-server[1136]: 2026/09/25 15:15:29 INFO Signed narinfos id=74 count=17031lb # [ 170.732857] niks3-server[1136]: 2026/09/25 15:15:29 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7032lb # [ 170.742357] niks3-server[1136]: 2026/09/25 15:15:29 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7033lb # [ 170.752166] niks3-server[1136]: 2026/09/25 15:15:29 INFO Received complete push request method=POST path=/api/pushes/74/complete7034worker1 # [ 170.760735] nix-grpc-daemon[1771]: time=2026-09-25T15:15:29.012Z level=INFO msg="Upload complete. (186ms)"7035lb # [ 170.759295] niks3-server[1136]: 2026/09/25 15:15:29 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7036worker1 # [ 170.763515] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=13 outputs=17037worker2 # [ 170.748423] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:29Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17038worker1 # [ 170.767420] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007039worker2 # [ 170.752591] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:29Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07040worker1 # [ 170.774318] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366007041worker1 # [ 170.777046] nix-daemon[848]: reaped child process 1927, status = succeeded7042worker1 # [ 170.780682] nix-grpc-daemon[1771]: time=2026-09-25T15:15:29.032Z level=INFO msg="All 1 paths already cached"7043worker1 # [ 170.783959] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366007044worker1 # [ 170.787987] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7045worker1 # [ 170.872264] nix-daemon[848]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)7046worker1 # [ 170.883766] nix-daemon[848]: reaped child process 1955, status = succeeded7047lb # [ 170.887603] niks3-server[1136]: 2026/09/25 15:15:30 INFO Received push request method=POST path=/api/pushes7048worker1 # [ 170.897207] nix-grpc-daemon[1771]: time=2026-09-25T15:15:29.149Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7049worker1 # [ 170.898697] nix-grpc-daemon[1771]: time=2026-09-25T15:15:29.150Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7050lb # [ 170.920552] niks3-server[1136]: 2026/09/25 15:15:30 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign7051lb # [ 170.929476] niks3-server[1136]: 2026/09/25 15:15:30 INFO Signed narinfos id=75 count=17052worker1 # [ 170.936433] nix-grpc-daemon[1771]: time=2026-09-25T15:15:29.188Z level=INFO msg="Uploading 1 narinfos"7053lb # [ 170.934601] niks3-server[1136]: 2026/09/25 15:15:30 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7054lb # [ 170.940912] niks3-server[1136]: 2026/09/25 15:15:30 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7055lb # [ 170.955508] niks3-server[1136]: 2026/09/25 15:15:30 INFO Received complete push request method=POST path=/api/pushes/75/complete7056worker1 # [ 170.964302] nix-grpc-daemon[1771]: time=2026-09-25T15:15:29.216Z level=INFO msg="Upload complete. (170ms)"7057lb # [ 170.962045] niks3-server[1136]: 2026/09/25 15:15:30 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7058worker1 # [ 170.966729] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=6247059worker1 # [ 170.970475] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007060worker1 # [ 170.977114] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7061worker1 # [ 170.979495] nix-daemon[848]: reaped child process 1950, status = succeeded7062worker1 # [ 170.984578] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7063worker1 # [ 172.019774] nix-daemon[848]: reaped child process 1959, status = succeeded7064worker1 # [ 172.097335] nix-daemon[848]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)7065worker1 # [ 172.108629] nix-daemon[848]: reaped child process 1978, status = succeeded7066lb # [ 172.113443] niks3-server[1136]: 2026/09/25 15:15:31 INFO Received push request method=POST path=/api/pushes7067worker1 # [ 172.125044] nix-grpc-daemon[1771]: time=2026-09-25T15:15:30.376Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7068worker1 # [ 172.126838] nix-grpc-daemon[1771]: time=2026-09-25T15:15:30.378Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7069lb # [ 172.151282] niks3-server[1136]: 2026/09/25 15:15:31 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign7070worker1 # [ 172.159386] nix-grpc-daemon[1771]: time=2026-09-25T15:15:30.411Z level=INFO msg="Uploading 1 narinfos"7071lb # [ 172.157115] niks3-server[1136]: 2026/09/25 15:15:31 INFO Signed narinfos id=76 count=17072lb # [ 172.165560] niks3-server[1136]: 2026/09/25 15:15:31 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7073lb # [ 172.172296] niks3-server[1136]: 2026/09/25 15:15:31 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7074lb # [ 172.178113] niks3-server[1136]: 2026/09/25 15:15:31 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7075lb # [ 172.188319] niks3-server[1136]: 2026/09/25 15:15:31 INFO Received complete push request method=POST path=/api/pushes/76/complete7076lb # [ 172.195416] niks3-server[1136]: 2026/09/25 15:15:31 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7077worker1 # [ 172.197636] nix-grpc-daemon[1771]: time=2026-09-25T15:15:30.449Z level=INFO msg="Upload complete. (205ms)"7078worker2 # [ 172.187985] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:30Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17079worker1 # [ 172.203859] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=14 outputs=17080worker2 # [ 172.192147] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:30Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07081worker1 # [ 172.212667] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007082worker1 # [ 172.224267] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366007083worker1 # [ 172.232528] nix-daemon[848]: reaped child process 1957, status = succeeded7084worker1 # [ 172.235657] nix-grpc-daemon[1771]: time=2026-09-25T15:15:30.486Z level=INFO msg="All 1 paths already cached"7085worker1 # [ 172.239859] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366007086worker1 # [ 172.244824] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7087worker1 # [ 172.329473] nix-daemon[848]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)7088worker1 # [ 172.341805] nix-daemon[848]: reaped child process 1985, status = succeeded7089lb # [ 172.346340] niks3-server[1136]: 2026/09/25 15:15:31 INFO Received push request method=POST path=/api/pushes7090worker1 # [ 172.357083] nix-grpc-daemon[1771]: time=2026-09-25T15:15:30.608Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7091worker1 # [ 172.358531] nix-grpc-daemon[1771]: time=2026-09-25T15:15:30.608Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7092lb # [ 172.373149] niks3-server[1136]: 2026/09/25 15:15:31 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign7093lb # [ 172.380151] niks3-server[1136]: 2026/09/25 15:15:31 INFO Signed narinfos id=77 count=17094worker1 # [ 172.385818] nix-grpc-daemon[1771]: time=2026-09-25T15:15:30.637Z level=INFO msg="Uploading 1 narinfos"7095lb # [ 172.390540] niks3-server[1136]: 2026/09/25 15:15:31 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7096lb # [ 172.394334] niks3-server[1136]: 2026/09/25 15:15:31 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7097lb # [ 172.405705] niks3-server[1136]: 2026/09/25 15:15:31 INFO Received complete push request method=POST path=/api/pushes/77/complete7098worker1 # [ 172.413233] nix-grpc-daemon[1771]: time=2026-09-25T15:15:30.665Z level=INFO msg="Upload complete. (163ms)"7099lb # [ 172.412276] niks3-server[1136]: 2026/09/25 15:15:31 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7100worker1 # [ 172.415399] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=6247101worker1 # [ 172.419221] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007102worker1 # [ 172.426371] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7103worker1 # [ 172.430201] nix-daemon[848]: reaped child process 1980, status = succeeded7104worker1 # [ 172.434517] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7105worker1 # [ 173.449100] nix-daemon[848]: reaped child process 1989, status = succeeded7106worker1 # [ 173.523237] nix-daemon[848]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)7107worker1 # [ 173.535907] nix-daemon[848]: reaped child process 2008, status = succeeded7108lb # [ 173.539886] niks3-server[1136]: 2026/09/25 15:15:32 INFO Received push request method=POST path=/api/pushes7109worker1 # [ 173.546826] nix-grpc-daemon[1771]: time=2026-09-25T15:15:31.798Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7110worker1 # [ 173.548420] nix-grpc-daemon[1771]: time=2026-09-25T15:15:31.800Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7111lb # [ 173.563644] niks3-server[1136]: 2026/09/25 15:15:32 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign7112worker1 # [ 173.568967] nix-grpc-daemon[1771]: time=2026-09-25T15:15:31.820Z level=INFO msg="Uploading 1 narinfos"7113lb # [ 173.567198] niks3-server[1136]: 2026/09/25 15:15:32 INFO Signed narinfos id=78 count=17114lb # [ 173.569539] niks3-server[1136]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7115lb # [ 173.572435] niks3-server[1136]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7116lb # [ 173.575819] niks3-server[1136]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7117lb # [ 173.578752] niks3-server[1136]: 2026/09/25 15:15:32 INFO Received complete push request method=POST path=/api/pushes/78/complete7118worker1 # [ 173.586295] nix-grpc-daemon[1771]: time=2026-09-25T15:15:31.838Z level=INFO msg="Upload complete. (145ms)"7119lb # [ 173.584567] niks3-server[1136]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7120worker1 # [ 173.588929] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=15 outputs=17121worker1 # [ 173.593222] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007122worker2 # [ 173.575453] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:31Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17123worker1 # [ 173.599836] nix-daemon[848]: reaped child process 1987, status = succeeded7124worker1 # [ 173.601652] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366007125worker2 # [ 173.583185] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:31Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07126worker1 # [ 173.607340] nix-grpc-daemon[1771]: time=2026-09-25T15:15:31.859Z level=INFO msg="All 1 paths already cached"7127worker1 # [ 173.611207] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366007128worker1 # [ 173.615865] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7129worker1 # [ 173.697939] nix-daemon[848]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)7130worker1 # [ 173.710922] nix-daemon[848]: reaped child process 2015, status = succeeded7131lb # [ 173.714256] niks3-server[1136]: 2026/09/25 15:15:32 INFO Received push request method=POST path=/api/pushes7132worker1 # [ 173.724706] nix-grpc-daemon[1771]: time=2026-09-25T15:15:31.976Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7133worker1 # [ 173.726770] nix-grpc-daemon[1771]: time=2026-09-25T15:15:31.978Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7134lb # [ 173.745143] niks3-server[1136]: 2026/09/25 15:15:32 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign7135lb # [ 173.752598] niks3-server[1136]: 2026/09/25 15:15:32 INFO Signed narinfos id=79 count=17136worker1 # [ 173.758311] nix-grpc-daemon[1771]: time=2026-09-25T15:15:32.010Z level=INFO msg="Uploading 1 narinfos"7137lb # [ 173.757793] niks3-server[1136]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7138lb # [ 173.765642] niks3-server[1136]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7139lb # [ 173.776685] niks3-server[1136]: 2026/09/25 15:15:32 INFO Received complete push request method=POST path=/api/pushes/79/complete7140worker1 # [ 173.790779] nix-grpc-daemon[1771]: time=2026-09-25T15:15:32.042Z level=INFO msg="Upload complete. (170ms)"7141lb # [ 173.789491] niks3-server[1136]: 2026/09/25 15:15:32 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7142worker1 # [ 173.795885] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=6247143worker1 # [ 173.801996] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007144worker1 # [ 173.811436] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7145worker1 # [ 173.822412] nix-daemon[848]: reaped child process 2010, status = succeeded7146worker1 # [ 173.825685] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7147worker1 # [ 175.041166] nix-daemon[848]: reaped child process 2019, status = succeeded7148worker1 # [ 175.112895] nix-daemon[848]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)7149worker1 # [ 175.125101] nix-daemon[848]: reaped child process 2038, status = succeeded7150lb # [ 175.132290] niks3-server[1136]: 2026/09/25 15:15:34 INFO Received push request method=POST path=/api/pushes7151worker1 # [ 175.141853] nix-grpc-daemon[1771]: time=2026-09-25T15:15:33.393Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7152worker1 # [ 175.143567] nix-grpc-daemon[1771]: time=2026-09-25T15:15:33.395Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7153lb # [ 175.169411] niks3-server[1136]: 2026/09/25 15:15:34 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7154lb # [ 175.179439] niks3-server[1136]: 2026/09/25 15:15:34 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign7155worker1 # [ 175.187632] nix-grpc-daemon[1771]: time=2026-09-25T15:15:33.439Z level=INFO msg="Uploading 1 narinfos"7156lb # [ 175.184688] niks3-server[1136]: 2026/09/25 15:15:34 INFO Signed narinfos id=80 count=17157lb # [ 175.191843] niks3-server[1136]: 2026/09/25 15:15:34 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7158lb # [ 175.196733] niks3-server[1136]: 2026/09/25 15:15:34 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7159lb # [ 175.207850] niks3-server[1136]: 2026/09/25 15:15:34 INFO Received complete push request method=POST path=/api/pushes/80/complete7160lb # [ 175.213750] niks3-server[1136]: 2026/09/25 15:15:34 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7161worker1 # [ 175.215430] nix-grpc-daemon[1771]: time=2026-09-25T15:15:33.467Z level=INFO msg="Upload complete. (183ms)"7162worker1 # [ 175.221636] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=16 outputs=17163worker2 # [ 175.205619] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:33Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17164worker1 # [ 175.230309] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007165worker2 # [ 175.214459] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:33Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07166worker1 # [ 175.242520] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366007167worker1 # [ 175.249282] nix-daemon[848]: reaped child process 2017, status = succeeded7168worker1 # [ 175.252240] nix-grpc-daemon[1771]: time=2026-09-25T15:15:33.503Z level=INFO msg="All 2 paths already cached"7169worker1 # [ 175.257417] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366007170worker1 # [ 175.265376] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7171worker1 # [ 175.351026] nix-daemon[848]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)7172worker1 # [ 175.363848] nix-daemon[848]: reaped child process 2045, status = succeeded7173lb # [ 175.368974] niks3-server[1136]: 2026/09/25 15:15:34 INFO Received push request method=POST path=/api/pushes7174worker1 # [ 175.379025] nix-grpc-daemon[1771]: time=2026-09-25T15:15:33.631Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7175worker1 # [ 175.381067] nix-grpc-daemon[1771]: time=2026-09-25T15:15:33.632Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7176lb # [ 175.398306] niks3-server[1136]: 2026/09/25 15:15:34 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign7177lb # [ 175.403855] niks3-server[1136]: 2026/09/25 15:15:34 INFO Signed narinfos id=81 count=17178worker1 # [ 175.412163] nix-grpc-daemon[1771]: time=2026-09-25T15:15:33.664Z level=INFO msg="Uploading 1 narinfos"7179lb # [ 175.410482] niks3-server[1136]: 2026/09/25 15:15:34 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7180lb # [ 175.419119] niks3-server[1136]: 2026/09/25 15:15:34 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7181lb # [ 175.433476] niks3-server[1136]: 2026/09/25 15:15:34 INFO Received complete push request method=POST path=/api/pushes/81/complete7182worker1 # [ 175.443617] nix-grpc-daemon[1771]: time=2026-09-25T15:15:33.695Z level=INFO msg="Upload complete. (173ms)"7183lb # [ 175.441961] niks3-server[1136]: 2026/09/25 15:15:34 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7184worker1 # [ 175.447516] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=7527185worker1 # [ 175.452254] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007186worker1 # [ 175.462105] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7187worker1 # [ 175.467221] nix-daemon[848]: reaped child process 2040, status = succeeded7188worker1 # [ 175.472157] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7189worker1 # [ 176.286678] nix-daemon[848]: reaped child process 2049, status = succeeded7190worker1 # [ 176.359489] nix-daemon[848]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)7191worker1 # [ 176.372886] nix-daemon[848]: reaped child process 2068, status = succeeded7192lb # [ 176.377328] niks3-server[1136]: 2026/09/25 15:15:35 INFO Received push request method=POST path=/api/pushes7193worker1 # [ 176.387826] nix-grpc-daemon[1771]: time=2026-09-25T15:15:34.639Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7194worker1 # [ 176.389322] nix-grpc-daemon[1771]: time=2026-09-25T15:15:34.641Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7195lb # [ 176.418764] niks3-server[1136]: 2026/09/25 15:15:35 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7196lb # [ 176.425843] niks3-server[1136]: 2026/09/25 15:15:35 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7197lb # [ 176.428678] niks3-server[1136]: 2026/09/25 15:15:35 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign7198lb # [ 176.432757] niks3-server[1136]: 2026/09/25 15:15:35 INFO Signed narinfos id=82 count=17199worker1 # [ 176.438290] nix-grpc-daemon[1771]: time=2026-09-25T15:15:34.690Z level=INFO msg="Uploading 1 narinfos"7200lb # [ 176.447359] niks3-server[1136]: 2026/09/25 15:15:35 INFO Received complete push request method=POST path=/api/pushes/82/complete7201lb # [ 176.451333] niks3-server[1136]: 2026/09/25 15:15:35 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7202lb # [ 176.455922] niks3-server[1136]: 2026/09/25 15:15:35 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7203worker1 # [ 176.457618] nix-grpc-daemon[1771]: time=2026-09-25T15:15:34.709Z level=INFO msg="Upload complete. (179ms)"7204worker1 # [ 176.462437] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=17 outputs=17205worker2 # [ 176.446925] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:34Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17206worker1 # [ 176.469239] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007207worker1 # [ 176.478205] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366007208worker1 # [ 176.483695] nix-daemon[848]: reaped child process 2047, status = succeeded7209worker1 # [ 176.485709] nix-grpc-daemon[1771]: time=2026-09-25T15:15:34.737Z level=INFO msg="All 2 paths already cached"7210worker1 # [ 176.489825] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366007211worker1 # [ 176.493734] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7212worker1 # [ 176.576072] nix-daemon[848]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)7213worker1 # [ 176.588481] nix-daemon[848]: reaped child process 2075, status = succeeded7214lb # [ 176.593393] niks3-server[1136]: 2026/09/25 15:15:35 INFO Received push request method=POST path=/api/pushes7215worker1 # [ 176.603450] nix-grpc-daemon[1771]: time=2026-09-25T15:15:34.855Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7216worker1 # [ 176.605382] nix-grpc-daemon[1771]: time=2026-09-25T15:15:34.856Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7217lb # [ 176.624444] niks3-server[1136]: 2026/09/25 15:15:35 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7218lb # [ 176.631530] niks3-server[1136]: 2026/09/25 15:15:35 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign7219lb # [ 176.639075] niks3-server[1136]: 2026/09/25 15:15:35 INFO Signed narinfos id=83 count=17220worker1 # [ 176.645711] nix-grpc-daemon[1771]: time=2026-09-25T15:15:34.897Z level=INFO msg="Uploading 1 narinfos"7221lb # [ 176.644887] niks3-server[1136]: 2026/09/25 15:15:35 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7222lb # [ 176.658408] niks3-server[1136]: 2026/09/25 15:15:35 INFO Received complete push request method=POST path=/api/pushes/83/complete7223lb # [ 176.665472] niks3-server[1136]: 2026/09/25 15:15:35 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7224worker1 # [ 176.671229] nix-grpc-daemon[1771]: time=2026-09-25T15:15:34.923Z level=INFO msg="Upload complete. (171ms)"7225worker1 # [ 176.673387] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=7527226worker1 # [ 176.676139] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007227worker1 # [ 176.684489] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7228worker1 # [ 176.685876] nix-daemon[848]: reaped child process 2070, status = succeeded7229worker1 # [ 176.691156] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7230worker1 # [ 177.530578] nix-daemon[848]: reaped child process 2079, status = succeeded7231worker1 # [ 177.610093] nix-daemon[848]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)7232worker1 # [ 177.621540] nix-daemon[848]: reaped child process 2098, status = succeeded7233lb # [ 177.625681] niks3-server[1136]: 2026/09/25 15:15:36 INFO Received push request method=POST path=/api/pushes7234worker1 # [ 177.636760] nix-grpc-daemon[1771]: time=2026-09-25T15:15:35.888Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7235worker1 # [ 177.639060] nix-grpc-daemon[1771]: time=2026-09-25T15:15:35.890Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7236lb # [ 177.670363] niks3-server[1136]: 2026/09/25 15:15:36 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign7237lb # [ 177.673156] niks3-server[1136]: 2026/09/25 15:15:36 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7238worker1 # [ 177.680946] nix-grpc-daemon[1771]: time=2026-09-25T15:15:35.932Z level=INFO msg="Uploading 1 narinfos"7239lb # [ 177.678493] niks3-server[1136]: 2026/09/25 15:15:36 INFO Signed narinfos id=84 count=17240lb # [ 177.684332] niks3-server[1136]: 2026/09/25 15:15:36 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7241lb # [ 177.689360] niks3-server[1136]: 2026/09/25 15:15:36 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7242lb # [ 177.701518] niks3-server[1136]: 2026/09/25 15:15:36 INFO Received complete push request method=POST path=/api/pushes/84/complete7243lb # [ 177.705499] niks3-server[1136]: 2026/09/25 15:15:36 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7244worker1 # [ 177.708049] nix-grpc-daemon[1771]: time=2026-09-25T15:15:35.959Z level=INFO msg="Upload complete. (208ms)"7245worker2 # [ 177.696486] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:36Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17246worker1 # [ 177.713759] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=18 outputs=17247worker1 # [ 177.721874] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007248worker1 # [ 177.732606] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366007249worker1 # [ 177.741382] nix-daemon[848]: reaped child process 2077, status = succeeded7250worker1 # [ 177.744453] nix-grpc-daemon[1771]: time=2026-09-25T15:15:35.995Z level=INFO msg="All 2 paths already cached"7251worker1 # [ 177.749732] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366007252worker1 # [ 177.756283] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7253worker1 # [ 177.833795] nix-daemon[848]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)7254worker1 # [ 177.847397] nix-daemon[848]: reaped child process 2105, status = succeeded7255lb # [ 177.849226] niks3-server[1136]: 2026/09/25 15:15:37 INFO Received push request method=POST path=/api/pushes7256worker1 # [ 177.859426] nix-grpc-daemon[1771]: time=2026-09-25T15:15:36.111Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7257worker1 # [ 177.861424] nix-grpc-daemon[1771]: time=2026-09-25T15:15:36.112Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7258lb # [ 177.880068] niks3-server[1136]: 2026/09/25 15:15:37 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7259lb # [ 177.884423] niks3-server[1136]: 2026/09/25 15:15:37 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign7260lb # [ 177.889122] niks3-server[1136]: 2026/09/25 15:15:37 INFO Signed narinfos id=85 count=17261worker1 # [ 177.894915] nix-grpc-daemon[1771]: time=2026-09-25T15:15:36.146Z level=INFO msg="Uploading 1 narinfos"7262lb # [ 177.903951] niks3-server[1136]: 2026/09/25 15:15:37 INFO Received complete push request method=POST path=/api/pushes/85/complete7263lb # [ 177.911976] niks3-server[1136]: 2026/09/25 15:15:37 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7264worker1 # [ 177.919139] nix-grpc-daemon[1771]: time=2026-09-25T15:15:36.171Z level=INFO msg="Upload complete. (158ms)"7265lb # [ 177.919446] niks3-server[1136]: 2026/09/25 15:15:37 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7266worker1 # [ 177.923588] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 paths=1 nar_bytes_in=7527267worker1 # [ 177.928601] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366007268worker1 # [ 177.940063] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7269worker1 # [ 177.944297] nix-daemon[848]: reaped child process 2100, status = succeeded7270worker1 # [ 177.948724] nix-daemon[848]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)7271worker1 # [ 178.562154] nix-daemon[848]: reaped child process 2109, status = succeeded7272worker1 # [ 178.636191] nix-daemon[848]: accepted connection from pid 2125, user nix-grpc-daemon (trusted)7273worker1 # [ 178.649142] nix-daemon[848]: reaped child process 2128, status = succeeded7274lb # [ 178.652854] niks3-server[1136]: 2026/09/25 15:15:37 INFO Received push request method=POST path=/api/pushes7275worker1 # [ 178.660275] nix-grpc-daemon[1771]: time=2026-09-25T15:15:36.911Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7276worker1 # [ 178.661687] nix-grpc-daemon[1771]: time=2026-09-25T15:15:36.911Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7277lb # [ 178.673806] niks3-server[1136]: 2026/09/25 15:15:37 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7278lb # [ 178.676274] niks3-server[1136]: 2026/09/25 15:15:37 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7279lb # [ 178.681521] niks3-server[1136]: 2026/09/25 15:15:37 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign7280worker1 # [ 178.687407] nix-grpc-daemon[1771]: time=2026-09-25T15:15:36.939Z level=INFO msg="Uploading 1 narinfos"7281lb # [ 178.685077] niks3-server[1136]: 2026/09/25 15:15:37 INFO Signed narinfos id=86 count=17282lb # [ 178.695784] niks3-server[1136]: 2026/09/25 15:15:37 INFO Received complete push request method=POST path=/api/pushes/86/complete7283lb # [ 178.697644] niks3-server[1136]: 2026/09/25 15:15:37 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7284lb # [ 178.702302] niks3-server[1136]: 2026/09/25 15:15:37 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7285worker1 # [ 178.708325] nix-grpc-daemon[1771]: time=2026-09-25T15:15:36.960Z level=INFO msg="Upload complete. (152ms)"7286worker1 # [ 178.710733] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36600 duration_s=0 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=19 outputs=17287worker1 # [ 178.717727] nix-daemon[848]: reaped child process 2107, status = succeeded7288client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 12.30 seconds)7289worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7290worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7291client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7292client # [ 178.824822] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7293client # Running as unit: gpu.service; invocation ID: ee560515718b40d094d27fa8908eaede7294client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7295lb: must succeed: curl -sf localhost:9901/clusters7296lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7297lb: must succeed: curl -sf localhost:9901/clusters7298lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7299worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7300worker2 # [ 178.950373] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:440007301client # [ 178.962594] nix[1597]: don't know how to build these paths:7302client # [ 178.963520] nix[1597]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7303worker1 # [ 178.976821] nix-grpc-daemon[1711]: ts=2026-09-25T15:15:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366007304worker2 # [ 178.977506] nix-grpc-daemon[1720]: ts=2026-09-25T15:15:37Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07305worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7306client # [ 178.983890] nix[1597]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7307lb: must succeed: curl -sf localhost:9901/clusters7308lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7309lb: must succeed: curl -sf localhost:9901/clusters7310lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7311worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7312worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7313client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7314lb # [ 187.076645] envoy[924]: conn peer=[2001:db8:1::1]:57776 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27038 tx=12225 ms=121267315client # [ 208.986467] nix[1597]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7316client # [ 208.993554] nix[1597]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7317client # [ 209.020598] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7318client # [ 209.024399] systemd[1]: gpu.service: Failed with result 'exit-code'.7319client # [ 209.028436] systemd[1]: gpu.service: Consumed 136ms CPU time over 30.199s wall clock time, 35.6M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7320client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.23 seconds)7321client: must succeed: journalctl -u gpu -o cat7322client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7323lb: must succeed: curl -sf localhost:9901/clusters7324lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7325lb: must succeed: curl -sf localhost:9901/clusters7326lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7327worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7328worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7329(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 43.03 seconds)7330subtest: low disk drains a worker and builds go to the other7331worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7332worker1 # [ 210.297124] nix-grpc-daemon[1711]: ts=2026-09-25T15:16:08Z level=info event=unhealthy reason=min_free7333worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.91 seconds)7334worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7335worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.03 seconds)7336client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27337client # don't know how to build these paths:7338client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7339worker2 # [ 210.563485] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:440007340worker2 # [ 210.569752] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440007341worker2 # [ 210.586188] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:08Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07342worker2 # [ 210.592377] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:08Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17343worker2 # [ 210.597289] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440007344worker2 # [ 210.607309] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440007345client # copying 1 paths to 'lb:50051'...7346client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7347worker2 # [ 210.621274] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440007348worker2 # [ 210.632786] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7349worker2 # [ 210.735922] nix-daemon[873]: accepted connection from pid 2087, user nix-grpc-daemon (trusted)7350worker2 # [ 210.750674] nix-daemon[873]: reaped child process 2090, status = succeeded7351lb # [ 210.774477] niks3-server[1136]: 2026/09/25 15:16:09 INFO Received push request method=POST path=/api/pushes7352worker2 # [ 210.769152] nix-grpc-daemon[1796]: time=2026-09-25T15:16:09.079Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7353worker2 # [ 210.770587] nix-grpc-daemon[1796]: time=2026-09-25T15:16:09.079Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7354lb # [ 210.803146] niks3-server[1136]: 2026/09/25 15:16:09 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign7355lb # [ 210.809327] niks3-server[1136]: 2026/09/25 15:16:10 INFO Signed narinfos id=87 count=17356worker2 # [ 210.797416] nix-grpc-daemon[1796]: time=2026-09-25T15:16:09.107Z level=INFO msg="Uploading 1 narinfos"7357lb # [ 210.819743] niks3-server[1136]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7358lb # [ 210.831792] niks3-server[1136]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7359lb # [ 210.846057] niks3-server[1136]: 2026/09/25 15:16:10 INFO Received complete push request method=POST path=/api/pushes/87/complete7360worker2 # [ 210.834419] nix-grpc-daemon[1796]: time=2026-09-25T15:16:09.145Z level=INFO msg="Upload complete. (190ms)"7361lb # [ 210.853415] niks3-server[1136]: 2026/09/25 15:16:10 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7362worker2 # [ 210.838522] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=0 paths=1 nar_bytes_in=5127363worker2 # [ 210.844727] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440007364worker2 # [ 210.858105] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7365worker2 # [ 210.865720] nix-daemon[873]: reaped child process 2085, status = succeeded7366worker2 # [ 210.869976] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7367client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7368client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7369client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7370client # farm-a> LOG-farm-a-drain7371worker2 # [ 211.699943] nix-daemon[873]: reaped child process 2094, status = succeeded7372worker2 # [ 211.784446] nix-daemon[873]: accepted connection from pid 2110, user nix-grpc-daemon (trusted)7373worker2 # [ 211.798641] nix-daemon[873]: reaped child process 2113, status = succeeded7374lb # [ 211.821195] niks3-server[1136]: 2026/09/25 15:16:11 INFO Received push request method=POST path=/api/pushes7375worker2 # [ 211.817112] nix-grpc-daemon[1796]: time=2026-09-25T15:16:10.127Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7376worker2 # [ 211.818615] nix-grpc-daemon[1796]: time=2026-09-25T15:16:10.129Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7377lb # [ 211.857978] niks3-server[1136]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7378lb # [ 211.868125] niks3-server[1136]: 2026/09/25 15:16:11 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign7379lb # [ 211.872345] niks3-server[1136]: 2026/09/25 15:16:11 INFO Signed narinfos id=88 count=17380worker2 # [ 211.860929] nix-grpc-daemon[1796]: time=2026-09-25T15:16:10.171Z level=INFO msg="Uploading 1 narinfos"7381lb # [ 211.879057] niks3-server[1136]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7382lb # [ 211.882986] niks3-server[1136]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7383lb # [ 211.887174] niks3-server[1136]: 2026/09/25 15:16:11 INFO Received complete push request method=POST path=/api/pushes/88/complete7384lb # [ 211.894258] niks3-server[1136]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7385worker2 # [ 211.878571] nix-grpc-daemon[1796]: time=2026-09-25T15:16:10.189Z level=INFO msg="Upload complete. (201ms)"7386worker2 # [ 211.885932] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=20 outputs=17387worker2 # [ 211.895747] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:10Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07388worker2 # [ 211.899743] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:10Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17389worker2 # [ 211.904263] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440007390worker2 # [ 211.912777] nix-daemon[873]: reaped child process 2092, status = succeeded7391worker2 # [ 211.914218] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440007392worker2 # [ 211.920389] nix-grpc-daemon[1796]: time=2026-09-25T15:16:10.231Z level=INFO msg="All 1 paths already cached"7393client # copying 1 paths to 'lb:50051'...7394worker2 # [ 211.925576] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440007395client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7396worker2 # [ 211.930858] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7397worker2 # [ 212.018295] nix-daemon[873]: accepted connection from pid 2119, user nix-grpc-daemon (trusted)7398worker2 # [ 212.032135] nix-daemon[873]: reaped child process 2122, status = succeeded7399lb # [ 212.054756] niks3-server[1136]: 2026/09/25 15:16:11 INFO Received push request method=POST path=/api/pushes7400worker2 # [ 212.048130] nix-grpc-daemon[1796]: time=2026-09-25T15:16:10.358Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7401worker2 # [ 212.050042] nix-grpc-daemon[1796]: time=2026-09-25T15:16:10.360Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7402lb # [ 212.087330] niks3-server[1136]: 2026/09/25 15:16:11 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7403lb # [ 212.091067] niks3-server[1136]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7404lb # [ 212.096592] niks3-server[1136]: 2026/09/25 15:16:11 INFO Signed narinfos id=89 count=17405worker2 # [ 212.083924] nix-grpc-daemon[1796]: time=2026-09-25T15:16:10.394Z level=INFO msg="Uploading 1 narinfos"7406lb # [ 212.107222] niks3-server[1136]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7407lb # [ 212.109042] niks3-server[1136]: 2026/09/25 15:16:11 INFO Received complete push request method=POST path=/api/pushes/89/complete7408lb # [ 212.116618] niks3-server[1136]: 2026/09/25 15:16:11 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7409worker2 # [ 212.100927] nix-grpc-daemon[1796]: time=2026-09-25T15:16:10.411Z level=INFO msg="Upload complete. (163ms)"7410worker2 # [ 212.104842] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=0 paths=1 nar_bytes_in=6247411worker2 # [ 212.109815] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440007412worker2 # [ 212.120438] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7413worker2 # [ 212.123466] nix-daemon[873]: reaped child process 2117, status = succeeded7414worker2 # [ 212.127576] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7415client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7416client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7417client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7418client # farm-b> LOG-farm-b-drain7419worker2 # [ 213.369238] nix-daemon[873]: reaped child process 2126, status = succeeded7420worker2 # [ 213.453763] nix-daemon[873]: accepted connection from pid 2142, user nix-grpc-daemon (trusted)7421worker2 # [ 213.467194] nix-daemon[873]: reaped child process 2145, status = succeeded7422lb # [ 213.490686] niks3-server[1136]: 2026/09/25 15:16:12 INFO Received push request method=POST path=/api/pushes7423worker2 # [ 213.484358] nix-grpc-daemon[1796]: time=2026-09-25T15:16:11.794Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7424worker2 # [ 213.487244] nix-grpc-daemon[1796]: time=2026-09-25T15:16:11.796Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7425lb # [ 213.533298] niks3-server[1136]: 2026/09/25 15:16:12 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7426lb # [ 213.538242] niks3-server[1136]: 2026/09/25 15:16:12 INFO Signed narinfos id=90 count=17427worker2 # [ 213.526353] nix-grpc-daemon[1796]: time=2026-09-25T15:16:11.836Z level=INFO msg="Uploading 1 narinfos"7428lb # [ 213.549791] niks3-server[1136]: 2026/09/25 15:16:12 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7429lb # [ 213.555297] niks3-server[1136]: 2026/09/25 15:16:12 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7430lb # [ 213.559469] niks3-server[1136]: 2026/09/25 15:16:12 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7431lb # [ 213.568615] niks3-server[1136]: 2026/09/25 15:16:12 INFO Received complete push request method=POST path=/api/pushes/90/complete7432lb # [ 213.575956] niks3-server[1136]: 2026/09/25 15:16:12 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7433worker2 # [ 213.559638] nix-grpc-daemon[1796]: time=2026-09-25T15:16:11.870Z level=INFO msg="Upload complete. (223ms)"7434worker2 # [ 213.569217] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=21 outputs=17435worker2 # [ 213.577634] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:11Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07436worker2 # [ 213.582221] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:11Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17437worker2 # [ 213.586224] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440007438worker2 # [ 213.597324] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440007439worker2 # [ 213.599652] nix-daemon[873]: reaped child process 2124, status = succeeded7440worker2 # [ 213.605391] nix-grpc-daemon[1796]: time=2026-09-25T15:16:11.916Z level=INFO msg="All 2 paths already cached"7441client # copying 1 paths to 'lb:50051'...7442worker2 # [ 213.610865] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440007443client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7444worker2 # [ 213.616235] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7445worker2 # [ 213.708838] nix-daemon[873]: accepted connection from pid 2151, user nix-grpc-daemon (trusted)7446worker2 # [ 213.724713] nix-daemon[873]: reaped child process 2154, status = succeeded7447lb # [ 213.746266] niks3-server[1136]: 2026/09/25 15:16:12 INFO Received push request method=POST path=/api/pushes7448worker2 # [ 213.739750] nix-grpc-daemon[1796]: time=2026-09-25T15:16:12.050Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7449worker2 # [ 213.741848] nix-grpc-daemon[1796]: time=2026-09-25T15:16:12.052Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7450lb # [ 213.778986] niks3-server[1136]: 2026/09/25 15:16:12 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7451worker2 # [ 213.770230] nix-grpc-daemon[1796]: time=2026-09-25T15:16:12.080Z level=INFO msg="Uploading 1 narinfos"7452lb # [ 213.784777] niks3-server[1136]: 2026/09/25 15:16:12 INFO Signed narinfos id=91 count=17453lb # [ 213.791556] niks3-server[1136]: 2026/09/25 15:16:12 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7454lb # [ 213.798359] niks3-server[1136]: 2026/09/25 15:16:12 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7455lb # [ 213.811238] niks3-server[1136]: 2026/09/25 15:16:13 INFO Received complete push request method=POST path=/api/pushes/91/complete7456worker2 # [ 213.804197] nix-grpc-daemon[1796]: time=2026-09-25T15:16:12.114Z level=INFO msg="Upload complete. (181ms)"7457lb # [ 213.821184] niks3-server[1136]: 2026/09/25 15:16:13 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7458worker2 # [ 213.808353] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=0 paths=1 nar_bytes_in=7527459worker2 # [ 213.813380] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440007460worker2 # [ 213.825160] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7461worker2 # [ 213.829305] nix-daemon[873]: reaped child process 2149, status = succeeded7462client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7463worker2 # [ 213.835272] nix-daemon[873]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)7464client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7465client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7466client # farm-top> LOG-farm-top-drain7467worker2 # [ 215.271796] nix-daemon[873]: reaped child process 2158, status = succeeded7468worker2 # [ 215.359082] nix-daemon[873]: accepted connection from pid 2174, user nix-grpc-daemon (trusted)7469worker2 # [ 215.372144] nix-daemon[873]: reaped child process 2177, status = succeeded7470lb # [ 215.395473] niks3-server[1136]: 2026/09/25 15:16:14 INFO Received push request method=POST path=/api/pushes7471worker2 # [ 215.390027] nix-grpc-daemon[1796]: time=2026-09-25T15:16:13.700Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7472worker2 # [ 215.392780] nix-grpc-daemon[1796]: time=2026-09-25T15:16:13.702Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7473lb # [ 215.432627] niks3-server[1136]: 2026/09/25 15:16:14 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7474lb # [ 215.444718] niks3-server[1136]: 2026/09/25 15:16:14 INFO Received sign narinfos request method=POST path=/api/pushes/92/sign7475worker2 # [ 215.436751] nix-grpc-daemon[1796]: time=2026-09-25T15:16:13.747Z level=INFO msg="Uploading 1 narinfos"7476lb # [ 215.452069] niks3-server[1136]: 2026/09/25 15:16:14 INFO Signed narinfos id=92 count=17477lb # [ 215.454302] niks3-server[1136]: 2026/09/25 15:16:14 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7478lb # [ 215.460644] niks3-server[1136]: 2026/09/25 15:16:14 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7479lb # [ 215.471095] niks3-server[1136]: 2026/09/25 15:16:14 INFO Received complete push request method=POST path=/api/pushes/92/complete7480lb # [ 215.477309] niks3-server[1136]: 2026/09/25 15:16:14 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7481worker2 # [ 215.461797] nix-grpc-daemon[1796]: time=2026-09-25T15:16:13.772Z level=INFO msg="Upload complete. (218ms)"7482worker2 # [ 215.470259] nix-grpc-daemon[1720]: ts=2026-09-25T15:16:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44000 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=22 outputs=17483worker2 # [ 215.483920] nix-daemon[873]: reaped child process 2156, status = succeeded7484client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.08 seconds)7485client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7486client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7487worker1: must succeed: rm /nix/.rw-store/fill7488worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7489worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7490worker1 # [ 216.300979] nix-grpc-daemon[1711]: ts=2026-09-25T15:16:14Z level=info event=healthy reason=min_free7491worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)7492(finished: subtest: low disk drains a worker and builds go to the other, in 7.39 seconds)7493(finished: run the VM test script, in 217.77 seconds)7494test script finished in 217.94s7495cleanup7496kill QemuMachine (pid 49)7497client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7498client # [2026-09-25T15:16:16Z INFO virtiofsd] Client disconnected, shutting down7499client # [2026-09-25T15:16:16Z INFO virtiofsd] Client disconnected, shutting down7500client # [2026-09-25T15:16:16Z INFO virtiofsd] Client disconnected, shutting down7501lb # [ 217.078630] envoy[924]: conn peer=[2001:db8:1::1]:33896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300497502lb # [ 217.082620] envoy[924]: conn peer=[2001:db8:1::1]:53634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9538 tx=4341 ms=49217503kill QemuMachine (pid 52)7504lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7505lb # [2026-09-25T15:16:16Z INFO virtiofsd] Client disconnected, shutting down7506lb # [2026-09-25T15:16:16Z INFO virtiofsd] Client disconnected, shutting down7507lb # [2026-09-25T15:16:16Z INFO virtiofsd] Client disconnected, shutting down7508kill QemuMachine (pid 50)7509worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7510worker1 # [2026-09-25T15:16:16Z INFO virtiofsd] Client disconnected, shutting down7511worker1 # [2026-09-25T15:16:16Z INFO virtiofsd] Client disconnected, shutting down7512worker1 # [2026-09-25T15:16:16Z INFO virtiofsd] Client disconnected, shutting down7513kill QemuMachine (pid 51)7514worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7515worker2 # [2026-09-25T15:16:17Z INFO virtiofsd] Client disconnected, shutting down7516worker2 # [2026-09-25T15:16:17Z INFO virtiofsd] Client disconnected, shutting down7517worker2 # [2026-09-25T15:16:17Z INFO virtiofsd] Client disconnected, shutting down7518(finished: cleanup, in 1.08 seconds)7519additionally exposed symbols:7520 client, lb, worker1, worker2,7521 vlan1,7522 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh