vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #493
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13worker2 # Disk image does not exist, creating the virtualisation disk image...14lb # Formatting '/build/vm-state-lb/tmp.TQd8FAYjbX', fmt=raw size=107374182415worker2 # Formatting '/build/vm-state-worker2/tmp.DbbHWWRtla', fmt=raw size=107374182416lb: QEMU running (pid 50)17worker2: QEMU running (pid 52)18lb # mke2fs 1.47.4 (6-Mar-2025)19worker2 # mke2fs 1.47.4 (6-Mar-2025)20lb # Discarding device blocks: 0/262144 done21worker2 # Discarding device blocks: 0/262144 done22lb # Creating filesystem with 262144 4k blocks and 65536 inodes23worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes24worker2 # Filesystem UUID: 437d16bf-d8ea-4905-9723-1b8593ed43aa25client: QEMU running (pid 49)26lb # Filesystem UUID: f75b2145-08c0-4451-8625-b6ee7a193ee227worker2 # Superblock backups stored on blocks:28lb # Superblock backups stored on blocks:29worker2 # 32768, 98304, 163840, 22937630lb # 32768, 98304, 163840, 22937631worker2 # 32lb # 33worker2 # Allocating group tables: 0/8 done34lb # Allocating group tables: 0/8 done35lb # Writing inode tables: 0/8 done36worker2 # Writing inode tables: 0/8 done37worker2 # Creating journal (8192 blocks): done38client # Disk image does not exist, creating the virtualisation disk image...39client # Formatting '/build/vm-state-client/tmp.NI4yzNUwRW', fmt=raw size=107374182440worker2 # Writing superblocks and filesystem accounting information: 0/8 done41client # mke2fs 1.47.4 (6-Mar-2025)42worker2 # 43client # Discarding device blocks: 0/262144 done44worker2 # Virtualisation disk image created.45client # Creating filesystem with 262144 4k blocks and 65536 inodes46worker2 # Starting virtiofs daemons...47client # Filesystem UUID: 9931793c-58a6-4dad-8989-ab163697f11a48worker2 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)49client # Superblock backups stored on blocks:50worker2 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether51client # 32768, 98304, 163840, 22937652worker2 # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...53client # 54worker2 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)55client # Allocating group tables: 0/8 done56worker2 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether57client # Writing inode tables: 0/8 done58worker2 # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...59client # Creating journal (8192 blocks): done60worker2 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)61client # Writing superblocks and filesystem accounting information: 0/8 done62worker2 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether63client # 64worker2 # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...65client # Virtualisation disk image created.66worker2 # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests67client # Starting virtiofs daemons...68worker2 # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests69client # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)70worker2 # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests71client # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72lb # Creating journal (8192 blocks): done73client # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...74lb # Writing superblocks and filesystem accounting information: 0/8 done75client # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76lb # 77client # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78lb # Virtualisation disk image created.79lb # Starting virtiofs daemons...80client # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...81lb # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)82client # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83lb # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84client # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether85lb # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...86lb # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)87client # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...88client # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests89lb # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90client # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests91lb # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)92client # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests93lb # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether94lb # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...95lb # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...96lb # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests97lb # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests98lb # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests99worker1: QEMU running (pid 51)100worker1 # Disk image does not exist, creating the virtualisation disk image...101worker1 # Formatting '/build/vm-state-worker1/tmp.TPvSPUfoxC', fmt=raw size=1073741824102worker1 # mke2fs 1.47.4 (6-Mar-2025)103(finished: start all VMs, in 0.25 seconds)104worker1 # Discarding device blocks: 0/262144 done105client: waiting for unit network-addresses-eth1.service106worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes107client: waiting for the VM to finish booting108worker1 # Filesystem UUID: 6b8fed44-35b0-4f04-a597-546cb7f1ebf0109worker1 # Superblock backups stored on blocks:110worker1 # 32768, 98304, 163840, 229376111worker1 # 112worker1 # Allocating group tables: 0/8 done113worker1 # Writing inode tables: 0/8 done114worker1 # Creating journal (8192 blocks): done115worker1 # Writing superblocks and filesystem accounting information: 0/8 done116worker1 # 117worker1 # Virtualisation disk image created.118worker1 # Starting virtiofs daemons...119worker1 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker1 # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker1 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker1 # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker1 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker1 # [2026-09-22T10:58:02Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker1 # [2026-09-22T10:58:02Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-22T10:58:02Z INFO virtiofsd] Client connected, servicing requests131lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135lb # 136lb # 137lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138worker2 # 139worker2 # 140worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...142lb # 143lb # 144lb # 145lb # 146lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00147client # 148client # 149worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...150worker2 # 151worker2 # 152worker2 # 153worker2 # 154client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00155worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...157lb # 158lb # 159worker1 # 160client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...161worker1 # 162client # 163client # 164client # 165client # 166worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00167client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00168worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169worker2 # 170worker2 # 171worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...172worker1 # 173worker1 # 174worker1 # 175worker1 # 176worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00177client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178client # 179client # 180worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker1 # 182worker1 # 183lb # Booting from ROM...184worker2 # Booting from ROM...185worker1 # Booting from ROM...186client # Booting from ROM...187lb # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g4s62ig20603prhr6rpvm83lvwh8ggjh-nixos-system-lb-test/init regInfo=/nix/store/mw0sy7zlar36zs4cd786mchwg970135k-closure-info/registration console=ttyS0,115200n8 console=tty0189lb # [ 0.000000] BIOS-provided physical RAM map:190lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194worker2 # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026195lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved196lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved197lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved198lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved199lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved200worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kalq4lkvbh0szzzv7dibj1wlyg3hwwzn-nixos-system-worker2-test/init regInfo=/nix/store/v4168pp3vxgkq1k0sdqrg69zswlp474l-closure-info/registration console=ttyS0,115200n8 console=tty0201lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved202worker2 # [ 0.000000] BIOS-provided physical RAM map:203lb # [ 0.000000] NX (Execute Disable) protection: active204worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable205lb # [ 0.000000] APIC: Static calls initialized206lb # [ 0.000000] SMBIOS 2.8 present.207worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved208lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014209worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved210lb # [ 0.000000] DMI: Memory slots populated: 1/1211worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable212lb # [ 0.000000] Hypervisor detected: KVM213lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000214worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved215lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00216worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved217lb # [ 0.000000] kvm-clock: using sched offset of 511351764 cycles218worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved219worker1 # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026220lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns221worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved222lb # [ 0.000005] tsc: Detected 2400.012 MHz processor223worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved224lb # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000225worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved226lb # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs227worker2 # [ 0.000000] NX (Execute Disable) protection: active228worker2 # [ 0.000000] APIC: Static calls initialized229worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4v49g7hxh8q1vdjhx1q5arqap8yynmc7-nixos-system-worker1-test/init regInfo=/nix/store/s71x21530plar9f2qqi7wxm3dwbphzch-closure-info/registration console=ttyS0,115200n8 console=tty0230lb # [ 0.000846] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT231worker2 # [ 0.000000] SMBIOS 2.8 present.232worker1 # [ 0.000000] BIOS-provided physical RAM map:233lb # [ 0.002744] found SMP MP-table at [mem 0x000f5450-0x000f545f]234lb # [ 0.002759] Using GB pages for direct mapping235worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable236worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014237lb # [ 0.002860] RAMDISK: [mem 0x5e36e000-0x5ffcffff]238worker2 # [ 0.000000] DMI: Memory slots populated: 1/1239worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved240lb # [ 0.002869] ACPI: Early table checksum verification disabled241worker2 # [ 0.000000] Hypervisor detected: KVM242worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved243lb # [ 0.002873] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )244worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000245worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00246worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable247lb # [ 0.002878] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)248worker2 # [ 0.000000] kvm-clock: using sched offset of 499534922 cycles249worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved250lb # [ 0.002883] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)251worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved252worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns253lb # [ 0.002891] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)254worker2 # [ 0.000006] tsc: Detected 2400.012 MHz processor255worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved256lb # [ 0.002893] ACPI: FACS 0x000000005FFE0000 000040257worker2 # [ 0.000816] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000258worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved259lb # [ 0.002895] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)260worker2 # [ 0.000844] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs261worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved262lb # [ 0.002896] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)263worker2 # [ 0.000847] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT264worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved265client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026266worker1 # [ 0.000000] NX (Execute Disable) protection: active267worker2 # [ 0.002736] found SMP MP-table at [mem 0x000f5450-0x000f545f]268lb # [ 0.002898] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)269worker1 # [ 0.000000] APIC: Static calls initialized270worker2 # [ 0.002750] Using GB pages for direct mapping271worker1 # [ 0.000000] SMBIOS 2.8 present.272worker2 # [ 0.002889] RAMDISK: [mem 0x5e36f000-0x5ffcffff]273lb # [ 0.002899] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)274worker2 # [ 0.002897] ACPI: Early table checksum verification disabled275worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014276lb # [ 0.002901] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]277worker2 # [ 0.002901] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )278worker1 # [ 0.000000] DMI: Memory slots populated: 1/1279worker1 # [ 0.000000] Hypervisor detected: KVM280lb # [ 0.002902] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]281client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test/init regInfo=/nix/store/z24i12fyjab3zng5p8xysghgchx1bnfn-closure-info/registration console=ttyS0,115200n8 console=tty0282worker2 # [ 0.002905] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)283worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000284client # [ 0.000000] BIOS-provided physical RAM map:285lb # [ 0.002902] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]286worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00287client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable288worker2 # [ 0.002910] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)289lb # [ 0.002903] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]290worker1 # [ 0.000000] kvm-clock: using sched offset of 495045075 cycles291client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved292lb # [ 0.002903] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]293worker2 # [ 0.002917] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)294worker2 # [ 0.002920] ACPI: FACS 0x000000005FFE0000 000040295client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved296worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns297lb # [ 0.002904] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]298worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor299client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable300worker2 # [ 0.002921] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)301lb # [ 0.002904] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]302worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000303lb # [ 0.003144] No NUMA configuration found304client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved305worker2 # [ 0.002923] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)306lb # [ 0.003144] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]307worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs308client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved309lb # [ 0.003147] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]310worker2 # [ 0.002924] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)311worker1 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT312lb # [ 0.005309] Zone ranges:313client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved314worker1 # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]315lb # [ 0.005311] DMA [mem 0x0000000000001000-0x0000000000ffffff]316worker2 # [ 0.002926] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)317client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved318worker1 # [ 0.002741] Using GB pages for direct mapping319lb # [ 0.005313] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]320worker2 # [ 0.002927] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]321worker1 # [ 0.002865] RAMDISK: [mem 0x5e370000-0x5ffcffff]322client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved323lb # [ 0.005315] Normal empty324lb # [ 0.005316] Device empty325worker1 # [ 0.002872] ACPI: Early table checksum verification disabled326worker2 # [ 0.002928] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]327client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved328lb # [ 0.005316] Movable zone start for each node329worker1 # [ 0.002875] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )330lb # [ 0.005317] Early memory node ranges331client # [ 0.000000] NX (Execute Disable) protection: active332worker2 # [ 0.002929] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]333client # [ 0.000000] APIC: Static calls initialized334lb # [ 0.005318] node 0: [mem 0x0000000000001000-0x000000000009efff]335client # [ 0.000000] SMBIOS 2.8 present.336worker1 # [ 0.002879] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)337worker2 # [ 0.002929] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]338lb # [ 0.005319] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]339worker2 # [ 0.002930] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]340worker1 # [ 0.002884] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)341client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014342lb # [ 0.005320] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]343client # [ 0.000000] DMI: Memory slots populated: 1/1344worker2 # [ 0.002930] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]345worker1 # [ 0.002891] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)346lb # [ 0.005342] On node 0, zone DMA: 1 pages in unavailable ranges347client # [ 0.000000] Hypervisor detected: KVM348worker1 # [ 0.002893] ACPI: FACS 0x000000005FFE0000 000040349worker2 # [ 0.002931] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]350client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000351lb # [ 0.005594] On node 0, zone DMA: 97 pages in unavailable ranges352worker2 # [ 0.003155] No NUMA configuration found353client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00354lb # [ 0.030547] On node 0, zone DMA32: 40 pages in unavailable ranges355worker1 # [ 0.002895] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)356worker2 # [ 0.003156] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]357lb # [ 0.031011] ACPI: PM-Timer IO Port: 0x608358client # [ 0.000000] kvm-clock: using sched offset of 519649322 cycles359worker2 # [ 0.003158] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]360worker1 # [ 0.002896] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)361lb # [ 0.031024] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])362worker2 # [ 0.005261] Zone ranges:363client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns364lb # [ 0.031052] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23365worker1 # [ 0.002898] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)366worker2 # [ 0.005262] DMA [mem 0x0000000000001000-0x0000000000ffffff]367client # [ 0.000004] tsc: Detected 2400.012 MHz processor368lb # [ 0.031055] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)369worker2 # [ 0.005263] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]370client # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000371worker1 # [ 0.002899] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)372worker2 # [ 0.005264] Normal empty373worker2 # [ 0.005265] Device empty374lb # [ 0.031056] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)375worker1 # [ 0.002901] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]376worker2 # [ 0.005266] Movable zone start for each node377client # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs378lb # [ 0.031058] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)379worker2 # [ 0.005267] Early memory node ranges380client # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT381worker1 # [ 0.002902] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]382worker2 # [ 0.005267] node 0: [mem 0x0000000000001000-0x000000000009efff]383lb # [ 0.031059] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)384worker1 # [ 0.002902] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]385client # [ 0.002727] found SMP MP-table at [mem 0x000f5450-0x000f545f]386worker2 # [ 0.005268] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]387lb # [ 0.031059] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)388client # [ 0.002738] Using GB pages for direct mapping389worker1 # [ 0.002903] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]390client # [ 0.002816] RAMDISK: [mem 0x5e370000-0x5ffcffff]391lb # [ 0.031062] ACPI: Using ACPI (MADT) for SMP configuration information392worker2 # [ 0.005269] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]393worker1 # [ 0.002903] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]394lb # [ 0.031070] ACPI: HPET id: 0x8086a201 base: 0xfed00000395worker2 # [ 0.005288] On node 0, zone DMA: 1 pages in unavailable ranges396client # [ 0.002826] ACPI: Early table checksum verification disabled397lb # [ 0.031074] TSC deadline timer available398client # [ 0.002829] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )399worker1 # [ 0.002904] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]400worker2 # [ 0.005541] On node 0, zone DMA: 97 pages in unavailable ranges401lb # [ 0.031078] CPU topo: Max. logical packages: 1402lb # [ 0.031079] CPU topo: Max. logical dies: 1403worker2 # [ 0.030555] On node 0, zone DMA32: 40 pages in unavailable ranges404worker1 # [ 0.002904] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]405client # [ 0.002832] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)406worker2 # [ 0.031014] ACPI: PM-Timer IO Port: 0x608407worker1 # [ 0.003124] No NUMA configuration found408lb # [ 0.031079] CPU topo: Max. dies per package: 1409lb # [ 0.031083] CPU topo: Max. threads per core: 1410worker2 # [ 0.031026] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])411client # [ 0.002836] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)412worker1 # [ 0.003125] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]413lb # [ 0.031083] CPU topo: Num. cores per package: 1414worker2 # [ 0.031052] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23415worker1 # [ 0.003127] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]416client # [ 0.002842] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)417lb # [ 0.031084] CPU topo: Num. threads per package: 1418worker1 # [ 0.005257] Zone ranges:419worker2 # [ 0.031055] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)420client # [ 0.002845] ACPI: FACS 0x000000005FFE0000 000040421lb # [ 0.031084] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs422worker1 # [ 0.005258] DMA [mem 0x0000000000001000-0x0000000000ffffff]423worker2 # [ 0.031056] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)424client # [ 0.002846] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)425lb # [ 0.031106] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()426worker1 # [ 0.005259] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]427worker1 # [ 0.005261] Normal empty428worker2 # [ 0.031057] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)429worker1 # [ 0.005261] Device empty430lb # [ 0.031139] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]431client # [ 0.002848] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)432worker1 # [ 0.005262] Movable zone start for each node433worker2 # [ 0.031058] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)434worker1 # [ 0.005263] Early memory node ranges435lb # [ 0.031141] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]436client # [ 0.002849] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)437worker2 # [ 0.031059] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)438worker1 # [ 0.005263] node 0: [mem 0x0000000000001000-0x000000000009efff]439lb # [ 0.031142] [mem 0x60000000-0xafffffff] available for PCI devices440worker2 # [ 0.031061] ACPI: Using ACPI (MADT) for SMP configuration information441client # [ 0.002851] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)442lb # [ 0.031144] Booting paravirtualized kernel on KVM443worker1 # [ 0.005264] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]444worker2 # [ 0.031062] ACPI: HPET id: 0x8086a201 base: 0xfed00000445client # [ 0.002852] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]446worker1 # [ 0.005265] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]447worker2 # [ 0.031066] TSC deadline timer available448lb # [ 0.031148] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns449worker2 # [ 0.031070] CPU topo: Max. logical packages: 1450client # [ 0.002853] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]451worker1 # [ 0.005284] On node 0, zone DMA: 1 pages in unavailable ranges452worker2 # [ 0.031071] CPU topo: Max. logical dies: 1453lb # [ 0.035613] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1454worker1 # [ 0.005529] On node 0, zone DMA: 97 pages in unavailable ranges455client # [ 0.002854] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]456worker2 # [ 0.031071] CPU topo: Max. dies per package: 1457lb # [ 0.037791] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152458worker1 # [ 0.030486] On node 0, zone DMA32: 40 pages in unavailable ranges459worker2 # [ 0.031075] CPU topo: Max. threads per core: 1460client # [ 0.002854] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]461lb # [ 0.037835] kvm-guest: PV spinlocks disabled, single CPU462worker1 # [ 0.030951] ACPI: PM-Timer IO Port: 0x608463worker2 # [ 0.031075] CPU topo: Num. cores per package: 1464client # [ 0.002855] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]465worker2 # [ 0.031075] CPU topo: Num. threads per package: 1466worker1 # [ 0.030965] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])467client # [ 0.002855] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]468worker2 # [ 0.031076] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs469worker1 # [ 0.030992] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23470worker2 # [ 0.031096] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()471client # [ 0.002856] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]472worker1 # [ 0.030994] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)473client # [ 0.003088] No NUMA configuration found474worker1 # [ 0.030996] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)475worker2 # [ 0.031127] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]476client # [ 0.003089] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]477lb # [ 0.037837] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/g4s62ig20603prhr6rpvm83lvwh8ggjh-nixos-system-lb-test/init regInfo=/nix/store/mw0sy7zlar36zs4cd786mchwg970135k-closure-info/registration console=ttyS0,115200n8 console=tty0478worker1 # [ 0.030997] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)479worker2 # [ 0.031129] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]480client # [ 0.003091] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]481client # [ 0.005534] Zone ranges:482worker1 # [ 0.030998] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)483worker2 # [ 0.031130] [mem 0x60000000-0xafffffff] available for PCI devices484client # [ 0.005535] DMA [mem 0x0000000000001000-0x0000000000ffffff]485worker2 # [ 0.031132] Booting paravirtualized kernel on KVM486worker1 # [ 0.030999] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)487lb # [ 0.037930] Unknown kernel command line parameters "regInfo=/nix/store/mw0sy7zlar36zs4cd786mchwg970135k-closure-info/registration", will be passed to user space.488lb # [ 0.037943] random: crng init done489client # [ 0.005537] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]490worker1 # [ 0.031002] ACPI: Using ACPI (MADT) for SMP configuration information491client # [ 0.005538] Normal empty492worker2 # [ 0.031135] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns493lb # [ 0.037944] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes494client # [ 0.005539] Device empty495worker1 # [ 0.031003] ACPI: HPET id: 0x8086a201 base: 0xfed00000496client # [ 0.005539] Movable zone start for each node497worker1 # [ 0.031007] TSC deadline timer available498worker2 # [ 0.035604] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1499lb # [ 0.039959] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)500client # [ 0.005540] Early memory node ranges501worker1 # [ 0.031011] CPU topo: Max. logical packages: 1502worker1 # [ 0.031012] CPU topo: Max. logical dies: 1503worker2 # [ 0.037673] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152504lb # [ 0.040965] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)505client # [ 0.005541] node 0: [mem 0x0000000000001000-0x000000000009efff]506worker1 # [ 0.031012] CPU topo: Max. dies per package: 1507lb # [ 0.041001] Fallback order for Node 0: 0508worker2 # [ 0.037717] kvm-guest: PV spinlocks disabled, single CPU509client # [ 0.005542] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]510worker1 # [ 0.031016] CPU topo: Max. threads per core: 1511lb # [ 0.041003] Built 1 zonelists, mobility grouping on. Total pages: 393078512client # [ 0.005542] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]513worker1 # [ 0.031017] CPU topo: Num. cores per package: 1514lb # [ 0.041005] Policy zone: DMA32515worker1 # [ 0.031017] CPU topo: Num. threads per package: 1516client # [ 0.005566] On node 0, zone DMA: 1 pages in unavailable ranges517lb # [ 0.043365] mem auto-init: stack:all(zero), heap alloc:on, heap free:off518worker1 # [ 0.031017] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs519client # [ 0.005854] On node 0, zone DMA: 97 pages in unavailable ranges520lb # [ 0.046483] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1521client # [ 0.032311] On node 0, zone DMA32: 40 pages in unavailable ranges522worker1 # [ 0.031038] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()523lb # [ 0.049762] allocated 3145728 bytes of page_ext524client # [ 0.032778] ACPI: PM-Timer IO Port: 0x608525worker2 # [ 0.037719] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kalq4lkvbh0szzzv7dibj1wlyg3hwwzn-nixos-system-worker2-test/init regInfo=/nix/store/v4168pp3vxgkq1k0sdqrg69zswlp474l-closure-info/registration console=ttyS0,115200n8 console=tty0526lb # [ 0.059592] ftrace: allocating 48787 entries in 192 pages527worker1 # [ 0.031071] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]528client # [ 0.032790] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])529lb # [ 0.059594] ftrace: allocated 192 pages with 2 groups530lb # [ 0.060441] Dynamic Preempt: lazy531worker1 # [ 0.031072] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]532client # [ 0.032820] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23533worker2 # [ 0.037812] Unknown kernel command line parameters "regInfo=/nix/store/v4168pp3vxgkq1k0sdqrg69zswlp474l-closure-info/registration", will be passed to user space.534lb # [ 0.060603] rcu: Preemptible hierarchical RCU implementation.535worker1 # [ 0.031074] [mem 0x60000000-0xafffffff] available for PCI devices536worker2 # [ 0.037825] random: crng init done537client # [ 0.032823] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)538lb # [ 0.060603] rcu: RCU event tracing is enabled.539worker1 # [ 0.031075] Booting paravirtualized kernel on KVM540worker2 # [ 0.037826] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes541client # [ 0.032825] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)542lb # [ 0.060604] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.543client # [ 0.032826] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)544lb # [ 0.060606] Trampoline variant of Tasks RCU enabled.545worker2 # [ 0.039976] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)546worker1 # [ 0.031079] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns547lb # [ 0.060606] Rude variant of Tasks RCU enabled.548client # [ 0.032827] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)549worker1 # [ 0.035570] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1550worker2 # [ 0.040986] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)551lb # [ 0.060606] Tracing variant of Tasks RCU enabled.552worker2 # [ 0.041019] Fallback order for Node 0: 0553client # [ 0.032827] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)554worker1 # [ 0.037624] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152555lb # [ 0.060607] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.556worker2 # [ 0.041022] Built 1 zonelists, mobility grouping on. Total pages: 393078557client # [ 0.032830] ACPI: Using ACPI (MADT) for SMP configuration information558worker1 # [ 0.037667] kvm-guest: PV spinlocks disabled, single CPU559worker2 # [ 0.041023] Policy zone: DMA32560lb # [ 0.060608] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1561client # [ 0.032831] ACPI: HPET id: 0x8086a201 base: 0xfed00000562client # [ 0.032835] TSC deadline timer available563worker2 # [ 0.043687] mem auto-init: stack:all(zero), heap alloc:on, heap free:off564lb # [ 0.060630] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.565client # [ 0.032839] CPU topo: Max. logical packages: 1566worker2 # [ 0.046772] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1567client # [ 0.032839] CPU topo: Max. logical dies: 1568worker2 # [ 0.050038] allocated 3145728 bytes of page_ext569lb # [ 0.060631] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.570client # [ 0.032840] CPU topo: Max. dies per package: 1571worker2 # [ 0.059811] ftrace: allocating 48787 entries in 192 pages572client # [ 0.032843] CPU topo: Max. threads per core: 1573worker2 # [ 0.059813] ftrace: allocated 192 pages with 2 groups574lb # [ 0.060632] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.575worker1 # [ 0.037669] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4v49g7hxh8q1vdjhx1q5arqap8yynmc7-nixos-system-worker1-test/init regInfo=/nix/store/s71x21530plar9f2qqi7wxm3dwbphzch-closure-info/registration console=ttyS0,115200n8 console=tty0576client # [ 0.032844] CPU topo: Num. cores per package: 1577worker2 # [ 0.060655] Dynamic Preempt: lazy578lb # [ 0.064973] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16579client # [ 0.032844] CPU topo: Num. threads per package: 1580worker2 # [ 0.060820] rcu: Preemptible hierarchical RCU implementation.581client # [ 0.032845] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs582lb # [ 0.065262] rcu: srcu_init: Setting srcu_struct sizes based on contention.583worker2 # [ 0.060820] rcu: RCU event tracing is enabled.584worker1 # [ 0.037762] Unknown kernel command line parameters "regInfo=/nix/store/s71x21530plar9f2qqi7wxm3dwbphzch-closure-info/registration", will be passed to user space.585worker1 # [ 0.037775] random: crng init done586client # [ 0.032863] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()587worker2 # [ 0.060821] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.588lb # [ 0.065270] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns589worker2 # [ 0.060822] Trampoline variant of Tasks RCU enabled.590worker1 # [ 0.037776] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes591client # [ 0.032897] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]592worker2 # [ 0.060823] Rude variant of Tasks RCU enabled.593lb # [ 0.065378] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)594worker1 # [ 0.039921] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)595client # [ 0.032898] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]596worker2 # [ 0.060823] Tracing variant of Tasks RCU enabled.597lb # [ 0.068895] Console: colour VGA+ 80x25598client # [ 0.032900] [mem 0x60000000-0xafffffff] available for PCI devices599lb # [ 0.068898] printk: legacy console [tty0] enabled600worker2 # [ 0.060824] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.601worker1 # [ 0.040927] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)602worker1 # [ 0.040960] Fallback order for Node 0: 0603client # [ 0.032902] Booting paravirtualized kernel on KVM604lb # [ 0.098329] printk: legacy console [ttyS0] enabled605worker2 # [ 0.060824] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1606lb # [ 0.193582] ACPI: Core revision 20250807607worker1 # [ 0.040963] Built 1 zonelists, mobility grouping on. Total pages: 393078608worker1 # [ 0.040964] Policy zone: DMA32609client # [ 0.032905] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns610worker2 # [ 0.060845] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.611worker1 # [ 0.043373] mem auto-init: stack:all(zero), heap alloc:on, heap free:off612client # [ 0.037401] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1613lb # [ 0.194406] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns614worker2 # [ 0.060847] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.615worker1 # [ 0.046406] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1616client # [ 0.039533] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152617lb # [ 0.195872] APIC: Switch to symmetric I/O mode setup618worker1 # [ 0.049540] allocated 3145728 bytes of page_ext619client # [ 0.039578] kvm-guest: PV spinlocks disabled, single CPU620worker2 # [ 0.060848] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.621lb # [ 0.196797] x2apic enabled622worker1 # [ 0.059082] ftrace: allocating 48787 entries in 192 pages623worker2 # [ 0.065155] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16624worker1 # [ 0.059085] ftrace: allocated 192 pages with 2 groups625worker1 # [ 0.060032] Dynamic Preempt: lazy626lb # [ 0.197525] APIC: Switched APIC routing to: physical x2apic627worker2 # [ 0.065429] rcu: srcu_init: Setting srcu_struct sizes based on contention.628worker1 # [ 0.060199] rcu: Preemptible hierarchical RCU implementation.629worker1 # [ 0.060199] rcu: RCU event tracing is enabled.630worker2 # [ 0.065434] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns631lb # [ 0.199282] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1632worker1 # [ 0.060200] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.633client # [ 0.039579] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test/init regInfo=/nix/store/z24i12fyjab3zng5p8xysghgchx1bnfn-closure-info/registration console=ttyS0,115200n8 console=tty0634worker2 # [ 0.065540] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)635worker1 # [ 0.060201] Trampoline variant of Tasks RCU enabled.636worker2 # [ 0.069052] Console: colour VGA+ 80x25637worker1 # [ 0.060202] Rude variant of Tasks RCU enabled.638worker2 # [ 0.069054] printk: legacy console [tty0] enabled639lb # [ 0.200216] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns640worker1 # [ 0.060202] Tracing variant of Tasks RCU enabled.641client # [ 0.039680] Unknown kernel command line parameters "regInfo=/nix/store/z24i12fyjab3zng5p8xysghgchx1bnfn-closure-info/registration", will be passed to user space.642worker2 # [ 0.098516] printk: legacy console [ttyS0] enabled643client # [ 0.039692] random: crng init done644worker1 # [ 0.060203] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.645lb # [ 0.201775] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)646worker2 # [ 0.194729] ACPI: Core revision 20250807647client # [ 0.039693] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes648worker1 # [ 0.060204] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1649lb # [ 0.203864] x86/cpu: User Mode Instruction Prevention (UMIP) activated650client # [ 0.041795] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)651worker1 # [ 0.060237] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.652worker2 # [ 0.195549] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns653lb # [ 0.204951] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127654client # [ 0.043023] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)655worker2 # [ 0.197024] APIC: Switch to symmetric I/O mode setup656worker1 # [ 0.060238] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.657client # [ 0.043055] Fallback order for Node 0: 0658worker2 # [ 0.197945] x2apic enabled659lb # [ 0.205773] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0660client # [ 0.043059] Built 1 zonelists, mobility grouping on. Total pages: 393078661worker1 # [ 0.060239] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.662client # [ 0.043060] Policy zone: DMA32663worker2 # [ 0.198666] APIC: Switched APIC routing to: physical x2apic664worker1 # [ 0.064525] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16665client # [ 0.045513] mem auto-init: stack:all(zero), heap alloc:on, heap free:off666lb # [ 0.206776] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto667worker1 # [ 0.064793] rcu: srcu_init: Setting srcu_struct sizes based on contention.668client # [ 0.048612] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1669client # [ 0.051891] allocated 3145728 bytes of page_ext670lb # [ 0.207772] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl671worker2 # [ 0.200442] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1672worker1 # [ 0.064798] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns673client # [ 0.061393] ftrace: allocating 48787 entries in 192 pages674lb # [ 0.209772] Transient Scheduler Attacks: Vulnerable: No microcode675client # [ 0.061395] ftrace: allocated 192 pages with 2 groups676worker1 # [ 0.064896] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)677client # [ 0.062224] Dynamic Preempt: lazy678worker2 # [ 0.201388] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns679lb # [ 0.210772] Spectre V2 : Mitigation: Enhanced / Automatic IBRS680worker1 # [ 0.068398] Console: colour VGA+ 80x25681client # [ 0.062377] rcu: Preemptible hierarchical RCU implementation.682worker1 # [ 0.068401] printk: legacy console [tty0] enabled683client # [ 0.062378] rcu: RCU event tracing is enabled.684lb # [ 0.211772] Speculative Return Stack Overflow: Mitigation: Safe RET685worker1 # [ 0.097799] printk: legacy console [ttyS0] enabled686worker2 # [ 0.202996] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)687client # [ 0.062378] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.688worker1 # [ 0.193268] ACPI: Core revision 20250807689worker2 # [ 0.205081] x86/cpu: User Mode Instruction Prevention (UMIP) activated690client # [ 0.062380] Trampoline variant of Tasks RCU enabled.691client # [ 0.062380] Rude variant of Tasks RCU enabled.692worker2 # [ 0.206117] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127693client # [ 0.062380] Tracing variant of Tasks RCU enabled.694worker1 # [ 0.194164] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns695worker2 # [ 0.206993] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0696client # [ 0.062381] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.697worker1 # [ 0.195643] APIC: Switch to symmetric I/O mode setup698lb # [ 0.212772] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization699worker1 # [ 0.196605] x2apic enabled700client # [ 0.062382] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1701lb # [ 0.214779] Spectre V2 : Enabling IBPB for BPF702worker2 # [ 0.207997] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto703client # [ 0.062411] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.704worker1 # [ 0.197335] APIC: Switched APIC routing to: physical x2apic705lb # [ 0.216291] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier706client # [ 0.062413] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.707worker2 # [ 0.208993] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl708lb # [ 0.216772] active return thunk: srso_alias_return_thunk709worker1 # [ 0.199098] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1710client # [ 0.062414] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.711worker2 # [ 0.209993] Transient Scheduler Attacks: Vulnerable: No microcode712lb # [ 0.218395] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'713client # [ 0.067162] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16714worker2 # [ 0.210993] Spectre V2 : Mitigation: Enhanced / Automatic IBRS715worker1 # [ 0.200056] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns716client # [ 0.067442] rcu: srcu_init: Setting srcu_struct sizes based on contention.717lb # [ 0.219631] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'718worker2 # [ 0.211993] Speculative Return Stack Overflow: Mitigation: Safe RET719lb # [ 0.220485] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'720client # [ 0.067449] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns721worker1 # [ 0.201668] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)722lb # [ 0.221483] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'723worker1 # [ 0.202750] x86/cpu: User Mode Instruction Prevention (UMIP) activated724client # [ 0.067552] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)725client # [ 0.071108] Console: colour VGA+ 80x25726lb # [ 0.221772] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'727client # [ 0.071111] printk: legacy console [tty0] enabled728worker1 # [ 0.203795] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127729lb # [ 0.222772] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'730client # [ 0.100617] printk: legacy console [ttyS0] enabled731worker1 # [ 0.204666] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0732worker2 # [ 0.212993] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization733client # [ 0.196228] ACPI: Core revision 20250807734lb # [ 0.223772] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'735worker2 # [ 0.215000] Spectre V2 : Enabling IBPB for BPF736lb # [ 0.224772] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256737worker1 # [ 0.205669] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto738client # [ 0.197162] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns739worker2 # [ 0.215994] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier740lb # [ 0.226471] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64741client # [ 0.198662] APIC: Switch to symmetric I/O mode setup742worker2 # [ 0.216993] active return thunk: srso_alias_return_thunk743worker1 # [ 0.206666] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl744client # [ 0.199588] x2apic enabled745lb # [ 0.226772] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512746worker1 # [ 0.207665] Transient Scheduler Attacks: Vulnerable: No microcode747worker2 # [ 0.218014] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'748lb # [ 0.227772] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024749client # [ 0.200327] APIC: Switched APIC routing to: physical x2apic750worker1 # [ 0.208665] Spectre V2 : Mitigation: Enhanced / Automatic IBRS751worker2 # [ 0.218993] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'752lb # [ 0.228772] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8753worker1 # [ 0.209665] Speculative Return Stack Overflow: Mitigation: Safe RET754worker2 # [ 0.219993] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'755client # [ 0.202099] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1756lb # [ 0.229772] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.757worker2 # [ 0.220993] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'758worker2 # [ 0.221993] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'759client # [ 0.203062] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns760worker2 # [ 0.222993] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'761client # [ 0.204764] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)762worker1 # [ 0.210665] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization763worker2 # [ 0.223993] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'764worker1 # [ 0.212672] Spectre V2 : Enabling IBPB for BPF765client # [ 0.205849] x86/cpu: User Mode Instruction Prevention (UMIP) activated766worker2 # [ 0.224994] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256767client # [ 0.206897] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127768worker2 # [ 0.225993] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64769client # [ 0.207761] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0770worker1 # [ 0.213666] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier771worker2 # [ 0.226993] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512772worker1 # [ 0.214678] active return thunk: srso_alias_return_thunk773worker2 # [ 0.227993] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024774client # [ 0.208764] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto775worker1 # [ 0.215684] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'776worker2 # [ 0.228993] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8777client # [ 0.209761] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl778worker1 # [ 0.216665] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'779client # [ 0.210760] Transient Scheduler Attacks: Vulnerable: No microcode780worker2 # [ 0.229993] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.781worker1 # [ 0.217665] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'782client # [ 0.211760] Spectre V2 : Mitigation: Enhanced / Automatic IBRS783worker1 # [ 0.218665] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'784client # [ 0.212760] Speculative Return Stack Overflow: Mitigation: Safe RET785worker1 # [ 0.219664] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'786worker1 # [ 0.220665] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'787worker1 # [ 0.221665] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'788worker1 # [ 0.222666] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256789client # [ 0.213760] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization790worker1 # [ 0.223665] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64791client # [ 0.215768] Spectre V2 : Enabling IBPB for BPF792worker1 # [ 0.224665] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512793client # [ 0.216761] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier794worker1 # [ 0.225665] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024795client # [ 0.217761] active return thunk: srso_alias_return_thunk796worker1 # [ 0.226665] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8797client # [ 0.218781] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'798worker1 # [ 0.227665] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.799client # [ 0.219760] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'800client # [ 0.220761] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'801client # [ 0.221760] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'802client # [ 0.223583] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'803client # [ 0.224566] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'804client # [ 0.225760] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'805client # [ 0.226761] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256806client # [ 0.227760] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807client # [ 0.228760] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808client # [ 0.229760] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809client # [ 0.230760] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810client # [ 0.231760] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811lb # [ 0.264159] Freeing SMP alternatives memory: 44K812lb # [ 0.264774] pid_max: default: 32768 minimum: 301813lb # [ 0.265826] LSM: initializing lsm=capability,landlock,yama,bpf,ima814lb # [ 0.266870] landlock: Up and running.815lb # [ 0.267513] Yama: becoming mindful.816lb # [ 0.268441] LSM support for eBPF active817lb # [ 0.268884] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818lb # [ 0.269812] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819lb # [ 0.272666] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820lb # [ 0.273267] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821worker2 # [ 0.264854] Freeing SMP alternatives memory: 44K822lb # [ 0.273778] ... version: 2823worker2 # [ 0.265569] pid_max: default: 32768 minimum: 301824lb # [ 0.274483] ... bit width: 48825lb # [ 0.274774] ... generic counters: 6826worker2 # [ 0.266084] LSM: initializing lsm=capability,landlock,yama,bpf,ima827lb # [ 0.275497] ... generic bitmap: 000000000000003f828worker2 # [ 0.267089] landlock: Up and running.829worker2 # [ 0.267993] Yama: becoming mindful.830lb # [ 0.275819] ... fixed-purpose counters: 0831worker2 # [ 0.269145] LSM support for eBPF active832lb # [ 0.276524] ... fixed-purpose bitmap: 0000000000000000833lb # [ 0.276774] ... value mask: 0000ffffffffffff834worker2 # [ 0.269896] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)835lb # [ 0.277688] ... max period: 00007fffffffffff836worker2 # [ 0.271029] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)837lb # [ 0.278480] ... global_ctrl mask: 000000000000003f838lb # [ 0.278883] signal: max sigframe size: 3376839lb # [ 0.279707] rcu: Hierarchical SRCU implementation.840worker2 # [ 0.273593] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)841lb # [ 0.280379] rcu: Max phase no-delay instances is 400.842worker1 # [ 0.262646] Freeing SMP alternatives memory: 44K843worker1 # [ 0.263226] pid_max: default: 32768 minimum: 301844worker2 # [ 0.274576] Performance Events: Fam17h+ core perfctr, AMD PMU driver.845worker1 # [ 0.263771] LSM: initializing lsm=capability,landlock,yama,bpf,ima846worker2 # [ 0.274999] ... version: 2847worker1 # [ 0.264762] landlock: Up and running.848worker2 # [ 0.275720] ... bit width: 48849worker1 # [ 0.265666] Yama: becoming mindful.850worker2 # [ 0.275995] ... generic counters: 6851worker1 # [ 0.266850] LSM support for eBPF active852worker2 # [ 0.276688] ... generic bitmap: 000000000000003f853lb # [ 0.285256] smp: Bringing up secondary CPUs ...854worker2 # [ 0.277000] ... fixed-purpose counters: 0855worker1 # [ 0.267605] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)856lb # [ 0.285788] smp: Brought up 1 node, 1 CPU857worker2 # [ 0.277713] ... fixed-purpose bitmap: 0000000000000000858lb # [ 0.286424] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)859worker1 # [ 0.268703] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)860worker2 # [ 0.277995] ... value mask: 0000ffffffffffff861worker2 # [ 0.278882] ... max period: 00007fffffffffff862worker2 # [ 0.279681] ... global_ctrl mask: 000000000000003f863lb # [ 0.286888] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)864worker1 # [ 0.271075] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)865worker2 # [ 0.280090] signal: max sigframe size: 3376866lb # [ 0.288008] devtmpfs: initialized867worker2 # [ 0.280866] rcu: Hierarchical SRCU implementation.868lb # [ 0.288763] x86/mm: Memory block size: 128MB869worker1 # [ 0.272308] Performance Events: Fam17h+ core perfctr, AMD PMU driver.870worker2 # [ 0.281575] rcu: Max phase no-delay instances is 400.871worker1 # [ 0.272670] ... version: 2872client # [ 0.266175] Freeing SMP alternatives memory: 44K873worker1 # [ 0.273383] ... bit width: 48874lb # [ 0.290352] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)875client # [ 0.266763] pid_max: default: 32768 minimum: 301876worker1 # [ 0.273668] ... generic counters: 6877worker1 # [ 0.274350] ... generic bitmap: 000000000000003f878lb # [ 0.290801] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).879client # [ 0.267854] LSM: initializing lsm=capability,landlock,yama,bpf,ima880worker1 # [ 0.274678] ... fixed-purpose counters: 0881client # [ 0.268856] landlock: Up and running.882lb # [ 0.291845] pinctrl core: initialized pinctrl subsystem883client # [ 0.269458] Yama: becoming mindful.884worker1 # [ 0.275391] ... fixed-purpose bitmap: 0000000000000000885lb # [ 0.292843] PM: RTC time: 10:58:02, date: 2026-09-22886client # [ 0.269972] LSM support for eBPF active887worker1 # [ 0.275668] ... value mask: 0000ffffffffffff888worker2 # [ 0.286528] smp: Bringing up secondary CPUs ...889worker2 # [ 0.287010] smp: Brought up 1 node, 1 CPU890worker1 # [ 0.276546] ... max period: 00007fffffffffff891client # [ 0.270900] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)892worker2 # [ 0.287658] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)893worker1 # [ 0.277310] ... global_ctrl mask: 000000000000003f894client # [ 0.271802] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)895worker1 # [ 0.277776] signal: max sigframe size: 3376896lb # [ 0.296186] NET: Registered PF_NETLINK/PF_ROUTE protocol family897worker1 # [ 0.278563] rcu: Hierarchical SRCU implementation.898worker2 # [ 0.288179] Memory: 1454616K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)899worker1 # [ 0.279262] rcu: Max phase no-delay instances is 400.900lb # [ 0.297260] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations901worker2 # [ 0.289228] devtmpfs: initialized902client # [ 0.274730] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)903lb # [ 0.297794] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations904worker2 # [ 0.290066] x86/mm: Memory block size: 128MB905client # [ 0.275297] Performance Events: Fam17h+ core perfctr, AMD PMU driver.906client # [ 0.275765] ... version: 2907lb # [ 0.299240] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations908client # [ 0.276463] ... bit width: 48909worker2 # [ 0.291781] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)910lb # [ 0.299784] audit: initializing netlink subsys (disabled)911client # [ 0.276763] ... generic counters: 6912worker2 # [ 0.292885] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).913client # [ 0.277488] ... generic bitmap: 000000000000003f914worker1 # [ 0.284248] smp: Bringing up secondary CPUs ...915lb # [ 0.300776] audit: type=2000 audit(1790074683.171:1): state=initialized audit_enabled=0 res=1916client # [ 0.277813] ... fixed-purpose counters: 0917worker1 # [ 0.284681] smp: Brought up 1 node, 1 CPU918worker2 # [ 0.294090] pinctrl core: initialized pinctrl subsystem919lb # [ 0.301869] thermal_sys: Registered thermal governor 'fair_share'920client # [ 0.278486] ... fixed-purpose bitmap: 0000000000000000921worker1 # [ 0.285352] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)922worker2 # [ 0.295209] PM: RTC time: 10:58:02, date: 2026-09-22923lb # [ 0.301871] thermal_sys: Registered thermal governor 'bang_bang'924client # [ 0.278763] ... value mask: 0000ffffffffffff925lb # [ 0.302775] thermal_sys: Registered thermal governor 'step_wise'926client # [ 0.279662] ... max period: 00007fffffffffff927lb # [ 0.303665] thermal_sys: Registered thermal governor 'user_space'928worker1 # [ 0.285785] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)929client # [ 0.280416] ... global_ctrl mask: 000000000000003f930worker1 # [ 0.286897] devtmpfs: initialized931lb # [ 0.304455] thermal_sys: Registered thermal governor 'power_allocator'932client # [ 0.280877] signal: max sigframe size: 3376933worker2 # [ 0.298721] NET: Registered PF_NETLINK/PF_ROUTE protocol family934lb # [ 0.304787] cpuidle: using governor menu935worker1 # [ 0.287661] x86/mm: Memory block size: 128MB936client # [ 0.281639] rcu: Hierarchical SRCU implementation.937client # [ 0.282312] rcu: Max phase no-delay instances is 400.938lb # [ 0.306764] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5939worker2 # [ 0.299470] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations940worker1 # [ 0.289215] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)941worker2 # [ 0.300014] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations942lb # [ 0.307788] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]943worker1 # [ 0.289692] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).944lb # [ 0.308777] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry945worker2 # [ 0.301255] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations946worker1 # [ 0.290760] pinctrl core: initialized pinctrl subsystem947lb # [ 0.309777] PCI: Using configuration type 1 for base access948worker2 # [ 0.302005] audit: initializing netlink subsys (disabled)949worker1 # [ 0.291947] PM: RTC time: 10:58:02, date: 2026-09-22950lb # [ 0.310801] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.951worker2 # [ 0.303110] thermal_sys: Registered thermal governor 'fair_share'952client # [ 0.287347] smp: Bringing up secondary CPUs ...953client # [ 0.287777] smp: Brought up 1 node, 1 CPU954worker2 # [ 0.303113] thermal_sys: Registered thermal governor 'bang_bang'955client # [ 0.288448] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)956worker2 # [ 0.303996] thermal_sys: Registered thermal governor 'step_wise'957worker2 # [ 0.304935] thermal_sys: Registered thermal governor 'user_space'958worker1 # [ 0.295369] NET: Registered PF_NETLINK/PF_ROUTE protocol family959client # [ 0.288987] Memory: 1454632K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)960worker2 # [ 0.305729] audit: type=2000 audit(1790074683.180:1): state=initialized audit_enabled=0 res=1961worker1 # [ 0.296125] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations962client # [ 0.290019] devtmpfs: initialized963worker2 # [ 0.306998] thermal_sys: Registered thermal governor 'power_allocator'964worker2 # [ 0.307012] cpuidle: using governor menu965client # [ 0.290847] x86/mm: Memory block size: 128MB966worker1 # [ 0.296686] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations967lb # [ 0.317790] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages968worker2 # [ 0.309195] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5969worker1 # [ 0.297926] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations970lb # [ 0.318774] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page971client # [ 0.292654] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)972worker1 # [ 0.298677] audit: initializing netlink subsys (disabled)973lb # [ 0.319739] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages974worker2 # [ 0.310229] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]975client # [ 0.293666] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).976worker1 # [ 0.299782] thermal_sys: Registered thermal governor 'fair_share'977worker2 # [ 0.310999] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry978client # [ 0.294845] pinctrl core: initialized pinctrl subsystem979worker1 # [ 0.299784] thermal_sys: Registered thermal governor 'bang_bang'980worker2 # [ 0.312007] PCI: Using configuration type 1 for base access981client # [ 0.295911] PM: RTC time: 10:58:02, date: 2026-09-22982worker1 # [ 0.300669] thermal_sys: Registered thermal governor 'step_wise'983worker2 # [ 0.313097] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.984worker1 # [ 0.301618] thermal_sys: Registered thermal governor 'user_space'985worker1 # [ 0.302440] audit: type=2000 audit(1790074683.189:1): state=initialized audit_enabled=0 res=1986lb # [ 0.324777] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page987worker1 # [ 0.303671] thermal_sys: Registered thermal governor 'power_allocator'988client # [ 0.299608] NET: Registered PF_NETLINK/PF_ROUTE protocol family989worker1 # [ 0.303685] cpuidle: using governor menu990client # [ 0.301040] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations991worker1 # [ 0.305923] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5992worker2 # [ 0.318413] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages993client # [ 0.301785] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations994worker1 # [ 0.306892] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]995worker2 # [ 0.318997] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page996client # [ 0.303050] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations997worker1 # [ 0.307671] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry998worker2 # [ 0.319996] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages999client # [ 0.303772] audit: initializing netlink subsys (disabled)1000worker1 # [ 0.308679] PCI: Using configuration type 1 for base access1001lb # [ 0.333151] ACPI: Added _OSI(Module Device)1002client # [ 0.304765] audit: type=2000 audit(1790074683.203:1): state=initialized audit_enabled=0 res=11003worker1 # [ 0.309812] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1004lb # [ 0.333776] ACPI: Added _OSI(Processor Device)1005client # [ 0.305900] thermal_sys: Registered thermal governor 'fair_share'1006lb # [ 0.334471] ACPI: Added _OSI(Processor Aggregator Device)1007client # [ 0.305902] thermal_sys: Registered thermal governor 'bang_bang'1008client # [ 0.306765] thermal_sys: Registered thermal governor 'step_wise'1009client # [ 0.307743] thermal_sys: Registered thermal governor 'user_space'1010worker2 # [ 0.326996] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1011client # [ 0.308560] thermal_sys: Registered thermal governor 'power_allocator'1012worker1 # [ 0.314912] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1013client # [ 0.309573] cpuidle: using governor menu1014worker1 # [ 0.315670] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1015client # [ 0.311310] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51016client # [ 0.312008] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1017lb # [ 0.342326] ACPI: 1 ACPI AML tables successfully acquired and loaded1018client # [ 0.312765] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1019client # [ 0.313775] PCI: Using configuration type 1 for base access1020worker1 # [ 0.320669] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1021client # [ 0.314906] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1022worker1 # [ 0.321668] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1023worker2 # [ 0.335407] ACPI: Added _OSI(Module Device)1024worker2 # [ 0.335997] ACPI: Added _OSI(Processor Device)1025lb # [ 0.346175] ACPI: Interpreter enabled1026worker2 # [ 0.336792] ACPI: Added _OSI(Processor Aggregator Device)1027lb # [ 0.346789] ACPI: PM: (supports S0 S3 S4 S5)1028lb # [ 0.347465] ACPI: Using IOAPIC for interrupt routing1029lb # [ 0.349844] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1030lb # [ 0.350775] PCI: Using E820 reservations for host bridge windows1031client # [ 0.321789] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1032client # [ 0.322763] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1033lb # [ 0.353819] ACPI: Enabled 2 GPEs in block 00 to 3F1034worker1 # [ 0.332063] ACPI: Added _OSI(Module Device)1035worker2 # [ 0.345586] ACPI: 1 ACPI AML tables successfully acquired and loaded1036worker1 # [ 0.332670] ACPI: Added _OSI(Processor Device)1037worker1 # [ 0.333387] ACPI: Added _OSI(Processor Aggregator Device)1038client # [ 0.327764] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1039client # [ 0.328764] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1040worker2 # [ 0.349393] ACPI: Interpreter enabled1041worker2 # [ 0.350010] ACPI: PM: (supports S0 S3 S4 S5)1042worker2 # [ 0.350734] ACPI: Using IOAPIC for interrupt routing1043lb # [ 0.361573] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1044lb # [ 0.362500] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1045worker2 # [ 0.353033] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1046worker1 # [ 0.341111] ACPI: 1 ACPI AML tables successfully acquired and loaded1047lb # [ 0.363852] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1048worker2 # [ 0.353996] PCI: Using E820 reservations for host bridge windows1049lb # [ 0.364896] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1050lb # [ 0.366120] PCI host bridge to bus 0000:001051lb # [ 0.366778] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1052worker2 # [ 0.357106] ACPI: Enabled 2 GPEs in block 00 to 3F1053worker1 # [ 0.345077] ACPI: Interpreter enabled1054lb # [ 0.367774] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1055worker1 # [ 0.345682] ACPI: PM: (supports S0 S3 S4 S5)1056client # [ 0.340002] ACPI: Added _OSI(Module Device)1057lb # [ 0.368774] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1058worker1 # [ 0.346404] ACPI: Using IOAPIC for interrupt routing1059client # [ 0.340704] ACPI: Added _OSI(Processor Device)1060client # [ 0.341317] ACPI: Added _OSI(Processor Aggregator Device)1061lb # [ 0.369774] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1062lb # [ 0.370774] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1063lb # [ 0.371774] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1064worker1 # [ 0.348726] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1065lb # [ 0.372774] pci_bus 0000:00: root bus resource [bus 00-ff]1066worker1 # [ 0.349668] PCI: Using E820 reservations for host bridge windows1067lb # [ 0.373689] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1068worker2 # [ 0.364626] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1069worker2 # [ 0.365002] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1070lb # [ 0.375153] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1071client # [ 0.347137] ACPI: 1 ACPI AML tables successfully acquired and loaded1072worker1 # [ 0.352806] ACPI: Enabled 2 GPEs in block 00 to 3F1073worker2 # [ 0.366076] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1074worker2 # [ 0.367119] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1075worker2 # [ 0.368358] PCI host bridge to bus 0000:001076client # [ 0.351184] ACPI: Interpreter enabled1077worker2 # [ 0.369000] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1078client # [ 0.351779] ACPI: PM: (supports S0 S3 S4 S5)1079worker2 # [ 0.369996] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1080client # [ 0.352521] ACPI: Using IOAPIC for interrupt routing1081worker2 # [ 0.370996] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1082lb # [ 0.377796] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1083worker2 # [ 0.371996] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1084lb # [ 0.378787] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1085worker1 # [ 0.360578] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1086lb # [ 0.379697] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1087worker2 # [ 0.372995] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1088client # [ 0.354823] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1089worker1 # [ 0.361452] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1090client # [ 0.355764] PCI: Using E820 reservations for host bridge windows1091lb # [ 0.380640] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1092worker2 # [ 0.373996] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1093worker1 # [ 0.362747] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1094worker2 # [ 0.375001] pci_bus 0000:00: root bus resource [bus 00-ff]1095worker1 # [ 0.363790] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1096lb # [ 0.382483] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1097worker2 # [ 0.376038] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1098client # [ 0.358828] ACPI: Enabled 2 GPEs in block 00 to 3F1099worker1 # [ 0.365019] PCI host bridge to bus 0000:001100worker1 # [ 0.365672] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1101worker2 # [ 0.377460] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1102worker1 # [ 0.366668] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1103lb # [ 0.385783] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1104lb # [ 0.386598] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1105worker1 # [ 0.367668] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1106lb # [ 0.387467] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1107worker1 # [ 0.368668] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1108lb # [ 0.387780] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1109worker1 # [ 0.369668] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1110lb # [ 0.389404] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1111worker1 # [ 0.370668] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1112client # [ 0.366328] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1113worker1 # [ 0.371668] pci_bus 0000:00: root bus resource [bus 00-ff]1114worker2 # [ 0.380116] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1115client # [ 0.366769] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1116worker2 # [ 0.381008] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1117worker1 # [ 0.372657] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1118client # [ 0.367841] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1119worker2 # [ 0.381947] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1120lb # [ 0.391782] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1121client # [ 0.368892] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1122worker1 # [ 0.374122] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1123lb # [ 0.392781] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1124worker2 # [ 0.382913] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1125client # [ 0.370121] PCI host bridge to bus 0000:001126lb # [ 0.393683] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1127client # [ 0.370767] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1128worker2 # [ 0.384703] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1129client # [ 0.371763] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1130lb # [ 0.395111] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1131client # [ 0.372763] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1132client # [ 0.373764] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1133worker2 # [ 0.387012] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1134lb # [ 0.396783] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1135worker2 # [ 0.387847] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1136lb # [ 0.397592] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1137client # [ 0.374763] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1138worker1 # [ 0.376729] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1139worker2 # [ 0.388725] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1140lb # [ 0.398462] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1141client # [ 0.375763] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1142worker1 # [ 0.377680] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1143worker2 # [ 0.389002] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1144client # [ 0.376763] pci_bus 0000:00: root bus resource [bus 00-ff]1145worker1 # [ 0.378647] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1146lb # [ 0.399325] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1147client # [ 0.377800] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1148worker2 # [ 0.390526] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1149worker1 # [ 0.379661] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1150lb # [ 0.400782] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1151client # [ 0.379213] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1152lb # [ 0.401594] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1153worker1 # [ 0.381423] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1154worker2 # [ 0.393020] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1155lb # [ 0.402480] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1156worker2 # [ 0.393859] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1157lb # [ 0.402781] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1158worker2 # [ 0.394744] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1159worker1 # [ 0.384706] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1160lb # [ 0.404242] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1161worker1 # [ 0.385603] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1162worker2 # [ 0.395562] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1163worker1 # [ 0.386452] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1164lb # [ 0.405786] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1165worker1 # [ 0.387543] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1166lb # [ 0.406695] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1167client # [ 0.381829] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1168worker2 # [ 0.398023] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1169client # [ 0.382776] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1170worker2 # [ 0.398855] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1171lb # [ 0.408350] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1172client # [ 0.383735] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1173worker1 # [ 0.389177] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1174worker2 # [ 0.399730] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1175lb # [ 0.409558] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1176client # [ 0.384709] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1177lb # [ 0.410475] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1178worker1 # [ 0.391527] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1179worker2 # [ 0.400558] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1180worker1 # [ 0.392285] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1181client # [ 0.386507] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1182lb # [ 0.411344] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1183worker1 # [ 0.392690] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1184worker2 # [ 0.402033] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1185worker2 # [ 0.402909] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1186lb # [ 0.412766] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1187worker1 # [ 0.394276] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1188client # [ 0.389782] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1189worker2 # [ 0.403770] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1190lb # [ 0.413718] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1191client # [ 0.390612] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1192worker2 # [ 0.404815] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1193client # [ 0.391524] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1194lb # [ 0.415386] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1195client # [ 0.391769] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1196worker2 # [ 0.406342] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1197lb # [ 0.416570] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1198worker1 # [ 0.396713] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1199worker1 # [ 0.398348] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1200lb # [ 0.417508] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1201client # [ 0.393460] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1202worker2 # [ 0.408627] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1203worker1 # [ 0.398690] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1204worker2 # [ 0.409019] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1205lb # [ 0.418314] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1206worker1 # [ 0.400203] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1207client # [ 0.396782] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1208worker2 # [ 0.410596] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1209lb # [ 0.419782] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1210client # [ 0.397705] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1211lb # [ 0.420621] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1212client # [ 0.398508] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1213worker2 # [ 0.412008] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1214worker1 # [ 0.402439] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1215worker2 # [ 0.412924] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1216lb # [ 0.421504] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1217worker1 # [ 0.403320] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1218client # [ 0.399314] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1219worker1 # [ 0.403721] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1220lb # [ 0.422316] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1221worker2 # [ 0.414429] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1222worker1 # [ 0.404674] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1223lb # [ 0.423782] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1224worker2 # [ 0.416619] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1225lb # [ 0.424697] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1226worker1 # [ 0.406390] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1227client # [ 0.402168] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1228worker2 # [ 0.417019] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1229lb # [ 0.425496] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1230client # [ 0.403388] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1231worker1 # [ 0.408680] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1232lb # [ 0.426320] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1233worker2 # [ 0.418563] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1234client # [ 0.403785] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1235worker1 # [ 0.409690] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1236lb # [ 0.427351] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1237client # [ 0.405308] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1238worker2 # [ 0.420008] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1239worker2 # [ 0.421006] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1240worker1 # [ 0.411416] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1241lb # [ 0.427974] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1242client # [ 0.407593] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1243lb # [ 0.429307] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1244worker2 # [ 0.422555] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1245client # [ 0.408377] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1246lb # [ 0.429982] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1247client # [ 0.408784] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1248worker1 # [ 0.413711] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1249lb # [ 0.431365] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1250worker1 # [ 0.414690] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1251worker2 # [ 0.425013] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1252client # [ 0.409769] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1253worker2 # [ 0.425920] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1254worker1 # [ 0.416245] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1255worker2 # [ 0.426779] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1256client # [ 0.411321] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1257worker1 # [ 0.417680] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1258worker2 # [ 0.428429] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1259client # [ 0.412786] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1260worker1 # [ 0.418684] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1261client # [ 0.413768] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1262worker2 # [ 0.430032] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1263worker1 # [ 0.420229] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1264worker2 # [ 0.430865] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1265client # [ 0.415355] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1266worker2 # [ 0.431728] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1267client # [ 0.416789] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1268worker1 # [ 0.422102] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1269worker2 # [ 0.433072] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1270worker1 # [ 0.423443] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1271lb # [ 0.432000] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1272client # [ 0.417750] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1273worker2 # [ 0.434711] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1274worker1 # [ 0.425087] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1275lb # [ 0.434250] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1276worker2 # [ 0.435190] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1277client # [ 0.419161] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1278worker2 # [ 0.436598] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1279lb # [ 0.435042] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1280worker1 # [ 0.427548] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1281client # [ 0.420787] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1282worker2 # [ 0.437222] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1283lb # [ 0.436109] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1284worker1 # [ 0.428309] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1285client # [ 0.421737] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1286worker1 # [ 0.428690] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1287worker2 # [ 0.438602] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1288lb # [ 0.437025] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1289client # [ 0.423149] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1290lb # [ 0.438582] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1291worker1 # [ 0.430231] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1292lb # [ 0.439455] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1293client # [ 0.424775] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1294lb # [ 0.440083] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1295worker1 # [ 0.431705] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1296client # [ 0.425786] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1297lb # [ 0.441373] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1298worker1 # [ 0.432593] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1299client # [ 0.427317] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1300worker1 # [ 0.433447] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1301worker1 # [ 0.435141] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1302client # [ 0.429491] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1303lb # [ 0.445102] ACPI: PCI: Interrupt link LNKA configured for IRQ 101304client # [ 0.429770] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1305worker1 # [ 0.436521] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1306lb # [ 0.445880] ACPI: PCI: Interrupt link LNKB configured for IRQ 101307client # [ 0.430723] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1308lb # [ 0.446872] ACPI: PCI: Interrupt link LNKC configured for IRQ 111309worker2 # [ 0.439205] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1310lb # [ 0.447869] ACPI: PCI: Interrupt link LNKD configured for IRQ 111311worker1 # [ 0.437521] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1312client # [ 0.432150] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1313lb # [ 0.448872] ACPI: PCI: Interrupt link LNKE configured for IRQ 101314worker1 # [ 0.439482] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1315worker2 # [ 0.442004] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1316lb # [ 0.449870] ACPI: PCI: Interrupt link LNKF configured for IRQ 101317lb # [ 0.450874] ACPI: PCI: Interrupt link LNKG configured for IRQ 111318worker2 # [ 0.443246] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1319worker1 # [ 0.440568] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1320lb # [ 0.451869] ACPI: PCI: Interrupt link LNKH configured for IRQ 111321client # [ 0.434782] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1322worker1 # [ 0.442401] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1323worker2 # [ 0.444283] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1324lb # [ 0.452811] ACPI: PCI: Interrupt link GSIA configured for IRQ 161325client # [ 0.435659] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1326lb # [ 0.453788] ACPI: PCI: Interrupt link GSIB configured for IRQ 171327client # [ 0.436473] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1328worker2 # [ 0.445237] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1329lb # [ 0.454750] ACPI: PCI: Interrupt link GSIC configured for IRQ 181330lb # [ 0.455550] ACPI: PCI: Interrupt link GSID configured for IRQ 191331client # [ 0.437324] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1332lb # [ 0.456516] ACPI: PCI: Interrupt link GSIE configured for IRQ 201333worker2 # [ 0.448055] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1334client # [ 0.438442] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1335lb # [ 0.456787] ACPI: PCI: Interrupt link GSIF configured for IRQ 211336worker2 # [ 0.448925] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1337lb # [ 0.457717] ACPI: PCI: Interrupt link GSIG configured for IRQ 221338client # [ 0.438961] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1339lb # [ 0.458482] ACPI: PCI: Interrupt link GSIH configured for IRQ 231340worker2 # [ 0.450317] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1341lb # [ 0.459791] iommu: Default domain type: Translated1342client # [ 0.440384] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1343worker2 # [ 0.451722] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1344lb # [ 0.460581] iommu: DMA domain TLB invalidation policy: lazy mode1345client # [ 0.440958] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1346lb # [ 0.461789] ACPI: bus type USB registered1347lb # [ 0.462523] usbcore: registered new interface driver usbfs1348client # [ 0.442565] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1349lb # [ 0.462790] usbcore: registered new interface driver hub1350worker2 # [ 0.455034] ACPI: PCI: Interrupt link LNKA configured for IRQ 101351worker1 # [ 0.442883] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1352lb # [ 0.463649] usbcore: registered new device driver usb1353worker2 # [ 0.456109] ACPI: PCI: Interrupt link LNKB configured for IRQ 101354lb # [ 0.465207] NetLabel: Initializing1355worker1 # [ 0.445476] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1356worker2 # [ 0.457099] ACPI: PCI: Interrupt link LNKC configured for IRQ 111357lb # [ 0.465775] NetLabel: domain hash size = 1281358lb # [ 0.466485] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1359worker2 # [ 0.458092] ACPI: PCI: Interrupt link LNKD configured for IRQ 111360worker1 # [ 0.446655] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1361lb # [ 0.466811] NetLabel: unlabeled traffic allowed by default1362worker2 # [ 0.459093] ACPI: PCI: Interrupt link LNKE configured for IRQ 101363lb # [ 0.467671] PCI: Using ACPI for IRQ routing1364worker1 # [ 0.447950] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1365worker2 # [ 0.460092] ACPI: PCI: Interrupt link LNKF configured for IRQ 101366worker2 # [ 0.461095] ACPI: PCI: Interrupt link LNKG configured for IRQ 111367worker1 # [ 0.448912] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1368worker2 # [ 0.462092] ACPI: PCI: Interrupt link LNKH configured for IRQ 111369worker2 # [ 0.463031] ACPI: PCI: Interrupt link GSIA configured for IRQ 161370worker1 # [ 0.450696] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1371worker2 # [ 0.464010] ACPI: PCI: Interrupt link GSIB configured for IRQ 171372worker1 # [ 0.451619] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1373worker2 # [ 0.465008] ACPI: PCI: Interrupt link GSIC configured for IRQ 181374client # [ 0.443618] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1375worker1 # [ 0.452744] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1376worker2 # [ 0.466007] ACPI: PCI: Interrupt link GSID configured for IRQ 191377worker2 # [ 0.466945] ACPI: PCI: Interrupt link GSIE configured for IRQ 201378client # [ 0.445491] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1379worker2 # [ 0.467773] ACPI: PCI: Interrupt link GSIF configured for IRQ 211380worker1 # [ 0.454302] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1381worker2 # [ 0.468712] ACPI: PCI: Interrupt link GSIG configured for IRQ 221382client # [ 0.446036] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1383worker2 # [ 0.469010] ACPI: PCI: Interrupt link GSIH configured for IRQ 231384client # [ 0.447077] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1385worker2 # [ 0.471085] iommu: Default domain type: Translated1386worker1 # [ 0.457750] ACPI: PCI: Interrupt link LNKA configured for IRQ 101387client # [ 0.448024] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1388worker2 # [ 0.471877] iommu: DMA domain TLB invalidation policy: lazy mode1389worker1 # [ 0.458773] ACPI: PCI: Interrupt link LNKB configured for IRQ 101390worker2 # [ 0.472990] ACPI: bus type USB registered1391worker1 # [ 0.459767] ACPI: PCI: Interrupt link LNKC configured for IRQ 111392client # [ 0.449805] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1393worker2 # [ 0.473744] usbcore: registered new interface driver usbfs1394worker1 # [ 0.460773] ACPI: PCI: Interrupt link LNKD configured for IRQ 111395client # [ 0.450660] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1396worker1 # [ 0.461776] ACPI: PCI: Interrupt link LNKE configured for IRQ 101397worker2 # [ 0.474013] usbcore: registered new interface driver hub1398client # [ 0.451832] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1399worker1 # [ 0.462767] ACPI: PCI: Interrupt link LNKF configured for IRQ 101400worker2 # [ 0.475009] usbcore: registered new device driver usb1401client # [ 0.453467] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1402worker1 # [ 0.463798] ACPI: PCI: Interrupt link LNKG configured for IRQ 111403worker2 # [ 0.476970] NetLabel: Initializing1404worker1 # [ 0.464767] ACPI: PCI: Interrupt link LNKH configured for IRQ 111405worker2 # [ 0.477431] NetLabel: domain hash size = 1281406worker2 # [ 0.477995] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1407worker1 # [ 0.465711] ACPI: PCI: Interrupt link GSIA configured for IRQ 161408worker2 # [ 0.478982] NetLabel: unlabeled traffic allowed by default1409worker1 # [ 0.466684] ACPI: PCI: Interrupt link GSIB configured for IRQ 171410worker2 # [ 0.479728] PCI: Using ACPI for IRQ routing1411worker1 # [ 0.467685] ACPI: PCI: Interrupt link GSIC configured for IRQ 181412worker1 # [ 0.468681] ACPI: PCI: Interrupt link GSID configured for IRQ 191413worker1 # [ 0.469681] ACPI: PCI: Interrupt link GSIE configured for IRQ 201414worker1 # [ 0.470690] ACPI: PCI: Interrupt link GSIF configured for IRQ 211415worker1 # [ 0.471680] ACPI: PCI: Interrupt link GSIG configured for IRQ 221416worker1 # [ 0.472681] ACPI: PCI: Interrupt link GSIH configured for IRQ 231417worker1 # [ 0.474686] iommu: Default domain type: Translated1418worker1 # [ 0.475538] iommu: DMA domain TLB invalidation policy: lazy mode1419client # [ 0.459459] ACPI: PCI: Interrupt link LNKA configured for IRQ 101420worker1 # [ 0.476734] ACPI: bus type USB registered1421client # [ 0.459878] ACPI: PCI: Interrupt link LNKB configured for IRQ 101422worker1 # [ 0.477514] usbcore: registered new interface driver usbfs1423client # [ 0.460864] ACPI: PCI: Interrupt link LNKC configured for IRQ 111424client # [ 0.461864] ACPI: PCI: Interrupt link LNKD configured for IRQ 111425worker1 # [ 0.478688] usbcore: registered new interface driver hub1426client # [ 0.462866] ACPI: PCI: Interrupt link LNKE configured for IRQ 101427worker1 # [ 0.479679] usbcore: registered new device driver usb1428client # [ 0.463863] ACPI: PCI: Interrupt link LNKF configured for IRQ 101429worker1 # [ 0.481435] NetLabel: Initializing1430worker1 # [ 0.481675] NetLabel: domain hash size = 1281431client # [ 0.464861] ACPI: PCI: Interrupt link LNKG configured for IRQ 111432worker1 # [ 0.482461] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1433client # [ 0.465864] ACPI: PCI: Interrupt link LNKH configured for IRQ 111434worker1 # [ 0.483474] NetLabel: unlabeled traffic allowed by default1435client # [ 0.466800] ACPI: PCI: Interrupt link GSIA configured for IRQ 161436worker1 # [ 0.484386] PCI: Using ACPI for IRQ routing1437client # [ 0.467778] ACPI: PCI: Interrupt link GSIB configured for IRQ 171438lb # [ 0.510158] pci 0000:00:01.0: vgaarb: setting as boot VGA device1439client # [ 0.468780] ACPI: PCI: Interrupt link GSIC configured for IRQ 181440lb # [ 0.510771] pci 0000:00:01.0: vgaarb: bridge control possible1441client # [ 0.469766] ACPI: PCI: Interrupt link GSID configured for IRQ 191442lb # [ 0.510771] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1443client # [ 0.470769] ACPI: PCI: Interrupt link GSIE configured for IRQ 201444lb # [ 0.510779] vgaarb: loaded1445client # [ 0.471747] ACPI: PCI: Interrupt link GSIF configured for IRQ 211446lb # [ 0.511509] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01447client # [ 0.472526] ACPI: PCI: Interrupt link GSIG configured for IRQ 221448lb # [ 0.511775] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1449client # [ 0.472783] ACPI: PCI: Interrupt link GSIH configured for IRQ 231450client # [ 0.475053] iommu: Default domain type: Translated1451lb # [ 0.513852] clocksource: Switched to clocksource kvm-clock1452client # [ 0.475770] iommu: DMA domain TLB invalidation policy: lazy mode1453lb # [ 0.515470] VFS: Disk quotas dquot_6.6.01454client # [ 0.477035] ACPI: bus type USB registered1455client # [ 0.477811] usbcore: registered new interface driver usbfs1456client # [ 0.478767] usbcore: registered new interface driver hub1457lb # [ 0.518022] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1458client # [ 0.479684] usbcore: registered new device driver usb1459lb # [ 0.519323] pnp: PnP ACPI init1460client # [ 0.481262] NetLabel: Initializing1461lb # [ 0.520115] ACPI: IRQ 4 override to edge(!), high(!)1462client # [ 0.481764] NetLabel: domain hash size = 1281463lb # [ 0.521033] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1464client # [ 0.482525] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1465lb # [ 0.522572] pnp: PnP ACPI: found 5 devices1466client # [ 0.482803] NetLabel: unlabeled traffic allowed by default1467client # [ 0.483706] PCI: Using ACPI for IRQ routing1468lb # [ 0.530127] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1469worker2 # [ 0.523327] pci 0000:00:01.0: vgaarb: setting as boot VGA device1470lb # [ 0.531550] clocksource: Switched to clocksource acpi_pm1471worker2 # [ 0.523992] pci 0000:00:01.0: vgaarb: bridge control possible1472lb # [ 0.532486] NET: Registered PF_INET protocol family1473worker2 # [ 0.523992] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1474worker2 # [ 0.524000] vgaarb: loaded1475lb # [ 0.533650] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1476worker2 # [ 0.524727] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01477worker2 # [ 0.524997] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1478worker2 # [ 0.530081] clocksource: Switched to clocksource kvm-clock1479worker2 # [ 0.533609] VFS: Disk quotas dquot_6.6.01480worker2 # [ 0.534308] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1481worker2 # [ 0.535705] pnp: PnP ACPI init1482worker2 # [ 0.536505] ACPI: IRQ 4 override to edge(!), high(!)1483worker2 # [ 0.537435] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1484worker1 # [ 0.527757] pci 0000:00:01.0: vgaarb: setting as boot VGA device1485worker2 # [ 0.539030] pnp: PnP ACPI: found 5 devices1486worker1 # [ 0.528664] pci 0000:00:01.0: vgaarb: bridge control possible1487lb # [ 0.550112] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1488worker1 # [ 0.528664] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1489lb # [ 0.551634] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1490worker1 # [ 0.528671] vgaarb: loaded1491worker1 # [ 0.529436] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01492lb # [ 0.552922] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1493worker1 # [ 0.529670] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1494lb # [ 0.554249] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1495lb # [ 0.555527] TCP: Hash tables configured (established 16384 bind 16384)1496lb # [ 0.556657] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1497worker2 # [ 0.546669] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1498lb # [ 0.557955] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1499worker2 # [ 0.548125] clocksource: Switched to clocksource acpi_pm1500worker1 # [ 0.534751] clocksource: Switched to clocksource kvm-clock1501lb # [ 0.559019] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1502worker2 # [ 0.549111] NET: Registered PF_INET protocol family1503lb # [ 0.560180] NET: Registered PF_UNIX/PF_LOCAL protocol family1504lb # [ 0.561143] NET: Registered PF_XDP protocol family1505worker2 # [ 0.550458] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1506lb # [ 0.561982] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1507worker1 # [ 0.538204] VFS: Disk quotas dquot_6.6.01508lb # [ 0.562985] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1509worker1 # [ 0.538909] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1510lb # [ 0.563999] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1511worker1 # [ 0.540287] pnp: PnP ACPI init1512lb # [ 0.565101] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1513worker1 # [ 0.541079] ACPI: IRQ 4 override to edge(!), high(!)1514lb # [ 0.566208] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1515client # [ 0.526496] pci 0000:00:01.0: vgaarb: setting as boot VGA device1516worker1 # [ 0.542079] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1517lb # [ 0.567299] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1518client # [ 0.526759] pci 0000:00:01.0: vgaarb: bridge control possible1519worker1 # [ 0.543597] pnp: PnP ACPI: found 5 devices1520lb # [ 0.568950] ACPI: \_SB_.GSIA: Enabled at IRQ 161521client # [ 0.526759] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1522client # [ 0.526767] vgaarb: loaded1523client # [ 0.527519] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01524lb # [ 0.571129] ACPI: \_SB_.GSIB: Enabled at IRQ 171525client # [ 0.527763] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1526lb # [ 0.573183] ACPI: \_SB_.GSIC: Enabled at IRQ 181527lb # [ 0.575143] ACPI: \_SB_.GSID: Enabled at IRQ 191528worker1 # [ 0.551174] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1529client # [ 0.531853] clocksource: Switched to clocksource kvm-clock1530lb # [ 0.576923] PCI: CLS 0 bytes, default 641531worker1 # [ 0.552661] clocksource: Switched to clocksource acpi_pm1532worker2 # [ 0.566296] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1533worker1 # [ 0.553633] NET: Registered PF_INET protocol family1534lb # [ 0.577784] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1535worker2 # [ 0.567839] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1536lb # [ 0.579478] Trying to unpack rootfs image as initramfs...1537worker1 # [ 0.555004] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1538client # [ 0.535402] VFS: Disk quotas dquot_6.6.01539worker2 # [ 0.569158] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1540client # [ 0.536073] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1541worker2 # [ 0.570489] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1542client # [ 0.537439] pnp: PnP ACPI init1543worker2 # [ 0.571798] TCP: Hash tables configured (established 16384 bind 16384)1544client # [ 0.538222] ACPI: IRQ 4 override to edge(!), high(!)1545worker2 # [ 0.573016] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1546client # [ 0.539211] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1547worker2 # [ 0.574396] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1548client # [ 0.540780] pnp: PnP ACPI: found 5 devices1549worker2 # [ 0.575606] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1550worker2 # [ 0.576864] NET: Registered PF_UNIX/PF_LOCAL protocol family1551worker2 # [ 0.577881] NET: Registered PF_XDP protocol family1552worker2 # [ 0.578786] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1553worker2 # [ 0.579866] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1554worker2 # [ 0.580930] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1555worker2 # [ 0.582126] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1556client # [ 0.548355] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1557worker2 # [ 0.583286] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1558client # [ 0.550052] clocksource: Switched to clocksource acpi_pm1559client # [ 0.551013] NET: Registered PF_INET protocol family1560worker2 # [ 0.584947] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1561worker1 # [ 0.571023] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1562worker2 # [ 0.586698] ACPI: \_SB_.GSIA: Enabled at IRQ 161563client # [ 0.552181] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1564worker1 # [ 0.572523] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1565worker1 # [ 0.573822] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1566worker2 # [ 0.588815] ACPI: \_SB_.GSIB: Enabled at IRQ 171567worker1 # [ 0.575256] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1568worker1 # [ 0.576513] TCP: Hash tables configured (established 16384 bind 16384)1569worker2 # [ 0.590878] ACPI: \_SB_.GSIC: Enabled at IRQ 181570worker1 # [ 0.577684] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1571worker2 # [ 0.592964] ACPI: \_SB_.GSID: Enabled at IRQ 191572worker1 # [ 0.579038] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1573worker1 # [ 0.580153] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1574worker2 # [ 0.594696] PCI: CLS 0 bytes, default 641575worker1 # [ 0.581405] NET: Registered PF_UNIX/PF_LOCAL protocol family1576worker2 # [ 0.595739] Trying to unpack rootfs image as initramfs...1577worker1 # [ 0.582407] NET: Registered PF_XDP protocol family1578worker1 # [ 0.583259] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1579worker1 # [ 0.584285] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1580worker1 # [ 0.585311] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1581worker1 # [ 0.586502] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1582worker1 # [ 0.587663] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1583worker2 # [ 0.601714] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1584worker1 # [ 0.588818] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1585client # [ 0.568971] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1586worker1 # [ 0.590604] ACPI: \_SB_.GSIA: Enabled at IRQ 161587client # [ 0.570557] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1588client # [ 0.571895] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1589worker1 # [ 0.593184] ACPI: \_SB_.GSIB: Enabled at IRQ 171590client # [ 0.573287] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1591worker1 # [ 0.595101] ACPI: \_SB_.GSIC: Enabled at IRQ 181592client # [ 0.574602] TCP: Hash tables configured (established 16384 bind 16384)1593client # [ 0.575709] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1594worker1 # [ 0.597237] ACPI: \_SB_.GSID: Enabled at IRQ 191595client # [ 0.577004] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1596lb # [ 0.623265] Initialise system trusted keyrings1597client # [ 0.578123] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1598worker1 # [ 0.598829] PCI: CLS 0 bytes, default 641599client # [ 0.579339] NET: Registered PF_UNIX/PF_LOCAL protocol family1600worker1 # [ 0.599699] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1601client # [ 0.580292] NET: Registered PF_XDP protocol family1602worker1 # [ 0.601472] Trying to unpack rootfs image as initramfs...1603client # [ 0.581159] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1604lb # [ 0.626607] workingset: timestamp_bits=40 max_order=19 bucket_order=01605client # [ 0.582155] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1606client # [ 0.583163] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1607client # [ 0.584305] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1608client # [ 0.585463] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1609client # [ 0.586600] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1610client # [ 0.588360] ACPI: \_SB_.GSIA: Enabled at IRQ 161611client # [ 0.590765] ACPI: \_SB_.GSIB: Enabled at IRQ 171612client # [ 0.592763] ACPI: \_SB_.GSIC: Enabled at IRQ 181613client # [ 0.594802] ACPI: \_SB_.GSID: Enabled at IRQ 191614client # [ 0.596518] PCI: CLS 0 bytes, default 641615client # [ 0.597439] Trying to unpack rootfs image as initramfs...1616client # [ 0.601466] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1617lb # [ 0.647692] Key type asymmetric registered1618lb # [ 0.648382] Asymmetric key parser 'x509' registered1619lb # [ 0.649251] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1620lb # [ 0.652660] io scheduler mq-deadline registered1621worker2 # [ 0.642104] Initialise system trusted keyrings1622lb # [ 0.653408] io scheduler kyber registered1623lb # [ 0.656288] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1624worker2 # [ 0.645641] workingset: timestamp_bits=40 max_order=19 bucket_order=01625lb # [ 0.659540] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1626lb # [ 0.663125] Linux agpgart interface v0.1031627lb # [ 0.663894] ACPI: bus type drm_connector registered1628lb # [ 0.666981] usbcore: registered new interface driver usbserial_generic1629lb # [ 0.668085] usbserial: USB Serial support registered for generic1630worker1 # [ 0.645626] Initialise system trusted keyrings1631lb # [ 0.671534] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1632lb # [ 0.671534] Please enable it if your BIOS has the CPPC option.1633worker1 # [ 0.648975] workingset: timestamp_bits=40 max_order=19 bucket_order=01634lb # [ 0.673797] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1635lb # [ 0.676660] drop_monitor: Initializing network drop monitor service1636worker2 # [ 0.666846] Key type asymmetric registered1637lb # [ 0.677825] NET: Registered PF_INET6 protocol family1638worker2 # [ 0.669593] Asymmetric key parser 'x509' registered1639worker2 # [ 0.670489] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1640lb # [ 0.682961] Segment Routing with IPv61641worker2 # [ 0.673692] io scheduler mq-deadline registered1642worker2 # [ 0.674505] io scheduler kyber registered1643lb # [ 0.685550] In-situ OAM (IOAM) with IPv61644lb # [ 0.686565] IPI shorthand broadcast: enabled1645client # [ 0.643161] Initialise system trusted keyrings1646worker2 # [ 0.678110] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1647worker2 # [ 0.679441] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1648client # [ 0.646394] workingset: timestamp_bits=40 max_order=19 bucket_order=01649worker2 # [ 0.683348] Linux agpgart interface v0.1031650worker1 # [ 0.670140] Key type asymmetric registered1651worker2 # [ 0.684135] ACPI: bus type drm_connector registered1652lb # [ 0.694791] sched_clock: Marking stable (562013546, 132502017)->(760181849, -65666286)1653worker1 # [ 0.670848] Asymmetric key parser 'x509' registered1654worker1 # [ 0.672949] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1655worker2 # [ 0.687019] usbcore: registered new interface driver usbserial_generic1656lb # [ 0.698608] registered taskstats version 11657worker2 # [ 0.688114] usbserial: USB Serial support registered for generic1658lb # [ 0.699575] Loading compiled-in X.509 certificates1659worker1 # [ 0.675946] io scheduler mq-deadline registered1660worker1 # [ 0.676757] io scheduler kyber registered1661worker2 # [ 0.690602] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1662worker2 # [ 0.690602] Please enable it if your BIOS has the CPPC option.1663worker2 # [ 0.692914] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1664worker1 # [ 0.679434] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1665worker1 # [ 0.680733] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1666worker2 # [ 0.698716] drop_monitor: Initializing network drop monitor service1667worker2 # [ 0.699933] NET: Registered PF_INET6 protocol family1668worker1 # [ 0.686675] Linux agpgart interface v0.1031669worker1 # [ 0.687482] ACPI: bus type drm_connector registered1670client # [ 0.667686] Key type asymmetric registered1671worker2 # [ 0.703067] Segment Routing with IPv61672worker1 # [ 0.690309] usbcore: registered new interface driver usbserial_generic1673client # [ 0.670343] Asymmetric key parser 'x509' registered1674worker1 # [ 0.691464] usbserial: USB Serial support registered for generic1675worker2 # [ 0.705611] In-situ OAM (IOAM) with IPv61676client # [ 0.671178] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1677worker2 # [ 0.706610] IPI shorthand broadcast: enabled1678lb # [ 0.718531] Demotion targets for Node 0: null1679lb # [ 0.719354] Key type .fscrypt registered1680client # [ 0.674490] io scheduler mq-deadline registered1681client # [ 0.675232] io scheduler kyber registered1682worker1 # [ 0.694912] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1683worker1 # [ 0.694912] Please enable it if your BIOS has the CPPC option.1684worker1 # [ 0.697048] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1685lb # [ 0.722522] Key type fscrypt-provisioning registered1686client # [ 0.677925] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1687lb # [ 0.723417] ima: No TPM chip found, activating TPM-bypass!1688lb # [ 0.724277] ima: Allocated hash algorithm: sha11689client # [ 0.679221] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1690worker1 # [ 0.700015] drop_monitor: Initializing network drop monitor service1691worker1 # [ 0.701180] NET: Registered PF_INET6 protocol family1692worker2 # [ 0.714729] sched_clock: Marking stable (581014311, 133557962)->(783287084, -68714811)1693lb # [ 0.726547] ima: No architecture policies found1694lb # [ 0.727423] PM: Magic number: 10:175:9831695worker1 # [ 0.704371] Segment Routing with IPv61696worker2 # [ 0.718812] registered taskstats version 11697client # [ 0.684950] Linux agpgart interface v0.1031698client # [ 0.685738] ACPI: bus type drm_connector registered1699worker2 # [ 0.719793] Loading compiled-in X.509 certificates1700lb # [ 0.730991] RAS: Correctable Errors collector initialized.1701worker1 # [ 0.706901] In-situ OAM (IOAM) with IPv61702worker1 # [ 0.707956] IPI shorthand broadcast: enabled1703client # [ 0.688812] usbcore: registered new interface driver usbserial_generic1704client # [ 0.689934] usbserial: USB Serial support registered for generic1705client # [ 0.692351] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1706lb # [ 0.738555] clk: Disabling unused clocks1707client # [ 0.692351] Please enable it if your BIOS has the CPPC option.1708lb # [ 0.739202] PM: genpd: Disabling unused power domains1709client # [ 0.694628] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1710worker1 # [ 0.716197] sched_clock: Marking stable (583014199, 132866846)->(787192114, -71311069)1711worker1 # [ 0.720125] registered taskstats version 11712client # [ 0.699500] drop_monitor: Initializing network drop monitor service1713client # [ 0.700683] NET: Registered PF_INET6 protocol family1714worker1 # [ 0.721086] Loading compiled-in X.509 certificates1715client # [ 0.703813] Segment Routing with IPv61716client # [ 0.704496] In-situ OAM (IOAM) with IPv61717worker2 # [ 0.738585] Demotion targets for Node 0: null1718worker2 # [ 0.739470] Key type .fscrypt registered1719client # [ 0.706637] IPI shorthand broadcast: enabled1720worker2 # [ 0.741583] Key type fscrypt-provisioning registered1721worker2 # [ 0.742584] ima: No TPM chip found, activating TPM-bypass!1722worker2 # [ 0.743521] ima: Allocated hash algorithm: sha11723worker2 # [ 0.747613] ima: No architecture policies found1724worker2 # [ 0.748624] PM: Magic number: 10:175:9831725client # [ 0.715049] sched_clock: Marking stable (581014191, 133310393)->(796401448, -82076864)1726worker2 # [ 0.750272] RAS: Correctable Errors collector initialized.1727worker1 # [ 0.739901] Demotion targets for Node 0: null1728worker1 # [ 0.740766] Key type .fscrypt registered1729client # [ 0.720440] registered taskstats version 11730client # [ 0.721387] Loading compiled-in X.509 certificates1731worker1 # [ 0.743886] Key type fscrypt-provisioning registered1732worker1 # [ 0.744859] ima: No TPM chip found, activating TPM-bypass!1733worker2 # [ 0.759232] clk: Disabling unused clocks1734worker1 # [ 0.745832] ima: Allocated hash algorithm: sha11735worker1 # [ 0.747937] ima: No architecture policies found1736worker1 # [ 0.748946] PM: Magic number: 10:175:9831737worker2 # [ 0.763618] PM: genpd: Disabling unused power domains1738worker1 # [ 0.750546] RAS: Correctable Errors collector initialized.1739client # [ 0.737906] Demotion targets for Node 0: null1740worker1 # [ 0.761555] clk: Disabling unused clocks1741client # [ 0.741371] Key type .fscrypt registered1742client # [ 0.742066] Key type fscrypt-provisioning registered1743client # [ 0.743046] ima: No TPM chip found, activating TPM-bypass!1744worker1 # [ 0.763915] PM: genpd: Disabling unused power domains1745client # [ 0.745337] ima: Allocated hash algorithm: sha11746client # [ 0.746153] ima: No architecture policies found1747client # [ 0.750343] PM: Magic number: 10:175:9831748client # [ 0.751999] RAS: Correctable Errors collector initialized.1749client # [ 0.760027] clk: Disabling unused clocks1750client # [ 0.764354] PM: genpd: Disabling unused power domains1751lb # [ 0.899103] Freeing initrd memory: 29064K1752lb # [ 0.902172] Freeing unused decrypted memory: 2028K1753lb # [ 0.904715] Freeing unused kernel image (initmem) memory: 3652K1754lb # [ 0.905732] Write protecting the kernel read-only data: 32768k1755lb # [ 0.907477] Freeing unused kernel image (text/rodata gap) memory: 1184K1756lb # [ 0.908889] Freeing unused kernel image (rodata/data gap) memory: 720K1757worker2 # [ 0.919843] Freeing initrd memory: 29060K1758worker2 # [ 0.922974] Freeing unused decrypted memory: 2028K1759worker2 # [ 0.925538] Freeing unused kernel image (initmem) memory: 3652K1760worker2 # [ 0.926708] Write protecting the kernel read-only data: 32768k1761worker2 # [ 0.928594] Freeing unused kernel image (text/rodata gap) memory: 1184K1762worker2 # [ 0.930118] Freeing unused kernel image (rodata/data gap) memory: 720K1763worker1 # [ 0.921982] Freeing initrd memory: 29056K1764worker1 # [ 0.925132] Freeing unused decrypted memory: 2028K1765worker1 # [ 0.927687] Freeing unused kernel image (initmem) memory: 3652K1766worker1 # [ 0.928810] Write protecting the kernel read-only data: 32768k1767worker1 # [ 0.930686] Freeing unused kernel image (text/rodata gap) memory: 1184K1768worker1 # [ 0.932245] Freeing unused kernel image (rodata/data gap) memory: 720K1769lb # [ 0.959642] x86/mm: Checked W+X mappings: passed, no W+X pages found.1770lb # [ 0.960688] Run /init as init process1771lb # [ 0.971259] systemd[1]: Inserted module 'autofs4'1772client # [ 0.927516] Freeing initrd memory: 29056K1773client # [ 0.930709] Freeing unused decrypted memory: 2028K1774client # [ 0.933301] Freeing unused kernel image (initmem) memory: 3652K1775client # [ 0.934404] Write protecting the kernel read-only data: 32768k1776client # [ 0.936250] Freeing unused kernel image (text/rodata gap) memory: 1184K1777client # [ 0.937801] Freeing unused kernel image (rodata/data gap) memory: 720K1778lb # [ 0.991068] fuse: init (API version 7.45)1779worker2 # [ 0.980968] x86/mm: Checked W+X mappings: passed, no W+X pages found.1780worker2 # [ 0.982095] Run /init as init process1781lb # [ 0.998414] ACPI: \_SB_.GSIG: Enabled at IRQ 221782lb # [ 1.000795] ACPI: \_SB_.GSIH: Enabled at IRQ 231783worker2 # [ 0.992534] systemd[1]: Inserted module 'autofs4'1784lb # [ 1.003622] ACPI: \_SB_.GSIE: Enabled at IRQ 201785lb # [ 1.005449] ACPI: \_SB_.GSIF: Enabled at IRQ 211786worker1 # [ 0.982891] x86/mm: Checked W+X mappings: passed, no W+X pages found.1787worker1 # [ 0.984002] Run /init as init process1788lb # [ 1.010200] virtiofs virtio5: discovered new tag: nix-store1789lb # [ 1.011825] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1790worker1 # [ 0.994367] systemd[1]: Inserted module 'autofs4'1791lb # [ 1.020244] virtiofs virtio6: discovered new tag: shared1792lb # [ 1.022071] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1793worker2 # [ 1.012265] fuse: init (API version 7.45)1794lb # [ 1.025680] virtiofs virtio7: discovered new tag: xchg1795lb # [ 1.027078] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1796worker2 # [ 1.019717] ACPI: \_SB_.GSIG: Enabled at IRQ 221797worker2 # [ 1.022138] ACPI: \_SB_.GSIH: Enabled at IRQ 231798client # [ 0.988826] x86/mm: Checked W+X mappings: passed, no W+X pages found.1799client # [ 0.989895] Run /init as init process1800worker2 # [ 1.025169] ACPI: \_SB_.GSIE: Enabled at IRQ 201801worker1 # [ 1.014108] fuse: init (API version 7.45)1802worker2 # [ 1.027968] ACPI: \_SB_.GSIF: Enabled at IRQ 211803client # [ 1.000582] systemd[1]: Inserted module 'autofs4'1804worker2 # [ 1.034899] virtiofs virtio5: discovered new tag: nix-store1805worker1 # [ 1.021629] ACPI: \_SB_.GSIG: Enabled at IRQ 221806worker2 # [ 1.036615] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1807worker1 # [ 1.024098] ACPI: \_SB_.GSIH: Enabled at IRQ 231808lb # [ 1.049295] systemd[1]: Successfully made /usr/ read-only.1809worker1 # [ 1.027183] ACPI: \_SB_.GSIE: Enabled at IRQ 201810worker1 # [ 1.029196] ACPI: \_SB_.GSIF: Enabled at IRQ 211811worker2 # [ 1.044224] virtiofs virtio6: discovered new tag: shared1812worker2 # [ 1.045785] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1813worker1 # [ 1.034501] virtiofs virtio5: discovered new tag: nix-store1814worker2 # [ 1.049329] virtiofs virtio7: discovered new tag: xchg1815worker1 # [ 1.036102] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1816worker2 # [ 1.050849] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1817client # [ 1.020421] fuse: init (API version 7.45)1818worker1 # [ 1.044042] virtiofs virtio6: discovered new tag: shared1819worker1 # [ 1.045812] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1820client # [ 1.027741] ACPI: \_SB_.GSIG: Enabled at IRQ 221821worker1 # [ 1.050305] virtiofs virtio7: discovered new tag: xchg1822client # [ 1.030461] ACPI: \_SB_.GSIH: Enabled at IRQ 231823worker1 # [ 1.051912] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1824client # [ 1.033698] ACPI: \_SB_.GSIE: Enabled at IRQ 201825client # [ 1.035792] ACPI: \_SB_.GSIF: Enabled at IRQ 211826worker2 # [ 1.073691] systemd[1]: Successfully made /usr/ read-only.1827client # [ 1.042603] virtiofs virtio5: discovered new tag: nix-store1828client # [ 1.044084] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1829client # [ 1.051284] virtiofs virtio6: discovered new tag: shared1830client # [ 1.052890] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1831worker1 # [ 1.074145] systemd[1]: Successfully made /usr/ read-only.1832client # [ 1.056363] virtiofs virtio7: discovered new tag: xchg1833client # [ 1.057810] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1834client # [ 1.080097] systemd[1]: Successfully made /usr/ read-only.1835lb # [ 1.384755] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836lb # [ 1.389396] systemd[1]: Detected virtualization kvm.1837lb # [ 1.390217] systemd[1]: Detected architecture x86-64.1838lb # [ 1.391045] systemd[1]: Running in initrd.1839lb # [ 1.392113] systemd[1]: Initializing machine ID from random generator.1840lb # [ 1.393240] systemd[1]: Hostname set to <lb>.1841worker2 # [ 1.410499] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842worker2 # [ 1.422732] systemd[1]: Detected virtualization kvm.1843worker2 # [ 1.424903] systemd[1]: Detected architecture x86-64.1844worker1 # [ 1.409789] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1845worker2 # [ 1.427040] systemd[1]: Running in initrd.1846worker1 # [ 1.414724] systemd[1]: Detected virtualization kvm.1847worker1 # [ 1.415585] systemd[1]: Detected architecture x86-64.1848worker1 # [ 1.416450] systemd[1]: Running in initrd.1849worker2 # [ 1.429493] systemd[1]: Initializing machine ID from random generator.1850worker1 # [ 1.417560] systemd[1]: Initializing machine ID from random generator.1851worker1 # [ 1.418813] systemd[1]: Hostname set to <worker1>.1852worker2 # [ 1.432400] systemd[1]: Hostname set to <worker2>.1853client # [ 1.416596] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854client # [ 1.428554] systemd[1]: Detected virtualization kvm.1855client # [ 1.430710] systemd[1]: Detected architecture x86-64.1856client # [ 1.432821] systemd[1]: Running in initrd.1857client # [ 1.435157] systemd[1]: Initializing machine ID from random generator.1858client # [ 1.438013] systemd[1]: Hostname set to <client>.1859lb # [ 1.598578] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860lb # [ 1.645100] systemd[1]: Queued start job for default target Initrd Default Target.1861worker1 # [ 1.622561] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862lb # [ 1.648805] systemd[1]: Created slice Slice /system/modprobe.1863lb # [ 1.649908] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1864lb # [ 1.651225] systemd[1]: Expecting device /dev/disk/by-label/nixos...1865lb # [ 1.652270] systemd[1]: Reached target Path Units.1866lb # [ 1.653144] systemd[1]: Reached target Slice Units.1867lb # [ 1.653977] systemd[1]: Reached target Swaps.1868lb # [ 1.654737] systemd[1]: Reached target Timer Units.1869lb # [ 1.655650] systemd[1]: Listening on D-Bus System Message Bus Socket.1870lb # [ 1.656786] systemd[1]: Listening on Journal Socket (/dev/log).1871lb # [ 1.657880] systemd[1]: Listening on Journal Sockets.1872lb # [ 1.658827] systemd[1]: Listening on udev Control Socket.1873lb # [ 1.659786] systemd[1]: Listening on udev Kernel Socket.1874lb # [ 1.660689] systemd[1]: Reached target Socket Units.1875lb # [ 1.662355] systemd[1]: Starting Create List of Static Device Nodes...1876worker2 # [ 1.652442] systemd[1]: bpf-restrict-fs: LSM BPF program attached1877lb # [ 1.666586] systemd[1]: Starting Load Kernel Module configfs...1878lb # [ 1.674588] systemd[1]: Starting Journal Service...1879worker1 # [ 1.675024] systemd[1]: Queued start job for default target Initrd Default Target.1880worker1 # [ 1.678288] systemd[1]: Created slice Slice /system/modprobe.1881worker2 # [ 1.691788] systemd[1]: Queued start job for default target Initrd Default Target.1882lb # [ 1.703727] systemd[1]: Starting Load Kernel Modules...1883client # [ 1.657556] systemd[1]: bpf-restrict-fs: LSM BPF program attached1884worker1 # [ 1.679454] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1885worker1 # [ 1.680838] systemd[1]: Expecting device /dev/disk/by-label/nixos...1886worker1 # [ 1.682006] systemd[1]: Reached target Path Units.1887worker2 # [ 1.695950] systemd[1]: Created slice Slice /system/modprobe.1888worker1 # [ 1.683011] systemd[1]: Reached target Slice Units.1889worker2 # [ 1.697101] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1890worker1 # [ 1.683890] systemd[1]: Reached target Swaps.1891lb # [ 1.707624] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1892worker1 # [ 1.684662] systemd[1]: Reached target Timer Units.1893worker2 # [ 1.698432] systemd[1]: Expecting device /dev/disk/by-label/nixos...1894worker2 # [ 1.699477] systemd[1]: Reached target Path Units.1895worker1 # [ 1.685634] systemd[1]: Listening on D-Bus System Message Bus Socket.1896worker2 # [ 1.700362] systemd[1]: Reached target Slice Units.1897worker1 # [ 1.686833] systemd[1]: Listening on Journal Socket (/dev/log).1898worker2 # [ 1.701194] systemd[1]: Reached target Swaps.1899worker1 # [ 1.688029] systemd[1]: Listening on Journal Sockets.1900worker2 # [ 1.701963] systemd[1]: Reached target Timer Units.1901worker1 # [ 1.689041] systemd[1]: Listening on udev Control Socket.1902worker2 # [ 1.702934] systemd[1]: Listening on D-Bus System Message Bus Socket.1903worker1 # [ 1.690078] systemd[1]: Listening on udev Kernel Socket.1904worker2 # [ 1.704103] systemd[1]: Listening on Journal Socket (/dev/log).1905worker1 # [ 1.691059] systemd[1]: Reached target Socket Units.1906lb # [ 1.715715] systemd[1]: Starting Coldplug All udev Devices...1907worker2 # [ 1.705257] systemd[1]: Listening on Journal Sockets.1908worker2 # [ 1.706224] systemd[1]: Listening on udev Control Socket.1909worker1 # [ 1.692963] systemd[1]: Starting Create List of Static Device Nodes...1910worker2 # [ 1.707194] systemd[1]: Listening on udev Kernel Socket.1911worker2 # [ 1.708100] systemd[1]: Reached target Socket Units.1912worker2 # [ 1.709912] systemd[1]: Starting Create List of Static Device Nodes...1913worker1 # [ 1.696744] systemd[1]: Starting Load Kernel Module configfs...1914worker2 # [ 1.713395] systemd[1]: Starting Load Kernel Module configfs...1915lb # [ 1.727320] systemd-journald[66]: Collecting audit messages is disabled.1916lb # [ 1.729089] systemd[1]: Finished Create List of Static Device Nodes.1917lb # [ 1.736620] systemd[1]: modprobe@configfs.service: Deactivated successfully.1918worker1 # [ 1.706630] systemd[1]: Starting Journal Service...1919lb # [ 1.742909] systemd[1]: Finished Load Kernel Module configfs.1920worker2 # [ 1.722641] systemd[1]: Starting Journal Service...1921lb # [ 1.748977] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1922worker1 # [ 1.731124] systemd[1]: Starting Load Kernel Modules...1923client # [ 1.711507] systemd[1]: Queued start job for default target Initrd Default Target.1924worker2 # [ 1.749773] systemd[1]: Starting Load Kernel Modules...1925client # [ 1.715642] systemd[1]: Created slice Slice /system/modprobe.1926lb # [ 1.760644] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1927client # [ 1.716795] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1928worker1 # [ 1.737014] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1929client # [ 1.718201] systemd[1]: Expecting device /dev/disk/by-label/nixos...1930client # [ 1.719337] systemd[1]: Reached target Path Units.1931client # [ 1.720204] systemd[1]: Reached target Slice Units.1932worker2 # [ 1.753689] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1933client # [ 1.721076] systemd[1]: Reached target Swaps.1934client # [ 1.721911] systemd[1]: Reached target Timer Units.1935client # [ 1.722909] systemd[1]: Listening on D-Bus System Message Bus Socket.1936client # [ 1.724141] systemd[1]: Listening on Journal Socket (/dev/log).1937client # [ 1.725362] systemd[1]: Listening on Journal Sockets.1938worker1 # [ 1.746098] systemd[1]: Starting Coldplug All udev Devices...1939client # [ 1.726413] systemd[1]: Listening on udev Control Socket.1940worker2 # [ 1.760819] systemd[1]: Starting Coldplug All udev Devices...1941client # [ 1.727432] systemd[1]: Listening on udev Kernel Socket.1942client # [ 1.728399] systemd[1]: Reached target Socket Units.1943lb # [ 1.773191] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1944client # [ 1.730169] systemd[1]: Starting Create List of Static Device Nodes...1945client # [ 1.734177] systemd[1]: Starting Load Kernel Module configfs...1946lb # [ 1.781788] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1947worker1 # [ 1.758281] systemd-journald[66]: Collecting audit messages is disabled.1948worker2 # [ 1.773210] systemd-journald[66]: Collecting audit messages is disabled.1949worker2 # [ 1.776853] systemd[1]: Finished Create List of Static Device Nodes.1950worker1 # [ 1.763222] systemd[1]: Finished Create List of Static Device Nodes.1951worker2 # [ 1.781087] systemd[1]: modprobe@configfs.service: Deactivated successfully.1952worker1 # [ 1.768654] systemd[1]: modprobe@configfs.service: Deactivated successfully.1953lb # [ 1.798130] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1954worker2 # [ 1.788981] systemd[1]: Finished Load Kernel Module configfs.1955client # [ 1.743841] systemd[1]: Starting Journal Service...1956worker1 # [ 1.777318] systemd[1]: Finished Load Kernel Module configfs.1957worker2 # [ 1.794021] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1958lb # [ 1.808734] systemd[1]: Starting Create Static Device Nodes in /dev...1959worker1 # [ 1.784410] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1960client # [ 1.769508] systemd[1]: Starting Load Kernel Modules...1961worker2 # [ 1.806789] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1962client # [ 1.773425] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1963lb # [ 1.822107] systemd[1]: Finished Load Kernel Modules.1964worker1 # [ 1.798235] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1965client # [ 1.782601] systemd[1]: Starting Coldplug All udev Devices...1966worker2 # [ 1.816945] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1967lb # [ 1.829763] systemd[1]: Starting Apply Kernel Variables...1968worker1 # [ 1.804655] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1969worker2 # [ 1.824651] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1970client # [ 1.792109] systemd-journald[66]: Collecting audit messages is disabled.1971client # [ 1.795435] systemd[1]: Finished Create List of Static Device Nodes.1972worker1 # [ 1.817166] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1973client # [ 1.802903] systemd[1]: modprobe@configfs.service: Deactivated successfully.1974lb # [ 1.850599] systemd[1]: Finished Create Static Device Nodes in /dev.1975client # [ 1.809754] systemd[1]: Finished Load Kernel Module configfs.1976lb # [ 1.855739] systemd[1]: Started Journal Service.1977lb # [ 1.721872] systemd-modules-load[68]: Inserted module 'dm_mod'1978lb # [ 1.726223] systemd-modules-load[68]: Inserted module 'virtio_balloon'1979lb # [ 1.728295] systemd-modules-load[68]: Inserted module 'virtio_gpu'1980worker2 # [ 1.850182] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1981client # [ 1.815779] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1982lb # [ 1.731287] systemd[1]: Reached target Preparation for Local File Systems.1983lb # [ 1.732398] systemd[1]: Reached target Local File Systems.1984lb # [ 1.737073] systemd[1]: Starting Create System Files and Directories...1985worker2 # [ 1.859797] systemd[1]: Starting Create Static Device Nodes in /dev...1986client # [ 1.829408] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1987lb # [ 1.744932] systemd[1]: Starting Rule-based Manager for Device Events and Files...1988worker2 # [ 1.871133] systemd[1]: Finished Load Kernel Modules.1989lb # [ 1.751820] systemd[1]: Finished Apply Kernel Variables.1990client # [ 1.839341] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1991worker1 # [ 1.861677] systemd[1]: Finished Load Kernel Modules.1992worker2 # [ 1.880805] systemd[1]: Starting Apply Kernel Variables...1993client # [ 1.846496] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1994worker1 # [ 1.868091] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1995worker1 # [ 1.880017] systemd[1]: Starting Apply Kernel Variables...1996lb # [ 1.773177] systemd[1]: Finished Create System Files and Directories.1997worker2 # [ 1.762545] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.897241] systemd[1]: Started Journal Service.1998worker2 # 1999worker2 # [ 1.765320] systemd-modules-load[68]: Inserted module 'virtio_balloon'2000worker2 # [ 1.769346] systemd-modules-load[68]: Inserted module 'virtio_gpu'
loading 2,432 hidden lines…
4433worker2 # [ 27.083356] nix-grpc-daemon[784]: ts=2026-09-22T10:58:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4434worker1 # [ 27.265804] nix-grpc-daemon[782]: ts=2026-09-22T10:58:30Z level=info event=scheduler_connected addr=lb:500514435worker2 # [ 27.282143] nix-grpc-daemon[784]: ts=2026-09-22T10:58:30Z level=info event=scheduler_connected addr=lb:500514436worker1 # [ 27.269837] nix-grpc-daemon[782]: ts=2026-09-22T10:58:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4437worker2 # [ 27.285316] nix-grpc-daemon[784]: ts=2026-09-22T10:58:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4438worker1 # [ 27.470473] nix-grpc-daemon[782]: ts=2026-09-22T10:58:30Z level=info event=scheduler_connected addr=lb:500514439worker2 # [ 27.483487] nix-grpc-daemon[784]: ts=2026-09-22T10:58:30Z level=info event=scheduler_connected addr=lb:500514440worker1 # [ 27.474263] nix-grpc-daemon[782]: ts=2026-09-22T10:58:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4441worker2 # [ 27.488597] nix-grpc-daemon[784]: ts=2026-09-22T10:58:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4442lb: must succeed: curl -sf localhost:9901/clusters4443lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4444worker1 # [ 27.676304] nix-grpc-daemon[782]: ts=2026-09-22T10:58:30Z level=info event=scheduler_connected addr=lb:500514445worker2 # [ 27.690892] nix-grpc-daemon[784]: ts=2026-09-22T10:58:30Z level=info event=scheduler_connected addr=lb:500514446worker1 # [ 27.679510] nix-grpc-daemon[782]: ts=2026-09-22T10:58:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4447worker2 # [ 27.694590] nix-grpc-daemon[784]: ts=2026-09-22T10:58:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4448worker1 # [ 27.876500] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514449worker1 # [ 27.880331] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4450worker2 # [ 27.895691] nix-grpc-daemon[784]: ts=2026-09-22T10:58:30Z level=info event=scheduler_connected addr=lb:500514451worker2 # [ 27.899493] nix-grpc-daemon[784]: ts=2026-09-22T10:58:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4452worker1 # [ 28.081188] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514453worker1 # [ 28.085170] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4454worker2 # [ 28.100641] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514455worker2 # [ 28.104503] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4456worker1 # [ 28.285876] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514457worker1 # [ 28.289791] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4458worker2 # [ 28.306354] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514459worker2 # [ 28.309471] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4460worker1 # [ 28.490502] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514461worker1 # [ 28.494317] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4462worker2 # [ 28.507616] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514463worker2 # [ 28.511319] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4464lb: must succeed: curl -sf localhost:9901/clusters4465lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4466worker1 # [ 28.694955] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514467worker1 # [ 28.696730] nix-grpc-daemon[782]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4468worker2 # [ 28.710816] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514469worker2 # [ 28.713933] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4470worker1 # [ 28.897398] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=scheduler_connected addr=lb:500514471worker1 # [ 28.899471] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4472worker2 # [ 28.911713] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_connected addr=lb:500514473worker2 # [ 28.915505] nix-grpc-daemon[784]: ts=2026-09-22T10:58:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4474worker1 # [ 29.099899] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=scheduler_connected addr=lb:500514475worker1 # [ 29.101612] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4476worker2 # [ 29.117428] nix-grpc-daemon[784]: ts=2026-09-22T10:58:32Z level=info event=scheduler_connected addr=lb:500514477worker2 # [ 29.120539] nix-grpc-daemon[784]: ts=2026-09-22T10:58:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4478worker1 # [ 29.302479] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=scheduler_connected addr=lb:500514479worker1 # [ 29.306437] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4480worker2 # [ 29.318867] nix-grpc-daemon[784]: ts=2026-09-22T10:58:32Z level=info event=scheduler_connected addr=lb:500514481worker2 # [ 29.322667] nix-grpc-daemon[784]: ts=2026-09-22T10:58:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4482worker1 # [ 29.399378] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=scheduler_take_over4483lb # [ 29.424352] niks3-server[971]: 2026/09/22 10:58:31 INFO lead: acquired remote=192.168.1.3:372564484worker1 # [ 29.506860] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=scheduler_connected addr=lb:500514485worker2 # [ 29.523783] nix-grpc-daemon[784]: ts=2026-09-22T10:58:32Z level=info event=scheduler_connected addr=lb:500514486worker1 # [ 29.513056] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:386264487worker1 # [ 29.514505] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04488worker1 # [ 29.520974] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=oidc_jwks provider=mock keys=14489worker1 # [ 29.522232] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:386264490worker1 # [ 29.523934] nix-grpc-daemon[782]: ts=2026-09-22T10:58:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04491lb: must succeed: curl -sf localhost:9901/clusters4492lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4493worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4494worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4495lb: must succeed: curl -sf localhost:9901/clusters4496lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4497lb: must succeed: curl -sf localhost:9901/clusters4498lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4499lb: must succeed: curl -sf localhost:9901/clusters4500lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4501lb: must succeed: curl -sf localhost:9901/clusters4502lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4503worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4504worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.02 seconds)4505worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4506worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4507lb: must succeed: curl -sf localhost:9901/clusters4508lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4509lb: must succeed: curl -sf localhost:9901/clusters4510lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4511worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4512worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4513worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4514worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4515worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4516worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4517worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4518worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4519(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 9.66 seconds)4520subtest: balancer auth: client cert, bearer token, nothing, unknown CN4521worker2 # [ 30.088844] nix-grpc-daemon[784]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:472084522worker2 # [ 30.099550] systemd[1]: Started Nix Daemon.4523worker2 # [ 30.212896] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4524worker2 # [ 30.228410] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)4525worker2 # [ 30.313775] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 2/5)4526worker2 # [ 30.479977] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 3/5)4527worker2 # [ 30.573562] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 694 ms (attempt 4/5)4528worker2 # [ 31.271752] nix-grpc-daemon[784]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4529worker2 # [ 31.275864] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4530worker1 # [ 31.290549] nix-grpc-daemon[782]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:386424531worker1 # [ 31.298904] systemd[1]: Started Nix Daemon.4532worker1 # [ 31.408253] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4533worker1 # [ 31.430591] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)4534worker1 # [ 31.469992] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 2/5)4535worker1 # [ 31.490031] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 332 ms (attempt 3/5)4536worker1 # [ 31.824088] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 4/5)4537worker1 # [ 31.918968] nix-grpc-daemon[782]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4538worker1 # [ 31.922354] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4539worker1 # [ 31.936164] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=debug event=path_info_miss path=8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test error="error: path '/nix/store/8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test' is required, but there is no substituter that can build it"4540worker2 # [ 31.969258] nix-grpc-daemon[784]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:472084541worker2 # [ 31.976376] nix-grpc-daemon[784]: ts=2026-09-22T10:58:34Z level=debug event=path_info_miss path=8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test error="error: path '/nix/store/8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test' is required, but there is no substituter that can build it"4542client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4543client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.05 seconds)4544client: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4545client: (finished: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.02 seconds)4546worker1 # [ 32.178272] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:386424547worker2 # [ 32.212944] nix-grpc-daemon[784]: ts=2026-09-22T10:58:35Z level=info event=oidc_jwks provider=mock keys=14548worker2 # [ 32.216075] nix-grpc-daemon[784]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:472084549worker2 # [ 32.225843] nix-grpc-daemon[784]: ts=2026-09-22T10:58:35Z level=debug event=path_info_miss path=8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test error="error: path '/nix/store/8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test' is required, but there is no substituter that can build it"4550worker2 # [ 32.235604] nix-grpc-daemon[784]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:472084551worker2 # [ 32.244583] nix-grpc-daemon[784]: ts=2026-09-22T10:58:35Z level=debug event=path_info_miss path=8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test error="error: path '/nix/store/8lmzg2p5143ngwl04jg0vbyckwpscl4i-nixos-system-client-test' is required, but there is no substituter that can build it"4552client: must succeed: rm -r /var/lib/nix-grpc-store4553client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.01 seconds)4554worker1 # [ 32.374558] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=info event=denied method=QueryMissing cn=- role=none4555worker2 # [ 32.513104] nix-grpc-daemon[784]: ts=2026-09-22T10:58:35Z level=info event=denied method=QueryMissing cn=stranger role=none4556(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.62 seconds)4557subtest: DAG build is scheduled across workers and lands in the cache4558client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24559worker1 # [ 32.633389] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:386424560client # don't know how to build these paths:4561client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4562worker2 # [ 32.658639] nix-grpc-daemon[784]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472084563worker1 # [ 32.670191] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04564worker1 # [ 32.675711] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14565worker1 # [ 32.714480] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424566worker1 # [ 32.725727] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386424567client # copying 1 paths to 'lb:50051'...4568worker1 # [ 32.736912] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386424569client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4570worker1 # [ 32.759277] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4571worker1 # [ 32.783199] nix-grpc-daemon[782]: ts=2026-09-22T10:58:35Z level=info event=niks3_push_started pid=8784572worker1 # [ 32.928484] nix-grpc-daemon[878]: time=2026-09-22T10:58:36.106Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4573worker1 # [ 33.016170] nix-daemon[864]: accepted connection from pid 891, user nix-grpc-daemon (trusted)4574worker1 # [ 33.030820] nix-daemon[864]: reaped child process 894, status = succeeded4575worker1 # [ 33.109960] nix-daemon[864]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4576worker1 # [ 33.122241] nix-daemon[864]: reaped child process 899, status = succeeded4577lb # [ 33.153820] niks3-server[971]: 2026/09/22 10:58:35 INFO Received uploads request method=POST path=/api/pending_closures4578worker1 # [ 33.136109] nix-grpc-daemon[878]: time=2026-09-22T10:58:36.315Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4579worker1 # [ 33.138889] nix-grpc-daemon[878]: time=2026-09-22T10:58:36.317Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4580lb # [ 33.209346] niks3-server[971]: 2026/09/22 10:58:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4581lb # [ 33.228603] niks3-server[971]: 2026/09/22 10:58:35 INFO Signed narinfos id=1 count=14582worker1 # [ 33.205834] nix-grpc-daemon[878]: time=2026-09-22T10:58:36.384Z level=INFO msg="Uploading 1 narinfos"4583lb # [ 33.240540] niks3-server[971]: 2026/09/22 10:58:35 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4584lb # [ 33.242988] niks3-server[971]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4585lb # [ 33.263723] niks3-server[971]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4586lb # [ 33.272090] niks3-server[971]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4587lb # [ 33.279372] niks3-server[971]: 2026/09/22 10:58:35 INFO Completed upload id=14588worker1 # [ 33.255196] nix-grpc-daemon[878]: time=2026-09-22T10:58:36.434Z level=INFO msg="Upload complete. (322ms)"4589worker1 # [ 33.260675] nix-grpc-daemon[782]: ts=2026-09-22T10:58:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 paths=1 nar_bytes_in=5044590worker1 # [ 33.267403] nix-grpc-daemon[782]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424591worker1 # [ 33.279168] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4592client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4593worker1 # [ 33.295956] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4594client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4595worker1 # [ 33.299862] nix-daemon[864]: reaped child process 876, status = succeeded4596client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4597client # farm-a> LOG-farm-a-t14598worker1 # [ 34.718180] nix-daemon[864]: reaped child process 903, status = succeeded4599worker1 # [ 34.792595] nix-daemon[864]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4600worker1 # [ 34.803977] nix-daemon[864]: reaped child process 923, status = succeeded4601lb # [ 34.837474] niks3-server[971]: 2026/09/22 10:58:37 INFO Received uploads request method=POST path=/api/pending_closures4602worker1 # [ 34.823374] nix-grpc-daemon[878]: time=2026-09-22T10:58:38.002Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4603worker1 # [ 34.825723] nix-grpc-daemon[878]: time=2026-09-22T10:58:38.004Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4604lb # [ 34.883706] niks3-server[971]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4605lb # [ 34.891210] niks3-server[971]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4606lb # [ 34.897662] niks3-server[971]: 2026/09/22 10:58:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4607lb # [ 34.900449] niks3-server[971]: 2026/09/22 10:58:37 INFO Signed narinfos id=2 count=14608worker1 # [ 34.876436] nix-grpc-daemon[878]: time=2026-09-22T10:58:38.055Z level=INFO msg="Uploading 1 narinfos"4609lb # [ 34.912268] niks3-server[971]: 2026/09/22 10:58:37 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4610lb # [ 34.915320] niks3-server[971]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4611lb # [ 34.919520] niks3-server[971]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4612lb # [ 34.922913] niks3-server[971]: 2026/09/22 10:58:37 INFO Completed upload id=24613worker1 # [ 34.898209] nix-grpc-daemon[878]: time=2026-09-22T10:58:38.077Z level=INFO msg="Upload complete. (189ms)"4614worker1 # [ 34.901989] nix-grpc-daemon[782]: ts=2026-09-22T10:58:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14615worker1 # [ 34.907653] nix-grpc-daemon[782]: ts=2026-09-22T10:58:38Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04616worker1 # [ 34.909552] nix-grpc-daemon[782]: ts=2026-09-22T10:58:38Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14617worker1 # [ 34.914700] nix-daemon[864]: reaped child process 901, status = succeeded4618worker1 # [ 34.951407] nix-grpc-daemon[782]: ts=2026-09-22T10:58:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424619worker1 # [ 34.958906] nix-grpc-daemon[782]: ts=2026-09-22T10:58:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386424620worker1 # [ 34.967478] nix-grpc-daemon[878]: time=2026-09-22T10:58:38.146Z level=INFO msg="All 1 paths already cached"4621client # copying 1 paths to 'lb:50051'...4622worker1 # [ 34.972943] nix-grpc-daemon[782]: ts=2026-09-22T10:58:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386424623client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4624worker1 # [ 34.994380] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4625worker1 # [ 35.076682] nix-daemon[864]: accepted connection from pid 929, user nix-grpc-daemon (trusted)4626worker1 # [ 35.089805] nix-daemon[864]: reaped child process 932, status = succeeded4627worker1 # [ 35.170844] nix-daemon[864]: accepted connection from pid 934, user nix-grpc-daemon (trusted)4628worker1 # [ 35.183506] nix-daemon[864]: reaped child process 937, status = succeeded4629lb # [ 35.215173] niks3-server[971]: 2026/09/22 10:58:37 INFO Received uploads request method=POST path=/api/pending_closures4630worker1 # [ 35.199781] nix-grpc-daemon[878]: time=2026-09-22T10:58:38.378Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4631worker1 # [ 35.201267] nix-grpc-daemon[878]: time=2026-09-22T10:58:38.378Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4632lb # [ 35.242363] niks3-server[971]: 2026/09/22 10:58:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4633lb # [ 35.249092] niks3-server[971]: 2026/09/22 10:58:37 INFO Signed narinfos id=3 count=14634worker1 # [ 35.227269] nix-grpc-daemon[878]: time=2026-09-22T10:58:38.406Z level=INFO msg="Uploading 1 narinfos"4635lb # [ 35.259588] niks3-server[971]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4636lb # [ 35.264719] niks3-server[971]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4637lb # [ 35.280290] niks3-server[971]: 2026/09/22 10:58:37 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4638lb # [ 35.285275] niks3-server[971]: 2026/09/22 10:58:37 INFO Completed upload id=34639worker1 # [ 35.261952] nix-grpc-daemon[878]: time=2026-09-22T10:58:38.441Z level=INFO msg="Upload complete. (261ms)"4640lb # [ 35.288851] niks3-server[971]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4641worker1 # [ 35.265473] nix-grpc-daemon[782]: ts=2026-09-22T10:58:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 paths=1 nar_bytes_in=6244642worker1 # [ 35.270407] nix-grpc-daemon[782]: ts=2026-09-22T10:58:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424643worker1 # [ 35.280991] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4644worker1 # [ 35.286111] nix-daemon[864]: reaped child process 927, status = succeeded4645worker1 # [ 35.289223] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4646client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4647client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4648client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4649client # farm-b> LOG-farm-b-t14650lb # [ 36.225141] envoy[908]: conn peer=[2001:db8:1::1]:50460 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=19174651lb # [ 36.228728] envoy[908]: conn peer=[2001:db8:1::1]:50474 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=624652lb # [ 36.232604] envoy[908]: conn peer=[2001:db8:1::1]:50482 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=84653lb # [ 36.235737] envoy[908]: conn peer=[2001:db8:1::1]:50494 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=94654worker1 # [ 36.346177] nix-daemon[864]: reaped child process 941, status = succeeded4655worker1 # [ 36.400150] nix-daemon[864]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4656worker1 # [ 36.411584] nix-daemon[864]: reaped child process 960, status = succeeded4657lb # [ 36.443624] niks3-server[971]: 2026/09/22 10:58:38 INFO Received uploads request method=POST path=/api/pending_closures4658worker1 # [ 36.423579] nix-grpc-daemon[878]: time=2026-09-22T10:58:39.602Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4659worker1 # [ 36.425055] nix-grpc-daemon[878]: time=2026-09-22T10:58:39.604Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4660lb # [ 36.461073] niks3-server[971]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4661lb # [ 36.468522] niks3-server[971]: 2026/09/22 10:58:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4662lb # [ 36.470474] niks3-server[971]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4663lb # [ 36.472580] niks3-server[971]: 2026/09/22 10:58:38 INFO Signed narinfos id=4 count=14664worker1 # [ 36.449166] nix-grpc-daemon[878]: time=2026-09-22T10:58:39.628Z level=INFO msg="Uploading 1 narinfos"4665lb # [ 36.477547] niks3-server[971]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4666lb # [ 36.481627] niks3-server[971]: 2026/09/22 10:58:38 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4667lb # [ 36.485963] niks3-server[971]: 2026/09/22 10:58:38 INFO Completed upload id=44668worker1 # [ 36.460989] nix-grpc-daemon[878]: time=2026-09-22T10:58:39.640Z level=INFO msg="Upload complete. (163ms)"4669lb # [ 36.488060] niks3-server[971]: 2026/09/22 10:58:38 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4670worker1 # [ 36.463429] nix-grpc-daemon[782]: ts=2026-09-22T10:58:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14671worker1 # [ 36.469956] nix-grpc-daemon[782]: ts=2026-09-22T10:58:39Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04672worker1 # [ 36.471886] nix-grpc-daemon[782]: ts=2026-09-22T10:58:39Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14673worker1 # [ 36.475943] nix-daemon[864]: reaped child process 939, status = succeeded4674worker1 # [ 36.513107] nix-grpc-daemon[782]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424675worker1 # [ 36.521837] nix-grpc-daemon[782]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386424676worker1 # [ 36.530111] nix-grpc-daemon[878]: time=2026-09-22T10:58:39.709Z level=INFO msg="All 2 paths already cached"4677client # copying 1 paths to 'lb:50051'...4678worker1 # [ 36.536394] nix-grpc-daemon[782]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386424679client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4680worker1 # [ 36.549153] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4681worker1 # [ 36.639419] nix-daemon[864]: accepted connection from pid 966, user nix-grpc-daemon (trusted)4682worker1 # [ 36.651431] nix-daemon[864]: reaped child process 969, status = succeeded4683worker1 # [ 36.734213] nix-daemon[864]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4684worker1 # [ 36.745766] nix-daemon[864]: reaped child process 974, status = succeeded4685lb # [ 36.778563] niks3-server[971]: 2026/09/22 10:58:39 INFO Received uploads request method=POST path=/api/pending_closures4686worker1 # [ 36.758279] nix-grpc-daemon[878]: time=2026-09-22T10:58:39.937Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4687worker1 # [ 36.759730] nix-grpc-daemon[878]: time=2026-09-22T10:58:39.937Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4688lb # [ 36.791299] niks3-server[971]: 2026/09/22 10:58:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4689lb # [ 36.793319] niks3-server[971]: 2026/09/22 10:58:39 INFO Signed narinfos id=5 count=14690worker1 # [ 36.771316] nix-grpc-daemon[878]: time=2026-09-22T10:58:39.950Z level=INFO msg="Uploading 1 narinfos"4691lb # [ 36.801648] niks3-server[971]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4692lb # [ 36.811579] niks3-server[971]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4693lb # [ 36.816903] niks3-server[971]: 2026/09/22 10:58:39 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4694worker1 # [ 36.796845] nix-grpc-daemon[878]: time=2026-09-22T10:58:39.976Z level=INFO msg="Upload complete. (237ms)"4695lb # [ 36.821782] niks3-server[971]: 2026/09/22 10:58:39 INFO Completed upload id=54696worker1 # [ 36.798757] nix-grpc-daemon[782]: ts=2026-09-22T10:58:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 paths=1 nar_bytes_in=7524697worker1 # [ 36.801797] nix-grpc-daemon[782]: ts=2026-09-22T10:58:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424698lb # [ 36.826777] niks3-server[971]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4699worker1 # [ 36.808404] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4700worker1 # [ 36.812089] nix-daemon[864]: reaped child process 964, status = succeeded4701worker1 # [ 36.815853] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4702client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4703client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4704client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4705client # farm-top> LOG-farm-top-t14706worker1 # [ 38.249840] nix-daemon[864]: reaped child process 978, status = succeeded4707worker1 # [ 38.332053] nix-daemon[864]: accepted connection from pid 995, user nix-grpc-daemon (trusted)4708worker1 # [ 38.343277] nix-daemon[864]: reaped child process 998, status = succeeded4709lb # [ 38.374939] niks3-server[971]: 2026/09/22 10:58:40 INFO Received uploads request method=POST path=/api/pending_closures4710worker1 # [ 38.361166] nix-grpc-daemon[878]: time=2026-09-22T10:58:41.540Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4711worker1 # [ 38.363631] nix-grpc-daemon[878]: time=2026-09-22T10:58:41.541Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4712lb # [ 38.399218] niks3-server[971]: 2026/09/22 10:58:40 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4713lb # [ 38.405103] niks3-server[971]: 2026/09/22 10:58:40 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4714lb # [ 38.406751] niks3-server[971]: 2026/09/22 10:58:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4715lb # [ 38.408992] niks3-server[971]: 2026/09/22 10:58:40 INFO Signed narinfos id=6 count=14716worker1 # [ 38.384661] nix-grpc-daemon[878]: time=2026-09-22T10:58:41.563Z level=INFO msg="Uploading 1 narinfos"4717lb # [ 38.416325] niks3-server[971]: 2026/09/22 10:58:40 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4718lb # [ 38.418798] niks3-server[971]: 2026/09/22 10:58:40 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4719lb # [ 38.423231] niks3-server[971]: 2026/09/22 10:58:40 INFO Completed upload id=64720worker1 # [ 38.398605] nix-grpc-daemon[878]: time=2026-09-22T10:58:41.577Z level=INFO msg="Upload complete. (173ms)"4721lb # [ 38.425842] niks3-server[971]: 2026/09/22 10:58:40 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4722worker1 # [ 38.401441] nix-grpc-daemon[782]: ts=2026-09-22T10:58:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14723worker1 # [ 38.408055] nix-daemon[864]: reaped child process 976, status = succeeded4724client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 5.92 seconds)4725client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4726client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.12 seconds)4727client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4728client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4729client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4730client # copying 3 paths...4731client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4732client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4733client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4734client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4735(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.27 seconds)4736subtest: repeat is answered Cached by the scheduler without building4737worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4738worker1 # finding garbage collector roots...4739worker1 # deleting specified paths...4740worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4741worker1 # deleting unused links...4742worker1 # note: hard linking is currently saving 0.0 KiB4743worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds)4744worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4745worker2 # finding garbage collector roots...4746worker2 # deleting specified paths...4747worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4748worker2 # deleting unused links...4749worker2 # note: hard linking is currently saving 0.0 KiB4750worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4751client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24752worker2 # [ 39.243140] nix-grpc-daemon[784]: ts=2026-09-22T10:58:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:472084753client # don't know how to build these paths:4754client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4755worker2 # [ 39.248889] nix-grpc-daemon[784]: ts=2026-09-22T10:58:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472084756worker1 # [ 39.260644] nix-grpc-daemon[782]: ts=2026-09-22T10:58:42Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14757client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.19 seconds)4758client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4759client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.09 seconds)4760lb: must succeed: curl -sf localhost:9901/clusters4761lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4762lb: must succeed: curl -sf localhost:9901/clusters4763lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4764worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4765worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4766(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.73 seconds)4767subtest: inputs already in the cache are not sent to the scheduler4768lb: must succeed: curl -sf localhost:9901/clusters4769lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4770lb: must succeed: curl -sf localhost:9901/clusters4771lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4772worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4773worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4774client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24775worker2 # [ 39.743166] nix-grpc-daemon[784]: ts=2026-09-22T10:58:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:472084776client # don't know how to build these paths:4777client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4778worker1 # [ 39.737552] nix-grpc-daemon[782]: ts=2026-09-22T10:58:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386424779worker1 # [ 39.749253] nix-grpc-daemon[878]: time=2026-09-22T10:58:42.926Z level=INFO msg="All 2 paths already cached"4780worker1 # [ 39.759720] nix-grpc-daemon[782]: ts=2026-09-22T10:58:42Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04781worker1 # [ 39.765310] nix-grpc-daemon[782]: ts=2026-09-22T10:58:42Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14782worker1 # [ 39.804500] nix-grpc-daemon[782]: ts=2026-09-22T10:58:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424783worker1 # [ 39.817506] nix-grpc-daemon[782]: ts=2026-09-22T10:58:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386424784worker1 # [ 39.830256] nix-grpc-daemon[878]: time=2026-09-22T10:58:43.009Z level=INFO msg="All 2 paths already cached"4785client # copying 1 paths to 'lb:50051'...4786client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4787worker1 # [ 39.839086] nix-grpc-daemon[782]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386424788worker1 # [ 39.850308] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4789worker1 # [ 39.949438] nix-daemon[864]: accepted connection from pid 1027, user nix-grpc-daemon (trusted)4790worker1 # [ 39.962924] nix-daemon[864]: reaped child process 1030, status = succeeded4791worker1 # [ 40.047631] nix-daemon[864]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4792worker1 # [ 40.060140] nix-daemon[864]: reaped child process 1035, status = succeeded4793lb # [ 40.092368] niks3-server[971]: 2026/09/22 10:58:42 INFO Received uploads request method=POST path=/api/pending_closures4794worker1 # [ 40.071972] nix-grpc-daemon[878]: time=2026-09-22T10:58:43.251Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4795worker1 # [ 40.073689] nix-grpc-daemon[878]: time=2026-09-22T10:58:43.251Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4796lb # [ 40.105951] niks3-server[971]: 2026/09/22 10:58:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4797lb # [ 40.108726] niks3-server[971]: 2026/09/22 10:58:42 INFO Signed narinfos id=7 count=14798worker1 # [ 40.084356] nix-grpc-daemon[878]: time=2026-09-22T10:58:43.263Z level=INFO msg="Uploading 1 narinfos"4799lb # [ 40.112414] niks3-server[971]: 2026/09/22 10:58:42 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4800lb # [ 40.115664] niks3-server[971]: 2026/09/22 10:58:42 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4801lb # [ 40.119275] niks3-server[971]: 2026/09/22 10:58:42 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4802lb # [ 40.124064] niks3-server[971]: 2026/09/22 10:58:42 INFO Completed upload id=74803worker1 # [ 40.100528] nix-grpc-daemon[878]: time=2026-09-22T10:58:43.279Z level=INFO msg="Upload complete. (238ms)"4804worker1 # [ 40.102452] nix-grpc-daemon[782]: ts=2026-09-22T10:58:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 paths=1 nar_bytes_in=7524805lb # [ 40.128261] niks3-server[971]: 2026/09/22 10:58:42 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4806worker1 # [ 40.106100] nix-grpc-daemon[782]: ts=2026-09-22T10:58:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424807worker1 # [ 40.112472] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4808worker1 # [ 40.114771] nix-daemon[864]: reaped child process 1025, status = succeeded4809worker1 # [ 40.120231] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4810client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4811client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4812client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4813client # farm-top> LOG-farm-top-t1b4814worker1 # [ 41.382089] nix-daemon[864]: reaped child process 1039, status = succeeded4815worker1 # [ 41.433555] nix-daemon[864]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)4816worker1 # [ 41.446913] nix-daemon[864]: reaped child process 1059, status = succeeded4817lb # [ 41.480422] niks3-server[971]: 2026/09/22 10:58:43 INFO Received uploads request method=POST path=/api/pending_closures4818worker1 # [ 41.464423] nix-grpc-daemon[878]: time=2026-09-22T10:58:44.643Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4819worker1 # [ 41.465918] nix-grpc-daemon[878]: time=2026-09-22T10:58:44.645Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4820lb # [ 41.521230] niks3-server[971]: 2026/09/22 10:58:43 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4821lb # [ 41.531095] niks3-server[971]: 2026/09/22 10:58:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4822lb # [ 41.536737] niks3-server[971]: 2026/09/22 10:58:43 INFO Signed narinfos id=8 count=14823worker1 # [ 41.513565] nix-grpc-daemon[878]: time=2026-09-22T10:58:44.692Z level=INFO msg="Uploading 1 narinfos"4824lb # [ 41.540872] niks3-server[971]: 2026/09/22 10:58:43 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4825lb # [ 41.545370] niks3-server[971]: 2026/09/22 10:58:43 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4826lb # [ 41.556657] niks3-server[971]: 2026/09/22 10:58:43 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4827lb # [ 41.560282] niks3-server[971]: 2026/09/22 10:58:43 INFO Completed upload id=84828lb # [ 41.562730] niks3-server[971]: 2026/09/22 10:58:43 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4829worker1 # [ 41.536856] nix-grpc-daemon[878]: time=2026-09-22T10:58:44.716Z level=INFO msg="Upload complete. (208ms)"4830worker1 # [ 41.543724] nix-grpc-daemon[782]: ts=2026-09-22T10:58:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14831worker1 # [ 41.557094] nix-daemon[864]: reaped child process 1037, status = succeeded4832client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.99 seconds)4833client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4834client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds)4835lb: must succeed: curl -sf localhost:9901/clusters4836lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4837lb: must succeed: curl -sf localhost:9901/clusters4838lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4839worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4840worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4841(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.25 seconds)4842subtest: two clients wanting the same drv build it once4843client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4844client # [ 41.792840] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4845client # Running as unit: dedup1.service; invocation ID: ab81045e1d4c4d37bbcdff301257df404846client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.06 seconds)4847client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4848worker2 # [ 41.961231] nix-grpc-daemon[784]: ts=2026-09-22T10:58:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:472084849client # [ 41.928456] nix[886]: don't know how to build these paths:4850client # [ 41.929392] nix[886]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4851worker2 # [ 41.972195] nix-grpc-daemon[784]: ts=2026-09-22T10:58:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472084852worker1 # [ 41.956921] nix-grpc-daemon[782]: ts=2026-09-22T10:58:45Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04853worker1 # [ 41.962424] nix-grpc-daemon[782]: ts=2026-09-22T10:58:45Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14854worker2 # [ 41.980346] nix-grpc-daemon[784]: ts=2026-09-22T10:58:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472084855client # [ 41.953075] nix[886]: copying 1 paths to 'lb:50051'...4856client # [ 41.953876] nix[886]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4857worker2 # [ 41.987432] nix-grpc-daemon[784]: ts=2026-09-22T10:58:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472084858worker2 # [ 41.993061] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4859worker2 # [ 42.004992] nix-grpc-daemon[784]: ts=2026-09-22T10:58:44Z level=info event=niks3_push_started pid=8804860worker2 # [ 42.123742] nix-grpc-daemon[880]: time=2026-09-22T10:58:45.070Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4861worker2 # [ 42.224386] nix-daemon[851]: accepted connection from pid 893, user nix-grpc-daemon (trusted)4862worker2 # [ 42.239958] nix-daemon[851]: reaped child process 896, status = succeeded4863worker2 # [ 42.323076] nix-daemon[851]: accepted connection from pid 898, user nix-grpc-daemon (trusted)4864worker2 # [ 42.337728] nix-daemon[851]: reaped child process 901, status = succeeded4865lb # [ 42.354051] niks3-server[971]: 2026/09/22 10:58:44 INFO Received uploads request method=POST path=/api/pending_closures4866worker2 # [ 42.346434] nix-grpc-daemon[880]: time=2026-09-22T10:58:45.294Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4867worker2 # [ 42.349461] nix-grpc-daemon[880]: time=2026-09-22T10:58:45.295Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4868lb # [ 42.374730] niks3-server[971]: 2026/09/22 10:58:44 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4869lb # [ 42.376391] niks3-server[971]: 2026/09/22 10:58:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4870lb # [ 42.378847] niks3-server[971]: 2026/09/22 10:58:44 INFO Signed narinfos id=9 count=14871worker2 # [ 42.367568] nix-grpc-daemon[880]: time=2026-09-22T10:58:45.315Z level=INFO msg="Uploading 1 narinfos"4872lb # [ 42.382683] niks3-server[971]: 2026/09/22 10:58:44 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4873lb # [ 42.386832] niks3-server[971]: 2026/09/22 10:58:44 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4874lb # [ 42.389562] niks3-server[971]: 2026/09/22 10:58:44 INFO Completed upload id=94875worker2 # [ 42.379060] nix-grpc-daemon[880]: time=2026-09-22T10:58:45.326Z level=INFO msg="Upload complete. (251ms)"4876lb # [ 42.392210] niks3-server[971]: 2026/09/22 10:58:44 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4877worker2 # [ 42.381298] nix-grpc-daemon[784]: ts=2026-09-22T10:58:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_in=4324878worker2 # [ 42.385251] nix-grpc-daemon[784]: ts=2026-09-22T10:58:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472084879worker2 # [ 42.391945] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4880worker2 # [ 42.394440] nix-daemon[851]: reaped child process 878, status = succeeded4881worker2 # [ 42.399592] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4882client # [ 42.365399] nix[886]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4883client # [ 42.855552] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4884client # Running as unit: dedup2.service; invocation ID: 14003aadbccb4612a696b6e480fcff054885client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4886worker1 # [ 43.034290] nix-grpc-daemon[782]: ts=2026-09-22T10:58:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:386424887client # [ 43.019249] nix[902]: don't know how to build these paths:4888client # [ 43.020171] nix[902]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4889worker1 # [ 43.046731] nix-grpc-daemon[782]: ts=2026-09-22T10:58:46Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04890client # [ 43.033357] nix[902]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4891worker2 # [ 43.065976] nix-grpc-daemon[784]: ts=2026-09-22T10:58:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472084892client: must succeed: sleep 24893client: (finished: must succeed: sleep 2, in 2.04 seconds)4894lb # [ 46.226150] envoy[908]: conn peer=[2001:db8:1::1]:50508 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9507 tx=4052 ms=57854895lb # [ 46.229939] envoy[908]: conn peer=[2001:db8:1::1]:45104 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=444896lb # [ 46.233446] envoy[908]: conn peer=[2001:db8:1::1]:45114 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4122 tx=1748 ms=18214897client: waiting for success: ! systemctl is-active dedup1 dedup24898worker2 # [ 46.441663] nix-daemon[851]: reaped child process 905, status = succeeded4899worker2 # [ 46.525099] nix-daemon[851]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4900worker2 # [ 46.537909] nix-daemon[851]: reaped child process 938, status = succeeded4901lb # [ 46.556508] niks3-server[971]: 2026/09/22 10:58:48 INFO Received uploads request method=POST path=/api/pending_closures4902worker2 # [ 46.555109] nix-grpc-daemon[880]: time=2026-09-22T10:58:49.502Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4903worker2 # [ 46.558869] nix-grpc-daemon[880]: time=2026-09-22T10:58:49.504Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4904lb # [ 46.597490] niks3-server[971]: 2026/09/22 10:58:48 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4905lb # [ 46.608632] niks3-server[971]: 2026/09/22 10:58:48 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4906lb # [ 46.617585] niks3-server[971]: 2026/09/22 10:58:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4907lb # [ 46.622550] niks3-server[971]: 2026/09/22 10:58:48 INFO Signed narinfos id=10 count=14908worker2 # [ 46.613967] nix-grpc-daemon[880]: time=2026-09-22T10:58:49.560Z level=INFO msg="Uploading 1 narinfos"4909lb # [ 46.627245] niks3-server[971]: 2026/09/22 10:58:48 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4910lb # [ 46.638055] niks3-server[971]: 2026/09/22 10:58:48 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4911lb # [ 46.641088] niks3-server[971]: 2026/09/22 10:58:48 INFO Completed upload id=104912worker2 # [ 46.629418] nix-grpc-daemon[880]: time=2026-09-22T10:58:49.577Z level=INFO msg="Upload complete. (217ms)"4913lb # [ 46.643409] niks3-server[971]: 2026/09/22 10:58:48 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4914worker2 # [ 46.635972] nix-grpc-daemon[784]: ts=2026-09-22T10:58:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14915worker2 # [ 46.643456] nix-grpc-daemon[784]: ts=2026-09-22T10:58:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14916worker2 # [ 46.654559] nix-daemon[851]: reaped child process 903, status = succeeded4917client # [ 46.631756] systemd[1]: dedup1.service: Deactivated successfully.4918client # [ 46.635977] systemd[1]: dedup1.service: Consumed 118ms CPU time over 4.842s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.5K outgoing IP traffic.4919client # [ 46.671973] systemd[1]: dedup2.service: Deactivated successfully.4920client # [ 46.674857] systemd[1]: dedup2.service: Consumed 142ms CPU time over 3.819s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4921client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4922client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4923client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4924worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4925worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4926worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4927worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.02 seconds)4928worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4929worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4930worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4931worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4932(finished: subtest: two clients wanting the same drv build it once, in 5.72 seconds)4933subtest: --timeout and --max-silent-time reach the builder4934worker2 # [ 47.632953] nix-grpc-daemon[784]: ts=2026-09-22T10:58:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:472084935worker1 # [ 47.632216] nix-grpc-daemon[782]: ts=2026-09-22T10:58:50Z level=debug event=want client=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=04936worker1 # [ 47.636218] nix-grpc-daemon[782]: ts=2026-09-22T10:58:50Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=14937worker1 # [ 47.675115] nix-grpc-daemon[782]: ts=2026-09-22T10:58:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424938worker1 # [ 47.687083] nix-grpc-daemon[782]: ts=2026-09-22T10:58:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386424939worker1 # [ 47.697684] nix-grpc-daemon[782]: ts=2026-09-22T10:58:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386424940worker1 # [ 47.708398] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4941worker1 # [ 47.805965] nix-daemon[864]: accepted connection from pid 1095, user nix-grpc-daemon (trusted)4942worker1 # [ 47.820096] nix-daemon[864]: reaped child process 1098, status = succeeded4943worker1 # [ 47.898466] nix-daemon[864]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)4944worker1 # [ 47.911145] nix-daemon[864]: reaped child process 1103, status = succeeded4945lb # [ 47.943448] niks3-server[971]: 2026/09/22 10:58:50 INFO Received uploads request method=POST path=/api/pending_closures4946worker1 # [ 47.924990] nix-grpc-daemon[878]: time=2026-09-22T10:58:51.104Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4947worker1 # [ 47.926669] nix-grpc-daemon[878]: time=2026-09-22T10:58:51.105Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"4948lb # [ 47.973841] niks3-server[971]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst4949lb # [ 47.981426] niks3-server[971]: 2026/09/22 10:58:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4950lb # [ 47.988390] niks3-server[971]: 2026/09/22 10:58:50 INFO Signed narinfos id=11 count=14951worker1 # [ 47.967107] nix-grpc-daemon[878]: time=2026-09-22T10:58:51.146Z level=INFO msg="Uploading 1 narinfos"4952lb # [ 47.997209] niks3-server[971]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls4953lb # [ 48.008130] niks3-server[971]: 2026/09/22 10:58:50 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4954lb # [ 48.013981] niks3-server[971]: 2026/09/22 10:58:50 INFO Completed upload id=114955worker1 # [ 47.989348] nix-grpc-daemon[878]: time=2026-09-22T10:58:51.168Z level=INFO msg="Upload complete. (269ms)"4956worker1 # [ 47.992531] nix-grpc-daemon[782]: ts=2026-09-22T10:58:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 paths=1 nar_bytes_in=4244957lb # [ 48.023142] niks3-server[971]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo4958worker1 # [ 47.997326] nix-grpc-daemon[782]: ts=2026-09-22T10:58:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386424959worker1 # [ 48.007841] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4960worker1 # [ 48.012175] nix-daemon[864]: reaped child process 1093, status = succeeded4961worker1 # [ 48.016676] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4962worker1 # [ 54.228926] nix-grpc-daemon[782]: ts=2026-09-22T10:58:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=6 outputs=04963worker1 # [ 54.253728] nix-daemon[864]: reaped child process 1105, status = succeeded4964worker1 # [ 54.255801] nix-daemon[864]: reaped child process 1107, status = succeeded4965worker1 # [ 54.398181] nix-grpc-daemon[782]: ts=2026-09-22T10:58:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:386424966worker1 # [ 54.406531] nix-grpc-daemon[782]: ts=2026-09-22T10:58:57Z level=debug event=want client=7 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=04967worker1 # [ 54.408705] nix-grpc-daemon[782]: ts=2026-09-22T10:58:57Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=14968worker2 # [ 54.428415] nix-grpc-daemon[784]: ts=2026-09-22T10:58:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472084969worker2 # [ 54.439870] nix-grpc-daemon[784]: ts=2026-09-22T10:58:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472084970worker2 # [ 54.451604] nix-grpc-daemon[784]: ts=2026-09-22T10:58:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472084971worker2 # [ 54.462601] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4972worker2 # [ 54.570870] nix-daemon[851]: accepted connection from pid 951, user nix-grpc-daemon (trusted)4973worker2 # [ 54.587045] nix-daemon[851]: reaped child process 954, status = succeeded4974worker2 # [ 54.667841] nix-daemon[851]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4975worker2 # [ 54.681761] nix-daemon[851]: reaped child process 959, status = succeeded4976lb # [ 54.700691] niks3-server[971]: 2026/09/22 10:58:57 INFO Received uploads request method=POST path=/api/pending_closures4977worker2 # [ 54.697072] nix-grpc-daemon[880]: time=2026-09-22T10:58:57.644Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4978worker2 # [ 54.699106] nix-grpc-daemon[880]: time=2026-09-22T10:58:57.646Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"4979lb # [ 54.733455] niks3-server[971]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst4980lb # [ 54.751792] niks3-server[971]: 2026/09/22 10:58:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4981worker2 # [ 54.745363] nix-grpc-daemon[880]: time=2026-09-22T10:58:57.692Z level=INFO msg="Uploading 1 narinfos"4982lb # [ 54.756775] niks3-server[971]: 2026/09/22 10:58:57 INFO Signed narinfos id=12 count=14983lb # [ 54.766345] niks3-server[971]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls4984lb # [ 54.774659] niks3-server[971]: 2026/09/22 10:58:57 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4985lb # [ 54.780578] niks3-server[971]: 2026/09/22 10:58:57 INFO Completed upload id=124986worker2 # [ 54.768977] nix-grpc-daemon[880]: time=2026-09-22T10:58:57.716Z level=INFO msg="Upload complete. (294ms)"4987lb # [ 54.783984] niks3-server[971]: 2026/09/22 10:58:57 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo4988worker2 # [ 54.773386] nix-grpc-daemon[784]: ts=2026-09-22T10:58:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_in=4244989worker2 # [ 54.779759] nix-grpc-daemon[784]: ts=2026-09-22T10:58:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472084990worker2 # [ 54.788557] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4991worker2 # [ 54.798804] nix-daemon[851]: reaped child process 949, status = succeeded4992worker2 # [ 54.803441] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4993lb # [ 56.226404] envoy[908]: conn peer=[2001:db8:1::1]:45128 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3011 tx=1426 ms=46904994lb # [ 56.230256] envoy[908]: conn peer=[2001:db8:1::1]:57414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1085 ms=36444995lb # [ 56.233917] envoy[908]: conn peer=[2001:db8:1::1]:57430 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2983 tx=975 ms=66264996worker2 # [ 61.016741] nix-grpc-daemon[784]: ts=2026-09-22T10:59:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=7 outputs=04997worker2 # [ 61.029249] nix-daemon[851]: reaped child process 963, status = succeeded4998worker2 # [ 61.030215] nix-daemon[851]: reaped child process 961, status = succeeded4999(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.63 seconds)5000subtest: the first client leaving does not fail the second5001client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5002client # [ 61.119654] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5003client # Running as unit: share1.service; invocation ID: 6c3c9682782241e09520a2ee1cc8436d5004client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.03 seconds)5005client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5006client # [ 61.238358] nix[974]: don't know how to build these paths:5007client # [ 61.239171] nix[974]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5008worker2 # [ 61.271384] nix-grpc-daemon[784]: ts=2026-09-22T10:59:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:472085009worker1 # [ 61.268853] nix-grpc-daemon[782]: ts=2026-09-22T10:59:04Z level=debug event=want client=8 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05010worker1 # [ 61.275425] nix-grpc-daemon[782]: ts=2026-09-22T10:59:04Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=15011worker1 # [ 61.279842] nix-grpc-daemon[782]: ts=2026-09-22T10:59:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386425012worker1 # [ 61.288519] nix-grpc-daemon[782]: ts=2026-09-22T10:59:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386425013client # [ 61.275777] nix[974]: copying 1 paths to 'lb:50051'...5014client # [ 61.276768] nix[974]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5015worker1 # [ 61.296909] nix-grpc-daemon[782]: ts=2026-09-22T10:59:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386425016worker1 # [ 61.307984] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5017worker1 # [ 61.404844] nix-daemon[864]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)5018worker1 # [ 61.417840] nix-daemon[864]: reaped child process 1137, status = succeeded5019worker1 # [ 61.499581] nix-daemon[864]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)5020worker1 # [ 61.513798] nix-daemon[864]: reaped child process 1142, status = succeeded5021lb # [ 61.543136] niks3-server[971]: 2026/09/22 10:59:03 INFO Received uploads request method=POST path=/api/pending_closures5022worker1 # [ 61.527385] nix-grpc-daemon[878]: time=2026-09-22T10:59:04.706Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5023worker1 # [ 61.528756] nix-grpc-daemon[878]: time=2026-09-22T10:59:04.706Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5024lb # [ 61.577532] niks3-server[971]: 2026/09/22 10:59:03 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5025lb # [ 61.586903] niks3-server[971]: 2026/09/22 10:59:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5026lb # [ 61.592789] niks3-server[971]: 2026/09/22 10:59:03 INFO Signed narinfos id=13 count=15027worker1 # [ 61.573860] nix-grpc-daemon[878]: time=2026-09-22T10:59:04.753Z level=INFO msg="Uploading 1 narinfos"5028lb # [ 61.608097] niks3-server[971]: 2026/09/22 10:59:03 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5029lb # [ 61.612096] niks3-server[971]: 2026/09/22 10:59:03 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5030lb # [ 61.621346] niks3-server[971]: 2026/09/22 10:59:03 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5031lb # [ 61.624856] niks3-server[971]: 2026/09/22 10:59:03 INFO Completed upload id=135032worker1 # [ 61.601521] nix-grpc-daemon[878]: time=2026-09-22T10:59:04.780Z level=INFO msg="Upload complete. (281ms)"5033worker1 # [ 61.605072] nix-grpc-daemon[782]: ts=2026-09-22T10:59:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 paths=1 nar_bytes_in=4325034worker1 # [ 61.610343] nix-grpc-daemon[782]: ts=2026-09-22T10:59:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386425035worker1 # [ 61.621119] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5036worker1 # [ 61.631166] nix-daemon[864]: reaped child process 1132, status = succeeded5037worker1 # [ 61.634522] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5038client # [ 61.613607] nix[974]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5039client # [ 62.160651] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5040client # Running as unit: share2.service; invocation ID: 97de826bf4be4007b1635c01c93828f65041client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.04 seconds)5042worker1 # [ 62.299602] nix-grpc-daemon[782]: ts=2026-09-22T10:59:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:386425043client # [ 62.292700] nix[990]: this derivation will be built:5044worker1 # [ 62.314705] nix-grpc-daemon[782]: ts=2026-09-22T10:59:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:386425045client # [ 62.299377] nix[990]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5046worker1 # [ 62.327866] nix-grpc-daemon[782]: ts=2026-09-22T10:59:05Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05047client # [ 62.311724] nix[990]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5048worker1 # [ 62.332388] nix-grpc-daemon[782]: ts=2026-09-22T10:59:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386425049client: must succeed: sleep 2; systemctl kill -s INT share15050client # [ 65.399484] systemd[1]: share1.service: Sent signal SIGINT to main process 974 (nix) on client request.5051client # [ 65.406665] nix[974]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5052client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5053client: waiting for success: ! systemctl is-active share15054client # [ 65.413574] nix[974]: error: interrupted by the user5055client # [ 65.454312] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5056client # [ 65.455648] systemd[1]: share1.service: Failed with result 'exit-code'.5057client # [ 65.457425] systemd[1]: share1.service: Consumed 110ms CPU time over 4.334s wall clock time, 43.5M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic.5058lb # [ 66.226927] envoy[908]: conn peer=[2001:db8:1::1]:54968 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=977 ms=66155059lb # [ 66.230682] envoy[908]: conn peer=[2001:db8:1::1]:54984 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2982 tx=788 ms=41845060client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5061client: waiting for success: ! systemctl is-active share25062worker1 # [ 66.875490] nix-daemon[864]: reaped child process 1146, status = succeeded5063worker1 # [ 66.961269] nix-daemon[864]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5064worker1 # [ 66.973183] nix-daemon[864]: reaped child process 1185, status = succeeded5065lb # [ 67.004942] niks3-server[971]: 2026/09/22 10:59:09 INFO Received uploads request method=POST path=/api/pending_closures5066worker1 # [ 66.991191] nix-grpc-daemon[878]: time=2026-09-22T10:59:10.170Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5067worker1 # [ 66.992939] nix-grpc-daemon[878]: time=2026-09-22T10:59:10.172Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5068lb # [ 67.049860] niks3-server[971]: 2026/09/22 10:59:09 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5069lb # [ 67.058455] niks3-server[971]: 2026/09/22 10:59:09 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5070lb # [ 67.067109] niks3-server[971]: 2026/09/22 10:59:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5071lb # [ 67.069733] niks3-server[971]: 2026/09/22 10:59:09 INFO Signed narinfos id=14 count=15072worker1 # [ 67.045405] nix-grpc-daemon[878]: time=2026-09-22T10:59:10.224Z level=INFO msg="Uploading 1 narinfos"5073lb # [ 67.072968] niks3-server[971]: 2026/09/22 10:59:09 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5074lb # [ 67.078117] niks3-server[971]: 2026/09/22 10:59:09 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5075lb # [ 67.082317] niks3-server[971]: 2026/09/22 10:59:09 INFO Completed upload id=145076worker1 # [ 67.057704] nix-grpc-daemon[878]: time=2026-09-22T10:59:10.236Z level=INFO msg="Upload complete. (211ms)"5077lb # [ 67.083840] niks3-server[971]: 2026/09/22 10:59:09 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5078worker1 # [ 67.064725] nix-grpc-daemon[782]: ts=2026-09-22T10:59:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=8 outputs=15079worker1 # [ 67.071312] nix-grpc-daemon[782]: ts=2026-09-22T10:59:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=8 outputs=15080worker1 # [ 67.082312] nix-daemon[864]: reaped child process 1144, status = succeeded5081client # [ 67.113699] systemd[1]: share2.service: Deactivated successfully.5082client # [ 67.117397] systemd[1]: share2.service: Consumed 112ms CPU time over 4.956s wall clock time, 35.5M memory peak, 6.2K incoming IP traffic, 6K outgoing IP traffic.5083client: (finished: waiting for success: ! systemctl is-active share2, in 1.06 seconds)5084client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5085client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.02 seconds)5086(finished: subtest: the first client leaving does not fail the second, in 6.64 seconds)5087subtest: build hook: nix-daemon with builders = grpc://lb5088client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5089client # [ 67.825720] systemd[1]: Started Nix Daemon.5090client # [ 67.905223] nix-daemon[1042]: accepted connection from pid 1039, user root (trusted)5091worker2 # [ 68.048688] nix-grpc-daemon[784]: ts=2026-09-22T10:59:10Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:472085092worker2 # [ 68.051212] nix-grpc-daemon[784]: ts=2026-09-22T10:59:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472085093worker2 # [ 68.062500] nix-grpc-daemon[784]: ts=2026-09-22T10:59:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472085094worker1 # [ 68.047450] nix-grpc-daemon[782]: ts=2026-09-22T10:59:11Z level=debug event=want client=10 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05095worker1 # [ 68.051845] nix-grpc-daemon[782]: ts=2026-09-22T10:59:11Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15096worker2 # [ 68.070501] nix-grpc-daemon[784]: ts=2026-09-22T10:59:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472085097worker2 # [ 68.077541] nix-grpc-daemon[784]: ts=2026-09-22T10:59:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472085098worker2 # [ 68.083047] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5099worker2 # [ 68.176069] nix-daemon[851]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)5100worker2 # [ 68.189877] nix-daemon[851]: reaped child process 1005, status = succeeded5101worker2 # [ 68.277404] nix-daemon[851]: accepted connection from pid 1007, user nix-grpc-daemon (trusted)5102worker2 # [ 68.292226] nix-daemon[851]: reaped child process 1010, status = succeeded5103lb # [ 68.309438] niks3-server[971]: 2026/09/22 10:59:10 INFO Received uploads request method=POST path=/api/pending_closures5104worker2 # [ 68.305955] nix-grpc-daemon[880]: time=2026-09-22T10:59:11.253Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5105worker2 # [ 68.307396] nix-grpc-daemon[880]: time=2026-09-22T10:59:11.253Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5106lb # [ 68.336872] niks3-server[971]: 2026/09/22 10:59:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5107lb # [ 68.344458] niks3-server[971]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5108lb # [ 68.350367] niks3-server[971]: 2026/09/22 10:59:10 INFO Signed narinfos id=15 count=15109worker2 # [ 68.342173] nix-grpc-daemon[880]: time=2026-09-22T10:59:11.289Z level=INFO msg="Uploading 1 narinfos"5110lb # [ 68.356876] niks3-server[971]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5111lb # [ 68.369474] niks3-server[971]: 2026/09/22 10:59:10 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5112lb # [ 68.374314] niks3-server[971]: 2026/09/22 10:59:10 INFO Completed upload id=155113worker2 # [ 68.365671] nix-grpc-daemon[880]: time=2026-09-22T10:59:11.313Z level=INFO msg="Upload complete. (277ms)"5114worker2 # [ 68.368052] nix-grpc-daemon[784]: ts=2026-09-22T10:59:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_in=5125115lb # [ 68.379571] niks3-server[971]: 2026/09/22 10:59:10 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5116worker2 # [ 68.374472] nix-grpc-daemon[784]: ts=2026-09-22T10:59:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472085117worker2 # [ 68.379059] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5118worker2 # [ 68.381378] nix-daemon[851]: reaped child process 1000, status = succeeded5119worker2 # [ 68.386864] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5120worker2 # [ 69.603270] nix-daemon[851]: reaped child process 1014, status = succeeded5121worker2 # [ 69.684411] nix-daemon[851]: accepted connection from pid 1030, user nix-grpc-daemon (trusted)5122worker2 # [ 69.697800] nix-daemon[851]: reaped child process 1033, status = succeeded5123lb # [ 69.718173] niks3-server[971]: 2026/09/22 10:59:12 INFO Received uploads request method=POST path=/api/pending_closures5124worker2 # [ 69.714384] nix-grpc-daemon[880]: time=2026-09-22T10:59:12.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5125worker2 # [ 69.716251] nix-grpc-daemon[880]: time=2026-09-22T10:59:12.663Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5126lb # [ 69.757447] niks3-server[971]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5127lb # [ 69.771194] niks3-server[971]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5128lb # [ 69.778258] niks3-server[971]: 2026/09/22 10:59:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5129lb # [ 69.785231] niks3-server[971]: 2026/09/22 10:59:12 INFO Signed narinfos id=16 count=15130worker2 # [ 69.774148] nix-grpc-daemon[880]: time=2026-09-22T10:59:12.721Z level=INFO msg="Uploading 1 narinfos"5131lb # [ 69.793456] niks3-server[971]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5132lb # [ 69.801634] niks3-server[971]: 2026/09/22 10:59:12 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5133lb # [ 69.806787] niks3-server[971]: 2026/09/22 10:59:12 INFO Completed upload id=165134worker2 # [ 69.795357] nix-grpc-daemon[880]: time=2026-09-22T10:59:12.742Z level=INFO msg="Upload complete. (201ms)"5135lb # [ 69.808936] niks3-server[971]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5136worker2 # [ 69.802401] nix-grpc-daemon[784]: ts=2026-09-22T10:59:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=9 outputs=15137worker1 # [ 69.792917] nix-grpc-daemon[782]: ts=2026-09-22T10:59:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:386425138worker2 # [ 69.815973] nix-daemon[851]: reaped child process 1012, status = succeeded5139worker1 # [ 69.802688] nix-grpc-daemon[782]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5140worker2 # [ 69.839198] nix-grpc-daemon[784]: ts=2026-09-22T10:59:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:472085141worker2 # [ 69.844376] nix-grpc-daemon[784]: ts=2026-09-22T10:59:12Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_out=1285142worker1 # [ 70.031824] nix-grpc-daemon[782]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:386425143worker2 # [ 70.052185] nix-grpc-daemon[784]: ts=2026-09-22T10:59:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472085144worker2 # [ 70.061782] nix-grpc-daemon[880]: time=2026-09-22T10:59:13.009Z level=INFO msg="All 1 paths already cached"5145worker1 # [ 70.062266] nix-grpc-daemon[782]: ts=2026-09-22T10:59:13Z level=debug event=want client=11 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05146worker1 # [ 70.066706] nix-grpc-daemon[782]: ts=2026-09-22T10:59:13Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15147worker2 # [ 70.119099] nix-grpc-daemon[784]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472085148worker2 # [ 70.130892] nix-grpc-daemon[784]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472085149worker2 # [ 70.143501] nix-grpc-daemon[880]: time=2026-09-22T10:59:13.091Z level=INFO msg="All 1 paths already cached"5150worker2 # [ 70.150643] nix-grpc-daemon[784]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472085151worker2 # [ 70.159343] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5152worker2 # [ 70.263604] nix-daemon[851]: accepted connection from pid 1037, user nix-grpc-daemon (trusted)5153worker2 # [ 70.280457] nix-daemon[851]: reaped child process 1040, status = succeeded5154worker2 # [ 70.367341] nix-daemon[851]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)5155worker2 # [ 70.381396] nix-daemon[851]: reaped child process 1045, status = succeeded5156lb # [ 70.400588] niks3-server[971]: 2026/09/22 10:59:12 INFO Received uploads request method=POST path=/api/pending_closures5157worker2 # [ 70.396492] nix-grpc-daemon[880]: time=2026-09-22T10:59:13.344Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5158worker2 # [ 70.397988] nix-grpc-daemon[880]: time=2026-09-22T10:59:13.345Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5159lb # [ 70.426522] niks3-server[971]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5160lb # [ 70.441582] niks3-server[971]: 2026/09/22 10:59:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5161worker2 # [ 70.434442] nix-grpc-daemon[880]: time=2026-09-22T10:59:13.382Z level=INFO msg="Uploading 1 narinfos"5162lb # [ 70.445827] niks3-server[971]: 2026/09/22 10:59:12 INFO Signed narinfos id=17 count=15163lb # [ 70.455274] niks3-server[971]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5164lb # [ 70.462068] niks3-server[971]: 2026/09/22 10:59:12 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5165worker2 # [ 70.458335] nix-grpc-daemon[880]: time=2026-09-22T10:59:13.406Z level=INFO msg="Upload complete. (286ms)"5166lb # [ 70.469961] niks3-server[971]: 2026/09/22 10:59:12 INFO Completed upload id=175167lb # [ 70.473831] niks3-server[971]: 2026/09/22 10:59:12 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5168worker2 # [ 70.460512] nix-grpc-daemon[784]: ts=2026-09-22T10:59:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_in=6245169worker2 # [ 70.465446] nix-grpc-daemon[784]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472085170worker2 # [ 70.474289] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5171worker2 # [ 70.479757] nix-daemon[851]: reaped child process 1035, status = succeeded5172worker2 # [ 70.485334] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5173worker2 # [ 71.326803] nix-daemon[851]: reaped child process 1049, status = succeeded5174worker2 # [ 71.404089] nix-daemon[851]: accepted connection from pid 1065, user nix-grpc-daemon (trusted)5175worker2 # [ 71.416765] nix-daemon[851]: reaped child process 1068, status = succeeded5176lb # [ 71.436668] niks3-server[971]: 2026/09/22 10:59:13 INFO Received uploads request method=POST path=/api/pending_closures5177worker2 # [ 71.432831] nix-grpc-daemon[880]: time=2026-09-22T10:59:14.380Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5178worker2 # [ 71.434289] nix-grpc-daemon[880]: time=2026-09-22T10:59:14.382Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5179lb # [ 71.472779] niks3-server[971]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5180lb # [ 71.485819] niks3-server[971]: 2026/09/22 10:59:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5181lb # [ 71.490355] niks3-server[971]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5182worker2 # [ 71.485848] nix-grpc-daemon[880]: time=2026-09-22T10:59:14.433Z level=INFO msg="Uploading 1 narinfos"5183lb # [ 71.498222] niks3-server[971]: 2026/09/22 10:59:13 INFO Signed narinfos id=18 count=15184lb # [ 71.506912] niks3-server[971]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5185lb # [ 71.514573] niks3-server[971]: 2026/09/22 10:59:13 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5186lb # [ 71.521230] niks3-server[971]: 2026/09/22 10:59:13 INFO Completed upload id=185187worker2 # [ 71.509419] nix-grpc-daemon[880]: time=2026-09-22T10:59:14.457Z level=INFO msg="Upload complete. (218ms)"5188lb # [ 71.525207] niks3-server[971]: 2026/09/22 10:59:13 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5189worker2 # [ 71.516156] nix-grpc-daemon[784]: ts=2026-09-22T10:59:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=10 outputs=15190worker1 # [ 71.508265] nix-grpc-daemon[782]: ts=2026-09-22T10:59:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:386425191worker2 # [ 71.530098] nix-daemon[851]: reaped child process 1047, status = succeeded5192worker1 # [ 71.517961] nix-grpc-daemon[782]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5193worker1 # [ 71.539407] nix-grpc-daemon[782]: ts=2026-09-22T10:59:14Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:386425194worker1 # [ 71.544547] nix-grpc-daemon[782]: ts=2026-09-22T10:59:14Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 paths=1 nar_bytes_out=1765195worker1 # [ 71.745177] nix-grpc-daemon[782]: ts=2026-09-22T10:59:14Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:386425196worker2 # [ 71.765743] nix-grpc-daemon[784]: ts=2026-09-22T10:59:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472085197worker2 # [ 71.775422] nix-grpc-daemon[880]: time=2026-09-22T10:59:14.723Z level=INFO msg="All 2 paths already cached"5198worker1 # [ 71.776227] nix-grpc-daemon[782]: ts=2026-09-22T10:59:14Z level=debug event=want client=12 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05199worker1 # [ 71.780606] nix-grpc-daemon[782]: ts=2026-09-22T10:59:14Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15200worker2 # [ 71.833928] nix-grpc-daemon[784]: ts=2026-09-22T10:59:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472085201worker2 # [ 71.846449] nix-grpc-daemon[784]: ts=2026-09-22T10:59:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472085202worker2 # [ 71.859445] nix-grpc-daemon[880]: time=2026-09-22T10:59:14.807Z level=INFO msg="All 2 paths already cached"5203worker2 # [ 71.866852] nix-grpc-daemon[784]: ts=2026-09-22T10:59:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472085204worker2 # [ 71.875314] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5205worker2 # [ 71.984073] nix-daemon[851]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)5206worker2 # [ 71.999113] nix-daemon[851]: reaped child process 1075, status = succeeded5207worker2 # [ 72.082659] nix-daemon[851]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)5208worker2 # [ 72.096723] nix-daemon[851]: reaped child process 1080, status = succeeded5209lb # [ 72.115848] niks3-server[971]: 2026/09/22 10:59:14 INFO Received uploads request method=POST path=/api/pending_closures5210worker2 # [ 72.108487] nix-grpc-daemon[880]: time=2026-09-22T10:59:15.056Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5211worker2 # [ 72.110415] nix-grpc-daemon[880]: time=2026-09-22T10:59:15.057Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5212lb # [ 72.131052] niks3-server[971]: 2026/09/22 10:59:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5213lb # [ 72.133498] niks3-server[971]: 2026/09/22 10:59:14 INFO Signed narinfos id=19 count=15214worker2 # [ 72.123515] nix-grpc-daemon[880]: time=2026-09-22T10:59:15.071Z level=INFO msg="Uploading 1 narinfos"5215lb # [ 72.137876] niks3-server[971]: 2026/09/22 10:59:14 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5216lb # [ 72.140306] niks3-server[971]: 2026/09/22 10:59:14 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5217lb # [ 72.146224] niks3-server[971]: 2026/09/22 10:59:14 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5218lb # [ 72.149946] niks3-server[971]: 2026/09/22 10:59:14 INFO Completed upload id=195219worker2 # [ 72.139693] nix-grpc-daemon[880]: time=2026-09-22T10:59:15.087Z level=INFO msg="Upload complete. (250ms)"5220lb # [ 72.152136] niks3-server[971]: 2026/09/22 10:59:14 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5221worker2 # [ 72.141900] nix-grpc-daemon[784]: ts=2026-09-22T10:59:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_in=7525222worker2 # [ 72.145188] nix-grpc-daemon[784]: ts=2026-09-22T10:59:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472085223worker2 # [ 72.152391] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5224worker2 # [ 72.154810] nix-daemon[851]: reaped child process 1070, status = succeeded5225worker2 # [ 72.159897] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5226worker2 # [ 73.204932] nix-daemon[851]: reaped child process 1084, status = succeeded5227worker2 # [ 73.277734] nix-daemon[851]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)5228worker2 # [ 73.291494] nix-daemon[851]: reaped child process 1103, status = succeeded5229lb # [ 73.311390] niks3-server[971]: 2026/09/22 10:59:15 INFO Received uploads request method=POST path=/api/pending_closures5230worker2 # [ 73.307645] nix-grpc-daemon[880]: time=2026-09-22T10:59:16.255Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5231worker2 # [ 73.309857] nix-grpc-daemon[880]: time=2026-09-22T10:59:16.256Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5232lb # [ 73.357948] niks3-server[971]: 2026/09/22 10:59:15 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5233lb # [ 73.366102] niks3-server[971]: 2026/09/22 10:59:15 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5234lb # [ 73.374964] niks3-server[971]: 2026/09/22 10:59:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5235worker2 # [ 73.369061] nix-grpc-daemon[880]: time=2026-09-22T10:59:16.316Z level=INFO msg="Uploading 1 narinfos"5236lb # [ 73.380590] niks3-server[971]: 2026/09/22 10:59:15 INFO Signed narinfos id=20 count=15237lb # [ 73.392948] niks3-server[971]: 2026/09/22 10:59:15 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5238lb # [ 73.397452] niks3-server[971]: 2026/09/22 10:59:15 INFO Completed upload id=205239lb # [ 73.399351] niks3-server[971]: 2026/09/22 10:59:15 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5240worker2 # [ 73.387558] nix-grpc-daemon[880]: time=2026-09-22T10:59:16.335Z level=INFO msg="Upload complete. (193ms)"5241lb # [ 73.401975] niks3-server[971]: 2026/09/22 10:59:15 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5242worker2 # [ 73.393858] nix-grpc-daemon[784]: ts=2026-09-22T10:59:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=11 outputs=15243worker1 # [ 73.385438] nix-grpc-daemon[782]: ts=2026-09-22T10:59:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:386425244worker2 # [ 73.407974] nix-daemon[851]: reaped child process 1082, status = succeeded5245worker1 # [ 73.394350] nix-grpc-daemon[782]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5246worker2 # [ 73.429143] nix-grpc-daemon[784]: ts=2026-09-22T10:59:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:472085247worker2 # [ 73.433757] nix-grpc-daemon[784]: ts=2026-09-22T10:59:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_out=2325248client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.76 seconds)5249client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5250client # [ 73.503820] nix-daemon[1042]: reaped child process 1044, status = succeeded5251client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.01 seconds)5252client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5253client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5254client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5255client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5256(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.81 seconds)5257subtest: an input only one worker has still reaches the builder5258client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5259client # this derivation will be built:5260client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5261client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5262client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds)5263client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5264client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5265worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5266worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.12 seconds)5267worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5268worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5269client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25270client # [ 74.143960] nix-daemon[1042]: accepted connection from pid 1125, user root (trusted)5271client # this derivation will be built:5272client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5273worker2 # [ 74.305849] nix-grpc-daemon[784]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:472085274client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5275worker1 # [ 74.295915] nix-grpc-daemon[782]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386425276worker1 # [ 74.401313] nix-daemon[864]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5277worker1 # [ 74.414141] nix-daemon[864]: reaped child process 1204, status = succeeded5278lb # [ 74.443184] niks3-server[971]: 2026/09/22 10:59:16 INFO Received uploads request method=POST path=/api/pending_closures5279worker1 # [ 74.426678] nix-grpc-daemon[878]: time=2026-09-22T10:59:17.605Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5280worker1 # [ 74.428457] nix-grpc-daemon[878]: time=2026-09-22T10:59:17.607Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5281lb # [ 74.467977] niks3-server[971]: 2026/09/22 10:59:16 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5282lb # [ 74.480251] niks3-server[971]: 2026/09/22 10:59:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5283worker1 # [ 74.462290] nix-grpc-daemon[878]: time=2026-09-22T10:59:17.641Z level=INFO msg="Uploading 1 narinfos"5284lb # [ 74.487375] niks3-server[971]: 2026/09/22 10:59:16 INFO Signed narinfos id=21 count=15285lb # [ 74.493886] niks3-server[971]: 2026/09/22 10:59:16 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5286lb # [ 74.508709] niks3-server[971]: 2026/09/22 10:59:16 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5287lb # [ 74.515237] niks3-server[971]: 2026/09/22 10:59:16 INFO Completed upload id=215288worker1 # [ 74.490346] nix-grpc-daemon[878]: time=2026-09-22T10:59:17.669Z level=INFO msg="Upload complete. (190ms)"5289lb # [ 74.517577] niks3-server[971]: 2026/09/22 10:59:16 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5290worker1 # [ 74.509906] nix-grpc-daemon[782]: ts=2026-09-22T10:59:17Z level=debug event=want client=13 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05291worker1 # [ 74.515508] nix-grpc-daemon[782]: ts=2026-09-22T10:59:17Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15292worker1 # [ 74.521324] nix-grpc-daemon[782]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386425293worker1 # [ 74.531553] nix-grpc-daemon[782]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386425294client # copying 2 paths to 'lb:50051'...5295worker1 # [ 74.543625] nix-grpc-daemon[782]: ts=2026-09-22T10:59:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386425296client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5297client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5298worker1 # [ 74.555208] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5299worker1 # [ 74.649572] nix-daemon[864]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)5300worker1 # [ 74.666093] nix-daemon[864]: reaped child process 1213, status = succeeded5301worker1 # [ 74.756436] nix-daemon[864]: accepted connection from pid 1215, user nix-grpc-daemon (trusted)5302worker1 # [ 74.771095] nix-daemon[864]: reaped child process 1218, status = succeeded5303lb # [ 74.803375] niks3-server[971]: 2026/09/22 10:59:17 INFO Received uploads request method=POST path=/api/pending_closures5304lb # [ 74.811587] niks3-server[971]: 2026/09/22 10:59:17 INFO Received uploads request method=POST path=/api/pending_closures5305worker1 # [ 74.794717] nix-grpc-daemon[878]: time=2026-09-22T10:59:17.973Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5306worker1 # [ 74.796222] nix-grpc-daemon[878]: time=2026-09-22T10:59:17.973Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5307worker1 # [ 74.797853] nix-grpc-daemon[878]: time=2026-09-22T10:59:17.973Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5308lb # [ 74.841345] niks3-server[971]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5309lb # [ 74.857751] niks3-server[971]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5310lb # [ 74.873188] niks3-server[971]: 2026/09/22 10:59:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5311lb # [ 74.876826] niks3-server[971]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5312lb # [ 74.881653] niks3-server[971]: 2026/09/22 10:59:17 INFO Signed narinfos id=23 count=25313lb # [ 74.883566] niks3-server[971]: 2026/09/22 10:59:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5314lb # [ 74.886073] niks3-server[971]: 2026/09/22 10:59:17 INFO Signed narinfos id=22 count=15315lb # [ 74.889129] niks3-server[971]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5316worker1 # [ 74.863916] nix-grpc-daemon[878]: time=2026-09-22T10:59:18.042Z level=INFO msg="Uploading 3 narinfos"5317lb # [ 74.899979] niks3-server[971]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5318lb # [ 74.905161] niks3-server[971]: 2026/09/22 10:59:17 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5319lb # [ 74.909661] niks3-server[971]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5320lb # [ 74.911994] niks3-server[971]: 2026/09/22 10:59:17 INFO Completed upload id=235321lb # [ 74.913831] niks3-server[971]: 2026/09/22 10:59:17 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5322lb # [ 74.918292] niks3-server[971]: 2026/09/22 10:59:17 INFO Completed upload id=225323worker1 # [ 74.893721] nix-grpc-daemon[878]: time=2026-09-22T10:59:18.072Z level=INFO msg="Upload complete. (328ms)"5324lb # [ 74.921343] niks3-server[971]: 2026/09/22 10:59:17 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5325worker1 # [ 74.898826] nix-grpc-daemon[782]: ts=2026-09-22T10:59:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 paths=2 nar_bytes_in=9765326worker1 # [ 74.905387] nix-grpc-daemon[782]: ts=2026-09-22T10:59:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386425327worker1 # [ 74.917225] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5328worker1 # [ 74.926221] nix-daemon[864]: reaped child process 1208, status = succeeded5329worker1 # [ 74.935034] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5330client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5331client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5332client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5333worker1 # [ 75.752870] nix-daemon[864]: reaped child process 1222, status = succeeded5334worker1 # [ 75.835565] nix-daemon[864]: accepted connection from pid 1238, user nix-grpc-daemon (trusted)5335worker1 # [ 75.849048] nix-daemon[864]: reaped child process 1241, status = succeeded5336lb # [ 75.881770] niks3-server[971]: 2026/09/22 10:59:18 INFO Received uploads request method=POST path=/api/pending_closures5337worker1 # [ 75.864821] nix-grpc-daemon[878]: time=2026-09-22T10:59:19.043Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5338lb # [ 75.912323] niks3-server[971]: 2026/09/22 10:59:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5339worker1 # [ 75.894187] nix-grpc-daemon[878]: time=2026-09-22T10:59:19.073Z level=INFO msg="Uploading 1 narinfos"5340lb # [ 75.919287] niks3-server[971]: 2026/09/22 10:59:18 INFO Signed narinfos id=24 count=15341lb # [ 75.929130] niks3-server[971]: 2026/09/22 10:59:18 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5342lb # [ 75.936832] niks3-server[971]: 2026/09/22 10:59:18 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5343lb # [ 75.942933] niks3-server[971]: 2026/09/22 10:59:18 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5344lb # [ 75.950643] niks3-server[971]: 2026/09/22 10:59:18 INFO Completed upload id=245345worker1 # [ 75.928146] nix-grpc-daemon[878]: time=2026-09-22T10:59:19.107Z level=INFO msg="Upload complete. (186ms)"5346worker1 # [ 75.933489] nix-grpc-daemon[782]: ts=2026-09-22T10:59:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=12 outputs=15347worker1 # [ 75.945130] nix-daemon[864]: reaped child process 1220, status = succeeded5348client # copying 1 paths...5349worker1 # [ 75.976894] nix-grpc-daemon[782]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:386425350client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5351worker2 # [ 76.008233] nix-grpc-daemon[784]: ts=2026-09-22T10:59:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:472085352worker2 # [ 76.019333] nix-grpc-daemon[784]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5353worker2 # [ 76.034644] nix-grpc-daemon[784]: ts=2026-09-22T10:59:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_out=1205354client # [ 76.095995] nix-daemon[1042]: reaped child process 1128, status = succeeded5355client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.08 seconds)5356client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25357lb # [ 76.227208] envoy[908]: conn peer=[2001:db8:1::1]:54986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1589 tx=1374 ms=48165358lb # [ 76.230973] envoy[908]: conn peer=[2001:db8:1::1]:40112 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=565359lb # [ 76.234628] envoy[908]: conn peer=[2001:db8:1::1]:40096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3405 tx=1808 ms=18465360lb # [ 76.238284] envoy[908]: conn peer=[2001:db8:1::1]:40128 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=2047 ms=15645361client # [ 76.197777] nix-daemon[1042]: accepted connection from pid 1147, user root (trusted)5362lb # [ 76.241910] envoy[908]: conn peer=[2001:db8:1::1]:40144 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=565363lb # [ 76.245525] envoy[908]: conn peer=[2001:db8:1::1]:40158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4327 tx=2276 ms=17275364lb # [ 76.249812] envoy[908]: conn peer=[2001:db8:1::1]:45114 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=565365client # this derivation will be built:5366client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5367lb # [ 76.253352] envoy[908]: conn peer=[2001:db8:1::1]:45126 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=795368lb # [ 76.256819] envoy[908]: conn peer=[2001:db8:1::1]:45116 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3877 tx=1842 ms=17805369worker2 # [ 76.358298] nix-grpc-daemon[784]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:472085370client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5371worker1 # [ 76.354129] nix-grpc-daemon[782]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386425372worker1 # [ 76.363357] nix-grpc-daemon[878]: time=2026-09-22T10:59:19.542Z level=INFO msg="All 1 paths already cached"5373worker1 # [ 76.377509] nix-grpc-daemon[782]: ts=2026-09-22T10:59:19Z level=debug event=want client=14 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05374worker2 # [ 76.393237] nix-grpc-daemon[784]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472085375worker1 # [ 76.382344] nix-grpc-daemon[782]: ts=2026-09-22T10:59:19Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15376worker2 # [ 76.404907] nix-grpc-daemon[784]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472085377client # copying 1 paths to 'lb:50051'...5378client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5379worker2 # [ 76.422784] nix-grpc-daemon[784]: ts=2026-09-22T10:59:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472085380worker2 # [ 76.434192] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5381worker2 # [ 76.450934] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5)5382worker2 # [ 76.538608] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 2/5)5383worker2 # [ 76.612706] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 226 ms (attempt 3/5)5384worker2 # [ 76.840887] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 129 ms (attempt 4/5)5385worker2 # [ 76.978375] nix-grpc-daemon[784]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5386worker2 # [ 76.981949] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5387worker2 # [ 76.987473] nix-grpc-daemon[784]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5388worker2 # [ 77.112890] nix-daemon[851]: accepted connection from pid 1116, user nix-grpc-daemon (trusted)5389worker2 # [ 77.129508] nix-daemon[851]: reaped child process 1119, status = succeeded5390worker2 # [ 77.203852] nix-daemon[851]: accepted connection from pid 1121, user nix-grpc-daemon (trusted)5391worker2 # [ 77.222074] nix-daemon[851]: reaped child process 1124, status = succeeded5392lb # [ 77.237957] niks3-server[971]: 2026/09/22 10:59:19 INFO Received uploads request method=POST path=/api/pending_closures5393worker2 # [ 77.234915] nix-grpc-daemon[880]: time=2026-09-22T10:59:20.182Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5394worker2 # [ 77.236733] nix-grpc-daemon[880]: time=2026-09-22T10:59:20.184Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5395lb # [ 77.267384] niks3-server[971]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5396lb # [ 77.282223] niks3-server[971]: 2026/09/22 10:59:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5397worker2 # [ 77.276114] nix-grpc-daemon[880]: time=2026-09-22T10:59:20.223Z level=INFO msg="Uploading 1 narinfos"5398lb # [ 77.287688] niks3-server[971]: 2026/09/22 10:59:19 INFO Signed narinfos id=25 count=15399lb # [ 77.293618] niks3-server[971]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5400lb # [ 77.307256] niks3-server[971]: 2026/09/22 10:59:19 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5401worker2 # [ 77.302166] nix-grpc-daemon[880]: time=2026-09-22T10:59:20.249Z level=INFO msg="Upload complete. (296ms)"5402lb # [ 77.313758] niks3-server[971]: 2026/09/22 10:59:19 INFO Completed upload id=255403lb # [ 77.317778] niks3-server[971]: 2026/09/22 10:59:19 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5404worker2 # [ 77.305535] nix-grpc-daemon[784]: ts=2026-09-22T10:59:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_in=5605405worker2 # [ 77.311260] nix-grpc-daemon[784]: ts=2026-09-22T10:59:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472085406worker2 # [ 77.321491] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5407worker2 # [ 77.331973] nix-daemon[851]: reaped child process 1109, status = succeeded5408worker2 # [ 77.336556] nix-grpc-daemon[784]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5409client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5410worker2 # [ 77.343285] nix-daemon[851]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5411client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5412client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5413worker2 # [ 78.376703] nix-daemon[851]: reaped child process 1129, status = succeeded5414worker2 # [ 78.452440] nix-daemon[851]: accepted connection from pid 1145, user nix-grpc-daemon (trusted)5415worker2 # [ 78.466048] nix-daemon[851]: reaped child process 1148, status = succeeded5416lb # [ 78.484530] niks3-server[971]: 2026/09/22 10:59:20 INFO Received uploads request method=POST path=/api/pending_closures5417worker2 # [ 78.477622] nix-grpc-daemon[880]: time=2026-09-22T10:59:21.425Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5418lb # [ 78.499169] niks3-server[971]: 2026/09/22 10:59:20 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5419lb # [ 78.503555] niks3-server[971]: 2026/09/22 10:59:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5420lb # [ 78.506310] niks3-server[971]: 2026/09/22 10:59:20 INFO Signed narinfos id=26 count=15421worker2 # [ 78.494650] nix-grpc-daemon[880]: time=2026-09-22T10:59:21.442Z level=INFO msg="Uploading 1 narinfos"5422lb # [ 78.507995] niks3-server[971]: 2026/09/22 10:59:20 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5423lb # [ 78.514499] niks3-server[971]: 2026/09/22 10:59:20 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5424lb # [ 78.517728] niks3-server[971]: 2026/09/22 10:59:20 INFO Completed upload id=265425worker2 # [ 78.506140] nix-grpc-daemon[880]: time=2026-09-22T10:59:21.453Z level=INFO msg="Upload complete. (153ms)"5426worker2 # [ 78.508581] nix-grpc-daemon[784]: ts=2026-09-22T10:59:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=13 outputs=15427lb # [ 78.522172] niks3-server[971]: 2026/09/22 10:59:20 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5428worker2 # [ 78.515692] nix-daemon[851]: reaped child process 1126, status = succeeded5429client # copying 1 paths...5430worker1 # [ 78.542515] nix-grpc-daemon[782]: ts=2026-09-22T10:59:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:386425431worker1 # [ 78.554177] nix-grpc-daemon[782]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5432client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5433worker2 # [ 78.598068] nix-grpc-daemon[784]: ts=2026-09-22T10:59:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:472085434worker2 # [ 78.601996] nix-grpc-daemon[784]: ts=2026-09-22T10:59:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_out=1205435client # [ 78.637809] nix-daemon[1042]: reaped child process 1150, status = succeeded5436client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.55 seconds)5437client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25438client # [ 78.749587] nix-daemon[1042]: accepted connection from pid 1169, user root (trusted)5439client # this derivation will be built:5440client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5441client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5442worker1 # [ 78.896347] nix-grpc-daemon[782]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:386425443worker1 # [ 78.902987] nix-grpc-daemon[782]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386425444worker1 # [ 78.908722] nix-grpc-daemon[878]: time=2026-09-22T10:59:22.086Z level=INFO msg="All 1 paths already cached"5445worker1 # [ 78.918748] nix-grpc-daemon[782]: ts=2026-09-22T10:59:22Z level=debug event=want client=15 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05446worker1 # [ 78.924303] nix-grpc-daemon[782]: ts=2026-09-22T10:59:22Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15447worker1 # [ 78.963683] nix-grpc-daemon[782]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386425448worker1 # [ 78.976625] nix-grpc-daemon[782]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:386425449worker1 # [ 78.988977] nix-grpc-daemon[878]: time=2026-09-22T10:59:22.168Z level=INFO msg="All 1 paths already cached"5450client # copying 1 paths to 'lb:50051'...5451client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5452worker1 # [ 78.997888] nix-grpc-daemon[782]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:386425453worker1 # [ 79.009957] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5454worker1 # [ 79.116558] nix-daemon[864]: accepted connection from pid 1250, user nix-grpc-daemon (trusted)5455worker1 # [ 79.131246] nix-daemon[864]: reaped child process 1253, status = succeeded5456worker1 # [ 79.216263] nix-daemon[864]: accepted connection from pid 1255, user nix-grpc-daemon (trusted)5457worker1 # [ 79.229707] nix-daemon[864]: reaped child process 1258, status = succeeded5458lb # [ 79.261259] niks3-server[971]: 2026/09/22 10:59:21 INFO Received uploads request method=POST path=/api/pending_closures5459worker1 # [ 79.245934] nix-grpc-daemon[878]: time=2026-09-22T10:59:22.424Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5460worker1 # [ 79.247311] nix-grpc-daemon[878]: time=2026-09-22T10:59:22.425Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5461lb # [ 79.288280] niks3-server[971]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5462lb # [ 79.290568] niks3-server[971]: 2026/09/22 10:59:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5463lb # [ 79.293085] niks3-server[971]: 2026/09/22 10:59:21 INFO Signed narinfos id=27 count=15464worker1 # [ 79.269829] nix-grpc-daemon[878]: time=2026-09-22T10:59:22.448Z level=INFO msg="Uploading 1 narinfos"5465lb # [ 79.296257] niks3-server[971]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5466lb # [ 79.303256] niks3-server[971]: 2026/09/22 10:59:21 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5467lb # [ 79.305796] niks3-server[971]: 2026/09/22 10:59:21 INFO Completed upload id=275468worker1 # [ 79.281170] nix-grpc-daemon[878]: time=2026-09-22T10:59:22.460Z level=INFO msg="Upload complete. (259ms)"5469worker1 # [ 79.283286] nix-grpc-daemon[782]: ts=2026-09-22T10:59:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 paths=1 nar_bytes_in=5605470lb # [ 79.310918] niks3-server[971]: 2026/09/22 10:59:21 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5471worker1 # [ 79.288500] nix-grpc-daemon[782]: ts=2026-09-22T10:59:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:386425472worker1 # [ 79.294109] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5473worker1 # [ 79.296408] nix-daemon[864]: reaped child process 1248, status = succeeded5474worker1 # [ 79.301795] nix-daemon[864]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5475client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5476client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5477client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5478worker1 # [ 79.914169] nix-daemon[864]: reaped child process 1262, status = succeeded5479worker1 # [ 79.991488] nix-daemon[864]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5480worker1 # [ 80.003623] nix-daemon[864]: reaped child process 1281, status = succeeded5481lb # [ 80.036980] niks3-server[971]: 2026/09/22 10:59:22 INFO Received uploads request method=POST path=/api/pending_closures5482worker1 # [ 80.019933] nix-grpc-daemon[878]: time=2026-09-22T10:59:23.199Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5483lb # [ 80.066828] niks3-server[971]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5484lb # [ 80.076529] niks3-server[971]: 2026/09/22 10:59:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5485lb # [ 80.081599] niks3-server[971]: 2026/09/22 10:59:22 INFO Signed narinfos id=28 count=15486worker1 # [ 80.060082] nix-grpc-daemon[878]: time=2026-09-22T10:59:23.239Z level=INFO msg="Uploading 1 narinfos"5487lb # [ 80.088648] niks3-server[971]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5488lb # [ 80.100815] niks3-server[971]: 2026/09/22 10:59:22 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5489lb # [ 80.107088] niks3-server[971]: 2026/09/22 10:59:22 INFO Completed upload id=285490worker1 # [ 80.082217] nix-grpc-daemon[878]: time=2026-09-22T10:59:23.261Z level=INFO msg="Upload complete. (174ms)"5491lb # [ 80.109858] niks3-server[971]: 2026/09/22 10:59:22 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5492worker1 # [ 80.086771] nix-grpc-daemon[782]: ts=2026-09-22T10:59:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38642 duration_s=0 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=14 outputs=15493client # copying 1 paths...5494worker1 # [ 80.101152] nix-daemon[864]: reaped child process 1260, status = succeeded5495worker2 # [ 80.111393] nix-grpc-daemon[784]: ts=2026-09-22T10:59:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:472085496worker2 # [ 80.120511] nix-grpc-daemon[784]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5497client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5498worker2 # [ 80.146283] nix-grpc-daemon[784]: ts=2026-09-22T10:59:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:472085499worker2 # [ 80.150899] nix-grpc-daemon[784]: ts=2026-09-22T10:59:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47208 duration_s=0 paths=1 nar_bytes_out=1205500client # [ 80.227577] nix-daemon[1042]: reaped child process 1172, status = succeeded5501client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.59 seconds)5502(finished: subtest: an input only one worker has still reaches the builder, in 6.68 seconds)5503subtest: upload whose reference the worker lacks is completed from the cache5504client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5505client # this derivation will be built:5506client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5507client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5508client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.26 seconds)5509client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5510client # this derivation will be built:5511client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5512client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5513client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.25 seconds)5514client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5515client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds)5516worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5517worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5518worker2 # [ 81.058158] systemd[1]: Stopping Nix worker-protocol over gRPC...5519worker2 # [ 81.429530] nix-grpc-daemon[784]: ts=2026-09-22T10:59:24Z level=info event=signal_exit5520worker2 # [ 83.417186] nix-grpc-daemon[784]: ts=2026-09-22T10:59:26Z level=info event=scheduler_restarting5521worker1 # [ 83.408754] nix-grpc-daemon[782]: ts=2026-09-22T10:59:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:386265522worker2 # [ 83.427195] nix-grpc-daemon[784]: ts=2026-09-22T10:59:26Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5523worker1 # [ 83.412618] nix-grpc-daemon[782]: ts=2026-09-22T10:59:26Z level=info event=worker_gone addr=192.168.1.4:500515524worker2 # [ 83.442163] nix-daemon[851]: reaped child process 853, status = succeeded5525worker2 # [ 83.561479] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5526worker2 # [ 83.565278] systemd[1]: Stopped Nix worker-protocol over gRPC.5527worker2 # [ 83.568841] systemd[1]: nix-grpc-daemon.service: Consumed 2.009s CPU time over 1min 7.298s wall clock time, 60.1M memory peak, 66.2K incoming IP traffic, 162.5K outgoing IP traffic.5528worker2 # [ 83.581789] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5529worker2 # [ 83.585426] systemd[1]: Closed Nix worker-protocol over gRPC.5530worker2 # [ 83.588058] systemd[1]: nix-grpc-daemon.socket: Consumed 334us CPU time over 1min 15.840s wall clock time, 256K memory peak, 74.2K incoming IP traffic, 60K outgoing IP traffic.5531lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5532lb: must succeed: curl -sf localhost:9901/clusters5533lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5534lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5535client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25536worker1 # [ 83.782142] nix-grpc-daemon[782]: ts=2026-09-22T10:59:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:386425537worker1 # [ 83.789605] nix-grpc-daemon[782]: ts=2026-09-22T10:59:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:386425538client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5539client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.18 seconds)5540worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5541worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5542worker2 # [ 83.889238] systemd[1]: Listening on Nix worker-protocol over gRPC.5543worker1 # [ 83.918984] systemd[1]: Stopping Nix worker-protocol over gRPC...5544worker1 # [ 84.393950] nix-grpc-daemon[782]: ts=2026-09-22T10:59:27Z level=info event=signal_exit5545lb # [ 84.425810] niks3-server[971]: 2026/09/22 10:59:26 INFO lead: released remote=192.168.1.3:372565546worker2 # [ 84.589982] systemd[1]: Starting scheduler-token.service...5547worker2 # [ 84.653305] systemd[1]: scheduler-token.service: Deactivated successfully.5548worker2 # [ 84.656556] systemd[1]: Finished scheduler-token.service.5549worker2 # [ 84.659298] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 66ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5550worker2 # [ 84.666940] systemd[1]: Starting Nix worker-protocol over gRPC...5551worker2 # [ 84.752371] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5552worker2 # [ 84.754569] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5553worker2 # [ 84.757822] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15554worker2 # [ 84.762054] nix-grpc-daemon[1173]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5555worker2 # [ 84.764162] nix-grpc-daemon[1173]: E0000 00:00:1790074767.709619 1173 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5556worker2 # [ 84.770075] nix-grpc-daemon[1173]: E0000 00:00:1790074767.715108 1173 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5557worker2 # [ 84.772234] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5558worker2 # [ 84.773803] systemd[1]: Started Nix worker-protocol over gRPC.5559worker2 # [ 84.781063] nix-grpc-daemon[1173]: E0000 00:00:1790074767.728596 1215 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5560lb # [ 84.794339] niks3-server[971]: 2026/09/22 10:59:27 INFO lead: acquired remote=192.168.1.4:374725561worker2 # [ 84.784237] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:27Z level=info event=scheduler_take_over5562worker2 # [ 84.796182] nix-grpc-daemon[1173]: E0000 00:00:1790074767.743853 1215 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5563worker2 # [ 84.800139] nix-grpc-daemon[1173]: E0000 00:00:1790074767.746123 1215 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5564worker2 # [ 84.810371] nix-grpc-daemon[1173]: E0000 00:00:1790074767.758049 1215 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5565worker2 # [ 84.820243] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:27Z level=info event=scheduler_connected addr=lb:500515566worker2 # [ 84.822114] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5567worker2 # [ 85.023634] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:27Z level=info event=scheduler_connected addr=lb:500515568worker2 # [ 85.027810] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5569worker2 # [ 85.228915] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=scheduler_connected addr=lb:500515570worker2 # [ 85.232925] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5571worker1 # [ 85.405823] nix-grpc-daemon[782]: ts=2026-09-22T10:59:28Z level=info event=scheduler_restarting5572worker1 # [ 85.408964] nix-grpc-daemon[782]: ts=2026-09-22T10:59:28Z level=info event=scheduler_restarting addr=lb:500515573worker1 # [ 85.413469] nix-grpc-daemon[782]: ts=2026-09-22T10:59:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:386265574worker1 # [ 85.416759] nix-grpc-daemon[782]: ts=2026-09-22T10:59:28Z level=info event=worker_gone addr=192.168.1.3:500515575worker2 # [ 85.434069] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=scheduler_connected addr=lb:500515576worker2 # [ 85.437947] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5577worker1 # [ 85.438648] nix-daemon[864]: reaped child process 866, status = succeeded5578worker1 # [ 85.504601] nix-grpc-daemon[782]: E0000 00:00:1790074768.683752 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5579worker1 # [ 85.537636] nix-grpc-daemon[782]: E0000 00:00:1790074768.716754 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5580worker1 # [ 85.615704] nix-grpc-daemon[782]: ts=2026-09-22T10:59:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5581worker2 # [ 85.639243] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=scheduler_connected addr=lb:500515582worker2 # [ 85.643119] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5583worker2 # [ 85.852109] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=scheduler_connected addr=lb:500515584worker2 # [ 85.856950] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=oidc_jwks provider=mock keys=15585worker2 # [ 85.860611] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:553965586worker2 # [ 85.864563] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05587lb # [ 86.227405] envoy[908]: conn peer=[2001:db8:1::1]:45132 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3754 tx=2015 ms=22555588lb # [ 86.228907] envoy[908]: conn peer=[2001:db8:1::1]:45144 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=175589lb # [ 86.230295] envoy[908]: conn peer=[2001:db8:1::1]:45156 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3712 tx=1971 ms=12935590lb # [ 86.231788] envoy[908]: conn peer=[2001:db8:1::1]:45172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575591lb # [ 86.233142] envoy[908]: conn peer=[2001:db8:1::4]:58344 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=54062 tx=2778 ms=669655592lb # [ 86.234622] envoy[908]: conn peer=[2001:db8:1::1]:35810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215593lb # [ 86.235964] envoy[908]: conn peer=[2001:db8:1::3]:34356 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=14870 tx=3111 ms=691855594worker1 # [ 87.535529] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5595worker1 # [ 87.539463] systemd[1]: Stopped Nix worker-protocol over gRPC.5596worker1 # [ 87.543613] systemd[1]: nix-grpc-daemon.service: Consumed 2.596s CPU time over 1min 11.336s wall clock time, 66.9M memory peak, 100.1K incoming IP traffic, 219.3K outgoing IP traffic.5597worker1 # [ 87.555580] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5598worker1 # [ 87.559639] systemd[1]: Closed Nix worker-protocol over gRPC.5599worker1 # [ 87.563758] systemd[1]: nix-grpc-daemon.socket: Consumed 302us CPU time over 1min 19.979s wall clock time, 256K memory peak, 101.9K incoming IP traffic, 84K outgoing IP traffic.5600lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5601lb: must succeed: curl -sf localhost:9901/clusters5602lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5603lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.04 seconds)5604client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25605worker2 # [ 87.778398] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:554025606worker2 # [ 87.786751] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5607worker2 # [ 87.805376] nix-grpc-daemon[1173]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5)5608worker2 # [ 87.830549] nix-grpc-daemon[1173]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 2/5)5609worker2 # [ 87.848201] nix-grpc-daemon[1173]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 349 ms (attempt 3/5)5610worker2 # [ 88.199401] nix-grpc-daemon[1173]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 753 ms (attempt 4/5)5611worker2 # [ 88.955800] nix-grpc-daemon[1173]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5612worker2 # [ 88.959734] nix-grpc-daemon[1173]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5613client # copying 2 paths...5614client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5615client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5616worker2 # [ 88.979727] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:554025617worker2 # [ 88.991113] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5618worker2 # [ 89.011546] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:31Z level=info event=niks3_push_started pid=12335619worker2 # [ 89.120666] nix-grpc-daemon[1233]: time=2026-09-22T10:59:32.067Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5620worker2 # [ 89.213570] nix-daemon[851]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5621worker2 # [ 89.226952] nix-daemon[851]: reaped child process 1249, status = succeeded5622lb # [ 89.245571] niks3-server[971]: 2026/09/22 10:59:31 INFO Received uploads request method=POST path=/api/pending_closures5623lb # [ 89.253623] niks3-server[971]: 2026/09/22 10:59:31 INFO Received uploads request method=POST path=/api/pending_closures5624worker2 # [ 89.249763] nix-grpc-daemon[1233]: time=2026-09-22T10:59:32.197Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5625worker2 # [ 89.251305] nix-grpc-daemon[1233]: time=2026-09-22T10:59:32.199Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5626worker2 # [ 89.253316] nix-grpc-daemon[1233]: time=2026-09-22T10:59:32.201Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5627lb # [ 89.299125] niks3-server[971]: 2026/09/22 10:59:31 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5628lb # [ 89.313761] niks3-server[971]: 2026/09/22 10:59:31 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5629lb # [ 89.329112] niks3-server[971]: 2026/09/22 10:59:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5630lb # [ 89.330780] niks3-server[971]: 2026/09/22 10:59:31 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5631lb # [ 89.334851] niks3-server[971]: 2026/09/22 10:59:31 INFO Signed narinfos id=29 count=15632lb # [ 89.337397] niks3-server[971]: 2026/09/22 10:59:31 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5633lb # [ 89.338779] niks3-server[971]: 2026/09/22 10:59:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5634lb # [ 89.340625] niks3-server[971]: 2026/09/22 10:59:31 INFO Signed narinfos id=30 count=25635worker2 # [ 89.330165] nix-grpc-daemon[1233]: time=2026-09-22T10:59:32.276Z level=INFO msg="Uploading 3 narinfos"5636lb # [ 89.357771] niks3-server[971]: 2026/09/22 10:59:31 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5637lb # [ 89.359724] niks3-server[971]: 2026/09/22 10:59:31 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5638lb # [ 89.362585] niks3-server[971]: 2026/09/22 10:59:31 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5639lb # [ 89.366176] niks3-server[971]: 2026/09/22 10:59:31 INFO Completed upload id=295640lb # [ 89.367905] niks3-server[971]: 2026/09/22 10:59:31 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5641lb # [ 89.370303] niks3-server[971]: 2026/09/22 10:59:31 INFO Completed upload id=305642worker2 # [ 89.358983] nix-grpc-daemon[1233]: time=2026-09-22T10:59:32.306Z level=INFO msg="Upload complete. (235ms)"5643lb # [ 89.374666] niks3-server[971]: 2026/09/22 10:59:31 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5644worker2 # [ 89.364409] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55402 duration_s=0 paths=2 nar_bytes_in=3125645worker2 # [ 89.377332] nix-daemon[851]: reaped child process 1231, status = succeeded5646client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.77 seconds)5647worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5648worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5649worker1 # [ 89.450449] systemd[1]: Listening on Nix worker-protocol over gRPC.5650worker1 # [ 89.458174] systemd[1]: Starting Nix worker-protocol over gRPC...5651worker1 # [ 89.550702] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5652worker1 # [ 89.552646] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5653worker1 # [ 89.555694] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15654worker1 # [ 89.560320] nix-grpc-daemon[1304]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5655worker1 # [ 89.561684] nix-grpc-daemon[1304]: E0000 00:00:1790074772.739157 1304 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5656worker1 # [ 89.568078] nix-grpc-daemon[1304]: E0000 00:00:1790074772.744599 1304 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5657worker1 # [ 89.572735] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5658worker1 # [ 89.576108] systemd[1]: Started Nix worker-protocol over gRPC.5659worker1 # [ 89.580148] nix-grpc-daemon[1304]: E0000 00:00:1790074772.758956 1345 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5660lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5661lb: must succeed: curl -sf localhost:9901/clusters5662worker1 # [ 89.596235] nix-grpc-daemon[1304]: E0000 00:00:1790074772.774231 1345 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5663worker1 # [ 89.599129] nix-grpc-daemon[1304]: E0000 00:00:1790074772.774340 1345 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5664worker1 # [ 89.609894] nix-grpc-daemon[1304]: E0000 00:00:1790074772.789007 1345 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5665worker1 # [ 89.618570] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:32Z level=info event=scheduler_connected addr=lb:500515666lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5667lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.03 seconds)5668worker2 # [ 89.633073] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:553965669(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.36 seconds)5670subtest: a niks3 restart does not cost the leader its role5671lb: must succeed: curl -sf localhost:9901/clusters5672worker2 # [ 89.637848] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05673lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5674lb: must succeed: curl -sf localhost:9901/clusters5675lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5676lb # [ 89.705269] systemd[1]: Stopping niks3 server...5677worker1 # [ 89.680282] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:32Z level=info event=scheduler_lock_lost status=2005678lb # [ 89.706212] niks3-server[971]: 2026/09/22 10:59:32 INFO Shutdown signal received, draining in-flight requests timeout=10s5679lb # [ 89.707518] niks3-server[971]: 2026/09/22 10:59:32 INFO lead: released remote=192.168.1.4:374725680worker2 # [ 89.694182] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:32Z level=info event=scheduler_lock_lost status=2005681lb # [ 89.720537] systemd[1]: niks3.service: Deactivated successfully.5682lb # [ 89.722076] systemd[1]: Stopped niks3 server.5683lb # [ 89.722742] systemd[1]: niks3.service: Consumed 571ms CPU time over 1min 10.497s wall clock time, 27.4M memory peak, 81.1K incoming IP traffic, 116.4K outgoing IP traffic.5684lb # [ 89.730364] systemd[1]: Starting niks3 server...5685lb # [ 89.780846] niks3-server[1104]: 2026/09/22 10:59:32 goose: no migrations to run. current version: 202609200000005686lb # [ 89.787175] niks3-server[1104]: 2026/09/22 10:59:32 OK 1_commit_pending_closure.sql (4.26ms)5687lb # [ 89.790687] niks3-server[1104]: 2026/09/22 10:59:32 OK 2_object_stats_trigger.sql (3.2ms)5688lb # [ 89.791773] niks3-server[1104]: 2026/09/22 10:59:32 goose: up to current file version: 25689lb # [ 89.793544] niks3-server[1104]: 2026/09/22 10:59:32 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5690lb # [ 89.803474] niks3-server[1104]: 2026/09/22 10:59:32 INFO Read proxy enabled — serving cache objects from S35691lb # [ 89.804814] niks3-server[1104]: 2026/09/22 10:59:32 INFO Using socket-activated listener address=0.0.0.0:57515692lb # [ 89.807168] niks3-server[1104]: 2026/09/22 10:59:32 INFO systemd watchdog enabled interval=15s5693lb # [ 89.808292] niks3-server[1104]: 2026/09/22 10:59:32 INFO Starting HTTP server address=0.0.0.0:57515694lb # [ 89.809652] systemd[1]: Started niks3 server.5695lb: waiting for unit niks3.service5696lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5697lb: must succeed: curl -sf localhost:9901/clusters5698lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5699worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5700worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5701lb: must succeed: curl -sf localhost:9901/clusters5702lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5703lb: must succeed: curl -sf localhost:9901/clusters5704lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5705worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5706worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5707client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25708worker2 # [ 90.129556] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:554025709client # don't know how to build these paths:5710client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5711worker2 # [ 90.138456] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:554025712worker2 # [ 90.160897] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:33Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05713worker2 # [ 90.166178] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:33Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15714worker2 # [ 90.204602] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554025715worker2 # [ 90.217327] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:554025716client # copying 1 paths to 'lb:50051'...5717client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5718worker2 # [ 90.229835] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:554025719worker2 # [ 90.240881] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5720worker2 # [ 90.347816] nix-daemon[851]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)5721worker2 # [ 90.361645] nix-daemon[851]: reaped child process 1269, status = succeeded5722worker2 # [ 90.455795] nix-daemon[851]: accepted connection from pid 1271, user nix-grpc-daemon (trusted)5723worker2 # [ 90.472422] nix-daemon[851]: reaped child process 1274, status = succeeded5724lb # [ 90.491525] niks3-server[1104]: 2026/09/22 10:59:32 INFO Received uploads request method=POST path=/api/pending_closures5725worker2 # [ 90.491517] nix-grpc-daemon[1233]: time=2026-09-22T10:59:33.438Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5726worker2 # [ 90.493087] nix-grpc-daemon[1233]: time=2026-09-22T10:59:33.438Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5727lb # [ 90.533152] niks3-server[1104]: 2026/09/22 10:59:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5728lb # [ 90.542596] niks3-server[1104]: 2026/09/22 10:59:32 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5729lb # [ 90.562298] niks3-server[1104]: 2026/09/22 10:59:32 INFO Signed narinfos id=31 count=15730worker2 # [ 90.554830] nix-grpc-daemon[1233]: time=2026-09-22T10:59:33.502Z level=INFO msg="Uploading 1 narinfos"5731lb # [ 90.569756] niks3-server[1104]: 2026/09/22 10:59:32 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5732lb # [ 90.573596] niks3-server[1104]: 2026/09/22 10:59:32 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5733lb # [ 90.579757] niks3-server[1104]: 2026/09/22 10:59:32 INFO Completed upload id=315734worker2 # [ 90.568319] nix-grpc-daemon[1233]: time=2026-09-22T10:59:33.515Z level=INFO msg="Upload complete. (316ms)"5735lb # [ 90.583239] niks3-server[1104]: 2026/09/22 10:59:32 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5736worker2 # [ 90.573358] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55402 duration_s=0 paths=1 nar_bytes_in=5365737worker2 # [ 90.580611] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554025738worker2 # [ 90.590885] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5739worker2 # [ 90.600109] nix-daemon[851]: reaped child process 1264, status = succeeded5740worker2 # [ 90.605540] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5741client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5742client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5743lb # [ 90.714440] niks3-server[1104]: 2026/09/22 10:59:33 INFO lead: acquired remote=192.168.1.4:375705744client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5745client # farm-a> LOG-farm-a-after-niks3-restart5746worker2 # [ 91.641397] nix-daemon[851]: reaped child process 1278, status = succeeded5747worker2 # [ 91.727151] nix-daemon[851]: accepted connection from pid 1296, user nix-grpc-daemon (trusted)5748worker2 # [ 91.741991] nix-daemon[851]: reaped child process 1299, status = succeeded5749lb # [ 91.760713] niks3-server[1104]: 2026/09/22 10:59:34 INFO Received uploads request method=POST path=/api/pending_closures5750worker2 # [ 91.760452] nix-grpc-daemon[1233]: time=2026-09-22T10:59:34.707Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5751worker2 # [ 91.761976] nix-grpc-daemon[1233]: time=2026-09-22T10:59:34.709Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5752lb # [ 91.807116] niks3-server[1104]: 2026/09/22 10:59:34 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5753lb # [ 91.817271] niks3-server[1104]: 2026/09/22 10:59:34 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5754lb # [ 91.824135] niks3-server[1104]: 2026/09/22 10:59:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5755lb # [ 91.834271] niks3-server[1104]: 2026/09/22 10:59:34 INFO Signed narinfos id=32 count=15756lb # [ 91.836151] niks3-server[1104]: 2026/09/22 10:59:34 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5757worker2 # [ 91.823562] nix-grpc-daemon[1233]: time=2026-09-22T10:59:34.770Z level=INFO msg="Uploading 1 narinfos"5758lb # [ 91.842457] niks3-server[1104]: 2026/09/22 10:59:34 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5759lb # [ 91.846207] niks3-server[1104]: 2026/09/22 10:59:34 INFO Completed upload id=325760worker2 # [ 91.834881] nix-grpc-daemon[1233]: time=2026-09-22T10:59:34.782Z level=INFO msg="Upload complete. (221ms)"5761lb # [ 91.849511] niks3-server[1104]: 2026/09/22 10:59:34 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5762worker2 # [ 91.841665] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55402 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15763worker2 # [ 91.856200] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:34Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05764worker2 # [ 91.859347] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:34Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15765worker2 # [ 91.868469] nix-daemon[851]: reaped child process 1276, status = succeeded5766worker2 # [ 91.899689] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554025767worker2 # [ 91.911339] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:554025768worker2 # [ 91.924805] nix-grpc-daemon[1233]: time=2026-09-22T10:59:34.872Z level=INFO msg="All 1 paths already cached"5769client # copying 1 paths to 'lb:50051'...5770client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5771worker2 # [ 91.932996] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:554025772worker2 # [ 91.941282] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5773worker2 # [ 92.035794] nix-daemon[851]: accepted connection from pid 1305, user nix-grpc-daemon (trusted)5774worker2 # [ 92.050072] nix-daemon[851]: reaped child process 1308, status = succeeded5775worker2 # [ 92.129047] nix-daemon[851]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5776worker2 # [ 92.143422] nix-daemon[851]: reaped child process 1313, status = succeeded5777lb # [ 92.160796] niks3-server[1104]: 2026/09/22 10:59:34 INFO Received uploads request method=POST path=/api/pending_closures5778worker2 # [ 92.157207] nix-grpc-daemon[1233]: time=2026-09-22T10:59:35.104Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5779worker2 # [ 92.158729] nix-grpc-daemon[1233]: time=2026-09-22T10:59:35.106Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5780lb # [ 92.194157] niks3-server[1104]: 2026/09/22 10:59:34 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5781lb # [ 92.201706] niks3-server[1104]: 2026/09/22 10:59:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5782worker2 # [ 92.197682] nix-grpc-daemon[1233]: time=2026-09-22T10:59:35.145Z level=INFO msg="Uploading 1 narinfos"5783lb # [ 92.209887] niks3-server[1104]: 2026/09/22 10:59:34 INFO Signed narinfos id=33 count=15784lb # [ 92.216659] niks3-server[1104]: 2026/09/22 10:59:34 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5785lb # [ 92.230563] niks3-server[1104]: 2026/09/22 10:59:34 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5786lb # [ 92.239392] niks3-server[1104]: 2026/09/22 10:59:34 INFO Completed upload id=335787worker2 # [ 92.228197] nix-grpc-daemon[1233]: time=2026-09-22T10:59:35.175Z level=INFO msg="Upload complete. (274ms)"5788lb # [ 92.242826] niks3-server[1104]: 2026/09/22 10:59:34 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5789worker2 # [ 92.232227] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55402 duration_s=0 paths=1 nar_bytes_in=6565790worker2 # [ 92.237825] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554025791worker2 # [ 92.246745] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5792worker2 # [ 92.255590] nix-daemon[851]: reaped child process 1303, status = succeeded5793worker2 # [ 92.261408] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5794client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5795client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5796client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5797client # farm-b> LOG-farm-b-after-niks3-restart5798worker2 # [ 93.078669] nix-daemon[851]: reaped child process 1317, status = succeeded5799worker2 # [ 93.147691] nix-daemon[851]: accepted connection from pid 1333, user nix-grpc-daemon (trusted)5800worker2 # [ 93.162887] nix-daemon[851]: reaped child process 1336, status = succeeded5801lb # [ 93.179388] niks3-server[1104]: 2026/09/22 10:59:35 INFO Received uploads request method=POST path=/api/pending_closures5802worker2 # [ 93.172507] nix-grpc-daemon[1233]: time=2026-09-22T10:59:36.120Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5803worker2 # [ 93.174836] nix-grpc-daemon[1233]: time=2026-09-22T10:59:36.121Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5804lb # [ 93.198530] niks3-server[1104]: 2026/09/22 10:59:35 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5805lb # [ 93.201836] niks3-server[1104]: 2026/09/22 10:59:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5806lb # [ 93.206243] niks3-server[1104]: 2026/09/22 10:59:35 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5807lb # [ 93.210760] niks3-server[1104]: 2026/09/22 10:59:35 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5808lb # [ 93.214173] niks3-server[1104]: 2026/09/22 10:59:35 INFO Signed narinfos id=34 count=15809worker2 # [ 93.202715] nix-grpc-daemon[1233]: time=2026-09-22T10:59:36.150Z level=INFO msg="Uploading 1 narinfos"5810lb # [ 93.219237] niks3-server[1104]: 2026/09/22 10:59:35 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5811lb # [ 93.224969] niks3-server[1104]: 2026/09/22 10:59:35 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5812lb # [ 93.230085] niks3-server[1104]: 2026/09/22 10:59:35 INFO Completed upload id=345813worker2 # [ 93.219729] nix-grpc-daemon[1233]: time=2026-09-22T10:59:36.167Z level=INFO msg="Upload complete. (153ms)"5814worker2 # [ 93.222964] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55402 duration_s=0 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15815worker2 # [ 93.229797] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:36Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05816worker2 # [ 93.231753] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:36Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15817worker2 # [ 93.236112] nix-daemon[851]: reaped child process 1315, status = succeeded5818worker2 # [ 93.273981] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554025819worker2 # [ 93.283543] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:554025820worker2 # [ 93.292562] nix-grpc-daemon[1233]: time=2026-09-22T10:59:36.240Z level=INFO msg="All 2 paths already cached"5821client # copying 1 paths to 'lb:50051'...5822client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5823worker2 # [ 93.299494] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:554025824worker2 # [ 93.308147] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5825worker2 # [ 93.395874] nix-daemon[851]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5826worker2 # [ 93.411392] nix-daemon[851]: reaped child process 1345, status = succeeded5827worker2 # [ 93.492972] nix-daemon[851]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5828worker2 # [ 93.506621] nix-daemon[851]: reaped child process 1350, status = succeeded5829lb # [ 93.526547] niks3-server[1104]: 2026/09/22 10:59:35 INFO Received uploads request method=POST path=/api/pending_closures5830worker2 # [ 93.524339] nix-grpc-daemon[1233]: time=2026-09-22T10:59:36.472Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5831worker2 # [ 93.526261] nix-grpc-daemon[1233]: time=2026-09-22T10:59:36.473Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5832lb # [ 93.560731] niks3-server[1104]: 2026/09/22 10:59:35 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5833lb # [ 93.568556] niks3-server[1104]: 2026/09/22 10:59:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5834lb # [ 93.575992] niks3-server[1104]: 2026/09/22 10:59:35 INFO Signed narinfos id=35 count=15835worker2 # [ 93.567559] nix-grpc-daemon[1233]: time=2026-09-22T10:59:36.515Z level=INFO msg="Uploading 1 narinfos"5836lb # [ 93.581945] niks3-server[1104]: 2026/09/22 10:59:35 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5837lb # [ 93.592429] niks3-server[1104]: 2026/09/22 10:59:35 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5838lb # [ 93.602863] niks3-server[1104]: 2026/09/22 10:59:35 INFO Completed upload id=355839worker2 # [ 93.591598] nix-grpc-daemon[1233]: time=2026-09-22T10:59:36.539Z level=INFO msg="Upload complete. (275ms)"5840lb # [ 93.605221] niks3-server[1104]: 2026/09/22 10:59:35 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5841worker2 # [ 93.595813] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55402 duration_s=0 paths=1 nar_bytes_in=7845842worker2 # [ 93.602305] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:554025843worker2 # [ 93.614106] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5844worker2 # [ 93.618322] nix-daemon[851]: reaped child process 1340, status = succeeded5845client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5846worker2 # [ 93.623492] nix-daemon[851]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5847client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5848client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5849client # farm-top> LOG-farm-top-after-niks3-restart5850worker2 # [ 94.842096] nix-daemon[851]: reaped child process 1354, status = succeeded5851worker2 # [ 94.917092] nix-daemon[851]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)5852worker2 # [ 94.930420] nix-daemon[851]: reaped child process 1373, status = succeeded5853lb # [ 94.948781] niks3-server[1104]: 2026/09/22 10:59:37 INFO Received uploads request method=POST path=/api/pending_closures5854worker2 # [ 94.943141] nix-grpc-daemon[1233]: time=2026-09-22T10:59:37.890Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5855worker2 # [ 94.944632] nix-grpc-daemon[1233]: time=2026-09-22T10:59:37.892Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5856lb # [ 94.969125] niks3-server[1104]: 2026/09/22 10:59:37 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5857lb # [ 94.975395] niks3-server[1104]: 2026/09/22 10:59:37 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5858lb # [ 94.983113] niks3-server[1104]: 2026/09/22 10:59:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5859lb # [ 94.984888] niks3-server[1104]: 2026/09/22 10:59:37 INFO Signed narinfos id=36 count=15860worker2 # [ 94.973515] nix-grpc-daemon[1233]: time=2026-09-22T10:59:37.921Z level=INFO msg="Uploading 1 narinfos"5861lb # [ 94.989137] niks3-server[1104]: 2026/09/22 10:59:37 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5862lb # [ 94.994055] niks3-server[1104]: 2026/09/22 10:59:37 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5863lb # [ 94.997935] niks3-server[1104]: 2026/09/22 10:59:37 INFO Completed upload id=365864worker2 # [ 94.986378] nix-grpc-daemon[1233]: time=2026-09-22T10:59:37.934Z level=INFO msg="Upload complete. (156ms)"5865lb # [ 95.001255] niks3-server[1104]: 2026/09/22 10:59:37 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5866worker2 # [ 94.989636] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55402 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15867worker2 # [ 94.997162] nix-daemon[851]: reaped child process 1352, status = succeeded5868client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.03 seconds)5869client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5870client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.10 seconds)5871(finished: subtest: a niks3 restart does not cost the leader its role, in 5.48 seconds)5872subtest: scheduler down: the other node takes the lock and keeps it5873lb: must succeed: curl -sf localhost:9901/clusters5874lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5875lb: must succeed: curl -sf localhost:9901/clusters5876lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5877lb: must succeed: curl -sf localhost:9901/clusters5878lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5879lb: must succeed: curl -sf localhost:9901/clusters5880lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5881worker2 # [ 95.245758] systemd[1]: Stopping Nix worker-protocol over gRPC...5882worker2 # [ 95.773600] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:38Z level=info event=signal_exit5883lb # [ 96.228550] envoy[908]: conn peer=[2001:db8:1::1]:35822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=15975884lb # [ 96.232600] envoy[908]: conn peer=[2001:db8:1::1]:35830 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9705 tx=4044 ms=48735885lb # [ 96.718455] niks3-server[1104]: 2026/09/22 10:59:39 INFO lead: released remote=192.168.1.4:375705886worker1 # [ 97.695748] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:40Z level=info event=scheduler_restarting addr=lb:500515887worker2 # [ 97.711593] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:40Z level=info event=scheduler_restarting5888worker2 # [ 97.715546] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:40Z level=info event=scheduler_restarting addr=lb:500515889worker2 # [ 97.719178] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:553965890worker2 # [ 97.722829] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:40Z level=info event=worker_gone addr=192.168.1.4:500515891worker2 # [ 97.726247] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:553965892worker2 # [ 97.729867] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:40Z level=info event=worker_gone addr=192.168.1.3:500515893worker2 # [ 97.746428] nix-daemon[851]: reaped child process 1225, status = succeeded5894worker2 # [ 97.816718] nix-grpc-daemon[1173]: E0000 00:00:1790074780.764052 1218 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5895worker2 # [ 97.821763] nix-grpc-daemon[1173]: E0000 00:00:1790074780.764342 1218 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5896worker1 # [ 97.895398] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_connected addr=lb:500515897worker1 # [ 97.899387] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5898worker2 # [ 97.914740] nix-grpc-daemon[1173]: ts=2026-09-22T10:59:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5899worker1 # [ 98.100218] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_connected addr=lb:500515900worker1 # [ 98.104289] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5901worker1 # [ 98.304988] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_connected addr=lb:500515902worker1 # [ 98.309192] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5903worker1 # [ 98.509859] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_connected addr=lb:500515904worker1 # [ 98.513794] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5905worker1 # [ 98.716135] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_connected addr=lb:500515906worker1 # [ 98.719333] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5907worker1 # [ 98.916763] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_connected addr=lb:500515908worker1 # [ 98.918615] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5909worker1 # [ 99.119586] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_connected addr=lb:500515910worker1 # [ 99.123511] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5911worker1 # [ 99.324419] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_connected addr=lb:500515912worker1 # [ 99.328356] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5913worker1 # [ 99.529200] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_connected addr=lb:500515914worker1 # [ 99.533181] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5915worker1 # [ 99.733697] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_connected addr=lb:500515916worker1 # [ 99.735432] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5917worker2 # [ 99.922737] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5918worker2 # [ 99.925900] systemd[1]: Stopped Nix worker-protocol over gRPC.5919worker2 # [ 99.929518] systemd[1]: nix-grpc-daemon.service: Consumed 1.094s CPU time over 15.255s wall clock time, 39M memory peak, 45.9K incoming IP traffic, 83.2K outgoing IP traffic.5920worker2 # [ 99.941084] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5921worker2 # [ 99.945412] systemd[1]: Closed Nix worker-protocol over gRPC.5922worker1 # [ 99.935988] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_connected addr=lb:500515923worker1 # [ 99.938099] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5924worker2 # [ 99.948505] systemd[1]: nix-grpc-daemon.socket: Consumed 356us CPU time over 16.059s wall clock time, 256K memory peak, 37.3K incoming IP traffic, 27.1K outgoing IP traffic.5925worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5926worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.03 seconds)5927worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5928worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5929worker1 # [ 100.138567] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_connected addr=lb:500515930worker1 # [ 100.140340] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5931worker1 # [ 100.340888] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_connected addr=lb:500515932worker1 # [ 100.342707] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5933worker1 # [ 100.543228] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_connected addr=lb:500515934worker1 # [ 100.545033] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5935worker1 # [ 100.686217] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_take_over5936lb # [ 100.711462] niks3-server[1104]: 2026/09/22 10:59:43 INFO lead: acquired remote=192.168.1.3:370425937worker1 # [ 100.746655] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_connected addr=lb:500515938worker1 # [ 100.750331] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5939worker1 # [ 100.948380] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=scheduler_connected addr=lb:500515940worker1 # [ 100.952423] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5941worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5942worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5943worker1 # [ 101.152991] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=scheduler_connected addr=lb:500515944worker1 # [ 101.156938] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5945worker1 # [ 101.357631] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=scheduler_connected addr=lb:500515946worker1 # [ 101.361644] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5947worker1 # [ 101.562532] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=scheduler_connected addr=lb:500515948worker1 # [ 101.566474] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5949worker1 # [ 101.767886] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=scheduler_connected addr=lb:500515950worker1 # [ 101.775090] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:580485951worker1 # [ 101.778760] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05952worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5953worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5954client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25955worker1 # [ 102.235195] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:580625956worker1 # [ 102.246775] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5957client # don't know how to build these paths:5958client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5959worker1 # [ 102.261266] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580625960worker1 # [ 102.271193] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 1/5)5961worker1 # [ 102.283925] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)5962worker1 # [ 102.447757] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 245 ms (attempt 3/5)5963worker1 # [ 102.694703] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 4/5)5964worker1 # [ 102.735460] nix-grpc-daemon[1304]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5965worker1 # [ 102.738949] nix-grpc-daemon[1304]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5966worker1 # [ 102.761934] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:45Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05967worker1 # [ 102.764058] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:45Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15968worker1 # [ 102.807788] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580625969worker1 # [ 102.820761] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580625970client # copying 1 paths to 'lb:50051'...5971client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5972worker1 # [ 102.832351] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580625973worker1 # [ 102.840598] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5974worker1 # [ 102.861982] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:46Z level=info event=niks3_push_started pid=13835975worker1 # [ 102.961208] nix-grpc-daemon[1383]: time=2026-09-22T10:59:46.140Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5976worker1 # [ 103.045374] nix-daemon[864]: accepted connection from pid 1396, user nix-grpc-daemon (trusted)5977worker1 # [ 103.057091] nix-daemon[864]: reaped child process 1399, status = succeeded5978worker1 # [ 103.137639] nix-daemon[864]: accepted connection from pid 1401, user nix-grpc-daemon (trusted)5979worker1 # [ 103.150351] nix-daemon[864]: reaped child process 1404, status = succeeded5980lb # [ 103.181304] niks3-server[1104]: 2026/09/22 10:59:45 INFO Received uploads request method=POST path=/api/pending_closures5981worker1 # [ 103.165755] nix-grpc-daemon[1383]: time=2026-09-22T10:59:46.345Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5982worker1 # [ 103.168766] nix-grpc-daemon[1383]: time=2026-09-22T10:59:46.346Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5983lb # [ 103.216764] niks3-server[1104]: 2026/09/22 10:59:45 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5984lb # [ 103.225836] niks3-server[1104]: 2026/09/22 10:59:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5985worker1 # [ 103.208590] nix-grpc-daemon[1383]: time=2026-09-22T10:59:46.387Z level=INFO msg="Uploading 1 narinfos"5986lb # [ 103.233405] niks3-server[1104]: 2026/09/22 10:59:45 INFO Signed narinfos id=37 count=15987lb # [ 103.245877] niks3-server[1104]: 2026/09/22 10:59:45 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5988lb # [ 103.252157] niks3-server[1104]: 2026/09/22 10:59:45 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5989worker1 # [ 103.233563] nix-grpc-daemon[1383]: time=2026-09-22T10:59:46.412Z level=INFO msg="Upload complete. (269ms)"5990lb # [ 103.258116] niks3-server[1104]: 2026/09/22 10:59:45 INFO Completed upload id=375991worker1 # [ 103.237074] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58062 duration_s=0 paths=1 nar_bytes_in=5205992lb # [ 103.264249] niks3-server[1104]: 2026/09/22 10:59:45 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5993worker1 # [ 103.239651] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580625994worker1 # [ 103.245615] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5995worker1 # [ 103.247888] nix-daemon[864]: reaped child process 1381, status = succeeded5996worker1 # [ 103.252861] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5997client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5998client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5999client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6000client # farm-a> LOG-farm-a-on-standby6001worker1 # [ 104.470541] nix-daemon[864]: reaped child process 1408, status = succeeded6002worker1 # [ 104.539574] nix-daemon[864]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6003worker1 # [ 104.552401] nix-daemon[864]: reaped child process 1427, status = succeeded6004lb # [ 104.583552] niks3-server[1104]: 2026/09/22 10:59:46 INFO Received uploads request method=POST path=/api/pending_closures6005worker1 # [ 104.563308] nix-grpc-daemon[1383]: time=2026-09-22T10:59:47.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6006worker1 # [ 104.564945] nix-grpc-daemon[1383]: time=2026-09-22T10:59:47.744Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6007lb # [ 104.620677] niks3-server[1104]: 2026/09/22 10:59:46 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6008lb # [ 104.631516] niks3-server[1104]: 2026/09/22 10:59:46 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6009lb # [ 104.639787] niks3-server[1104]: 2026/09/22 10:59:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6010worker1 # [ 104.619721] nix-grpc-daemon[1383]: time=2026-09-22T10:59:47.799Z level=INFO msg="Uploading 1 narinfos"6011lb # [ 104.643954] niks3-server[1104]: 2026/09/22 10:59:46 INFO Signed narinfos id=38 count=16012lb # [ 104.655443] niks3-server[1104]: 2026/09/22 10:59:46 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6013lb # [ 104.660350] niks3-server[1104]: 2026/09/22 10:59:46 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6014lb # [ 104.663617] niks3-server[1104]: 2026/09/22 10:59:46 INFO Completed upload id=386015worker1 # [ 104.639227] nix-grpc-daemon[1383]: time=2026-09-22T10:59:47.818Z level=INFO msg="Upload complete. (179ms)"6016lb # [ 104.665824] niks3-server[1104]: 2026/09/22 10:59:46 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6017worker1 # [ 104.643362] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58062 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16018worker1 # [ 104.651123] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:47Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06019worker1 # [ 104.653674] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:47Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=16020worker1 # [ 104.658380] nix-daemon[864]: reaped child process 1406, status = succeeded6021worker1 # [ 104.694673] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580626022worker1 # [ 104.703757] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580626023worker1 # [ 104.712701] nix-grpc-daemon[1383]: time=2026-09-22T10:59:47.892Z level=INFO msg="All 1 paths already cached"6024client # copying 1 paths to 'lb:50051'...6025client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6026worker1 # [ 104.718519] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580626027worker1 # [ 104.725129] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)6028worker1 # [ 104.808431] nix-daemon[864]: accepted connection from pid 1433, user nix-grpc-daemon (trusted)6029worker1 # [ 104.820651] nix-daemon[864]: reaped child process 1436, status = succeeded6030worker1 # [ 104.901168] nix-daemon[864]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)6031worker1 # [ 104.914309] nix-daemon[864]: reaped child process 1441, status = succeeded6032lb # [ 104.943778] niks3-server[1104]: 2026/09/22 10:59:47 INFO Received uploads request method=POST path=/api/pending_closures6033worker1 # [ 104.927671] nix-grpc-daemon[1383]: time=2026-09-22T10:59:48.106Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6034worker1 # [ 104.929752] nix-grpc-daemon[1383]: time=2026-09-22T10:59:48.108Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6035lb # [ 104.975475] niks3-server[1104]: 2026/09/22 10:59:47 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6036lb # [ 104.985364] niks3-server[1104]: 2026/09/22 10:59:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6037worker1 # [ 104.966962] nix-grpc-daemon[1383]: time=2026-09-22T10:59:48.146Z level=INFO msg="Uploading 1 narinfos"6038lb # [ 104.991582] niks3-server[1104]: 2026/09/22 10:59:47 INFO Signed narinfos id=39 count=16039lb # [ 104.996641] niks3-server[1104]: 2026/09/22 10:59:47 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6040lb # [ 105.006685] niks3-server[1104]: 2026/09/22 10:59:47 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6041lb # [ 105.010192] niks3-server[1104]: 2026/09/22 10:59:47 INFO Completed upload id=396042worker1 # [ 104.986766] nix-grpc-daemon[1383]: time=2026-09-22T10:59:48.166Z level=INFO msg="Upload complete. (256ms)"6043lb # [ 105.013262] niks3-server[1104]: 2026/09/22 10:59:47 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6044worker1 # [ 104.988799] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58062 duration_s=0 paths=1 nar_bytes_in=6406045worker1 # [ 104.993396] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580626046worker1 # [ 104.998776] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)6047worker1 # [ 105.001075] nix-daemon[864]: reaped child process 1431, status = succeeded6048worker1 # [ 105.006228] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)6049client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6050client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6051client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6052client # farm-b> LOG-farm-b-on-standby6053worker1 # [ 106.044685] nix-daemon[864]: reaped child process 1445, status = succeeded6054worker1 # [ 106.126213] nix-daemon[864]: accepted connection from pid 1461, user nix-grpc-daemon (trusted)6055worker1 # [ 106.137468] nix-daemon[864]: reaped child process 1464, status = succeeded6056lb # [ 106.169541] niks3-server[1104]: 2026/09/22 10:59:48 INFO Received uploads request method=POST path=/api/pending_closures6057worker1 # [ 106.152970] nix-grpc-daemon[1383]: time=2026-09-22T10:59:49.332Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6058worker1 # [ 106.154626] nix-grpc-daemon[1383]: time=2026-09-22T10:59:49.333Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6059lb # [ 106.205134] niks3-server[1104]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6060lb # [ 106.215479] niks3-server[1104]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6061lb # [ 106.225386] niks3-server[1104]: 2026/09/22 10:59:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6062worker1 # [ 106.208692] nix-grpc-daemon[1383]: time=2026-09-22T10:59:49.387Z level=INFO msg="Uploading 1 narinfos"6063lb # [ 106.233534] envoy[908]: conn peer=[2001:db8:1::4]:34584 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5880 tx=625 ms=130956064lb # [ 106.236718] niks3-server[1104]: 2026/09/22 10:59:48 INFO Signed narinfos id=40 count=16065lb # [ 106.239482] niks3-server[1104]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6066lb # [ 106.249974] niks3-server[1104]: 2026/09/22 10:59:48 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6067lb # [ 106.252693] niks3-server[1104]: 2026/09/22 10:59:48 INFO Completed upload id=406068worker1 # [ 106.227879] nix-grpc-daemon[1383]: time=2026-09-22T10:59:49.407Z level=INFO msg="Upload complete. (214ms)"6069lb # [ 106.256326] niks3-server[1104]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6070worker1 # [ 106.233528] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58062 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16071worker1 # [ 106.246408] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:49Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06072worker1 # [ 106.249937] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:49Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16073worker1 # [ 106.259212] nix-daemon[864]: reaped child process 1443, status = succeeded6074worker1 # [ 106.289548] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580626075worker1 # [ 106.301547] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580626076worker1 # [ 106.313461] nix-grpc-daemon[1383]: time=2026-09-22T10:59:49.492Z level=INFO msg="All 2 paths already cached"6077client # copying 1 paths to 'lb:50051'...6078client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6079worker1 # [ 106.320889] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580626080worker1 # [ 106.330337] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)6081worker1 # [ 106.418072] nix-daemon[864]: accepted connection from pid 1470, user nix-grpc-daemon (trusted)6082worker1 # [ 106.431754] nix-daemon[864]: reaped child process 1473, status = succeeded6083worker1 # [ 106.502819] nix-daemon[864]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6084worker1 # [ 106.514611] nix-daemon[864]: reaped child process 1478, status = succeeded6085lb # [ 106.545150] niks3-server[1104]: 2026/09/22 10:59:48 INFO Received uploads request method=POST path=/api/pending_closures6086worker1 # [ 106.528542] nix-grpc-daemon[1383]: time=2026-09-22T10:59:49.707Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6087worker1 # [ 106.529906] nix-grpc-daemon[1383]: time=2026-09-22T10:59:49.707Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6088lb # [ 106.574553] niks3-server[1104]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6089lb # [ 106.584131] niks3-server[1104]: 2026/09/22 10:59:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6090lb # [ 106.589340] niks3-server[1104]: 2026/09/22 10:59:48 INFO Signed narinfos id=41 count=16091worker1 # [ 106.567646] nix-grpc-daemon[1383]: time=2026-09-22T10:59:49.746Z level=INFO msg="Uploading 1 narinfos"6092lb # [ 106.597200] niks3-server[1104]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6093lb # [ 106.610481] niks3-server[1104]: 2026/09/22 10:59:48 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6094lb # [ 106.616636] niks3-server[1104]: 2026/09/22 10:59:48 INFO Completed upload id=416095worker1 # [ 106.594187] nix-grpc-daemon[1383]: time=2026-09-22T10:59:49.773Z level=INFO msg="Upload complete. (252ms)"6096lb # [ 106.622222] niks3-server[1104]: 2026/09/22 10:59:48 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6097worker1 # [ 106.597963] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58062 duration_s=0 paths=1 nar_bytes_in=7686098worker1 # [ 106.603713] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580626099worker1 # [ 106.616462] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)6100worker1 # [ 106.617809] nix-daemon[864]: reaped child process 1468, status = succeeded6101client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6102worker1 # [ 106.622875] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)6103client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6104client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6105client # farm-top> LOG-farm-top-on-standby6106worker1 # [ 107.664457] nix-daemon[864]: reaped child process 1482, status = succeeded6107worker1 # [ 107.740279] nix-daemon[864]: accepted connection from pid 1498, user nix-grpc-daemon (trusted)6108worker1 # [ 107.751610] nix-daemon[864]: reaped child process 1501, status = succeeded6109lb # [ 107.784264] niks3-server[1104]: 2026/09/22 10:59:50 INFO Received uploads request method=POST path=/api/pending_closures6110worker1 # [ 107.766403] nix-grpc-daemon[1383]: time=2026-09-22T10:59:50.945Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6111worker1 # [ 107.768063] nix-grpc-daemon[1383]: time=2026-09-22T10:59:50.947Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6112lb # [ 107.811980] niks3-server[1104]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6113lb # [ 107.825209] niks3-server[1104]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6114lb # [ 107.831790] niks3-server[1104]: 2026/09/22 10:59:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6115lb # [ 107.833551] niks3-server[1104]: 2026/09/22 10:59:50 INFO Signed narinfos id=42 count=16116worker1 # [ 107.810468] nix-grpc-daemon[1383]: time=2026-09-22T10:59:50.989Z level=INFO msg="Uploading 1 narinfos"6117lb # [ 107.837098] niks3-server[1104]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6118lb # [ 107.843602] niks3-server[1104]: 2026/09/22 10:59:50 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6119lb # [ 107.848075] niks3-server[1104]: 2026/09/22 10:59:50 INFO Completed upload id=426120worker1 # [ 107.824676] nix-grpc-daemon[1383]: time=2026-09-22T10:59:51.003Z level=INFO msg="Upload complete. (194ms)"6121lb # [ 107.850948] niks3-server[1104]: 2026/09/22 10:59:50 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6122worker1 # [ 107.829420] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58062 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16123worker1 # [ 107.840682] nix-daemon[864]: reaped child process 1480, status = succeeded6124client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.78 seconds)6125client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6126client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6127worker2 # [ 108.029894] systemd[1]: Listening on Nix worker-protocol over gRPC.6128worker2 # [ 108.036909] systemd[1]: Starting scheduler-token.service...6129worker2 # [ 108.078992] systemd[1]: scheduler-token.service: Deactivated successfully.6130worker2 # [ 108.080735] systemd[1]: Finished scheduler-token.service.6131worker2 # [ 108.082323] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 44ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6132worker2 # [ 108.086591] systemd[1]: Starting Nix worker-protocol over gRPC...6133worker2 # [ 108.161381] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6134worker2 # [ 108.163489] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6135worker2 # [ 108.167144] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16136worker2 # [ 108.171117] nix-grpc-daemon[1394]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6137worker2 # [ 108.174181] nix-grpc-daemon[1394]: E0000 00:00:1790074791.118558 1394 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6138worker2 # [ 108.179738] nix-grpc-daemon[1394]: E0000 00:00:1790074791.121783 1394 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6139worker2 # [ 108.183603] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6140worker2 # [ 108.189088] systemd[1]: Started Nix worker-protocol over gRPC.6141worker2 # [ 108.189968] nix-grpc-daemon[1394]: E0000 00:00:1790074791.135071 1433 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6142worker2 # [ 108.202118] nix-grpc-daemon[1394]: E0000 00:00:1790074791.149063 1433 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6143worker2 # [ 108.204244] nix-grpc-daemon[1394]: E0000 00:00:1790074791.149190 1433 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6144lb: must succeed: curl -sf localhost:9901/clusters6145worker2 # [ 108.216567] nix-grpc-daemon[1394]: E0000 00:00:1790074791.164248 1433 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6146worker2 # [ 108.224327] nix-grpc-daemon[1394]: ts=2026-09-22T10:59:51Z level=info event=scheduler_connected addr=lb:500516147worker1 # [ 108.218176] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=info event=oidc_jwks provider=mock keys=16148lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6149worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6150worker1 # [ 108.221921] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:580486151worker1 # [ 108.225963] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06152worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6153lb: must succeed: curl -sf localhost:9901/clusters6154lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6155lb: must succeed: curl -sf localhost:9901/clusters6156lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6157worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6158worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6159(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.24 seconds)6160subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6161client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6162client # [ 108.346885] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6163client # Running as unit: rs.service; invocation ID: 0a46383339a74f398511a6e466636a756164client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.03 seconds)6165client # [ 108.472041] nix[1295]: don't know how to build these paths:6166client # [ 108.472928] nix[1295]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6167worker1 # [ 108.491607] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:580626168worker1 # [ 108.501854] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06169worker1 # [ 108.507240] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16170worker1 # [ 108.545831] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580626171worker1 # [ 108.557457] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580626172client # [ 108.546146] nix[1295]: copying 1 paths to 'lb:50051'...6173client # [ 108.546992] nix[1295]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6174worker1 # [ 108.567815] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580626175worker1 # [ 108.578162] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)6176worker1 # [ 108.675100] nix-daemon[864]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6177worker1 # [ 108.687733] nix-daemon[864]: reaped child process 1517, status = succeeded6178worker1 # [ 108.766435] nix-daemon[864]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6179worker1 # [ 108.778684] nix-daemon[864]: reaped child process 1522, status = succeeded6180lb # [ 108.810520] niks3-server[1104]: 2026/09/22 10:59:51 INFO Received uploads request method=POST path=/api/pending_closures6181worker1 # [ 108.794039] nix-grpc-daemon[1383]: time=2026-09-22T10:59:51.973Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6182worker1 # [ 108.795821] nix-grpc-daemon[1383]: time=2026-09-22T10:59:51.974Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6183lb # [ 108.837648] niks3-server[1104]: 2026/09/22 10:59:51 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6184lb # [ 108.850120] niks3-server[1104]: 2026/09/22 10:59:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6185worker1 # [ 108.830529] nix-grpc-daemon[1383]: time=2026-09-22T10:59:52.009Z level=INFO msg="Uploading 1 narinfos"6186lb # [ 108.855410] niks3-server[1104]: 2026/09/22 10:59:51 INFO Signed narinfos id=43 count=16187lb # [ 108.870606] niks3-server[1104]: 2026/09/22 10:59:51 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6188lb # [ 108.879818] niks3-server[1104]: 2026/09/22 10:59:51 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6189lb # [ 108.884775] niks3-server[1104]: 2026/09/22 10:59:51 INFO Completed upload id=436190worker1 # [ 108.859780] nix-grpc-daemon[1383]: time=2026-09-22T10:59:52.039Z level=INFO msg="Upload complete. (270ms)"6191lb # [ 108.887712] niks3-server[1104]: 2026/09/22 10:59:51 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6192worker1 # [ 108.864511] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58062 duration_s=0 paths=1 nar_bytes_in=4166193worker1 # [ 108.869420] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580626194worker1 # [ 108.881122] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)6195worker1 # [ 108.886121] nix-daemon[864]: reaped child process 1512, status = succeeded6196worker1 # [ 108.889455] nix-daemon[864]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)6197client # [ 108.868426] nix[1295]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6198lb: must succeed: curl -sf localhost:9901/clusters6199lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6200lb: must succeed: curl -sf localhost:9901/clusters6201lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6202lb: must succeed: curl -sf localhost:9901/clusters6203lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6204lb: must succeed: curl -sf localhost:9901/clusters6205lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6206worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6207worker2 # [ 111.010470] systemd-journald[284]: Received client request to rotate journal, rotating.6208worker2 # [ 111.028229] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/fb857a668fa94e63b944f3729bde791e.6209worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6210worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6211worker1 # [ 110.923567] systemd[1]: Stopping Nix worker-protocol over gRPC...6212worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6213client: must succeed: sleep 5.06214worker1 # [ 111.574337] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:54Z level=info event=draining6215client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6216worker1 # [ 116.108850] nix-daemon[864]: reaped child process 1526, status = succeeded6217worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6218worker1 # [ 116.181300] nix-daemon[864]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6219worker1 # [ 116.194139] nix-daemon[864]: reaped child process 1570, status = succeeded6220lb # [ 116.225820] niks3-server[1104]: 2026/09/22 10:59:58 INFO Received uploads request method=POST path=/api/pending_closures6221lb # [ 116.233421] envoy[908]: conn peer=[2001:db8:1::1]:59008 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9586 tx=4064 ms=56116222worker1 # [ 116.213485] nix-grpc-daemon[1383]: time=2026-09-22T10:59:59.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6223worker1 # [ 116.215224] nix-grpc-daemon[1383]: time=2026-09-22T10:59:59.394Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6224lb # [ 116.270476] niks3-server[1104]: 2026/09/22 10:59:58 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6225lb # [ 116.280650] niks3-server[1104]: 2026/09/22 10:59:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6226lb # [ 116.286442] niks3-server[1104]: 2026/09/22 10:59:58 INFO Signed narinfos id=44 count=16227worker1 # [ 116.264856] nix-grpc-daemon[1383]: time=2026-09-22T10:59:59.443Z level=INFO msg="Uploading 1 narinfos"6228lb # [ 116.295502] niks3-server[1104]: 2026/09/22 10:59:58 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6229lb # [ 116.302671] niks3-server[1104]: 2026/09/22 10:59:58 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6230lb # [ 116.309961] niks3-server[1104]: 2026/09/22 10:59:58 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6231lb # [ 116.313940] niks3-server[1104]: 2026/09/22 10:59:58 INFO Completed upload id=446232worker1 # [ 116.289216] nix-grpc-daemon[1383]: time=2026-09-22T10:59:59.468Z level=INFO msg="Upload complete. (188ms)"6233lb # [ 116.315570] niks3-server[1104]: 2026/09/22 10:59:58 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6234worker1 # [ 116.293620] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58062 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16235worker1 # [ 116.300892] nix-daemon[864]: reaped child process 1524, status = succeeded6236client # [ 116.296696] systemd[1]: rs.service: Deactivated successfully.6237client # [ 116.299114] systemd[1]: rs.service: Consumed 123ms CPU time over 7.951s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6238worker1 # [ 116.575911] nix-grpc-daemon[1304]: ts=2026-09-22T10:59:59Z level=info event=signal_exit6239lb # [ 116.713246] niks3-server[1104]: 2026/09/22 10:59:59 INFO lead: released remote=192.168.1.3:370426240worker2 # [ 117.702693] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:00Z level=info event=scheduler_restarting addr=lb:500516241worker1 # [ 117.693697] nix-grpc-daemon[1304]: ts=2026-09-22T11:00:00Z level=info event=scheduler_restarting6242worker1 # [ 117.697384] nix-grpc-daemon[1304]: ts=2026-09-22T11:00:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:580486243worker1 # [ 117.701587] nix-grpc-daemon[1304]: ts=2026-09-22T11:00:00Z level=info event=worker_gone addr=192.168.1.4:500516244worker1 # [ 117.704758] nix-grpc-daemon[1304]: ts=2026-09-22T11:00:00Z level=info event=scheduler_restarting addr=lb:500516245worker1 # [ 117.708182] nix-grpc-daemon[1304]: ts=2026-09-22T11:00:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:580486246worker1 # [ 117.711971] nix-grpc-daemon[1304]: ts=2026-09-22T11:00:00Z level=info event=worker_gone addr=192.168.1.3:500516247worker1 # [ 117.729881] nix-daemon[864]: reaped child process 1373, status = succeeded6248worker2 # [ 117.903668] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:00Z level=info event=scheduler_connected addr=lb:500516249worker2 # [ 117.907767] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6250worker1 # [ 117.904268] nix-grpc-daemon[1304]: ts=2026-09-22T11:00:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6251worker2 # [ 118.108968] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_connected addr=lb:500516252worker2 # [ 118.112910] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6253lb # [ 118.200889] niks3-server[1104]: 2026/09/22 11:00:00 INFO lead: acquired remote=192.168.1.4:464406254worker2 # [ 118.192373] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_take_over6255worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.11 seconds)6256client: waiting for success: ! systemctl is-active rs6257client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6258client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6259worker2 # [ 118.314115] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_connected addr=lb:500516260worker1 # [ 118.302716] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6261worker1 # [ 118.305835] systemd[1]: Stopped Nix worker-protocol over gRPC.6262worker2 # [ 118.318142] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6263worker1 # [ 118.308535] systemd[1]: nix-grpc-daemon.service: Consumed 1.235s CPU time over 28.845s wall clock time, 39.1M memory peak, 50.1K incoming IP traffic, 98.4K outgoing IP traffic.6264client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6265client: must fail: journalctl -u rs | grep -q 'reconnecting'6266worker1 # [ 118.323375] systemd[1]: Starting Nix worker-protocol over gRPC...6267client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6268lb: must succeed: curl -sf localhost:9901/clusters6269lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6270worker1 # [ 118.406222] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6271worker1 # [ 118.408217] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6272worker1 # [ 118.411342] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16273worker1 # [ 118.415751] nix-grpc-daemon[1577]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6274worker1 # [ 118.417143] nix-grpc-daemon[1577]: E0000 00:00:1790074801.594850 1577 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6275worker1 # [ 118.423057] nix-grpc-daemon[1577]: E0000 00:00:1790074801.600201 1577 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6276worker1 # [ 118.426281] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6277worker1 # [ 118.427893] systemd[1]: Started Nix worker-protocol over gRPC.6278worker1 # [ 118.431069] nix-grpc-daemon[1577]: E0000 00:00:1790074801.610025 1618 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6279worker1 # [ 118.442847] nix-grpc-daemon[1577]: E0000 00:00:1790074801.622153 1618 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6280worker1 # [ 118.446141] nix-grpc-daemon[1577]: E0000 00:00:1790074801.624427 1618 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6281worker1 # [ 118.459978] nix-grpc-daemon[1577]: E0000 00:00:1790074801.639226 1618 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6282worker1 # [ 118.470240] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:01Z level=info event=scheduler_connected addr=lb:500516283worker1 # [ 118.472217] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6284worker2 # [ 118.519471] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_connected addr=lb:500516285worker2 # [ 118.523356] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6286worker1 # [ 118.672772] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:01Z level=info event=scheduler_connected addr=lb:500516287worker1 # [ 118.674497] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6288worker2 # [ 118.725727] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_connected addr=lb:500516289worker2 # [ 118.728946] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6290worker1 # [ 118.874956] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:02Z level=info event=scheduler_connected addr=lb:500516291worker2 # [ 118.894629] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:515126292worker2 # [ 118.899276] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06293worker2 # [ 118.926435] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=scheduler_connected addr=lb:500516294worker2 # [ 118.936154] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=oidc_jwks provider=mock keys=16295worker2 # [ 118.939931] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:515126296worker2 # [ 118.943931] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:01Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06297lb: must succeed: curl -sf localhost:9901/clusters6298lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6299worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6300worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6301(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.13 seconds)6302subtest: scheduler restart mid-build: build finishes, no double build6303client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6304client # [ 119.497699] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6305client # Running as unit: mid.service; invocation ID: d82e219eadd1409c93bcf795a6426d856306client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6307worker2 # [ 119.682179] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:515246308worker2 # [ 119.694247] nix-daemon[851]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)6309client # [ 119.673487] nix[1331]: don't know how to build these paths:6310client # [ 119.674474] nix[1331]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6311worker2 # [ 119.714419] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:02Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06312worker2 # [ 119.718813] nix-grpc-daemon[1394]: ts=2026-09-22T11:00:02Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16313worker1 # [ 119.763788] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329146314worker1 # [ 119.774776] nix-daemon[864]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6315worker1 # [ 119.790860] nix-grpc-daemon[1577]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 1/5)6316worker1 # [ 119.863283] nix-grpc-daemon[1577]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 200 ms (attempt 2/5)6317worker1 # [ 120.065469] nix-grpc-daemon[1577]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 225 ms (attempt 3/5)6318worker1 # [ 120.292432] nix-grpc-daemon[1577]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 383 ms (attempt 4/5)6319worker1 # [ 120.679117] nix-grpc-daemon[1577]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6320worker1 # [ 120.683210] nix-grpc-daemon[1577]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6321worker1 # [ 120.698308] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329146322client # [ 120.686415] nix[1331]: copying 1 paths to 'lb:50051'...6323client # [ 120.687933] nix[1331]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6324worker1 # [ 120.709577] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329146325worker1 # [ 120.715903] nix-daemon[864]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6326worker1 # [ 120.732633] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:03Z level=info event=niks3_push_started pid=16456327worker1 # [ 120.813427] nix-grpc-daemon[1645]: time=2026-09-22T11:00:03.992Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6328worker1 # [ 120.891513] nix-daemon[864]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6329worker1 # [ 120.905536] nix-daemon[864]: reaped child process 1660, status = succeeded6330worker1 # [ 120.977991] nix-daemon[864]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)6331worker1 # [ 120.991321] nix-daemon[864]: reaped child process 1665, status = succeeded6332lb # [ 121.020337] niks3-server[1104]: 2026/09/22 11:00:03 INFO Received uploads request method=POST path=/api/pending_closures6333worker1 # [ 121.004906] nix-grpc-daemon[1645]: time=2026-09-22T11:00:04.184Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6334worker1 # [ 121.007395] nix-grpc-daemon[1645]: time=2026-09-22T11:00:04.185Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6335lb # [ 121.050651] niks3-server[1104]: 2026/09/22 11:00:03 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6336lb # [ 121.062151] niks3-server[1104]: 2026/09/22 11:00:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6337lb # [ 121.069998] niks3-server[1104]: 2026/09/22 11:00:03 INFO Signed narinfos id=45 count=16338worker1 # [ 121.048572] nix-grpc-daemon[1645]: time=2026-09-22T11:00:04.227Z level=INFO msg="Uploading 1 narinfos"6339lb # [ 121.092115] niks3-server[1104]: 2026/09/22 11:00:03 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6340lb # [ 121.096133] niks3-server[1104]: 2026/09/22 11:00:03 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6341lb # [ 121.107596] niks3-server[1104]: 2026/09/22 11:00:03 INFO Completed upload id=456342worker1 # [ 121.083146] nix-grpc-daemon[1645]: time=2026-09-22T11:00:04.262Z level=INFO msg="Upload complete. (266ms)"6343lb # [ 121.111056] niks3-server[1104]: 2026/09/22 11:00:03 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6344worker1 # [ 121.089122] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=4246345worker1 # [ 121.096299] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329146346worker1 # [ 121.103566] nix-daemon[864]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6347worker1 # [ 121.114987] nix-daemon[864]: reaped child process 1643, status = succeeded6348worker1 # [ 121.118614] nix-daemon[864]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6349client # [ 121.096908] nix[1331]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6350lb: must succeed: curl -sf localhost:9901/clusters6351lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6352lb: must succeed: curl -sf localhost:9901/clusters6353lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6354worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6355worker2 # [ 123.096874] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1394 (nix-grpc-daemon) on client request.6356lb # [ 123.126248] niks3-server[1104]: 2026/09/22 11:00:05 INFO lead: released remote=192.168.1.4:464406357worker2 # [ 123.107930] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6358worker1 # [ 123.104096] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:06Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6359client # [ 123.084472] nix[1331]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6360worker2 # [ 123.127720] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6361worker2 # [ 123.129230] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6362worker2 # [ 123.130467] systemd[1]: nix-grpc-daemon.service: Consumed 211ms CPU time over 15.035s wall clock time, 13.6M memory peak, 11.3K incoming IP traffic, 21K outgoing IP traffic.6363worker2 # [ 123.143158] systemd[1]: Starting scheduler-token.service...6364worker2 # [ 123.145302] nix-daemon[851]: reaped child process 1503, status = succeeded6365worker2 # [ 123.173770] systemd[1]: scheduler-token.service: Deactivated successfully.6366worker2 # [ 123.174909] systemd[1]: Finished scheduler-token.service.6367worker2 # [ 123.176318] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 32ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6368worker2 # [ 123.179179] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6369worker2 # [ 123.182914] systemd[1]: Starting scheduler-token.service...6370worker2 # [ 123.211146] systemd[1]: scheduler-token.service: Deactivated successfully.6371worker2 # [ 123.212740] systemd[1]: Finished scheduler-token.service.6372worker2 # [ 123.215237] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 29ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6373worker2 # [ 123.218543] systemd[1]: Starting Nix worker-protocol over gRPC...6374worker2 # [ 123.296135] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6375worker2 # [ 123.298340] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6376worker2 # [ 123.301862] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16377worker2 # [ 123.306055] nix-grpc-daemon[1537]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6378worker2 # [ 123.308168] nix-grpc-daemon[1537]: E0000 00:00:1790074806.253548 1537 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6379worker2 # [ 123.312204] nix-grpc-daemon[1537]: E0000 00:00:1790074806.259098 1537 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6380worker1 # [ 123.304337] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:06Z level=info event=scheduler_connected addr=lb:500516381worker2 # [ 123.318517] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6382worker2 # [ 123.322255] systemd[1]: Started Nix worker-protocol over gRPC.6383lb # [ 123.337053] niks3-server[1104]: 2026/09/22 11:00:05 INFO lead: acquired remote=192.168.1.4:329986384worker2 # [ 123.326083] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=scheduler_take_over6385worker2 # [ 123.330228] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:526146386worker2 # [ 123.335101] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=16387worker2 # [ 123.337554] nix-grpc-daemon[1537]: E0000 00:00:1790074806.276159 1578 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6388worker2 # [ 123.346441] nix-grpc-daemon[1537]: E0000 00:00:1790074806.294095 1578 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6389worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.28 seconds)6390client: must succeed: sleep 5.06391worker2 # [ 123.350390] nix-grpc-daemon[1537]: E0000 00:00:1790074806.296360 1578 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6392worker2 # [ 123.361025] nix-grpc-daemon[1537]: E0000 00:00:1790074806.308684 1578 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6393worker2 # [ 123.368186] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=scheduler_connected addr=lb:500516394worker2 # [ 123.372432] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=oidc_jwks provider=mock keys=16395worker2 # [ 123.373778] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:526146396worker2 # [ 123.375421] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06397lb # [ 126.231337] envoy[908]: conn peer=[2001:db8:1::1]:44870 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2960 tx=1427 ms=78166398lb # [ 126.235698] envoy[908]: conn peer=[2001:db8:1::3]:45526 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5377 tx=1241 ms=282826399lb # [ 126.239634] envoy[908]: conn peer=[2001:db8:1::4]:55968 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6128 tx=495 ms=148956400client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6401worker1 # [ 128.537920] nix-daemon[864]: reaped child process 1669, status = succeeded6402client: waiting for success: ! systemctl is-active mid6403worker1 # [ 128.610934] nix-daemon[864]: accepted connection from pid 1706, user nix-grpc-daemon (trusted)6404worker1 # [ 128.623267] nix-daemon[864]: reaped child process 1709, status = succeeded6405lb # [ 128.655545] niks3-server[1104]: 2026/09/22 11:00:10 INFO Received uploads request method=POST path=/api/pending_closures6406worker1 # [ 128.640737] nix-grpc-daemon[1645]: time=2026-09-22T11:00:11.819Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6407worker1 # [ 128.642492] nix-grpc-daemon[1645]: time=2026-09-22T11:00:11.821Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6408lb # [ 128.703483] niks3-server[1104]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6409lb # [ 128.713041] niks3-server[1104]: 2026/09/22 11:00:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6410lb # [ 128.720445] niks3-server[1104]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6411lb # [ 128.730979] niks3-server[1104]: 2026/09/22 11:00:11 INFO Signed narinfos id=46 count=16412worker1 # [ 128.707148] nix-grpc-daemon[1645]: time=2026-09-22T11:00:11.885Z level=INFO msg="Uploading 1 narinfos"6413lb # [ 128.738223] niks3-server[1104]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6414lb # [ 128.741770] niks3-server[1104]: 2026/09/22 11:00:11 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6415lb # [ 128.748551] niks3-server[1104]: 2026/09/22 11:00:11 INFO Completed upload id=466416lb # [ 128.750883] niks3-server[1104]: 2026/09/22 11:00:11 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6417worker1 # [ 128.724199] nix-grpc-daemon[1645]: time=2026-09-22T11:00:11.903Z level=INFO msg="Upload complete. (193ms)"6418worker1 # [ 128.729854] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=7 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16419worker1 # [ 128.743465] nix-daemon[864]: reaped child process 1667, status = succeeded6420client # [ 128.742617] systemd[1]: mid.service: Deactivated successfully.6421client # [ 128.746350] systemd[1]: mid.service: Consumed 146ms CPU time over 9.248s wall clock time, 41.4M memory peak, 7.7K incoming IP traffic, 8.8K outgoing IP traffic.6422client: (finished: waiting for success: ! systemctl is-active mid, in 1.06 seconds)6423client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6424client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.02 seconds)6425(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.16 seconds)6426subtest: interrupting the client stops the build on the worker6427client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6428client # [ 129.628529] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6429client # Running as unit: intr.service; invocation ID: 9707c555508047eb90a91b9953e69ef56430client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6431worker1 # [ 129.777258] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:329146432client # [ 129.762438] nix[1367]: don't know how to build these paths:6433client # [ 129.763308] nix[1367]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6434worker2 # [ 129.806281] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:12Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06435worker2 # [ 129.810526] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:12Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16436worker2 # [ 129.852181] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526366437worker2 # [ 129.863586] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6438worker2 # [ 129.880109] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5)6439worker2 # [ 129.980850] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5)6440worker2 # [ 130.159390] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 393 ms (attempt 3/5)6441worker2 # [ 130.554546] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 468 ms (attempt 4/5)6442worker2 # [ 131.025866] nix-grpc-daemon[1537]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6443worker2 # [ 131.029828] nix-grpc-daemon[1537]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6444worker2 # [ 131.045688] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526366445client # [ 131.022848] nix[1367]: copying 1 paths to 'lb:50051'...6446client # [ 131.027214] nix[1367]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6447worker2 # [ 131.060394] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526366448worker2 # [ 131.071353] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6449worker2 # [ 131.090385] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:14Z level=info event=niks3_push_started pid=16156450worker2 # [ 131.192857] nix-grpc-daemon[1615]: time=2026-09-22T11:00:14.139Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6451worker2 # [ 131.277691] nix-daemon[851]: accepted connection from pid 1628, user nix-grpc-daemon (trusted)6452worker2 # [ 131.291361] nix-daemon[851]: reaped child process 1631, status = succeeded6453worker2 # [ 131.375846] nix-daemon[851]: accepted connection from pid 1633, user nix-grpc-daemon (trusted)6454worker2 # [ 131.389393] nix-daemon[851]: reaped child process 1636, status = succeeded6455lb # [ 131.408546] niks3-server[1104]: 2026/09/22 11:00:13 INFO Received uploads request method=POST path=/api/pending_closures6456worker2 # [ 131.405135] nix-grpc-daemon[1615]: time=2026-09-22T11:00:14.352Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6457worker2 # [ 131.408160] nix-grpc-daemon[1615]: time=2026-09-22T11:00:14.354Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6458lb # [ 131.446641] niks3-server[1104]: 2026/09/22 11:00:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6459lb # [ 131.451839] niks3-server[1104]: 2026/09/22 11:00:13 INFO Signed narinfos id=47 count=16460worker2 # [ 131.443637] nix-grpc-daemon[1615]: time=2026-09-22T11:00:14.391Z level=INFO msg="Uploading 1 narinfos"6461lb # [ 131.457140] niks3-server[1104]: 2026/09/22 11:00:13 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6462lb # [ 131.471551] niks3-server[1104]: 2026/09/22 11:00:13 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6463lb # [ 131.478519] niks3-server[1104]: 2026/09/22 11:00:13 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6464worker2 # [ 131.474727] nix-grpc-daemon[1615]: time=2026-09-22T11:00:14.422Z level=INFO msg="Upload complete. (279ms)"6465lb # [ 131.486268] niks3-server[1104]: 2026/09/22 11:00:13 INFO Completed upload id=476466worker2 # [ 131.476891] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=0 paths=1 nar_bytes_in=4246467lb # [ 131.490078] niks3-server[1104]: 2026/09/22 11:00:13 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6468worker2 # [ 131.481648] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526366469worker2 # [ 131.491299] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6470worker2 # [ 131.497356] nix-daemon[851]: reaped child process 1613, status = succeeded6471worker2 # [ 131.501631] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6472client # [ 131.467614] nix[1367]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6473client: must succeed: systemctl kill -s INT intr6474client # [ 133.038320] systemd[1]: intr.service: Sent signal SIGINT to main process 1367 (nix) on client request.6475client # [ 133.041313] nix[1367]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6476client: (finished: must succeed: systemctl kill -s INT intr, in 0.02 seconds)6477worker2 # [ 133.076922] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:16Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6478client # [ 133.044519] nix[1367]: error: interrupted by the user6479client # [ 133.064289] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6480client # [ 133.065689] systemd[1]: intr.service: Failed with result 'exit-code'.6481client # [ 133.067878] systemd[1]: intr.service: Consumed 120ms CPU time over 3.438s wall clock time, 43.4M memory peak, 6.5K incoming IP traffic, 8.2K outgoing IP traffic.6482worker2 # [ 133.121808] nix-daemon[851]: reaped child process 1638, status = succeeded6483worker2 # [ 133.123961] nix-daemon[1640]: unexpected Nix daemon error: error: interrupted by the user6484worker2 # [ 133.130644] nix-daemon[851]: reaped child process 1640, status = failed with exit code 16485(finished: subtest: interrupting the client stops the build on the worker, in 3.51 seconds)6486subtest: a deploy mid-build drains: the build finishes, the new generation takes over6487client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6488client # [ 133.138750] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6489client # Running as unit: sw.service; invocation ID: 164b37d2ed58402a8ecfa0a739a746fc6490client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.03 seconds)6491worker2 # [ 133.296821] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:526366492client # [ 133.269119] nix[1387]: don't know how to build these paths:6493client # [ 133.269976] nix[1387]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6494worker2 # [ 133.311989] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:16Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06495worker2 # [ 133.316902] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:16Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16496worker1 # [ 133.343563] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329146497worker1 # [ 133.353098] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329146498client # [ 133.340181] nix[1387]: copying 1 paths to 'lb:50051'...6499client # [ 133.341049] nix[1387]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6500worker1 # [ 133.361636] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:329146501worker1 # [ 133.372362] nix-daemon[864]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6502worker1 # [ 133.469279] nix-daemon[864]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6503worker1 # [ 133.481436] nix-daemon[864]: reaped child process 1740, status = succeeded6504worker1 # [ 133.561174] nix-daemon[864]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6505worker1 # [ 133.573925] nix-daemon[864]: reaped child process 1745, status = succeeded6506lb # [ 133.605662] niks3-server[1104]: 2026/09/22 11:00:15 INFO Received uploads request method=POST path=/api/pending_closures6507worker1 # [ 133.593028] nix-grpc-daemon[1645]: time=2026-09-22T11:00:16.771Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6508worker1 # [ 133.594437] nix-grpc-daemon[1645]: time=2026-09-22T11:00:16.771Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6509lb # [ 133.634785] niks3-server[1104]: 2026/09/22 11:00:15 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6510lb # [ 133.645708] niks3-server[1104]: 2026/09/22 11:00:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6511worker1 # [ 133.630715] nix-grpc-daemon[1645]: time=2026-09-22T11:00:16.809Z level=INFO msg="Uploading 1 narinfos"6512lb # [ 133.655068] niks3-server[1104]: 2026/09/22 11:00:15 INFO Signed narinfos id=48 count=16513lb # [ 133.666314] niks3-server[1104]: 2026/09/22 11:00:15 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6514lb # [ 133.672432] niks3-server[1104]: 2026/09/22 11:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6515lb # [ 133.677974] niks3-server[1104]: 2026/09/22 11:00:16 INFO Completed upload id=486516worker1 # [ 133.653196] nix-grpc-daemon[1645]: time=2026-09-22T11:00:16.832Z level=INFO msg="Upload complete. (269ms)"6517lb # [ 133.682782] niks3-server[1104]: 2026/09/22 11:00:16 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6518worker1 # [ 133.656293] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=0 paths=1 nar_bytes_in=4166519worker1 # [ 133.661267] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:329146520worker1 # [ 133.671222] nix-daemon[864]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6521worker1 # [ 133.675971] nix-daemon[864]: reaped child process 1735, status = succeeded6522worker1 # [ 133.680573] nix-daemon[864]: accepted connection from pid 1577, user nix-grpc-daemon (trusted)6523client # [ 133.659707] nix[1387]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6524worker1: must succeed: systemctl reload nix-grpc-daemon.service6525worker1 # [ 135.545258] systemd[1]: Reloading Nix worker-protocol over gRPC...6526worker1 # [ 135.572939] systemd[1]: Reloaded Nix worker-protocol over gRPC.6527worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6528worker1: must succeed: pgrep -f 'read -t [9]0 x'6529worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6530client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26531worker1 # [ 135.764679] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:329146532client # don't know how to build these paths:6533client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6534worker1 # [ 135.770785] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:329146535worker2 # [ 135.805917] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:18Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06536worker2 # [ 135.811460] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:18Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16537worker2 # [ 135.860307] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526366538worker2 # [ 135.871861] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526366539client # copying 1 paths to 'lb:50051'...6540worker2 # [ 135.884492] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526366541client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6542worker2 # [ 135.903917] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6543worker2 # [ 136.010796] nix-daemon[851]: accepted connection from pid 1691, user nix-grpc-daemon (trusted)6544worker2 # [ 136.026203] nix-daemon[851]: reaped child process 1694, status = succeeded6545worker2 # [ 136.106338] nix-daemon[851]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6546worker2 # [ 136.120866] nix-daemon[851]: reaped child process 1699, status = succeeded6547lb # [ 136.139699] niks3-server[1104]: 2026/09/22 11:00:18 INFO Received uploads request method=POST path=/api/pending_closures6548worker2 # [ 136.138523] nix-grpc-daemon[1615]: time=2026-09-22T11:00:19.085Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6549worker2 # [ 136.140154] nix-grpc-daemon[1615]: time=2026-09-22T11:00:19.087Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6550lb # [ 136.171960] niks3-server[1104]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6551lb # [ 136.190330] niks3-server[1104]: 2026/09/22 11:00:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6552worker2 # [ 136.184895] nix-grpc-daemon[1615]: time=2026-09-22T11:00:19.132Z level=INFO msg="Uploading 1 narinfos"6553lb # [ 136.196314] niks3-server[1104]: 2026/09/22 11:00:18 INFO Signed narinfos id=49 count=16554lb # [ 136.209511] niks3-server[1104]: 2026/09/22 11:00:18 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6555lb # [ 136.216951] niks3-server[1104]: 2026/09/22 11:00:18 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6556lb # [ 136.223468] niks3-server[1104]: 2026/09/22 11:00:18 INFO Completed upload id=496557worker2 # [ 136.211529] nix-grpc-daemon[1615]: time=2026-09-22T11:00:19.159Z level=INFO msg="Upload complete. (292ms)"6558worker2 # [ 136.216155] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=0 paths=1 nar_bytes_in=5286559lb # [ 136.232365] envoy[908]: conn peer=[2001:db8:1::1]:47934 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3087 tx=1518 ms=90836560lb # [ 136.233814] envoy[908]: conn peer=[2001:db8:1::1]:45212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2967 tx=852 ms=32976561worker2 # [ 136.221605] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526366562worker2 # [ 136.232482] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6563worker2 # [ 136.242057] nix-daemon[851]: reaped child process 1689, status = succeeded6564worker2 # [ 136.246842] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6565client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6566client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6567worker1 # [ 136.426672] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:19Z level=info event=draining6568client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6569client # farm-a> LOG-farm-a-during-drain6570worker2 # [ 137.294379] nix-daemon[851]: reaped child process 1703, status = succeeded6571worker2 # [ 137.359633] nix-daemon[851]: accepted connection from pid 1719, user nix-grpc-daemon (trusted)6572worker2 # [ 137.371853] nix-daemon[851]: reaped child process 1722, status = succeeded6573lb # [ 137.390507] niks3-server[1104]: 2026/09/22 11:00:19 INFO Received uploads request method=POST path=/api/pending_closures6574worker2 # [ 137.383447] nix-grpc-daemon[1615]: time=2026-09-22T11:00:20.330Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6575worker2 # [ 137.385027] nix-grpc-daemon[1615]: time=2026-09-22T11:00:20.332Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6576lb # [ 137.411871] niks3-server[1104]: 2026/09/22 11:00:19 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6577lb # [ 137.416774] niks3-server[1104]: 2026/09/22 11:00:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6578lb # [ 137.418475] niks3-server[1104]: 2026/09/22 11:00:19 INFO Signed narinfos id=50 count=16579worker2 # [ 137.406886] nix-grpc-daemon[1615]: time=2026-09-22T11:00:20.354Z level=INFO msg="Uploading 1 narinfos"6580lb # [ 137.420529] niks3-server[1104]: 2026/09/22 11:00:19 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6581lb # [ 137.423962] niks3-server[1104]: 2026/09/22 11:00:19 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6582lb # [ 137.428490] niks3-server[1104]: 2026/09/22 11:00:19 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6583lb # [ 137.432418] niks3-server[1104]: 2026/09/22 11:00:19 INFO Completed upload id=506584worker2 # [ 137.421925] nix-grpc-daemon[1615]: time=2026-09-22T11:00:20.369Z level=INFO msg="Upload complete. (167ms)"6585lb # [ 137.434148] niks3-server[1104]: 2026/09/22 11:00:19 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6586worker2 # [ 137.426198] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=3 outputs=16587worker2 # [ 137.431619] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:20Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06588worker2 # [ 137.433550] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:20Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16589worker2 # [ 137.439059] nix-daemon[851]: reaped child process 1701, status = succeeded6590worker2 # [ 137.477980] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526366591worker2 # [ 137.486316] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526366592worker2 # [ 137.495330] nix-grpc-daemon[1615]: time=2026-09-22T11:00:20.443Z level=INFO msg="All 1 paths already cached"6593client # copying 1 paths to 'lb:50051'...6594client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6595worker2 # [ 137.500620] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526366596worker2 # [ 137.509892] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6597worker2 # [ 137.599050] nix-daemon[851]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)6598worker2 # [ 137.613739] nix-daemon[851]: reaped child process 1731, status = succeeded6599worker2 # [ 137.694735] nix-daemon[851]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6600worker2 # [ 137.708360] nix-daemon[851]: reaped child process 1736, status = succeeded6601lb # [ 137.726062] niks3-server[1104]: 2026/09/22 11:00:20 INFO Received uploads request method=POST path=/api/pending_closures6602worker2 # [ 137.718531] nix-grpc-daemon[1615]: time=2026-09-22T11:00:20.666Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6603worker2 # [ 137.720142] nix-grpc-daemon[1615]: time=2026-09-22T11:00:20.667Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6604lb # [ 137.744054] niks3-server[1104]: 2026/09/22 11:00:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6605lb # [ 137.748386] niks3-server[1104]: 2026/09/22 11:00:20 INFO Signed narinfos id=51 count=16606worker2 # [ 137.738525] nix-grpc-daemon[1615]: time=2026-09-22T11:00:20.685Z level=INFO msg="Uploading 1 narinfos"6607lb # [ 137.749995] niks3-server[1104]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6608lb # [ 137.753050] niks3-server[1104]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6609lb # [ 137.759121] niks3-server[1104]: 2026/09/22 11:00:20 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6610lb # [ 137.762346] niks3-server[1104]: 2026/09/22 11:00:20 INFO Completed upload id=516611worker2 # [ 137.750742] nix-grpc-daemon[1615]: time=2026-09-22T11:00:20.698Z level=INFO msg="Upload complete. (231ms)"6612lb # [ 137.765260] niks3-server[1104]: 2026/09/22 11:00:20 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6613worker2 # [ 137.752805] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=0 paths=1 nar_bytes_in=6406614worker2 # [ 137.755697] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526366615worker2 # [ 137.763260] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6616worker2 # [ 137.766156] nix-daemon[851]: reaped child process 1726, status = succeeded6617client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6618client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6619worker2 # [ 137.770847] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6620client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6621client # farm-b> LOG-farm-b-during-drain6622worker2 # [ 138.588114] nix-daemon[851]: reaped child process 1740, status = succeeded6623worker2 # [ 138.661083] nix-daemon[851]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6624worker2 # [ 138.675471] nix-daemon[851]: reaped child process 1759, status = succeeded6625lb # [ 138.693258] niks3-server[1104]: 2026/09/22 11:00:21 INFO Received uploads request method=POST path=/api/pending_closures6626worker2 # [ 138.686034] nix-grpc-daemon[1615]: time=2026-09-22T11:00:21.633Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6627worker2 # [ 138.687574] nix-grpc-daemon[1615]: time=2026-09-22T11:00:21.635Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6628lb # [ 138.710233] niks3-server[1104]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6629lb # [ 138.715753] niks3-server[1104]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6630lb # [ 138.720238] niks3-server[1104]: 2026/09/22 11:00:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6631lb # [ 138.722033] niks3-server[1104]: 2026/09/22 11:00:21 INFO Signed narinfos id=52 count=16632worker2 # [ 138.710479] nix-grpc-daemon[1615]: time=2026-09-22T11:00:21.658Z level=INFO msg="Uploading 1 narinfos"6633lb # [ 138.725263] niks3-server[1104]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6634lb # [ 138.730834] niks3-server[1104]: 2026/09/22 11:00:21 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6635worker2 # [ 138.721820] nix-grpc-daemon[1615]: time=2026-09-22T11:00:21.669Z level=INFO msg="Upload complete. (144ms)"6636lb # [ 138.733624] niks3-server[1104]: 2026/09/22 11:00:21 INFO Completed upload id=526637lb # [ 138.735716] niks3-server[1104]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6638worker2 # [ 138.724385] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=4 outputs=16639worker2 # [ 138.729972] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:21Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06640worker2 # [ 138.732152] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:21Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16641worker2 # [ 138.737760] nix-daemon[851]: reaped child process 1738, status = succeeded6642worker2 # [ 138.773179] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526366643worker2 # [ 138.783754] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526366644worker2 # [ 138.796172] nix-grpc-daemon[1615]: time=2026-09-22T11:00:21.743Z level=INFO msg="All 2 paths already cached"6645client # copying 1 paths to 'lb:50051'...6646client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6647worker2 # [ 138.803441] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526366648worker2 # [ 138.811145] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6649worker2 # [ 138.899354] nix-daemon[851]: accepted connection from pid 1765, user nix-grpc-daemon (trusted)6650worker2 # [ 138.913172] nix-daemon[851]: reaped child process 1768, status = succeeded6651worker2 # [ 138.996650] nix-daemon[851]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6652worker2 # [ 139.011069] nix-daemon[851]: reaped child process 1773, status = succeeded6653lb # [ 139.029244] niks3-server[1104]: 2026/09/22 11:00:21 INFO Received uploads request method=POST path=/api/pending_closures6654worker2 # [ 139.026029] nix-grpc-daemon[1615]: time=2026-09-22T11:00:21.973Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6655worker2 # [ 139.027495] nix-grpc-daemon[1615]: time=2026-09-22T11:00:21.975Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6656lb # [ 139.059141] niks3-server[1104]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6657lb # [ 139.072741] niks3-server[1104]: 2026/09/22 11:00:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6658worker2 # [ 139.068591] nix-grpc-daemon[1615]: time=2026-09-22T11:00:22.016Z level=INFO msg="Uploading 1 narinfos"6659lb # [ 139.080857] niks3-server[1104]: 2026/09/22 11:00:21 INFO Signed narinfos id=53 count=16660lb # [ 139.085624] niks3-server[1104]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6661lb # [ 139.097470] niks3-server[1104]: 2026/09/22 11:00:21 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6662lb # [ 139.105762] niks3-server[1104]: 2026/09/22 11:00:21 INFO Completed upload id=536663worker2 # [ 139.094152] nix-grpc-daemon[1615]: time=2026-09-22T11:00:22.041Z level=INFO msg="Upload complete. (274ms)"6664lb # [ 139.107653] niks3-server[1104]: 2026/09/22 11:00:21 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6665worker2 # [ 139.096583] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=0 paths=1 nar_bytes_in=7686666worker2 # [ 139.102280] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526366667worker2 # [ 139.111339] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6668worker2 # [ 139.117696] nix-daemon[851]: reaped child process 1763, status = succeeded6669client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6670worker2 # [ 139.122697] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)6671client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6672client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6673client # farm-top> LOG-farm-top-during-drain6674worker2 # [ 139.766278] nix-daemon[851]: reaped child process 1777, status = succeeded6675worker2 # [ 139.839739] nix-daemon[851]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6676worker2 # [ 139.854319] nix-daemon[851]: reaped child process 1796, status = succeeded6677lb # [ 139.872332] niks3-server[1104]: 2026/09/22 11:00:22 INFO Received uploads request method=POST path=/api/pending_closures6678worker2 # [ 139.864749] nix-grpc-daemon[1615]: time=2026-09-22T11:00:22.812Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6679worker2 # [ 139.866527] nix-grpc-daemon[1615]: time=2026-09-22T11:00:22.813Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6680lb # [ 139.890410] niks3-server[1104]: 2026/09/22 11:00:22 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6681lb # [ 139.895746] niks3-server[1104]: 2026/09/22 11:00:22 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6682lb # [ 139.898117] niks3-server[1104]: 2026/09/22 11:00:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6683lb # [ 139.900072] niks3-server[1104]: 2026/09/22 11:00:22 INFO Signed narinfos id=54 count=16684worker2 # [ 139.889884] nix-grpc-daemon[1615]: time=2026-09-22T11:00:22.837Z level=INFO msg="Uploading 1 narinfos"6685lb # [ 139.902635] niks3-server[1104]: 2026/09/22 11:00:22 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6686lb # [ 139.908909] niks3-server[1104]: 2026/09/22 11:00:22 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6687lb # [ 139.912090] niks3-server[1104]: 2026/09/22 11:00:22 INFO Completed upload id=546688worker2 # [ 139.900378] nix-grpc-daemon[1615]: time=2026-09-22T11:00:22.848Z level=INFO msg="Upload complete. (171ms)"6689lb # [ 139.915049] niks3-server[1104]: 2026/09/22 11:00:22 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6690worker2 # [ 139.903410] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=0 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=5 outputs=16691worker2 # [ 139.910695] nix-daemon[851]: reaped child process 1775, status = succeeded6692client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 4.31 seconds)6693client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6694client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6695worker1: must succeed: pkill -f 'read -t [9]0 x'6696worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6697client: waiting for success: ! systemctl is-active sw6698worker1 # [ 140.102733] nix-daemon[864]: reaped child process 1749, status = succeeded6699worker1 # [ 140.173092] nix-daemon[864]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)6700worker1 # [ 140.184546] nix-daemon[864]: reaped child process 1794, status = succeeded6701lb # [ 140.215577] niks3-server[1104]: 2026/09/22 11:00:22 INFO Received uploads request method=POST path=/api/pending_closures6702worker1 # [ 140.199537] nix-grpc-daemon[1645]: time=2026-09-22T11:00:23.378Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6703worker1 # [ 140.201464] nix-grpc-daemon[1645]: time=2026-09-22T11:00:23.380Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6704lb # [ 140.252545] niks3-server[1104]: 2026/09/22 11:00:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6705lb # [ 140.256959] niks3-server[1104]: 2026/09/22 11:00:22 INFO Signed narinfos id=55 count=16706worker1 # [ 140.236080] nix-grpc-daemon[1645]: time=2026-09-22T11:00:23.415Z level=INFO msg="Uploading 1 narinfos"6707lb # [ 140.269347] niks3-server[1104]: 2026/09/22 11:00:22 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6708lb # [ 140.274119] niks3-server[1104]: 2026/09/22 11:00:22 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6709lb # [ 140.282322] niks3-server[1104]: 2026/09/22 11:00:22 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6710lb # [ 140.290426] niks3-server[1104]: 2026/09/22 11:00:22 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6711lb # [ 140.295568] niks3-server[1104]: 2026/09/22 11:00:22 INFO Completed upload id=556712worker1 # [ 140.272467] nix-grpc-daemon[1645]: time=2026-09-22T11:00:23.451Z level=INFO msg="Upload complete. (180ms)"6713lb # [ 140.298756] niks3-server[1104]: 2026/09/22 11:00:22 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6714worker1 # [ 140.277650] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32914 duration_s=6 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=2 outputs=16715worker1 # [ 140.290258] nix-daemon[864]: reaped child process 1747, status = succeeded6716client # [ 140.287721] systemd[1]: sw.service: Deactivated successfully.6717client # [ 140.292388] systemd[1]: sw.service: Consumed 126ms CPU time over 7.151s wall clock time, 43.4M memory peak, 7K incoming IP traffic, 8.2K outgoing IP traffic.6718worker1 # [ 140.427403] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:23Z level=info event=signal_exit6719client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6720client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6721client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6722worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6723worker2 # [ 141.453875] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:526146724worker2 # [ 141.457733] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:24Z level=info event=worker_gone addr=192.168.1.3:500516725worker1 # [ 141.445536] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:24Z level=info event=scheduler_restarting6726worker1 # [ 141.453543] nix-grpc-daemon[1577]: ts=2026-09-22T11:00:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6727worker1 # [ 141.473794] nix-daemon[864]: reaped child process 1633, status = succeeded6728worker1 # [ 141.681386] nix-grpc-daemon[1577]: E0000 00:00:1790074824.860453 1622 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6729worker1 # [ 141.686268] nix-grpc-daemon[1577]: E0000 00:00:1790074824.860592 1622 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6730worker1 # [ 142.446217] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6731worker1 # [ 142.447914] systemd[1]: nix-grpc-daemon.service: Consumed 712ms CPU time over 24.124s wall clock time, 36.8M memory peak, 24.3K incoming IP traffic, 34.2K outgoing IP traffic.6732worker1 # [ 142.454704] systemd[1]: Starting Nix worker-protocol over gRPC...6733worker1 # [ 142.530841] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6734worker1 # [ 142.532855] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6735worker1 # [ 142.535988] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16736worker1 # [ 142.540057] nix-grpc-daemon[1809]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6737worker1 # [ 142.543169] nix-grpc-daemon[1809]: E0000 00:00:1790074825.719290 1809 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6738worker1 # [ 142.545446] nix-grpc-daemon[1809]: E0000 00:00:1790074825.722326 1809 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6739worker1 # [ 142.552241] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6740worker1 # [ 142.555289] systemd[1]: Started Nix worker-protocol over gRPC.6741worker1 # [ 142.559082] nix-grpc-daemon[1809]: E0000 00:00:1790074825.736150 1848 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6742worker1 # [ 142.571081] nix-grpc-daemon[1809]: E0000 00:00:1790074825.749590 1848 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6743worker1 # [ 142.574142] nix-grpc-daemon[1809]: E0000 00:00:1790074825.749699 1848 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6744worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.42 seconds)6745lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6746lb: must succeed: curl -sf localhost:9901/clusters6747worker1 # [ 142.585517] nix-grpc-daemon[1809]: E0000 00:00:1790074825.764781 1848 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6748worker1 # [ 142.600139] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:25Z level=info event=scheduler_connected addr=lb:500516749worker2 # [ 142.617268] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:526146750worker2 # [ 142.621617] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06751lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6752lb: must succeed: curl -sf localhost:9901/clusters6753lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6754lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)6755lb: must succeed: curl -sf localhost:9901/clusters6756lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6757worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6758worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6759(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 10.62 seconds)6760subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6761client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16762worker2 # [ 143.908352] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:526366763worker1 # [ 143.902225] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589766764worker1 # [ 143.913753] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6765worker1 # [ 143.929255] nix-grpc-daemon[1809]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 1/5)6766worker1 # [ 144.022065] nix-grpc-daemon[1809]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5)6767worker1 # [ 144.055993] nix-grpc-daemon[1809]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 3/5)6768worker1 # [ 144.227381] nix-grpc-daemon[1809]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 731 ms (attempt 4/5)6769worker1 # [ 144.962197] nix-grpc-daemon[1809]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6770worker1 # [ 144.965585] nix-grpc-daemon[1809]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6771worker2 # [ 145.011724] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:27Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06772worker2 # [ 145.016696] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:27Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16773worker2 # [ 145.020986] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:27Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06774worker2 # [ 145.024785] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:27Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06775worker1 # [ 145.046376] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766776worker1 # [ 145.053197] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589766777worker1 # [ 145.058586] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589766778worker1 # [ 145.062614] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6779worker1 # [ 145.074581] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:28Z level=info event=niks3_push_started pid=18696780worker1 # [ 145.163306] nix-grpc-daemon[1869]: time=2026-09-22T11:00:28.342Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6781worker1 # [ 145.247464] nix-daemon[864]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)6782worker1 # [ 145.258902] nix-daemon[864]: reaped child process 1885, status = succeeded6783worker1 # [ 145.339075] nix-daemon[864]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6784worker1 # [ 145.351651] nix-daemon[864]: reaped child process 1890, status = succeeded6785lb # [ 145.384246] niks3-server[1104]: 2026/09/22 11:00:27 INFO Received uploads request method=POST path=/api/pending_closures6786worker1 # [ 145.367193] nix-grpc-daemon[1869]: time=2026-09-22T11:00:28.546Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6787worker1 # [ 145.368928] nix-grpc-daemon[1869]: time=2026-09-22T11:00:28.547Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6788lb # [ 145.425793] niks3-server[1104]: 2026/09/22 11:00:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6789worker1 # [ 145.406700] nix-grpc-daemon[1869]: time=2026-09-22T11:00:28.585Z level=INFO msg="Uploading 1 narinfos"6790lb # [ 145.432125] niks3-server[1104]: 2026/09/22 11:00:27 INFO Signed narinfos id=56 count=16791lb # [ 145.435210] niks3-server[1104]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6792lb # [ 145.449088] niks3-server[1104]: 2026/09/22 11:00:27 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6793worker1 # [ 145.432847] nix-grpc-daemon[1869]: time=2026-09-22T11:00:28.612Z level=INFO msg="Upload complete. (266ms)"6794lb # [ 145.457536] niks3-server[1104]: 2026/09/22 11:00:27 INFO Completed upload id=566795worker1 # [ 145.434867] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 paths=1 nar_bytes_in=5046796worker1 # [ 145.439840] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766797lb # [ 145.466623] niks3-server[1104]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6798lb # [ 145.471435] niks3-server[1104]: 2026/09/22 11:00:27 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6799worker1 # [ 145.450151] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6800worker1 # [ 145.456162] nix-daemon[864]: reaped child process 1867, status = succeeded6801worker1 # [ 145.459478] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6802lb # [ 146.232946] envoy[908]: conn peer=[2001:db8:1::1]:52192 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9486 tx=4054 ms=41476803lb # [ 146.234419] envoy[908]: conn peer=[2001:db8:1::1]:52176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2958 tx=1386 ms=70006804lb # [ 146.235785] envoy[908]: conn peer=[2001:db8:1::3]:34300 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1563 tx=475 ms=229786805worker1 # [ 146.675805] nix-daemon[864]: reaped child process 1894, status = succeeded6806worker1 # [ 146.751426] nix-daemon[864]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6807worker1 # [ 146.764325] nix-daemon[864]: reaped child process 1913, status = succeeded6808lb # [ 146.795950] niks3-server[1104]: 2026/09/22 11:00:29 INFO Received uploads request method=POST path=/api/pending_closures6809worker1 # [ 146.780508] nix-grpc-daemon[1869]: time=2026-09-22T11:00:29.959Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6810worker1 # [ 146.782048] nix-grpc-daemon[1869]: time=2026-09-22T11:00:29.961Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6811lb # [ 146.834879] niks3-server[1104]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6812lb # [ 146.849398] niks3-server[1104]: 2026/09/22 11:00:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6813lb # [ 146.855342] niks3-server[1104]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6814worker1 # [ 146.837092] nix-grpc-daemon[1869]: time=2026-09-22T11:00:30.015Z level=INFO msg="Uploading 1 narinfos"6815lb # [ 146.861432] niks3-server[1104]: 2026/09/22 11:00:29 INFO Signed narinfos id=57 count=16816lb # [ 146.872703] niks3-server[1104]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6817lb # [ 146.876074] niks3-server[1104]: 2026/09/22 11:00:29 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6818lb # [ 146.879030] niks3-server[1104]: 2026/09/22 11:00:29 INFO Completed upload id=576819worker1 # [ 146.853933] nix-grpc-daemon[1869]: time=2026-09-22T11:00:30.033Z level=INFO msg="Upload complete. (187ms)"6820lb # [ 146.881600] niks3-server[1104]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6821worker1 # [ 146.859600] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=6 outputs=16822worker2 # [ 146.874412] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:29Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16823worker2 # [ 146.883779] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:29Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06824worker1 # [ 146.873261] nix-daemon[864]: reaped child process 1892, status = succeeded6825worker1 # [ 146.904824] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766826worker1 # [ 146.916940] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589766827worker1 # [ 146.927993] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589766828worker1 # [ 146.936159] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6829worker1 # [ 147.032443] nix-daemon[864]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6830worker1 # [ 147.045167] nix-daemon[864]: reaped child process 1920, status = succeeded6831worker1 # [ 147.126202] nix-daemon[864]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6832worker1 # [ 147.138559] nix-daemon[864]: reaped child process 1925, status = succeeded6833lb # [ 147.170214] niks3-server[1104]: 2026/09/22 11:00:29 INFO Received uploads request method=POST path=/api/pending_closures6834worker1 # [ 147.154479] nix-grpc-daemon[1869]: time=2026-09-22T11:00:30.333Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6835worker1 # [ 147.155809] nix-grpc-daemon[1869]: time=2026-09-22T11:00:30.333Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6836lb # [ 147.198387] niks3-server[1104]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6837lb # [ 147.206059] niks3-server[1104]: 2026/09/22 11:00:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6838worker1 # [ 147.186469] nix-grpc-daemon[1869]: time=2026-09-22T11:00:30.365Z level=INFO msg="Uploading 1 narinfos"6839lb # [ 147.211138] niks3-server[1104]: 2026/09/22 11:00:29 INFO Signed narinfos id=58 count=16840lb # [ 147.222779] niks3-server[1104]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6841lb # [ 147.231099] niks3-server[1104]: 2026/09/22 11:00:29 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6842worker1 # [ 147.213862] nix-grpc-daemon[1869]: time=2026-09-22T11:00:30.393Z level=INFO msg="Upload complete. (265ms)"6843lb # [ 147.238631] niks3-server[1104]: 2026/09/22 11:00:29 INFO Completed upload id=586844worker1 # [ 147.215829] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 paths=1 nar_bytes_in=5046845lb # [ 147.244700] niks3-server[1104]: 2026/09/22 11:00:29 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6846worker1 # [ 147.221266] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766847worker1 # [ 147.229125] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6848worker1 # [ 147.233884] nix-daemon[864]: reaped child process 1915, status = succeeded6849worker1 # [ 147.238415] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6850worker1 # [ 148.455773] nix-daemon[864]: reaped child process 1929, status = succeeded6851worker1 # [ 148.525090] nix-daemon[864]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)6852worker1 # [ 148.536420] nix-daemon[864]: reaped child process 1948, status = succeeded6853lb # [ 148.567071] niks3-server[1104]: 2026/09/22 11:00:30 INFO Received uploads request method=POST path=/api/pending_closures6854worker1 # [ 148.550412] nix-grpc-daemon[1869]: time=2026-09-22T11:00:31.729Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6855worker1 # [ 148.552551] nix-grpc-daemon[1869]: time=2026-09-22T11:00:31.731Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6856lb # [ 148.606299] niks3-server[1104]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6857lb # [ 148.612106] niks3-server[1104]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6858lb # [ 148.622760] niks3-server[1104]: 2026/09/22 11:00:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6859worker1 # [ 148.603542] nix-grpc-daemon[1869]: time=2026-09-22T11:00:31.782Z level=INFO msg="Uploading 1 narinfos"6860lb # [ 148.628053] niks3-server[1104]: 2026/09/22 11:00:30 INFO Signed narinfos id=59 count=16861lb # [ 148.636780] niks3-server[1104]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6862lb # [ 148.646066] niks3-server[1104]: 2026/09/22 11:00:30 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6863lb # [ 148.649160] niks3-server[1104]: 2026/09/22 11:00:30 INFO Completed upload id=596864worker1 # [ 148.624601] nix-grpc-daemon[1869]: time=2026-09-22T11:00:31.803Z level=INFO msg="Upload complete. (179ms)"6865lb # [ 148.651677] niks3-server[1104]: 2026/09/22 11:00:30 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6866worker1 # [ 148.629920] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=7 outputs=16867worker2 # [ 148.644652] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:31Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16868worker2 # [ 148.653884] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:31Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06869worker1 # [ 148.644546] nix-daemon[864]: reaped child process 1927, status = succeeded6870worker1 # [ 148.674707] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766871worker1 # [ 148.686136] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589766872worker1 # [ 148.696888] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589766873worker1 # [ 148.707252] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6874worker1 # [ 148.794940] nix-daemon[864]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)6875worker1 # [ 148.807231] nix-daemon[864]: reaped child process 1955, status = succeeded6876worker1 # [ 148.879506] nix-daemon[864]: accepted connection from pid 1957, user nix-grpc-daemon (trusted)6877worker1 # [ 148.893620] nix-daemon[864]: reaped child process 1960, status = succeeded6878lb # [ 148.920846] niks3-server[1104]: 2026/09/22 11:00:31 INFO Received uploads request method=POST path=/api/pending_closures6879worker1 # [ 148.905301] nix-grpc-daemon[1869]: time=2026-09-22T11:00:32.084Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6880worker1 # [ 148.907155] nix-grpc-daemon[1869]: time=2026-09-22T11:00:32.086Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6881lb # [ 148.950951] niks3-server[1104]: 2026/09/22 11:00:31 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6882lb # [ 148.963689] niks3-server[1104]: 2026/09/22 11:00:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6883lb # [ 148.971189] niks3-server[1104]: 2026/09/22 11:00:31 INFO Signed narinfos id=60 count=16884worker1 # [ 148.949806] nix-grpc-daemon[1869]: time=2026-09-22T11:00:32.128Z level=INFO msg="Uploading 1 narinfos"6885lb # [ 148.974913] niks3-server[1104]: 2026/09/22 11:00:31 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6886lb # [ 148.989805] niks3-server[1104]: 2026/09/22 11:00:31 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6887worker1 # [ 148.972839] nix-grpc-daemon[1869]: time=2026-09-22T11:00:32.152Z level=INFO msg="Upload complete. (255ms)"6888lb # [ 148.997530] niks3-server[1104]: 2026/09/22 11:00:31 INFO Completed upload id=606889worker1 # [ 148.975197] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 paths=1 nar_bytes_in=5046890lb # [ 149.000794] niks3-server[1104]: 2026/09/22 11:00:31 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6891worker1 # [ 148.979129] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766892worker1 # [ 148.984109] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6893worker1 # [ 148.990208] nix-daemon[864]: reaped child process 1950, status = succeeded6894worker1 # [ 148.995254] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6895worker1 # [ 150.212657] nix-daemon[864]: reaped child process 1964, status = succeeded6896worker1 # [ 150.284289] nix-daemon[864]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)6897worker1 # [ 150.295446] nix-daemon[864]: reaped child process 1983, status = succeeded6898lb # [ 150.326676] niks3-server[1104]: 2026/09/22 11:00:32 INFO Received uploads request method=POST path=/api/pending_closures6899worker1 # [ 150.306172] nix-grpc-daemon[1869]: time=2026-09-22T11:00:33.485Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6900worker1 # [ 150.307976] nix-grpc-daemon[1869]: time=2026-09-22T11:00:33.487Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6901lb # [ 150.343886] niks3-server[1104]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6902lb # [ 150.349688] niks3-server[1104]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6903lb # [ 150.353090] niks3-server[1104]: 2026/09/22 11:00:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6904lb # [ 150.355951] niks3-server[1104]: 2026/09/22 11:00:32 INFO Signed narinfos id=61 count=16905worker1 # [ 150.332473] nix-grpc-daemon[1869]: time=2026-09-22T11:00:33.511Z level=INFO msg="Uploading 1 narinfos"6906lb # [ 150.358676] niks3-server[1104]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6907lb # [ 150.367567] niks3-server[1104]: 2026/09/22 11:00:32 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6908lb # [ 150.373809] niks3-server[1104]: 2026/09/22 11:00:32 INFO Completed upload id=616909worker1 # [ 150.349163] nix-grpc-daemon[1869]: time=2026-09-22T11:00:33.528Z level=INFO msg="Upload complete. (146ms)"6910lb # [ 150.377586] niks3-server[1104]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6911worker1 # [ 150.351654] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=8 outputs=16912worker1 # [ 150.358343] nix-daemon[864]: reaped child process 1962, status = succeeded6913worker2 # [ 150.366595] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:33Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16914worker2 # [ 150.376915] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:33Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06915worker1 # [ 150.403465] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766916worker1 # [ 150.409880] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589766917worker1 # [ 150.415878] nix-grpc-daemon[1869]: time=2026-09-22T11:00:33.595Z level=INFO msg="All 1 paths already cached"6918worker1 # [ 150.419768] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589766919worker1 # [ 150.423492] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6920worker1 # [ 150.506346] nix-daemon[864]: accepted connection from pid 1987, user nix-grpc-daemon (trusted)6921worker1 # [ 150.518731] nix-daemon[864]: reaped child process 1990, status = succeeded6922worker1 # [ 150.598882] nix-daemon[864]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6923worker1 # [ 150.611576] nix-daemon[864]: reaped child process 1995, status = succeeded6924lb # [ 150.641553] niks3-server[1104]: 2026/09/22 11:00:32 INFO Received uploads request method=POST path=/api/pending_closures6925worker1 # [ 150.621733] nix-grpc-daemon[1869]: time=2026-09-22T11:00:33.800Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6926worker1 # [ 150.623095] nix-grpc-daemon[1869]: time=2026-09-22T11:00:33.800Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6927lb # [ 150.654200] niks3-server[1104]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6928lb # [ 150.658168] niks3-server[1104]: 2026/09/22 11:00:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6929lb # [ 150.661078] niks3-server[1104]: 2026/09/22 11:00:32 INFO Signed narinfos id=62 count=16930worker1 # [ 150.636233] nix-grpc-daemon[1869]: time=2026-09-22T11:00:33.815Z level=INFO msg="Uploading 1 narinfos"6931lb # [ 150.664839] niks3-server[1104]: 2026/09/22 11:00:32 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6932lb # [ 150.669415] niks3-server[1104]: 2026/09/22 11:00:33 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6933lb # [ 150.673445] niks3-server[1104]: 2026/09/22 11:00:33 INFO Completed upload id=626934worker1 # [ 150.648482] nix-grpc-daemon[1869]: time=2026-09-22T11:00:33.827Z level=INFO msg="Upload complete. (219ms)"6935lb # [ 150.675662] niks3-server[1104]: 2026/09/22 11:00:33 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6936worker1 # [ 150.650535] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 paths=1 nar_bytes_in=6246937worker1 # [ 150.653289] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766938worker1 # [ 150.659870] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6939worker1 # [ 150.662121] nix-daemon[864]: reaped child process 1985, status = succeeded6940worker1 # [ 150.667160] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6941worker1 # [ 151.883567] nix-daemon[864]: reaped child process 1999, status = succeeded6942worker1 # [ 151.954214] nix-daemon[864]: accepted connection from pid 2015, user nix-grpc-daemon (trusted)6943worker1 # [ 151.966727] nix-daemon[864]: reaped child process 2018, status = succeeded6944lb # [ 151.998058] niks3-server[1104]: 2026/09/22 11:00:34 INFO Received uploads request method=POST path=/api/pending_closures6945worker1 # [ 151.978388] nix-grpc-daemon[1869]: time=2026-09-22T11:00:35.157Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6946worker1 # [ 151.979975] nix-grpc-daemon[1869]: time=2026-09-22T11:00:35.159Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6947lb # [ 152.021319] niks3-server[1104]: 2026/09/22 11:00:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6948lb # [ 152.023683] niks3-server[1104]: 2026/09/22 11:00:34 INFO Signed narinfos id=63 count=16949worker1 # [ 151.998856] nix-grpc-daemon[1869]: time=2026-09-22T11:00:35.178Z level=INFO msg="Uploading 1 narinfos"6950lb # [ 152.028282] niks3-server[1104]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6951lb # [ 152.030947] niks3-server[1104]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6952lb # [ 152.034701] niks3-server[1104]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6953lb # [ 152.039216] niks3-server[1104]: 2026/09/22 11:00:34 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6954lb # [ 152.044631] niks3-server[1104]: 2026/09/22 11:00:34 INFO Completed upload id=636955worker1 # [ 152.019953] nix-grpc-daemon[1869]: time=2026-09-22T11:00:35.199Z level=INFO msg="Upload complete. (145ms)"6956lb # [ 152.046305] niks3-server[1104]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6957worker1 # [ 152.022519] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=9 outputs=16958worker1 # [ 152.029527] nix-daemon[864]: reaped child process 1997, status = succeeded6959worker2 # [ 152.038924] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:34Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16960worker1 # [ 152.032278] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766961worker2 # [ 152.048089] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:34Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06962worker1 # [ 152.038222] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589766963worker1 # [ 152.043772] nix-grpc-daemon[1869]: time=2026-09-22T11:00:35.223Z level=INFO msg="All 1 paths already cached"6964worker1 # [ 152.047446] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589766965worker1 # [ 152.051908] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6966worker1 # [ 152.132800] nix-daemon[864]: accepted connection from pid 2022, user nix-grpc-daemon (trusted)6967worker1 # [ 152.145296] nix-daemon[864]: reaped child process 2025, status = succeeded6968worker1 # [ 152.225412] nix-daemon[864]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)6969worker1 # [ 152.238628] nix-daemon[864]: reaped child process 2030, status = succeeded6970lb # [ 152.268595] niks3-server[1104]: 2026/09/22 11:00:34 INFO Received uploads request method=POST path=/api/pending_closures6971worker1 # [ 152.253618] nix-grpc-daemon[1869]: time=2026-09-22T11:00:35.432Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6972worker1 # [ 152.254973] nix-grpc-daemon[1869]: time=2026-09-22T11:00:35.432Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6973lb # [ 152.296903] niks3-server[1104]: 2026/09/22 11:00:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6974lb # [ 152.302881] niks3-server[1104]: 2026/09/22 11:00:34 INFO Signed narinfos id=64 count=16975worker1 # [ 152.282116] nix-grpc-daemon[1869]: time=2026-09-22T11:00:35.461Z level=INFO msg="Uploading 1 narinfos"6976lb # [ 152.310696] niks3-server[1104]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6977lb # [ 152.319707] niks3-server[1104]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6978lb # [ 152.328532] niks3-server[1104]: 2026/09/22 11:00:34 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6979lb # [ 152.331739] niks3-server[1104]: 2026/09/22 11:00:34 INFO Completed upload id=646980worker1 # [ 152.308238] nix-grpc-daemon[1869]: time=2026-09-22T11:00:35.487Z level=INFO msg="Upload complete. (251ms)"6981lb # [ 152.334682] niks3-server[1104]: 2026/09/22 11:00:34 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6982worker1 # [ 152.310801] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 paths=1 nar_bytes_in=6246983worker1 # [ 152.314133] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589766984worker1 # [ 152.322366] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6985worker1 # [ 152.325133] nix-daemon[864]: reaped child process 2020, status = succeeded6986worker1 # [ 152.330816] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6987worker1 # [ 153.568302] nix-daemon[864]: reaped child process 2034, status = succeeded6988worker1 # [ 153.647759] nix-daemon[864]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)6989worker1 # [ 153.660147] nix-daemon[864]: reaped child process 2053, status = succeeded6990lb # [ 153.692339] niks3-server[1104]: 2026/09/22 11:00:36 INFO Received uploads request method=POST path=/api/pending_closures6991worker1 # [ 153.676421] nix-grpc-daemon[1869]: time=2026-09-22T11:00:36.855Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6992worker1 # [ 153.678141] nix-grpc-daemon[1869]: time=2026-09-22T11:00:36.857Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6993lb # [ 153.730428] niks3-server[1104]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6994lb # [ 153.740871] niks3-server[1104]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6995lb # [ 153.753382] niks3-server[1104]: 2026/09/22 11:00:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6996lb # [ 153.756821] niks3-server[1104]: 2026/09/22 11:00:36 INFO Signed narinfos id=65 count=16997worker1 # [ 153.734773] nix-grpc-daemon[1869]: time=2026-09-22T11:00:36.913Z level=INFO msg="Uploading 1 narinfos"6998lb # [ 153.766756] niks3-server[1104]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6999lb # [ 153.774425] niks3-server[1104]: 2026/09/22 11:00:36 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7000lb # [ 153.777322] niks3-server[1104]: 2026/09/22 11:00:36 INFO Completed upload id=657001lb # [ 153.780381] niks3-server[1104]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7002worker1 # [ 153.754236] nix-grpc-daemon[1869]: time=2026-09-22T11:00:36.933Z level=INFO msg="Upload complete. (213ms)"7003worker2 # [ 153.774441] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:36Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17004worker1 # [ 153.761914] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=10 outputs=17005worker1 # [ 153.769619] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589767006worker2 # [ 153.783893] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:36Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07007worker1 # [ 153.780509] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589767008worker1 # [ 153.789743] nix-daemon[864]: reaped child process 2032, status = succeeded7009worker1 # [ 153.792508] nix-grpc-daemon[1869]: time=2026-09-22T11:00:36.971Z level=INFO msg="All 1 paths already cached"7010worker1 # [ 153.797142] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589767011worker1 # [ 153.802447] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7012worker1 # [ 153.883652] nix-daemon[864]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)7013worker1 # [ 153.897478] nix-daemon[864]: reaped child process 2060, status = succeeded7014worker1 # [ 153.976277] nix-daemon[864]: accepted connection from pid 2062, user nix-grpc-daemon (trusted)7015worker1 # [ 153.987835] nix-daemon[864]: reaped child process 2065, status = succeeded7016lb # [ 154.018213] niks3-server[1104]: 2026/09/22 11:00:36 INFO Received uploads request method=POST path=/api/pending_closures7017worker1 # [ 154.000928] nix-grpc-daemon[1869]: time=2026-09-22T11:00:37.180Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7018worker1 # [ 154.002470] nix-grpc-daemon[1869]: time=2026-09-22T11:00:37.181Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7019lb # [ 154.045595] niks3-server[1104]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7020lb # [ 154.059443] niks3-server[1104]: 2026/09/22 11:00:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7021lb # [ 154.063659] niks3-server[1104]: 2026/09/22 11:00:36 INFO Signed narinfos id=66 count=17022worker1 # [ 154.041755] nix-grpc-daemon[1869]: time=2026-09-22T11:00:37.221Z level=INFO msg="Uploading 1 narinfos"7023lb # [ 154.070213] niks3-server[1104]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7024lb # [ 154.082831] niks3-server[1104]: 2026/09/22 11:00:36 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7025worker1 # [ 154.063822] nix-grpc-daemon[1869]: time=2026-09-22T11:00:37.243Z level=INFO msg="Upload complete. (256ms)"7026lb # [ 154.088883] niks3-server[1104]: 2026/09/22 11:00:36 INFO Completed upload id=667027worker1 # [ 154.065997] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 paths=1 nar_bytes_in=6247028lb # [ 154.094105] niks3-server[1104]: 2026/09/22 11:00:36 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7029worker1 # [ 154.072414] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589767030worker1 # [ 154.076277] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7031worker1 # [ 154.078550] nix-daemon[864]: reaped child process 2055, status = succeeded7032worker1 # [ 154.083666] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7033worker1 # [ 154.696800] nix-daemon[864]: reaped child process 2069, status = succeeded7034worker1 # [ 154.770640] nix-daemon[864]: accepted connection from pid 2085, user nix-grpc-daemon (trusted)7035worker1 # [ 154.781578] nix-daemon[864]: reaped child process 2088, status = succeeded7036lb # [ 154.814329] niks3-server[1104]: 2026/09/22 11:00:37 INFO Received uploads request method=POST path=/api/pending_closures7037worker1 # [ 154.798535] nix-grpc-daemon[1869]: time=2026-09-22T11:00:37.977Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7038worker1 # [ 154.800192] nix-grpc-daemon[1869]: time=2026-09-22T11:00:37.979Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7039lb # [ 154.858642] niks3-server[1104]: 2026/09/22 11:00:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7040lb # [ 154.864741] niks3-server[1104]: 2026/09/22 11:00:37 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7041lb # [ 154.869395] niks3-server[1104]: 2026/09/22 11:00:37 INFO Signed narinfos id=67 count=17042worker1 # [ 154.847828] nix-grpc-daemon[1869]: time=2026-09-22T11:00:38.026Z level=INFO msg="Uploading 1 narinfos"7043lb # [ 154.874589] niks3-server[1104]: 2026/09/22 11:00:37 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7044lb # [ 154.879120] niks3-server[1104]: 2026/09/22 11:00:37 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7045lb # [ 154.889952] niks3-server[1104]: 2026/09/22 11:00:37 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7046lb # [ 154.893710] niks3-server[1104]: 2026/09/22 11:00:37 INFO Completed upload id=677047worker1 # [ 154.870553] nix-grpc-daemon[1869]: time=2026-09-22T11:00:38.049Z level=INFO msg="Upload complete. (181ms)"7048lb # [ 154.896753] niks3-server[1104]: 2026/09/22 11:00:37 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7049worker1 # [ 154.877675] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=11 outputs=17050worker2 # [ 154.892797] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:37Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17051worker1 # [ 154.884723] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589767052worker2 # [ 154.899387] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:37Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07053worker1 # [ 154.894809] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589767054worker1 # [ 154.903625] nix-grpc-daemon[1869]: time=2026-09-22T11:00:38.082Z level=INFO msg="All 2 paths already cached"7055worker1 # [ 154.907252] nix-daemon[864]: reaped child process 2067, status = succeeded7056worker1 # [ 154.910511] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589767057worker1 # [ 154.914812] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7058worker1 # [ 154.995787] nix-daemon[864]: accepted connection from pid 2092, user nix-grpc-daemon (trusted)7059worker1 # [ 155.009460] nix-daemon[864]: reaped child process 2095, status = succeeded7060worker1 # [ 155.083840] nix-daemon[864]: accepted connection from pid 2097, user nix-grpc-daemon (trusted)7061worker1 # [ 155.097827] nix-daemon[864]: reaped child process 2100, status = succeeded7062lb # [ 155.128303] niks3-server[1104]: 2026/09/22 11:00:37 INFO Received uploads request method=POST path=/api/pending_closures7063worker1 # [ 155.111696] nix-grpc-daemon[1869]: time=2026-09-22T11:00:38.291Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7064worker1 # [ 155.113199] nix-grpc-daemon[1869]: time=2026-09-22T11:00:38.292Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7065lb # [ 155.163560] niks3-server[1104]: 2026/09/22 11:00:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7066lb # [ 155.167241] niks3-server[1104]: 2026/09/22 11:00:37 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7067lb # [ 155.173365] niks3-server[1104]: 2026/09/22 11:00:37 INFO Signed narinfos id=68 count=17068worker1 # [ 155.151763] nix-grpc-daemon[1869]: time=2026-09-22T11:00:38.331Z level=INFO msg="Uploading 1 narinfos"7069lb # [ 155.179829] niks3-server[1104]: 2026/09/22 11:00:37 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7070lb # [ 155.194508] niks3-server[1104]: 2026/09/22 11:00:37 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7071worker1 # [ 155.175882] nix-grpc-daemon[1869]: time=2026-09-22T11:00:38.355Z level=INFO msg="Upload complete. (256ms)"7072lb # [ 155.200482] niks3-server[1104]: 2026/09/22 11:00:37 INFO Completed upload id=687073lb # [ 155.203318] niks3-server[1104]: 2026/09/22 11:00:37 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7074worker1 # [ 155.178580] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 paths=1 nar_bytes_in=7527075worker1 # [ 155.185325] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589767076worker1 # [ 155.197057] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7077worker1 # [ 155.198671] nix-daemon[864]: reaped child process 2090, status = succeeded7078worker1 # [ 155.203715] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7079worker1 # [ 156.669328] nix-daemon[864]: reaped child process 2104, status = succeeded7080worker1 # [ 156.712472] nix-daemon[864]: accepted connection from pid 2120, user nix-grpc-daemon (trusted)7081worker1 # [ 156.724985] nix-daemon[864]: reaped child process 2123, status = succeeded7082lb # [ 156.756120] niks3-server[1104]: 2026/09/22 11:00:39 INFO Received uploads request method=POST path=/api/pending_closures7083worker1 # [ 156.738337] nix-grpc-daemon[1869]: time=2026-09-22T11:00:39.917Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7084worker1 # [ 156.739862] nix-grpc-daemon[1869]: time=2026-09-22T11:00:39.919Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7085lb # [ 156.794446] niks3-server[1104]: 2026/09/22 11:00:39 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7086lb # [ 156.803384] niks3-server[1104]: 2026/09/22 11:00:39 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7087lb # [ 156.812769] niks3-server[1104]: 2026/09/22 11:00:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7088lb # [ 156.819452] niks3-server[1104]: 2026/09/22 11:00:39 INFO Signed narinfos id=69 count=17089worker1 # [ 156.797174] nix-grpc-daemon[1869]: time=2026-09-22T11:00:39.976Z level=INFO msg="Uploading 1 narinfos"7090lb # [ 156.825216] niks3-server[1104]: 2026/09/22 11:00:39 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7091lb # [ 156.836058] niks3-server[1104]: 2026/09/22 11:00:39 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7092lb # [ 156.838730] niks3-server[1104]: 2026/09/22 11:00:39 INFO Completed upload id=697093worker1 # [ 156.814324] nix-grpc-daemon[1869]: time=2026-09-22T11:00:39.993Z level=INFO msg="Upload complete. (202ms)"7094lb # [ 156.841045] niks3-server[1104]: 2026/09/22 11:00:39 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7095worker1 # [ 156.819792] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=12 outputs=17096worker2 # [ 156.833929] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:39Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17097worker1 # [ 156.834781] nix-daemon[864]: reaped child process 2102, status = succeeded7098worker1 # [ 156.864240] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589767099worker1 # [ 156.876564] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589767100worker1 # [ 156.888969] nix-grpc-daemon[1869]: time=2026-09-22T11:00:40.068Z level=INFO msg="All 2 paths already cached"7101worker1 # [ 156.896478] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589767102worker1 # [ 156.906170] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7103worker1 # [ 156.999723] nix-daemon[864]: accepted connection from pid 2127, user nix-grpc-daemon (trusted)7104worker1 # [ 157.012858] nix-daemon[864]: reaped child process 2130, status = succeeded7105worker1 # [ 157.092565] nix-daemon[864]: accepted connection from pid 2132, user nix-grpc-daemon (trusted)7106worker1 # [ 157.105849] nix-daemon[864]: reaped child process 2135, status = succeeded7107lb # [ 157.136172] niks3-server[1104]: 2026/09/22 11:00:39 INFO Received uploads request method=POST path=/api/pending_closures7108worker1 # [ 157.119641] nix-grpc-daemon[1869]: time=2026-09-22T11:00:40.298Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7109worker1 # [ 157.121048] nix-grpc-daemon[1869]: time=2026-09-22T11:00:40.298Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7110lb # [ 157.161129] niks3-server[1104]: 2026/09/22 11:00:39 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7111lb # [ 157.172691] niks3-server[1104]: 2026/09/22 11:00:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7112lb # [ 157.179168] niks3-server[1104]: 2026/09/22 11:00:39 INFO Signed narinfos id=70 count=17113worker1 # [ 157.158085] nix-grpc-daemon[1869]: time=2026-09-22T11:00:40.337Z level=INFO msg="Uploading 1 narinfos"7114lb # [ 157.187193] niks3-server[1104]: 2026/09/22 11:00:39 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7115lb # [ 157.199248] niks3-server[1104]: 2026/09/22 11:00:39 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7116worker1 # [ 157.182457] nix-grpc-daemon[1869]: time=2026-09-22T11:00:40.361Z level=INFO msg="Upload complete. (264ms)"7117lb # [ 157.207272] niks3-server[1104]: 2026/09/22 11:00:39 INFO Completed upload id=707118lb # [ 157.210587] niks3-server[1104]: 2026/09/22 11:00:39 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7119worker1 # [ 157.184652] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 paths=1 nar_bytes_in=7527120worker1 # [ 157.189477] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589767121worker1 # [ 157.201076] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7122worker1 # [ 157.204679] nix-daemon[864]: reaped child process 2125, status = succeeded7123worker1 # [ 157.209474] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7124worker1 # [ 158.224616] nix-daemon[864]: reaped child process 2139, status = succeeded7125worker1 # [ 158.296205] nix-daemon[864]: accepted connection from pid 2155, user nix-grpc-daemon (trusted)7126worker1 # [ 158.308669] nix-daemon[864]: reaped child process 2158, status = succeeded7127lb # [ 158.339984] niks3-server[1104]: 2026/09/22 11:00:40 INFO Received uploads request method=POST path=/api/pending_closures7128worker1 # [ 158.319589] nix-grpc-daemon[1869]: time=2026-09-22T11:00:41.498Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7129worker1 # [ 158.321096] nix-grpc-daemon[1869]: time=2026-09-22T11:00:41.500Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7130lb # [ 158.359155] niks3-server[1104]: 2026/09/22 11:00:40 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7131lb # [ 158.361817] niks3-server[1104]: 2026/09/22 11:00:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7132lb # [ 158.363822] niks3-server[1104]: 2026/09/22 11:00:40 INFO Signed narinfos id=71 count=17133worker1 # [ 158.339180] nix-grpc-daemon[1869]: time=2026-09-22T11:00:41.518Z level=INFO msg="Uploading 1 narinfos"7134lb # [ 158.367972] niks3-server[1104]: 2026/09/22 11:00:40 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7135lb # [ 158.370118] niks3-server[1104]: 2026/09/22 11:00:40 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7136lb # [ 158.375422] niks3-server[1104]: 2026/09/22 11:00:40 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7137lb # [ 158.378388] niks3-server[1104]: 2026/09/22 11:00:40 INFO Completed upload id=717138worker1 # [ 158.353364] nix-grpc-daemon[1869]: time=2026-09-22T11:00:41.532Z level=INFO msg="Upload complete. (137ms)"7139lb # [ 158.381084] niks3-server[1104]: 2026/09/22 11:00:40 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7140worker1 # [ 158.355749] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=13 outputs=17141worker2 # [ 158.370790] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:41Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17142worker1 # [ 158.362706] nix-daemon[864]: reaped child process 2137, status = succeeded7143worker1 # [ 158.400732] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589767144worker1 # [ 158.408672] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589767145worker1 # [ 158.417132] nix-grpc-daemon[1869]: time=2026-09-22T11:00:41.596Z level=INFO msg="All 2 paths already cached"7146worker1 # [ 158.422161] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589767147worker1 # [ 158.430143] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7148worker1 # [ 158.514618] nix-daemon[864]: accepted connection from pid 2162, user nix-grpc-daemon (trusted)7149worker1 # [ 158.526857] nix-daemon[864]: reaped child process 2165, status = succeeded7150worker1 # [ 158.606863] nix-daemon[864]: accepted connection from pid 2167, user nix-grpc-daemon (trusted)7151worker1 # [ 158.618724] nix-daemon[864]: reaped child process 2170, status = succeeded7152lb # [ 158.651062] niks3-server[1104]: 2026/09/22 11:00:40 INFO Received uploads request method=POST path=/api/pending_closures7153worker1 # [ 158.633534] nix-grpc-daemon[1869]: time=2026-09-22T11:00:41.812Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7154worker1 # [ 158.635424] nix-grpc-daemon[1869]: time=2026-09-22T11:00:41.814Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7155lb # [ 158.679441] niks3-server[1104]: 2026/09/22 11:00:41 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7156lb # [ 158.685985] niks3-server[1104]: 2026/09/22 11:00:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7157lb # [ 158.693449] niks3-server[1104]: 2026/09/22 11:00:41 INFO Signed narinfos id=72 count=17158worker1 # [ 158.671693] nix-grpc-daemon[1869]: time=2026-09-22T11:00:41.850Z level=INFO msg="Uploading 1 narinfos"7159lb # [ 158.703705] niks3-server[1104]: 2026/09/22 11:00:41 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7160lb # [ 158.714181] niks3-server[1104]: 2026/09/22 11:00:41 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7161worker1 # [ 158.695325] nix-grpc-daemon[1869]: time=2026-09-22T11:00:41.874Z level=INFO msg="Upload complete. (257ms)"7162lb # [ 158.720331] niks3-server[1104]: 2026/09/22 11:00:41 INFO Completed upload id=727163worker1 # [ 158.697511] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=0 paths=1 nar_bytes_in=7527164lb # [ 158.723909] niks3-server[1104]: 2026/09/22 11:00:41 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7165worker1 # [ 158.703347] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589767166worker1 # [ 158.713867] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7167worker1 # [ 158.716169] nix-daemon[864]: reaped child process 2160, status = succeeded7168worker1 # [ 158.721513] nix-daemon[864]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)7169worker1 # [ 159.936956] nix-daemon[864]: reaped child process 2174, status = succeeded7170worker1 # [ 160.010774] nix-daemon[864]: accepted connection from pid 2190, user nix-grpc-daemon (trusted)7171worker1 # [ 160.021951] nix-daemon[864]: reaped child process 2193, status = succeeded7172lb # [ 160.054673] niks3-server[1104]: 2026/09/22 11:00:42 INFO Received uploads request method=POST path=/api/pending_closures7173worker1 # [ 160.037841] nix-grpc-daemon[1869]: time=2026-09-22T11:00:43.216Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7174worker1 # [ 160.039566] nix-grpc-daemon[1869]: time=2026-09-22T11:00:43.218Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7175lb # [ 160.089638] niks3-server[1104]: 2026/09/22 11:00:42 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7176lb # [ 160.103574] niks3-server[1104]: 2026/09/22 11:00:42 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7177lb # [ 160.111198] niks3-server[1104]: 2026/09/22 11:00:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7178lb # [ 160.118242] niks3-server[1104]: 2026/09/22 11:00:42 INFO Signed narinfos id=73 count=17179worker1 # [ 160.094267] nix-grpc-daemon[1869]: time=2026-09-22T11:00:43.273Z level=INFO msg="Uploading 1 narinfos"7180lb # [ 160.123750] niks3-server[1104]: 2026/09/22 11:00:42 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7181lb # [ 160.133514] niks3-server[1104]: 2026/09/22 11:00:42 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7182lb # [ 160.138905] niks3-server[1104]: 2026/09/22 11:00:42 INFO Completed upload id=737183lb # [ 160.140516] niks3-server[1104]: 2026/09/22 11:00:42 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7184worker1 # [ 160.115543] nix-grpc-daemon[1869]: time=2026-09-22T11:00:43.294Z level=INFO msg="Upload complete. (186ms)"7185worker1 # [ 160.122220] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58976 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=14 outputs=17186worker1 # [ 160.136362] nix-daemon[864]: reaped child process 2172, status = succeeded7187client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.40 seconds)7188worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7189worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)7190client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7191client # [ 160.180687] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7192client # Running as unit: gpu.service; invocation ID: 961616a1c000466eafc9f4a8bc697a4c7193client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.03 seconds)7194lb: must succeed: curl -sf localhost:9901/clusters7195lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7196lb: must succeed: curl -sf localhost:9901/clusters7197lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7198worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7199worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7200worker1 # [ 160.328952] nix-grpc-daemon[1809]: ts=2026-09-22T11:00:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:589767201client # [ 160.314454] nix[1467]: don't know how to build these paths:7202client # [ 160.315321] nix[1467]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7203worker2 # [ 160.352679] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526367204worker2 # [ 160.369159] nix-grpc-daemon[1537]: ts=2026-09-22T11:00:43Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07205client # [ 160.340027] nix[1467]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker7206lb: must succeed: curl -sf localhost:9901/clusters7207lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7208lb: must succeed: curl -sf localhost:9901/clusters7209lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7210worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7211worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7212client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7213lb # [ 166.235370] envoy[908]: conn peer=[2001:db8:1::1]:50710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27111 tx=11318 ms=162437214client # [ 190.341666] nix[1467]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}7215client # [ 190.349594] nix[1467]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7216client # [ 190.382155] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7217client # [ 190.385596] systemd[1]: gpu.service: Failed with result 'exit-code'.7218client # [ 190.390176] systemd[1]: gpu.service: Consumed 139ms CPU time over 30.208s wall clock time, 37.6M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7219client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.27 seconds)7220client: must succeed: journalctl -u gpu -o cat7221client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7222lb: must succeed: curl -sf localhost:9901/clusters7223lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7224lb: must succeed: curl -sf localhost:9901/clusters7225lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7226worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7227worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7228(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.09 seconds)7229subtest: low disk drains a worker and builds go to the other7230worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7231worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.99 seconds)7232worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7233worker1 # [ 192.565847] nix-grpc-daemon[1809]: ts=2026-09-22T11:01:15Z level=info event=unhealthy reason=min_free7234worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.05 seconds)7235client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27236client # don't know how to build these paths:7237worker1 # [ 193.029583] nix-grpc-daemon[1809]: ts=2026-09-22T11:01:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:589767238client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7239worker2 # [ 193.047495] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526367240worker2 # [ 193.062070] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:16Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07241worker2 # [ 193.064432] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:16Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17242worker2 # [ 193.105412] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526367243worker2 # [ 193.113103] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526367244client # copying 1 paths to 'lb:50051'...7245worker2 # [ 193.121256] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526367246client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7247worker2 # [ 193.126410] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)7248worker2 # [ 193.224941] nix-daemon[851]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)7249worker2 # [ 193.240541] nix-daemon[851]: reaped child process 1833, status = succeeded7250worker2 # [ 193.324129] nix-daemon[851]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)7251worker2 # [ 193.338428] nix-daemon[851]: reaped child process 1838, status = succeeded7252lb # [ 193.356948] niks3-server[1104]: 2026/09/22 11:01:15 INFO Received uploads request method=POST path=/api/pending_closures7253worker2 # [ 193.353730] nix-grpc-daemon[1615]: time=2026-09-22T11:01:16.301Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7254worker2 # [ 193.355595] nix-grpc-daemon[1615]: time=2026-09-22T11:01:16.303Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7255lb # [ 193.390142] niks3-server[1104]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7256lb # [ 193.402458] niks3-server[1104]: 2026/09/22 11:01:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7257worker2 # [ 193.398265] nix-grpc-daemon[1615]: time=2026-09-22T11:01:16.345Z level=INFO msg="Uploading 1 narinfos"7258lb # [ 193.410056] niks3-server[1104]: 2026/09/22 11:01:15 INFO Signed narinfos id=74 count=17259lb # [ 193.425660] niks3-server[1104]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7260lb # [ 193.435600] niks3-server[1104]: 2026/09/22 11:01:15 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7261lb # [ 193.439628] niks3-server[1104]: 2026/09/22 11:01:15 INFO Completed upload id=747262worker2 # [ 193.428151] nix-grpc-daemon[1615]: time=2026-09-22T11:01:16.375Z level=INFO msg="Upload complete. (293ms)"7263lb # [ 193.440979] niks3-server[1104]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7264worker2 # [ 193.432165] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=0 paths=1 nar_bytes_in=5127265worker2 # [ 193.438608] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526367266worker2 # [ 193.450111] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)7267worker2 # [ 193.460238] nix-daemon[851]: reaped child process 1828, status = succeeded7268worker2 # [ 193.463820] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)7269client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7270client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7271client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7272client # farm-a> LOG-farm-a-drain7273worker2 # [ 194.896672] nix-daemon[851]: reaped child process 1842, status = succeeded7274worker2 # [ 194.988403] nix-daemon[851]: accepted connection from pid 1858, user nix-grpc-daemon (trusted)7275worker2 # [ 195.000575] nix-daemon[851]: reaped child process 1861, status = succeeded7276lb # [ 195.019612] niks3-server[1104]: 2026/09/22 11:01:17 INFO Received uploads request method=POST path=/api/pending_closures7277worker2 # [ 195.017520] nix-grpc-daemon[1615]: time=2026-09-22T11:01:17.965Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7278worker2 # [ 195.019431] nix-grpc-daemon[1615]: time=2026-09-22T11:01:17.966Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7279lb # [ 195.057081] niks3-server[1104]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7280lb # [ 195.069145] niks3-server[1104]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7281lb # [ 195.083195] niks3-server[1104]: 2026/09/22 11:01:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7282lb # [ 195.087371] niks3-server[1104]: 2026/09/22 11:01:17 INFO Signed narinfos id=75 count=17283worker2 # [ 195.078300] nix-grpc-daemon[1615]: time=2026-09-22T11:01:18.025Z level=INFO msg="Uploading 1 narinfos"7284lb # [ 195.094539] niks3-server[1104]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7285lb # [ 195.099065] niks3-server[1104]: 2026/09/22 11:01:17 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7286lb # [ 195.102191] niks3-server[1104]: 2026/09/22 11:01:17 INFO Completed upload id=757287worker2 # [ 195.090806] nix-grpc-daemon[1615]: time=2026-09-22T11:01:18.038Z level=INFO msg="Upload complete. (216ms)"7288lb # [ 195.105470] niks3-server[1104]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7289worker2 # [ 195.100851] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=15 outputs=17290worker2 # [ 195.108646] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:18Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07291worker2 # [ 195.113357] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:18Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17292worker2 # [ 195.122643] nix-daemon[851]: reaped child process 1840, status = succeeded7293worker2 # [ 195.153385] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526367294worker2 # [ 195.167951] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526367295worker2 # [ 195.181697] nix-grpc-daemon[1615]: time=2026-09-22T11:01:18.129Z level=INFO msg="All 1 paths already cached"7296client # copying 1 paths to 'lb:50051'...7297client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7298worker2 # [ 195.190602] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526367299worker2 # [ 195.199551] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)7300worker2 # [ 195.301413] nix-daemon[851]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)7301worker2 # [ 195.315747] nix-daemon[851]: reaped child process 1870, status = succeeded7302worker2 # [ 195.399566] nix-daemon[851]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)7303worker2 # [ 195.413948] nix-daemon[851]: reaped child process 1875, status = succeeded7304lb # [ 195.429834] niks3-server[1104]: 2026/09/22 11:01:17 INFO Received uploads request method=POST path=/api/pending_closures7305worker2 # [ 195.427243] nix-grpc-daemon[1615]: time=2026-09-22T11:01:18.374Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7306worker2 # [ 195.428735] nix-grpc-daemon[1615]: time=2026-09-22T11:01:18.374Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7307lb # [ 195.457572] niks3-server[1104]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7308lb # [ 195.466923] niks3-server[1104]: 2026/09/22 11:01:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7309lb # [ 195.471371] niks3-server[1104]: 2026/09/22 11:01:17 INFO Signed narinfos id=76 count=17310worker2 # [ 195.463095] nix-grpc-daemon[1615]: time=2026-09-22T11:01:18.410Z level=INFO msg="Uploading 1 narinfos"7311lb # [ 195.482612] niks3-server[1104]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7312lb # [ 195.492061] niks3-server[1104]: 2026/09/22 11:01:17 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7313lb # [ 195.497959] niks3-server[1104]: 2026/09/22 11:01:17 INFO Completed upload id=767314worker2 # [ 195.488668] nix-grpc-daemon[1615]: time=2026-09-22T11:01:18.436Z level=INFO msg="Upload complete. (276ms)"7315lb # [ 195.503600] niks3-server[1104]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7316worker2 # [ 195.491404] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=0 paths=1 nar_bytes_in=6247317worker2 # [ 195.497778] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526367318worker2 # [ 195.508297] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)7319worker2 # [ 195.510748] nix-daemon[851]: reaped child process 1865, status = succeeded7320worker2 # [ 195.516327] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)7321client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7322client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7323client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7324client # farm-b> LOG-farm-b-drain7325lb # [ 196.236365] envoy[908]: conn peer=[2001:db8:1::1]:40158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300497326worker2 # [ 196.337186] nix-daemon[851]: reaped child process 1879, status = succeeded7327worker2 # [ 196.415794] nix-daemon[851]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)7328worker2 # [ 196.429171] nix-daemon[851]: reaped child process 1898, status = succeeded7329lb # [ 196.448192] niks3-server[1104]: 2026/09/22 11:01:18 INFO Received uploads request method=POST path=/api/pending_closures7330worker2 # [ 196.445775] nix-grpc-daemon[1615]: time=2026-09-22T11:01:19.393Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7331worker2 # [ 196.448058] nix-grpc-daemon[1615]: time=2026-09-22T11:01:19.395Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7332lb # [ 196.494470] niks3-server[1104]: 2026/09/22 11:01:18 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7333lb # [ 196.500165] niks3-server[1104]: 2026/09/22 11:01:18 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7334lb # [ 196.509260] niks3-server[1104]: 2026/09/22 11:01:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7335lb # [ 196.514237] niks3-server[1104]: 2026/09/22 11:01:18 INFO Signed narinfos id=77 count=17336worker2 # [ 196.505757] nix-grpc-daemon[1615]: time=2026-09-22T11:01:19.453Z level=INFO msg="Uploading 1 narinfos"7337lb # [ 196.520805] niks3-server[1104]: 2026/09/22 11:01:18 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7338lb # [ 196.529530] niks3-server[1104]: 2026/09/22 11:01:18 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7339lb # [ 196.532939] niks3-server[1104]: 2026/09/22 11:01:18 INFO Completed upload id=777340worker2 # [ 196.521606] nix-grpc-daemon[1615]: time=2026-09-22T11:01:19.469Z level=INFO msg="Upload complete. (198ms)"7341lb # [ 196.535700] niks3-server[1104]: 2026/09/22 11:01:18 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7342worker2 # [ 196.527852] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=16 outputs=17343worker2 # [ 196.541477] nix-daemon[851]: reaped child process 1877, status = succeeded7344worker2 # [ 196.574206] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:19Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07345worker2 # [ 196.579601] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:19Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17346worker2 # [ 196.585440] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526367347worker2 # [ 196.597445] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526367348worker2 # [ 196.607332] nix-grpc-daemon[1615]: time=2026-09-22T11:01:19.554Z level=INFO msg="All 2 paths already cached"7349client # copying 1 paths to 'lb:50051'...7350client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7351worker2 # [ 196.615185] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526367352worker2 # [ 196.623402] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)7353worker2 # [ 196.724622] nix-daemon[851]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)7354worker2 # [ 196.740929] nix-daemon[851]: reaped child process 1907, status = succeeded7355worker2 # [ 196.822400] nix-daemon[851]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)7356worker2 # [ 196.836175] nix-daemon[851]: reaped child process 1912, status = succeeded7357lb # [ 196.853154] niks3-server[1104]: 2026/09/22 11:01:19 INFO Received uploads request method=POST path=/api/pending_closures7358worker2 # [ 196.846521] nix-grpc-daemon[1615]: time=2026-09-22T11:01:19.793Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7359worker2 # [ 196.847938] nix-grpc-daemon[1615]: time=2026-09-22T11:01:19.794Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7360lb # [ 196.868053] niks3-server[1104]: 2026/09/22 11:01:19 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7361lb # [ 196.871494] niks3-server[1104]: 2026/09/22 11:01:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7362lb # [ 196.873700] niks3-server[1104]: 2026/09/22 11:01:19 INFO Signed narinfos id=78 count=17363worker2 # [ 196.864381] nix-grpc-daemon[1615]: time=2026-09-22T11:01:19.811Z level=INFO msg="Uploading 1 narinfos"7364lb # [ 196.878303] niks3-server[1104]: 2026/09/22 11:01:19 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7365lb # [ 196.883646] niks3-server[1104]: 2026/09/22 11:01:19 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7366lb # [ 196.888554] niks3-server[1104]: 2026/09/22 11:01:19 INFO Completed upload id=787367worker2 # [ 196.878198] nix-grpc-daemon[1615]: time=2026-09-22T11:01:19.825Z level=INFO msg="Upload complete. (243ms)"7368lb # [ 196.891307] niks3-server[1104]: 2026/09/22 11:01:19 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7369worker2 # [ 196.880333] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=0 paths=1 nar_bytes_in=7527370worker2 # [ 196.884210] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526367371worker2 # [ 196.890710] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)7372worker2 # [ 196.893077] nix-daemon[851]: reaped child process 1902, status = succeeded7373worker2 # [ 196.898490] nix-daemon[851]: accepted connection from pid 1537, user nix-grpc-daemon (trusted)7374client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7375client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7376client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7377client # farm-top> LOG-farm-top-drain7378worker2 # [ 197.739590] nix-daemon[851]: reaped child process 1916, status = succeeded7379worker2 # [ 197.820571] nix-daemon[851]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)7380worker2 # [ 197.833499] nix-daemon[851]: reaped child process 1935, status = succeeded7381lb # [ 197.851227] niks3-server[1104]: 2026/09/22 11:01:20 INFO Received uploads request method=POST path=/api/pending_closures7382worker2 # [ 197.850342] nix-grpc-daemon[1615]: time=2026-09-22T11:01:20.797Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7383worker2 # [ 197.852290] nix-grpc-daemon[1615]: time=2026-09-22T11:01:20.799Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7384lb # [ 197.890518] niks3-server[1104]: 2026/09/22 11:01:20 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7385lb # [ 197.901977] niks3-server[1104]: 2026/09/22 11:01:20 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7386lb # [ 197.909990] niks3-server[1104]: 2026/09/22 11:01:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7387worker2 # [ 197.904567] nix-grpc-daemon[1615]: time=2026-09-22T11:01:20.851Z level=INFO msg="Uploading 1 narinfos"7388lb # [ 197.915610] niks3-server[1104]: 2026/09/22 11:01:20 INFO Signed narinfos id=79 count=17389lb # [ 197.921156] niks3-server[1104]: 2026/09/22 11:01:20 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7390lb # [ 197.930532] niks3-server[1104]: 2026/09/22 11:01:20 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7391lb # [ 197.934617] niks3-server[1104]: 2026/09/22 11:01:20 INFO Completed upload id=797392lb # [ 197.935888] niks3-server[1104]: 2026/09/22 11:01:20 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7393worker2 # [ 197.923290] nix-grpc-daemon[1615]: time=2026-09-22T11:01:20.870Z level=INFO msg="Upload complete. (217ms)"7394worker2 # [ 197.931476] nix-grpc-daemon[1537]: ts=2026-09-22T11:01:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52636 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=17 outputs=17395worker2 # [ 197.944981] nix-daemon[851]: reaped child process 1914, status = succeeded7396client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.06 seconds)7397client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7398client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7399worker1: must succeed: rm /nix/.rw-store/fill7400worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.11 seconds)7401worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7402worker1 # [ 198.570172] nix-grpc-daemon[1809]: ts=2026-09-22T11:01:21Z level=info event=healthy reason=min_free7403worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.05 seconds)7404(finished: subtest: low disk drains a worker and builds go to the other, in 8.43 seconds)7405(finished: run the VM test script, in 200.22 seconds)7406test script finished in 200.38s7407cleanup7408kill QemuMachine (pid 49)7409client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7410client # [2026-09-22T11:01:22Z INFO virtiofsd] Client disconnected, shutting down7411client # [2026-09-22T11:01:22Z INFO virtiofsd] Client disconnected, shutting down7412client # [2026-09-22T11:01:22Z INFO virtiofsd] Client disconnected, shutting down7413kill QemuMachine (pid 50)7414lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7415lb # [2026-09-22T11:01:22Z INFO virtiofsd] Client disconnected, shutting down7416lb # [2026-09-22T11:01:22Z INFO virtiofsd] Client disconnected, shutting down7417lb # [2026-09-22T11:01:22Z INFO virtiofsd] Client disconnected, shutting down7418kill QemuMachine (pid 51)7419worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7420worker1 # [2026-09-22T11:01:23Z INFO virtiofsd] Client disconnected, shutting down7421worker1 # [2026-09-22T11:01:23Z INFO virtiofsd] Client disconnected, shutting down7422worker1 # [2026-09-22T11:01:23Z INFO virtiofsd] Client disconnected, shutting down7423kill QemuMachine (pid 52)7424worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7425worker2 # [2026-09-22T11:01:23Z INFO virtiofsd] Client disconnected, shutting down7426worker2 # [2026-09-22T11:01:23Z INFO virtiofsd] Client disconnected, shutting down7427worker2 # [2026-09-22T11:01:23Z INFO virtiofsd] Client disconnected, shutting down7428(finished: cleanup, in 0.98 seconds)7429additionally exposed symbols:7430 client, lb, worker1, worker2,7431 vlan1,7432 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh