vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #513
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb: QEMU running (pid 49)13lb # Disk image does not exist, creating the virtualisation disk image...14lb # Formatting '/build/vm-state-lb/tmp.qml3liuXEP', fmt=raw size=107374182415lb # mke2fs 1.47.4 (6-Mar-2025)16lb # Discarding device blocks: 0/262144 done17client # Disk image does not exist, creating the virtualisation disk image...18lb # Creating filesystem with 262144 4k blocks and 65536 inodes19client # Formatting '/build/vm-state-client/tmp.g9Qg20BDIv', fmt=raw size=107374182420lb # Filesystem UUID: 9235bbd0-5e56-4e7e-9705-a7600a7783e421client: QEMU running (pid 50)22client # mke2fs 1.47.4 (6-Mar-2025)23lb # Superblock backups stored on blocks:24worker2 # Disk image does not exist, creating the virtualisation disk image...25worker2: QEMU running (pid 52)26client # Discarding device blocks: 0/262144 done27worker2 # Formatting '/build/vm-state-worker2/tmp.emgRknR60X', fmt=raw size=107374182428lb # 32768, 98304, 163840, 22937629client # Creating filesystem with 262144 4k blocks and 65536 inodes30worker2 # mke2fs 1.47.4 (6-Mar-2025)31lb # 32client # Filesystem UUID: aa7e6a8b-accf-4dca-8a13-28c620627c1533worker2 # Discarding device blocks: 0/262144 done34lb # Allocating group tables: 0/8 done35client # Superblock backups stored on blocks:36worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes37lb # Writing inode tables: 0/8 done38worker1: QEMU running (pid 51)39worker2 # Filesystem UUID: 05fdbd80-b498-48ce-95d5-8c595845813340lb # Creating journal (8192 blocks): done41client # 32768, 98304, 163840, 22937642worker1 # Disk image does not exist, creating the virtualisation disk image...43worker2 # Superblock backups stored on blocks:44lb # Writing superblocks and filesystem accounting information: 0/8 done45worker1 # Formatting '/build/vm-state-worker1/tmp.2kUnJk98D3', fmt=raw size=107374182446worker2 # 32768, 98304, 163840, 22937647lb # 48worker1 # mke2fs 1.47.4 (6-Mar-2025)49worker2 # 50lb # Virtualisation disk image created.51worker1 # Discarding device blocks: 0/262144 done52worker2 # Allocating group tables: 0/8 done53lb # Starting virtiofs daemons...54worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes55worker2 # Writing inode tables: 0/8 done56lb # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)57worker1 # Filesystem UUID: 12f3882e-0173-4aba-8b26-dc31b8bf8bf358worker2 # Creating journal (8192 blocks): done59lb # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether60worker1 # Superblock backups stored on blocks:61worker2 # Writing superblocks and filesystem accounting information: 0/8 done62lb # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...63lb # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)64worker1 # 32768, 98304, 163840, 22937665(finished: start all VMs, in 0.31 seconds)66client # 67worker1 # 68client: waiting for unit network-addresses-eth1.service69client # Allocating group tables: 0/8 done70worker1 # Allocating group tables: 0/8 done71client: waiting for the VM to finish booting72client # Writing inode tables: 0/8 done73worker1 # Writing inode tables: 0/8 done74worker2 # 75worker1 # Creating journal (8192 blocks): done76worker2 # Virtualisation disk image created.77client # Creating journal (8192 blocks): done78lb # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79client # Writing superblocks and filesystem accounting information: 0/8 done80worker2 # Starting virtiofs daemons...81worker2 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)82lb # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...83client # 84worker1 # Writing superblocks and filesystem accounting information: 0/8 done85worker2 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether86lb # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)87client # Virtualisation disk image created.88worker1 # 89worker2 # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...90lb # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether91client # Starting virtiofs daemons...92worker1 # Virtualisation disk image created.93worker2 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94lb # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...95lb # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests96lb # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests97lb # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests98worker2 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether99worker1 # Starting virtiofs daemons...100worker2 # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...101worker1 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)102worker2 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)103worker1 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104worker2 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether105worker1 # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...106worker2 # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...107worker1 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)108worker2 # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests109worker1 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether110client # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)111worker1 # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...112client # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether113worker1 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)114client # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...115client # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)116worker1 # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether117worker2 # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests118worker1 # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...119worker2 # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests120worker1 # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests121client # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether122worker1 # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests123client # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...124client # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)125client # [2026-09-23T04:41:31Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether126client # [2026-09-23T04:41:31Z INFO virtiofsd] Waiting for vhost-user socket connection...127client # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests128worker1 # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-23T04:41:31Z INFO virtiofsd] Client connected, servicing requests131client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135client # 136client # 137client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138lb # 139lb # 140client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...141client # 142client # 143client # 144client # 145lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00146worker2 # 147worker2 # 148client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00149worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00150lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...151lb # 152lb # 153lb # 154lb # 155lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...157client # 158client # 159worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...160worker2 # 161worker2 # 162worker2 # 163worker2 # 164worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00165lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...166lb # 167lb # 168worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169worker2 # 170worker2 # 171worker1 # 172worker1 # 173worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00174worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...175worker1 # 176worker1 # 177worker1 # 178worker1 # 179worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180client # Booting from ROM...181lb # Booting from ROM...182worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...183worker1 # 184worker1 # 185worker2 # Booting from ROM...186worker1 # Booting from ROM...187lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0189lb # [ 0.000000] BIOS-provided physical RAM map:190lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200lb # [ 0.000000] NX (Execute Disable) protection: active201worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026202lb # [ 0.000000] APIC: Static calls initialized203lb # [ 0.000000] SMBIOS 2.8 present.204lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014205client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026206lb # [ 0.000000] DMI: Memory slots populated: 1/1207lb # [ 0.000000] Hypervisor detected: KVM208worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9ddfrc7c8w69kr1zczx1vf02afx7i8bp-nixos-system-worker2-test/init regInfo=/nix/store/liiyg21dddxspzy1m75waqzhgqq549bm-closure-info/registration console=ttyS0,115200n8 console=tty0209lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000210worker2 # [ 0.000000] BIOS-provided physical RAM map:211lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00212worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable213lb # [ 0.000000] kvm-clock: using sched offset of 454883244 cycles214worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved215client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qhqikc52z3k5ifiwyi2zsl4rf7zd53vq-nixos-system-client-test/init regInfo=/nix/store/fcb5v5vxajs7m326m62jh72759wys78k-closure-info/registration console=ttyS0,115200n8 console=tty0216client # [ 0.000000] BIOS-provided physical RAM map:217lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns218worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved219lb # [ 0.000005] tsc: Detected 2400.012 MHz processor220client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable221worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable222lb # [ 0.000806] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000223client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved224worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved225worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved226client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved227lb # [ 0.000834] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs228worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved229client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable230lb # [ 0.000836] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT231worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved232lb # [ 0.002726] found SMP MP-table at [mem 0x000f5450-0x000f545f]233client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved234lb # [ 0.002738] Using GB pages for direct mapping235worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved236client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved237lb # [ 0.002894] RAMDISK: [mem 0x5e36e000-0x5ffcffff]238worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved239client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved240lb # [ 0.002902] ACPI: Early table checksum verification disabled241worker2 # [ 0.000000] NX (Execute Disable) protection: active242worker2 # [ 0.000000] APIC: Static calls initialized243client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved244lb # [ 0.002905] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )245worker2 # [ 0.000000] SMBIOS 2.8 present.246client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved247lb # [ 0.002910] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)248worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014249client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved250worker2 # [ 0.000000] DMI: Memory slots populated: 1/1251lb # [ 0.002925] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)252worker2 # [ 0.000000] Hypervisor detected: KVM253client # [ 0.000000] NX (Execute Disable) protection: active254client # [ 0.000000] APIC: Static calls initialized255worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000256client # [ 0.000000] SMBIOS 2.8 present.257lb # [ 0.002933] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)258worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00259lb # [ 0.002935] ACPI: FACS 0x000000005FFE0000 000040260worker2 # [ 0.000000] kvm-clock: using sched offset of 462476387 cycles261client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014262client # [ 0.000000] DMI: Memory slots populated: 1/1263lb # [ 0.002936] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)264worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns265client # [ 0.000000] Hypervisor detected: KVM266worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor267lb # [ 0.002938] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)268client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000269worker2 # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000270client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00271lb # [ 0.002939] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)272worker2 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs273client # [ 0.000001] kvm-clock: using sched offset of 468719606 cycles274worker2 # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT275lb # [ 0.002941] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)276client # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns277worker2 # [ 0.002726] found SMP MP-table at [mem 0x000f5450-0x000f545f]278lb # [ 0.002942] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]279worker2 # [ 0.002738] Using GB pages for direct mapping280client # [ 0.000006] tsc: Detected 2400.012 MHz processor281worker2 # [ 0.002898] RAMDISK: [mem 0x5e36f000-0x5ffcffff]282client # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000283lb # [ 0.002943] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]284worker2 # [ 0.002906] ACPI: Early table checksum verification disabled285lb # [ 0.002944] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]286client # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs287worker2 # [ 0.002910] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )288lb # [ 0.002944] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]289client # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT290worker2 # [ 0.002914] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)291client # [ 0.002739] found SMP MP-table at [mem 0x000f5450-0x000f545f]292lb # [ 0.002945] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]293worker2 # [ 0.002918] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)294client # [ 0.002752] Using GB pages for direct mapping295lb # [ 0.002945] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]296client # [ 0.002891] RAMDISK: [mem 0x5e370000-0x5ffcffff]297worker2 # [ 0.002926] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)298client # [ 0.002900] ACPI: Early table checksum verification disabled299worker2 # [ 0.002928] ACPI: FACS 0x000000005FFE0000 000040300lb # [ 0.002946] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]301lb # [ 0.003174] No NUMA configuration found302client # [ 0.002903] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )303worker2 # [ 0.002929] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)304lb # [ 0.003175] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]305client # [ 0.002907] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)306worker2 # [ 0.002931] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)307lb # [ 0.003177] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]308lb # [ 0.005399] Zone ranges:309client # [ 0.002912] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)310worker2 # [ 0.002933] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)311lb # [ 0.005400] DMA [mem 0x0000000000001000-0x0000000000ffffff]312worker2 # [ 0.002934] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)313client # [ 0.002919] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)314lb # [ 0.005401] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]315client # [ 0.002921] ACPI: FACS 0x000000005FFE0000 000040316lb # [ 0.005402] Normal empty317worker2 # [ 0.002935] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]318lb # [ 0.005403] Device empty319worker2 # [ 0.002937] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]320lb # [ 0.005404] Movable zone start for each node321client # [ 0.002923] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)322lb # [ 0.005405] Early memory node ranges323worker2 # [ 0.002937] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]324client # [ 0.002925] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)325lb # [ 0.005405] node 0: [mem 0x0000000000001000-0x000000000009efff]326worker2 # [ 0.002938] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]327client # [ 0.002926] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)328lb # [ 0.005406] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]329worker2 # [ 0.002938] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]330lb # [ 0.005407] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]331worker2 # [ 0.002939] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]332client # [ 0.002928] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)333worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026334lb # [ 0.005427] On node 0, zone DMA: 1 pages in unavailable ranges335worker2 # [ 0.002939] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]336client # [ 0.002929] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]337worker2 # [ 0.003162] No NUMA configuration found338lb # [ 0.005688] On node 0, zone DMA: 97 pages in unavailable ranges339client # [ 0.002930] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]340lb # [ 0.031750] On node 0, zone DMA32: 40 pages in unavailable ranges341worker2 # [ 0.003162] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]342lb # [ 0.032216] ACPI: PM-Timer IO Port: 0x608343client # [ 0.002931] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]344worker2 # [ 0.003165] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]345worker2 # [ 0.005536] Zone ranges:346lb # [ 0.032230] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])347client # [ 0.002931] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]348worker2 # [ 0.005537] DMA [mem 0x0000000000001000-0x0000000000ffffff]349worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/iw1slifxbpr8i9q8sa6lz50n4d12ifxq-nixos-system-worker1-test/init regInfo=/nix/store/dwxwx816vd5bhxzb3g8qmghndnh53gjs-closure-info/registration console=ttyS0,115200n8 console=tty0350lb # [ 0.032254] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23351client # [ 0.002932] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]352worker2 # [ 0.005539] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]353worker1 # [ 0.000000] BIOS-provided physical RAM map:354worker2 # [ 0.005540] Normal empty355lb # [ 0.032257] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)356worker2 # [ 0.005541] Device empty357worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable358client # [ 0.002932] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]359lb # [ 0.032259] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)360worker2 # [ 0.005542] Movable zone start for each node361worker2 # [ 0.005543] Early memory node ranges362client # [ 0.002933] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]363worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved364lb # [ 0.032260] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)365client # [ 0.003201] No NUMA configuration found366worker2 # [ 0.005543] node 0: [mem 0x0000000000001000-0x000000000009efff]367worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved368worker2 # [ 0.005544] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]369lb # [ 0.032261] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)370client # [ 0.003202] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]371worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable372client # [ 0.003204] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]373lb # [ 0.032262] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)374worker2 # [ 0.005545] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]375client # [ 0.005759] Zone ranges:376worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved377worker2 # [ 0.005565] On node 0, zone DMA: 1 pages in unavailable ranges378lb # [ 0.032264] ACPI: Using ACPI (MADT) for SMP configuration information379client # [ 0.005761] DMA [mem 0x0000000000001000-0x0000000000ffffff]380worker2 # [ 0.005983] On node 0, zone DMA: 97 pages in unavailable ranges381worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved382lb # [ 0.032265] ACPI: HPET id: 0x8086a201 base: 0xfed00000383client # [ 0.005762] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]384worker2 # [ 0.034752] On node 0, zone DMA32: 40 pages in unavailable ranges385lb # [ 0.032270] TSC deadline timer available386worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved387client # [ 0.005763] Normal empty388worker2 # [ 0.035208] ACPI: PM-Timer IO Port: 0x608389lb # [ 0.032274] CPU topo: Max. logical packages: 1390client # [ 0.005764] Device empty391worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved392client # [ 0.005765] Movable zone start for each node393lb # [ 0.032275] CPU topo: Max. logical dies: 1394worker2 # [ 0.035220] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])395client # [ 0.005766] Early memory node ranges396lb # [ 0.032275] CPU topo: Max. dies per package: 1397worker2 # [ 0.035245] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23398worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved399client # [ 0.005766] node 0: [mem 0x0000000000001000-0x000000000009efff]400lb # [ 0.032278] CPU topo: Max. threads per core: 1401worker2 # [ 0.035249] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)402worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved403lb # [ 0.032279] CPU topo: Num. cores per package: 1404client # [ 0.005767] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]405worker1 # [ 0.000000] NX (Execute Disable) protection: active406worker2 # [ 0.035250] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)407lb # [ 0.032279] CPU topo: Num. threads per package: 1408worker1 # [ 0.000000] APIC: Static calls initialized409client # [ 0.005768] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]410worker2 # [ 0.035251] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)411worker1 # [ 0.000000] SMBIOS 2.8 present.412lb # [ 0.032280] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs413client # [ 0.005793] On node 0, zone DMA: 1 pages in unavailable ranges414worker2 # [ 0.035252] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)415lb # [ 0.032302] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()416client # [ 0.006094] On node 0, zone DMA: 97 pages in unavailable ranges417worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014418worker2 # [ 0.035253] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)419worker1 # [ 0.000000] DMI: Memory slots populated: 1/1420lb # [ 0.032335] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]421client # [ 0.036095] On node 0, zone DMA32: 40 pages in unavailable ranges422worker1 # [ 0.000000] Hypervisor detected: KVM423worker2 # [ 0.035255] ACPI: Using ACPI (MADT) for SMP configuration information424client # [ 0.036573] ACPI: PM-Timer IO Port: 0x608425worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000426worker2 # [ 0.035257] ACPI: HPET id: 0x8086a201 base: 0xfed00000427lb # [ 0.032337] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]428client # [ 0.036586] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])429worker2 # [ 0.035261] TSC deadline timer available430worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00431lb # [ 0.032338] [mem 0x60000000-0xafffffff] available for PCI devices432worker2 # [ 0.035265] CPU topo: Max. logical packages: 1433client # [ 0.036617] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23434worker1 # [ 0.000001] kvm-clock: using sched offset of 497981280 cycles435lb # [ 0.032340] Booting paravirtualized kernel on KVM436worker2 # [ 0.035266] CPU topo: Max. logical dies: 1437client # [ 0.036620] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)438worker2 # [ 0.035266] CPU topo: Max. dies per package: 1439worker2 # [ 0.035269] CPU topo: Max. threads per core: 1440worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns441lb # [ 0.032343] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns442client # [ 0.036621] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)443worker2 # [ 0.035270] CPU topo: Num. cores per package: 1444worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor445lb # [ 0.036817] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1446worker2 # [ 0.035270] CPU topo: Num. threads per package: 1447client # [ 0.036623] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)448worker1 # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000449worker2 # [ 0.035271] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs450client # [ 0.036624] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)451lb # [ 0.038938] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152452worker1 # [ 0.000843] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs453lb # [ 0.038982] kvm-guest: PV spinlocks disabled, single CPU454worker2 # [ 0.035290] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()455client # [ 0.036624] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)456worker1 # [ 0.000845] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT457worker2 # [ 0.035324] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]458client # [ 0.036627] ACPI: Using ACPI (MADT) for SMP configuration information459worker1 # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]460client # [ 0.036628] ACPI: HPET id: 0x8086a201 base: 0xfed00000461worker1 # [ 0.002748] Using GB pages for direct mapping462worker2 # [ 0.035325] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]463client # [ 0.036632] TSC deadline timer available464worker2 # [ 0.035327] [mem 0x60000000-0xafffffff] available for PCI devices465worker1 # [ 0.002893] RAMDISK: [mem 0x5e370000-0x5ffcffff]466client # [ 0.036636] CPU topo: Max. logical packages: 1467worker1 # [ 0.002902] ACPI: Early table checksum verification disabled468worker2 # [ 0.035329] Booting paravirtualized kernel on KVM469client # [ 0.036637] CPU topo: Max. logical dies: 1470worker1 # [ 0.002905] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )471client # [ 0.036637] CPU topo: Max. dies per package: 1472lb # [ 0.038984] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0473worker2 # [ 0.035332] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns474client # [ 0.036641] CPU topo: Max. threads per core: 1475worker1 # [ 0.002909] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)476client # [ 0.036641] CPU topo: Num. cores per package: 1477worker2 # [ 0.039830] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1478client # [ 0.036641] CPU topo: Num. threads per package: 1479lb # [ 0.039078] Unknown kernel command line parameters "regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration", will be passed to user space.480worker2 # [ 0.042117] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152481worker1 # [ 0.002913] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)482lb # [ 0.039091] random: crng init done483worker2 # [ 0.042160] kvm-guest: PV spinlocks disabled, single CPU484client # [ 0.036642] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs485worker1 # [ 0.002919] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)486lb # [ 0.039092] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes487client # [ 0.036661] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()488worker1 # [ 0.002921] ACPI: FACS 0x000000005FFE0000 000040489lb # [ 0.041167] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)490client # [ 0.036693] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]491worker1 # [ 0.002923] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)492lb # [ 0.042470] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)493client # [ 0.036695] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]494lb # [ 0.042503] Fallback order for Node 0: 0495worker1 # [ 0.002924] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)496client # [ 0.036696] [mem 0x60000000-0xafffffff] available for PCI devices497worker2 # [ 0.042162] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9ddfrc7c8w69kr1zczx1vf02afx7i8bp-nixos-system-worker2-test/init regInfo=/nix/store/liiyg21dddxspzy1m75waqzhgqq549bm-closure-info/registration console=ttyS0,115200n8 console=tty0498lb # [ 0.042506] Built 1 zonelists, mobility grouping on. Total pages: 393078499client # [ 0.036698] Booting paravirtualized kernel on KVM500worker1 # [ 0.002926] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)501lb # [ 0.042507] Policy zone: DMA32502worker2 # [ 0.042260] Unknown kernel command line parameters "regInfo=/nix/store/liiyg21dddxspzy1m75waqzhgqq549bm-closure-info/registration", will be passed to user space.503lb # [ 0.045061] mem auto-init: stack:all(zero), heap alloc:on, heap free:off504worker1 # [ 0.002927] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)505client # [ 0.036701] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns506worker2 # [ 0.042272] random: crng init done507lb # [ 0.048259] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1508worker1 # [ 0.002929] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]509client # [ 0.041194] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1510worker2 # [ 0.042273] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes511lb # [ 0.051522] allocated 3145728 bytes of page_ext512worker1 # [ 0.002930] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]513client # [ 0.043499] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152514lb # [ 0.061539] ftrace: allocating 48787 entries in 192 pages515worker2 # [ 0.044534] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)516client # [ 0.043546] kvm-guest: PV spinlocks disabled, single CPU517worker1 # [ 0.002931] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]518lb # [ 0.061541] ftrace: allocated 192 pages with 2 groups519worker2 # [ 0.045672] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)520lb # [ 0.062387] Dynamic Preempt: lazy521worker2 # [ 0.045704] Fallback order for Node 0: 0522worker1 # [ 0.002931] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]523lb # [ 0.062547] rcu: Preemptible hierarchical RCU implementation.524worker2 # [ 0.045707] Built 1 zonelists, mobility grouping on. Total pages: 393078525worker1 # [ 0.002932] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]526lb # [ 0.062548] rcu: RCU event tracing is enabled.527worker2 # [ 0.045708] Policy zone: DMA32528worker1 # [ 0.002932] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]529lb # [ 0.062548] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.530worker2 # [ 0.048338] mem auto-init: stack:all(zero), heap alloc:on, heap free:off531lb # [ 0.062550] Trampoline variant of Tasks RCU enabled.532worker1 # [ 0.002933] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]533worker2 # [ 0.052136] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1534worker1 # [ 0.003188] No NUMA configuration found535lb # [ 0.062550] Rude variant of Tasks RCU enabled.536worker2 # [ 0.055668] allocated 3145728 bytes of page_ext537client # [ 0.043548] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qhqikc52z3k5ifiwyi2zsl4rf7zd53vq-nixos-system-client-test/init regInfo=/nix/store/fcb5v5vxajs7m326m62jh72759wys78k-closure-info/registration console=ttyS0,115200n8 console=tty0538lb # [ 0.062551] Tracing variant of Tasks RCU enabled.539worker2 # [ 0.065956] ftrace: allocating 48787 entries in 192 pages540worker1 # [ 0.003189] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]541worker2 # [ 0.065958] ftrace: allocated 192 pages with 2 groups542worker1 # [ 0.003191] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]543lb # [ 0.062551] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.544worker2 # [ 0.066836] Dynamic Preempt: lazy545client # [ 0.043645] Unknown kernel command line parameters "regInfo=/nix/store/fcb5v5vxajs7m326m62jh72759wys78k-closure-info/registration", will be passed to user space.546worker1 # [ 0.005976] Zone ranges:547client # [ 0.043660] random: crng init done548lb # [ 0.062552] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1549worker2 # [ 0.066986] rcu: Preemptible hierarchical RCU implementation.550worker1 # [ 0.005977] DMA [mem 0x0000000000001000-0x0000000000ffffff]551worker2 # [ 0.066986] rcu: RCU event tracing is enabled.552client # [ 0.043660] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes553worker1 # [ 0.005979] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]554lb # [ 0.062579] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.555worker2 # [ 0.066987] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.556worker1 # [ 0.005980] Normal empty557client # [ 0.045929] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)558worker1 # [ 0.005981] Device empty559worker2 # [ 0.066988] Trampoline variant of Tasks RCU enabled.560lb # [ 0.062581] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.561worker1 # [ 0.005981] Movable zone start for each node562worker2 # [ 0.066989] Rude variant of Tasks RCU enabled.563client # [ 0.047063] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)564worker1 # [ 0.005982] Early memory node ranges565worker2 # [ 0.066989] Tracing variant of Tasks RCU enabled.566client # [ 0.047097] Fallback order for Node 0: 0567lb # [ 0.062582] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.568worker1 # [ 0.005982] node 0: [mem 0x0000000000001000-0x000000000009efff]569worker2 # [ 0.066990] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.570client # [ 0.047100] Built 1 zonelists, mobility grouping on. Total pages: 393078571lb # [ 0.066980] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16572worker1 # [ 0.005983] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]573client # [ 0.047101] Policy zone: DMA32574worker2 # [ 0.066991] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1575lb # [ 0.067592] rcu: srcu_init: Setting srcu_struct sizes based on contention.576worker1 # [ 0.005984] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]577client # [ 0.050051] mem auto-init: stack:all(zero), heap alloc:on, heap free:off578worker2 # [ 0.067050] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.579worker1 # [ 0.006004] On node 0, zone DMA: 1 pages in unavailable ranges580client # [ 0.053531] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1581lb # [ 0.067598] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns582worker1 # [ 0.006293] On node 0, zone DMA: 97 pages in unavailable ranges583worker2 # [ 0.067051] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.584client # [ 0.057071] allocated 3145728 bytes of page_ext585worker1 # [ 0.035229] On node 0, zone DMA32: 40 pages in unavailable ranges586client # [ 0.067318] ftrace: allocating 48787 entries in 192 pages587lb # [ 0.067708] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)588worker2 # [ 0.067052] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.589worker1 # [ 0.035733] ACPI: PM-Timer IO Port: 0x608590lb # [ 0.071478] Console: colour VGA+ 80x25591client # [ 0.067320] ftrace: allocated 192 pages with 2 groups592worker2 # [ 0.071498] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16593client # [ 0.068204] Dynamic Preempt: lazy594worker1 # [ 0.035745] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])595lb # [ 0.071481] printk: legacy console [tty0] enabled596client # [ 0.068352] rcu: Preemptible hierarchical RCU implementation.597worker2 # [ 0.071776] rcu: srcu_init: Setting srcu_struct sizes based on contention.598lb # [ 0.102435] printk: legacy console [ttyS0] enabled599worker1 # [ 0.035773] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23600client # [ 0.068353] rcu: RCU event tracing is enabled.601lb # [ 0.208463] ACPI: Core revision 20250807602worker1 # [ 0.035776] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)603worker2 # [ 0.071781] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns604client # [ 0.068353] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.605worker1 # [ 0.035778] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)606client # [ 0.068355] Trampoline variant of Tasks RCU enabled.607worker2 # [ 0.071897] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)608client # [ 0.068355] Rude variant of Tasks RCU enabled.609worker1 # [ 0.035779] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)610lb # [ 0.209450] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns611worker2 # [ 0.075422] Console: colour VGA+ 80x25612client # [ 0.068356] Tracing variant of Tasks RCU enabled.613worker2 # [ 0.075426] printk: legacy console [tty0] enabled614worker1 # [ 0.035780] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)615lb # [ 0.211147] APIC: Switch to symmetric I/O mode setup616worker2 # [ 0.104875] printk: legacy console [ttyS0] enabled617client # [ 0.068356] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.618lb # [ 0.212183] x2apic enabled619worker1 # [ 0.035780] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)620worker2 # [ 0.199860] ACPI: Core revision 20250807621client # [ 0.068357] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1622worker1 # [ 0.035783] ACPI: Using ACPI (MADT) for SMP configuration information623lb # [ 0.212982] APIC: Switched APIC routing to: physical x2apic624worker1 # [ 0.035784] ACPI: HPET id: 0x8086a201 base: 0xfed00000625worker2 # [ 0.200700] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns626client # [ 0.068424] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.627worker1 # [ 0.035787] TSC deadline timer available628worker1 # [ 0.035791] CPU topo: Max. logical packages: 1629worker2 # [ 0.202179] APIC: Switch to symmetric I/O mode setup630client # [ 0.068426] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.631worker1 # [ 0.035792] CPU topo: Max. logical dies: 1632lb # [ 0.214973] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1633worker2 # [ 0.203104] x2apic enabled634worker1 # [ 0.035792] CPU topo: Max. dies per package: 1635client # [ 0.068427] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.636worker1 # [ 0.035796] CPU topo: Max. threads per core: 1637worker2 # [ 0.203840] APIC: Switched APIC routing to: physical x2apic638client # [ 0.072837] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16639worker1 # [ 0.035797] CPU topo: Num. cores per package: 1640lb # [ 0.215980] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns641worker1 # [ 0.035797] CPU topo: Num. threads per package: 1642client # [ 0.073125] rcu: srcu_init: Setting srcu_struct sizes based on contention.643worker1 # [ 0.035797] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs644worker2 # [ 0.205690] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1645lb # [ 0.217723] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)646client # [ 0.073132] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns647worker1 # [ 0.035816] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()648lb # [ 0.218820] x86/cpu: User Mode Instruction Prevention (UMIP) activated649worker2 # [ 0.206669] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns650worker1 # [ 0.035847] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]651client # [ 0.073244] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)652client # [ 0.077153] Console: colour VGA+ 80x25653lb # [ 0.219870] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127654worker1 # [ 0.035849] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]655worker2 # [ 0.208330] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)656client # [ 0.077156] printk: legacy console [tty0] enabled657worker1 # [ 0.035850] [mem 0x60000000-0xafffffff] available for PCI devices658lb # [ 0.220721] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0659client # [ 0.107705] printk: legacy console [ttyS0] enabled660worker2 # [ 0.209414] x86/cpu: User Mode Instruction Prevention (UMIP) activated661worker1 # [ 0.035852] Booting paravirtualized kernel on KVM662client # [ 0.211227] ACPI: Core revision 20250807663worker2 # [ 0.210471] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127664lb # [ 0.221724] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto665worker1 # [ 0.035854] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns666worker2 # [ 0.211327] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0667client # [ 0.212123] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns668worker1 # [ 0.040357] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1669lb # [ 0.222721] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl670client # [ 0.213702] APIC: Switch to symmetric I/O mode setup671worker1 # [ 0.042711] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152672client # [ 0.214749] x2apic enabled673worker2 # [ 0.212332] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto674worker1 # [ 0.042756] kvm-guest: PV spinlocks disabled, single CPU675lb # [ 0.223721] Transient Scheduler Attacks: Vulnerable: No microcode676lb # [ 0.224720] Spectre V2 : Mitigation: Enhanced / Automatic IBRS677client # [ 0.215491] APIC: Switched APIC routing to: physical x2apic678worker2 # [ 0.214327] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl679lb # [ 0.226436] Speculative Return Stack Overflow: Mitigation: Safe RET680worker2 # [ 0.215328] Transient Scheduler Attacks: Vulnerable: No microcode681client # [ 0.217380] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1682worker2 # [ 0.216327] Spectre V2 : Mitigation: Enhanced / Automatic IBRS683worker1 # [ 0.042758] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/iw1slifxbpr8i9q8sa6lz50n4d12ifxq-nixos-system-worker1-test/init regInfo=/nix/store/dwxwx816vd5bhxzb3g8qmghndnh53gjs-closure-info/registration console=ttyS0,115200n8 console=tty0684worker2 # [ 0.217327] Speculative Return Stack Overflow: Mitigation: Safe RET685client # [ 0.218457] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns686worker1 # [ 0.042851] Unknown kernel command line parameters "regInfo=/nix/store/dwxwx816vd5bhxzb3g8qmghndnh53gjs-closure-info/registration", will be passed to user space.687worker1 # [ 0.042864] random: crng init done688client # [ 0.220279] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)689lb # [ 0.227513] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization690lb # [ 0.228728] Spectre V2 : Enabling IBPB for BPF691worker1 # [ 0.042865] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes692client # [ 0.222365] x86/cpu: User Mode Instruction Prevention (UMIP) activated693worker2 # [ 0.218327] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization694worker1 # [ 0.045201] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)695client # [ 0.223400] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127696lb # [ 0.229519] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier697worker2 # [ 0.220334] Spectre V2 : Enabling IBPB for BPF698worker1 # [ 0.046364] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)699lb # [ 0.230721] active return thunk: srso_alias_return_thunk700worker1 # [ 0.046400] Fallback order for Node 0: 0701client # [ 0.224277] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0702worker2 # [ 0.221328] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier703worker1 # [ 0.046403] Built 1 zonelists, mobility grouping on. Total pages: 393078704lb # [ 0.231746] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'705worker1 # [ 0.046404] Policy zone: DMA32706worker2 # [ 0.222329] active return thunk: srso_alias_return_thunk707client # [ 0.225280] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto708worker1 # [ 0.049376] mem auto-init: stack:all(zero), heap alloc:on, heap free:off709lb # [ 0.232720] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'710worker2 # [ 0.223356] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'711worker1 # [ 0.052728] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1712client # [ 0.226276] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl713lb # [ 0.233720] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'714worker1 # [ 0.056367] allocated 3145728 bytes of page_ext715worker2 # [ 0.224327] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'716worker1 # [ 0.066599] ftrace: allocating 48787 entries in 192 pages717client # [ 0.227277] Transient Scheduler Attacks: Vulnerable: No microcode718lb # [ 0.234720] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'719worker2 # [ 0.225327] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'720worker1 # [ 0.066601] ftrace: allocated 192 pages with 2 groups721worker1 # [ 0.067499] Dynamic Preempt: lazy722client # [ 0.228276] Spectre V2 : Mitigation: Enhanced / Automatic IBRS723lb # [ 0.235720] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'724worker2 # [ 0.226327] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'725worker1 # [ 0.067643] rcu: Preemptible hierarchical RCU implementation.726client # [ 0.229276] Speculative Return Stack Overflow: Mitigation: Safe RET727worker1 # [ 0.067643] rcu: RCU event tracing is enabled.728worker2 # [ 0.227327] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'729lb # [ 0.237524] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'730worker1 # [ 0.067644] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.731worker2 # [ 0.228327] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'732lb # [ 0.238553] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'733worker1 # [ 0.067646] Trampoline variant of Tasks RCU enabled.734worker1 # [ 0.067646] Rude variant of Tasks RCU enabled.735worker2 # [ 0.229327] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'736lb # [ 0.239721] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256737worker1 # [ 0.067646] Tracing variant of Tasks RCU enabled.738worker2 # [ 0.230328] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256739lb # [ 0.240720] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64740worker1 # [ 0.067647] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.741client # [ 0.230276] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization742worker2 # [ 0.231327] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64743worker1 # [ 0.067648] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1744lb # [ 0.241720] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512745client # [ 0.232284] Spectre V2 : Enabling IBPB for BPF746worker2 # [ 0.232327] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512747worker1 # [ 0.067707] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.748lb # [ 0.242720] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024749worker2 # [ 0.233327] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024750client # [ 0.233832] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier751worker1 # [ 0.067710] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.752lb # [ 0.243721] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8753worker2 # [ 0.234327] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8754client # [ 0.234277] active return thunk: srso_alias_return_thunk755worker1 # [ 0.067710] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.756lb # [ 0.244720] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.757client # [ 0.235163] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'758worker2 # [ 0.235327] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker1 # [ 0.072208] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16760worker1 # [ 0.072501] rcu: srcu_init: Setting srcu_struct sizes based on contention.761client # [ 0.236276] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'762client # [ 0.237276] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'763worker1 # [ 0.072509] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns764client # [ 0.238276] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'765worker1 # [ 0.072618] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)766worker1 # [ 0.076413] Console: colour VGA+ 80x25767client # [ 0.239276] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'768worker1 # [ 0.076415] printk: legacy console [tty0] enabled769worker1 # [ 0.108595] printk: legacy console [ttyS0] enabled770client # [ 0.240276] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'771worker1 # [ 0.210805] ACPI: Core revision 20250807772client # [ 0.241276] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'773client # [ 0.242276] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256774worker1 # [ 0.211736] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns775worker1 # [ 0.213333] APIC: Switch to symmetric I/O mode setup776client # [ 0.243276] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64777worker1 # [ 0.214373] x2apic enabled778client # [ 0.244276] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512779worker1 # [ 0.215117] APIC: Switched APIC routing to: physical x2apic780client # [ 0.245276] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024781client # [ 0.246276] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8782worker1 # [ 0.217041] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1783client # [ 0.247276] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.784worker1 # [ 0.218075] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns785worker1 # [ 0.219792] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)786worker1 # [ 0.220877] x86/cpu: User Mode Instruction Prevention (UMIP) activated787worker1 # [ 0.222918] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127788worker1 # [ 0.223789] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0789worker1 # [ 0.224793] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto790worker1 # [ 0.225789] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl791worker1 # [ 0.226789] Transient Scheduler Attacks: Vulnerable: No microcode792worker1 # [ 0.227788] Spectre V2 : Mitigation: Enhanced / Automatic IBRS793worker1 # [ 0.228789] Speculative Return Stack Overflow: Mitigation: Safe RET794worker1 # [ 0.229788] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization795worker1 # [ 0.231797] Spectre V2 : Enabling IBPB for BPF796worker1 # [ 0.232789] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier797worker1 # [ 0.233790] active return thunk: srso_alias_return_thunk798worker1 # [ 0.234685] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'799lb # [ 0.279192] Freeing SMP alternatives memory: 44K800lb # [ 0.279723] pid_max: default: 32768 minimum: 301801worker2 # [ 0.271368] Freeing SMP alternatives memory: 44K802worker1 # [ 0.235788] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'803worker2 # [ 0.272144] pid_max: default: 32768 minimum: 301804lb # [ 0.280810] LSM: initializing lsm=capability,landlock,yama,bpf,ima805worker1 # [ 0.236788] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'806lb # [ 0.281835] landlock: Up and running.807worker2 # [ 0.272950] LSM: initializing lsm=capability,landlock,yama,bpf,ima808worker1 # [ 0.237788] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'809lb # [ 0.282721] Yama: becoming mindful.810worker2 # [ 0.273435] landlock: Up and running.811worker2 # [ 0.274328] Yama: becoming mindful.812worker1 # [ 0.238788] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'813lb # [ 0.283558] LSM support for eBPF active814worker2 # [ 0.275449] LSM support for eBPF active815worker1 # [ 0.239788] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'816lb # [ 0.284286] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)817worker2 # [ 0.276179] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818worker1 # [ 0.240789] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'819lb # [ 0.284759] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)820worker2 # [ 0.277370] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)821worker1 # [ 0.241789] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256822worker1 # [ 0.243788] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64823lb # [ 0.288071] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)824worker1 # [ 0.244788] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512825worker2 # [ 0.279667] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)826worker1 # [ 0.245788] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024827lb # [ 0.289291] Performance Events: Fam17h+ core perfctr, AMD PMU driver.828worker2 # [ 0.280906] Performance Events: Fam17h+ core perfctr, AMD PMU driver.829worker1 # [ 0.246788] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8830lb # [ 0.289730] ... version: 2831worker2 # [ 0.281336] ... version: 2832lb # [ 0.290490] ... bit width: 48833worker2 # [ 0.282024] ... bit width: 48834worker1 # [ 0.247789] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.835worker2 # [ 0.282331] ... generic counters: 6836lb # [ 0.290724] ... generic counters: 6837client # [ 0.283734] Freeing SMP alternatives memory: 44K838worker2 # [ 0.283014] ... generic bitmap: 000000000000003f839lb # [ 0.291456] ... generic bitmap: 000000000000003f840worker2 # [ 0.283329] ... fixed-purpose counters: 0841client # [ 0.284281] pid_max: default: 32768 minimum: 301842lb # [ 0.291723] ... fixed-purpose counters: 0843worker2 # [ 0.283996] ... fixed-purpose bitmap: 0000000000000000844lb # [ 0.292488] ... fixed-purpose bitmap: 0000000000000000845client # [ 0.285382] LSM: initializing lsm=capability,landlock,yama,bpf,ima846worker2 # [ 0.284329] ... value mask: 0000ffffffffffff847lb # [ 0.292724] ... value mask: 0000ffffffffffff848client # [ 0.286413] landlock: Up and running.849worker2 # [ 0.285186] ... max period: 00007fffffffffff850lb # [ 0.293659] ... max period: 00007fffffffffff851client # [ 0.287820] Yama: becoming mindful.852worker2 # [ 0.285984] ... global_ctrl mask: 000000000000003f853lb # [ 0.294447] ... global_ctrl mask: 000000000000003f854worker2 # [ 0.286433] signal: max sigframe size: 3376855client # [ 0.288531] LSM support for eBPF active856lb # [ 0.294820] signal: max sigframe size: 3376857worker2 # [ 0.287223] rcu: Hierarchical SRCU implementation.858lb # [ 0.295657] rcu: Hierarchical SRCU implementation.859worker2 # [ 0.287902] rcu: Max phase no-delay instances is 400.860client # [ 0.289455] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)861lb # [ 0.296354] rcu: Max phase no-delay instances is 400.862client # [ 0.291326] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)863client # [ 0.294703] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)864worker2 # [ 0.293017] smp: Bringing up secondary CPUs ...865worker2 # [ 0.293345] smp: Brought up 1 node, 1 CPU866lb # [ 0.301364] smp: Bringing up secondary CPUs ...867client # [ 0.295969] Performance Events: Fam17h+ core perfctr, AMD PMU driver.868lb # [ 0.301737] smp: Brought up 1 node, 1 CPU869worker2 # [ 0.293991] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)870client # [ 0.296283] ... version: 2871lb # [ 0.302456] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)872client # [ 0.297072] ... bit width: 48873client # [ 0.297279] ... generic counters: 6874worker2 # [ 0.294549] Memory: 1454592K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)875client # [ 0.298036] ... generic bitmap: 000000000000003f876worker2 # [ 0.295636] devtmpfs: initialized877lb # [ 0.302898] Memory: 1454592K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)878client # [ 0.298339] ... fixed-purpose counters: 0879lb # [ 0.303961] devtmpfs: initialized880worker2 # [ 0.296468] x86/mm: Memory block size: 128MB881client # [ 0.299121] ... fixed-purpose bitmap: 0000000000000000882lb # [ 0.304743] x86/mm: Memory block size: 128MB883client # [ 0.300028] ... value mask: 0000ffffffffffff884worker2 # [ 0.298273] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)885client # [ 0.300279] ... max period: 00007fffffffffff886lb # [ 0.306606] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)887client # [ 0.301260] ... global_ctrl mask: 000000000000003f888worker2 # [ 0.299224] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).889client # [ 0.302153] signal: max sigframe size: 3376890worker2 # [ 0.300447] pinctrl core: initialized pinctrl subsystem891lb # [ 0.307648] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).892client # [ 0.302896] rcu: Hierarchical SRCU implementation.893worker2 # [ 0.301515] PM: RTC time: 04:41:31, date: 2026-09-23894lb # [ 0.308817] pinctrl core: initialized pinctrl subsystem895client # [ 0.303282] rcu: Max phase no-delay instances is 400.896lb # [ 0.309993] PM: RTC time: 04:41:31, date: 2026-09-23897worker2 # [ 0.305570] NET: Registered PF_NETLINK/PF_ROUTE protocol family898lb # [ 0.313441] NET: Registered PF_NETLINK/PF_ROUTE protocol family899worker2 # [ 0.306850] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations900client # [ 0.308909] smp: Bringing up secondary CPUs ...901lb # [ 0.314233] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations902client # [ 0.309293] smp: Brought up 1 node, 1 CPU903worker2 # [ 0.307356] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations904lb # [ 0.314745] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations905client # [ 0.310039] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)906worker2 # [ 0.308618] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations907worker2 # [ 0.309340] audit: initializing netlink subsys (disabled)908lb # [ 0.315997] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations909worker1 # [ 0.283612] Freeing SMP alternatives memory: 44K910lb # [ 0.316733] audit: initializing netlink subsys (disabled)911client # [ 0.310490] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)912worker2 # [ 0.310483] thermal_sys: Registered thermal governor 'fair_share'913worker1 # [ 0.284364] pid_max: default: 32768 minimum: 301914client # [ 0.311562] devtmpfs: initialized915worker2 # [ 0.310486] thermal_sys: Registered thermal governor 'bang_bang'916lb # [ 0.317940] thermal_sys: Registered thermal governor 'fair_share'917worker1 # [ 0.284880] LSM: initializing lsm=capability,landlock,yama,bpf,ima918worker2 # [ 0.311331] thermal_sys: Registered thermal governor 'step_wise'919client # [ 0.312499] x86/mm: Memory block size: 128MB920lb # [ 0.317942] thermal_sys: Registered thermal governor 'bang_bang'921worker1 # [ 0.285898] landlock: Up and running.922worker2 # [ 0.312237] thermal_sys: Registered thermal governor 'user_space'923worker1 # [ 0.287262] Yama: becoming mindful.924lb # [ 0.318724] thermal_sys: Registered thermal governor 'step_wise'925worker2 # [ 0.313025] audit: type=2000 audit(1790138492.663:1): state=initialized audit_enabled=0 res=1926client # [ 0.314307] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)927worker1 # [ 0.288004] LSM support for eBPF active928lb # [ 0.319711] thermal_sys: Registered thermal governor 'user_space'929worker2 # [ 0.314332] thermal_sys: Registered thermal governor 'power_allocator'930worker1 # [ 0.288897] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)931worker2 # [ 0.314349] cpuidle: using governor menu932lb # [ 0.320516] audit: type=2000 audit(1790138492.646:1): state=initialized audit_enabled=0 res=1933client # [ 0.315307] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).934client # [ 0.316380] pinctrl core: initialized pinctrl subsystem935lb # [ 0.322725] thermal_sys: Registered thermal governor 'power_allocator'936worker1 # [ 0.289833] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)937worker2 # [ 0.316531] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5938lb # [ 0.322739] cpuidle: using governor menu939client # [ 0.317551] PM: RTC time: 04:41:31, date: 2026-09-23940worker2 # [ 0.317622] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]941lb # [ 0.325084] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5942worker1 # [ 0.292776] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)943worker2 # [ 0.318333] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry944worker2 # [ 0.319345] PCI: Using configuration type 1 for base access945lb # [ 0.325990] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]946worker1 # [ 0.293373] Performance Events: Fam17h+ core perfctr, AMD PMU driver.947worker1 # [ 0.293793] ... version: 2948client # [ 0.321226] NET: Registered PF_NETLINK/PF_ROUTE protocol family949lb # [ 0.326726] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry950worker2 # [ 0.320470] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.951worker1 # [ 0.294512] ... bit width: 48952lb # [ 0.327735] PCI: Using configuration type 1 for base access953worker1 # [ 0.294791] ... generic counters: 6954client # [ 0.322579] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations955worker1 # [ 0.295529] ... generic bitmap: 000000000000003f956lb # [ 0.328878] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.957client # [ 0.323305] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations958worker1 # [ 0.295802] ... fixed-purpose counters: 0959worker1 # [ 0.296534] ... fixed-purpose bitmap: 0000000000000000960client # [ 0.324579] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations961worker1 # [ 0.296791] ... value mask: 0000ffffffffffff962client # [ 0.325290] audit: initializing netlink subsys (disabled)963worker1 # [ 0.297691] ... max period: 00007fffffffffff964worker2 # [ 0.325613] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages965worker1 # [ 0.298438] ... global_ctrl mask: 000000000000003f966client # [ 0.326280] audit: type=2000 audit(1790138492.669:1): state=initialized audit_enabled=0 res=1967worker2 # [ 0.326331] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page968worker1 # [ 0.298904] signal: max sigframe size: 3376969client # [ 0.327502] thermal_sys: Registered thermal governor 'fair_share'970lb # [ 0.333978] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages971worker1 # [ 0.299729] rcu: Hierarchical SRCU implementation.972worker1 # [ 0.300394] rcu: Max phase no-delay instances is 400.973client # [ 0.327504] thermal_sys: Registered thermal governor 'bang_bang'974lb # [ 0.334725] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page975client # [ 0.328280] thermal_sys: Registered thermal governor 'step_wise'976client # [ 0.329280] thermal_sys: Registered thermal governor 'user_space'977worker2 # [ 0.331331] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages978client # [ 0.330278] thermal_sys: Registered thermal governor 'power_allocator'979client # [ 0.331292] cpuidle: using governor menu980worker2 # [ 0.332330] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page981lb # [ 0.339724] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages982client # [ 0.333526] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5983worker1 # [ 0.305755] smp: Bringing up secondary CPUs ...984lb # [ 0.340723] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page985worker1 # [ 0.306388] smp: Brought up 1 node, 1 CPU986client # [ 0.334539] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]987worker1 # [ 0.306792] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)988client # [ 0.335282] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry989client # [ 0.336290] PCI: Using configuration type 1 for base access990worker1 # [ 0.307999] Memory: 1454628K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)991worker1 # [ 0.309055] devtmpfs: initialized992client # [ 0.337407] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.993worker1 # [ 0.309943] x86/mm: Memory block size: 128MB994worker2 # [ 0.342696] ACPI: Added _OSI(Module Device)995worker1 # [ 0.311765] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)996worker2 # [ 0.343331] ACPI: Added _OSI(Processor Device)997worker1 # [ 0.312696] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).998worker2 # [ 0.344083] ACPI: Added _OSI(Processor Aggregator Device)999client # [ 0.342532] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1000worker1 # [ 0.313862] pinctrl core: initialized pinctrl subsystem1001lb # [ 0.351142] ACPI: Added _OSI(Module Device)1002client # [ 0.343280] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1003lb # [ 0.351725] ACPI: Added _OSI(Processor Device)1004worker1 # [ 0.314984] PM: RTC time: 04:41:31, date: 2026-09-231005lb # [ 0.352534] ACPI: Added _OSI(Processor Aggregator Device)1006worker1 # [ 0.318670] NET: Registered PF_NETLINK/PF_ROUTE protocol family1007client # [ 0.348280] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1008worker1 # [ 0.320078] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations1009client # [ 0.349279] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1010worker1 # [ 0.320815] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1011worker2 # [ 0.352399] ACPI: 1 ACPI AML tables successfully acquired and loaded1012lb # [ 0.359653] ACPI: 1 ACPI AML tables successfully acquired and loaded1013worker1 # [ 0.322090] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1014worker1 # [ 0.322802] audit: initializing netlink subsys (disabled)1015worker1 # [ 0.323966] thermal_sys: Registered thermal governor 'fair_share'1016worker2 # [ 0.356759] ACPI: Interpreter enabled1017worker1 # [ 0.323969] thermal_sys: Registered thermal governor 'bang_bang'1018worker2 # [ 0.357347] ACPI: PM: (supports S0 S3 S4 S5)1019worker1 # [ 0.324792] thermal_sys: Registered thermal governor 'step_wise'1020worker2 # [ 0.358073] ACPI: Using IOAPIC for interrupt routing1021worker1 # [ 0.325735] thermal_sys: Registered thermal governor 'user_space'1022lb # [ 0.365694] ACPI: Interpreter enabled1023lb # [ 0.366217] ACPI: PM: (supports S0 S3 S4 S5)1024worker1 # [ 0.326537] audit: type=2000 audit(1790138492.700:1): state=initialized audit_enabled=0 res=11025lb # [ 0.366724] ACPI: Using IOAPIC for interrupt routing1026worker1 # [ 0.327794] thermal_sys: Registered thermal governor 'power_allocator'1027client # [ 0.359703] ACPI: Added _OSI(Module Device)1028worker2 # [ 0.360370] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1029worker1 # [ 0.327806] cpuidle: using governor menu1030client # [ 0.360280] ACPI: Added _OSI(Processor Device)1031worker2 # [ 0.361330] PCI: Using E820 reservations for host bridge windows1032lb # [ 0.367644] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1033client # [ 0.361041] ACPI: Added _OSI(Processor Aggregator Device)1034worker1 # [ 0.330041] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51035worker1 # [ 0.331029] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1036worker2 # [ 0.364395] ACPI: Enabled 2 GPEs in block 00 to 3F1037worker1 # [ 0.331794] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1038worker1 # [ 0.332803] PCI: Using configuration type 1 for base access1039lb # [ 0.372724] PCI: Using E820 reservations for host bridge windows1040lb # [ 0.373861] ACPI: Enabled 2 GPEs in block 00 to 3F1041worker1 # [ 0.333963] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1042client # [ 0.369486] ACPI: 1 ACPI AML tables successfully acquired and loaded1043worker2 # [ 0.371973] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1044worker2 # [ 0.372336] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1045worker2 # [ 0.373408] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1046lb # [ 0.381391] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1047worker1 # [ 0.341102] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1048client # [ 0.373698] ACPI: Interpreter enabled1049worker2 # [ 0.374453] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1050client # [ 0.374294] ACPI: PM: (supports S0 S3 S4 S5)1051worker1 # [ 0.341792] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1052lb # [ 0.381731] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1053client # [ 0.375001] ACPI: Using IOAPIC for interrupt routing1054worker2 # [ 0.375714] PCI host bridge to bus 0000:001055lb # [ 0.382801] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1056worker2 # [ 0.376335] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1057lb # [ 0.383847] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1058worker2 # [ 0.377330] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1059lb # [ 0.385092] PCI host bridge to bus 0000:001060worker2 # [ 0.378330] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1061client # [ 0.377331] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1062lb # [ 0.385727] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1063worker1 # [ 0.346793] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1064client # [ 0.378279] PCI: Using E820 reservations for host bridge windows1065worker2 # [ 0.379330] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1066worker1 # [ 0.347792] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1067lb # [ 0.386723] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1068worker2 # [ 0.380330] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1069lb # [ 0.387723] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1070worker2 # [ 0.381330] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1071client # [ 0.381416] ACPI: Enabled 2 GPEs in block 00 to 3F1072lb # [ 0.388723] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1073worker2 # [ 0.382330] pci_bus 0000:00: root bus resource [bus 00-ff]1074lb # [ 0.389723] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1075worker2 # [ 0.383280] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1076lb # [ 0.390723] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1077worker2 # [ 0.384794] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1078lb # [ 0.391724] pci_bus 0000:00: root bus resource [bus 00-ff]1079lb # [ 0.392725] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1080worker1 # [ 0.357119] ACPI: Added _OSI(Module Device)1081lb # [ 0.394182] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1082client # [ 0.388756] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1083worker1 # [ 0.357792] ACPI: Added _OSI(Processor Device)1084worker1 # [ 0.358550] ACPI: Added _OSI(Processor Aggregator Device)1085client # [ 0.389285] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1086client # [ 0.390358] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1087client # [ 0.391409] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1088client # [ 0.392649] PCI host bridge to bus 0000:001089client # [ 0.393283] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1090worker2 # [ 0.387430] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1091client # [ 0.394279] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1092lb # [ 0.396912] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1093worker2 # [ 0.388484] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1094worker1 # [ 0.365383] ACPI: 1 ACPI AML tables successfully acquired and loaded1095client # [ 0.395278] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1096lb # [ 0.397740] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1097worker2 # [ 0.389355] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1098lb # [ 0.398791] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1099client # [ 0.396279] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1100worker2 # [ 0.390734] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1101client # [ 0.397278] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1102lb # [ 0.400048] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1103client # [ 0.398279] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1104worker1 # [ 0.369211] ACPI: Interpreter enabled1105worker2 # [ 0.392134] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1106worker1 # [ 0.369806] ACPI: PM: (supports S0 S3 S4 S5)1107client # [ 0.399284] pci_bus 0000:00: root bus resource [bus 00-ff]1108worker1 # [ 0.370551] ACPI: Using IOAPIC for interrupt routing1109lb # [ 0.401528] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1110client # [ 0.400330] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1111worker2 # [ 0.395368] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1112client # [ 0.401838] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1113worker2 # [ 0.396268] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1114lb # [ 0.404760] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1115worker1 # [ 0.372836] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1116lb # [ 0.405677] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1117worker2 # [ 0.397080] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1118worker1 # [ 0.373792] PCI: Using E820 reservations for host bridge windows1119lb # [ 0.406496] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1120worker2 # [ 0.398190] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1121lb # [ 0.407595] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1122worker1 # [ 0.376915] ACPI: Enabled 2 GPEs in block 00 to 3F1123worker2 # [ 0.399702] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1124lb # [ 0.409784] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1125worker2 # [ 0.402379] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1126worker2 # [ 0.403295] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1127lb # [ 0.412689] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1128worker2 # [ 0.404123] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1129client # [ 0.404360] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1130lb # [ 0.413400] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1131client # [ 0.405294] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1132lb # [ 0.413745] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1133client # [ 0.406303] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1134worker2 # [ 0.405874] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1135worker1 # [ 0.385016] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1136lb # [ 0.415304] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1137client # [ 0.407463] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1138worker1 # [ 0.385798] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1139worker2 # [ 0.408276] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1140worker1 # [ 0.386870] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1141client # [ 0.409020] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1142worker2 # [ 0.409020] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1143worker1 # [ 0.387919] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1144worker2 # [ 0.409352] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1145lb # [ 0.418708] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1146worker1 # [ 0.389139] PCI host bridge to bus 0000:001147lb # [ 0.419435] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1148worker1 # [ 0.389795] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1149client # [ 0.411309] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1150worker2 # [ 0.410891] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1151lb # [ 0.419748] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1152worker1 # [ 0.390791] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1153client # [ 0.412205] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1154worker1 # [ 0.391791] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1155lb # [ 0.421370] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1156client # [ 0.413139] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1157worker1 # [ 0.392791] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1158client # [ 0.414284] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1159worker1 # [ 0.393791] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1160worker2 # [ 0.413367] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1161lb # [ 0.422754] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1162client # [ 0.415858] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1163worker2 # [ 0.414962] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1164worker1 # [ 0.394791] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1165lb # [ 0.423664] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1166worker1 # [ 0.395791] pci_bus 0000:00: root bus resource [bus 00-ff]1167worker2 # [ 0.415352] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1168worker2 # [ 0.416336] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1169client # [ 0.417308] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1170worker1 # [ 0.396781] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1171lb # [ 0.424754] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1172client # [ 0.418203] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1173lb # [ 0.425730] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1174worker2 # [ 0.417877] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1175worker1 # [ 0.398253] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1176client # [ 0.419099] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1177lb # [ 0.427323] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1178worker2 # [ 0.419342] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1179client # [ 0.420826] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1180worker2 # [ 0.420273] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1181lb # [ 0.428759] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1182lb # [ 0.429745] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1183client # [ 0.422323] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1184worker2 # [ 0.421818] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1185client # [ 0.423231] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1186lb # [ 0.431353] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1187client # [ 0.424112] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1188worker2 # [ 0.423342] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1189worker2 # [ 0.424335] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1190client # [ 0.425764] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1191worker1 # [ 0.400934] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1192lb # [ 0.433390] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1193worker2 # [ 0.425910] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1194worker1 # [ 0.401805] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1195lb # [ 0.433752] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1196worker1 # [ 0.402818] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1197worker2 # [ 0.427372] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1198worker1 # [ 0.404137] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1199client # [ 0.428315] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1200worker2 # [ 0.428352] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1201lb # [ 0.436239] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1202client # [ 0.429287] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1203worker1 # [ 0.405583] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1204client # [ 0.430301] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1205worker2 # [ 0.430429] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1206lb # [ 0.438743] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1207client # [ 0.431286] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1208lb # [ 0.439747] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1209client # [ 0.432916] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1210worker1 # [ 0.407800] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1211worker2 # [ 0.432382] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1212worker1 # [ 0.408655] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1213worker2 # [ 0.433352] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1214lb # [ 0.442073] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1215client # [ 0.434267] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1216worker1 # [ 0.409575] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1217worker2 # [ 0.434909] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1218client # [ 0.435080] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1219worker1 # [ 0.410798] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1220lb # [ 0.443737] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1221lb # [ 0.444749] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1222worker1 # [ 0.412385] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1223client # [ 0.436834] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1224worker2 # [ 0.437306] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1225lb # [ 0.446294] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1226client # [ 0.439280] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1227worker1 # [ 0.413800] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1228worker2 # [ 0.438003] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1229client # [ 0.440284] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1230worker1 # [ 0.414648] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1231worker2 # [ 0.438352] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1232worker1 # [ 0.415538] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1233lb # [ 0.448690] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1234client # [ 0.441871] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1235worker2 # [ 0.439874] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1236worker1 # [ 0.416395] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1237lb # [ 0.449723] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1238client # [ 0.443293] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1239worker2 # [ 0.441338] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1240lb # [ 0.450745] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1241worker2 # [ 0.442187] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1242client # [ 0.444294] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1243worker2 # [ 0.443117] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1244worker1 # [ 0.418800] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1245lb # [ 0.452309] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1246worker1 # [ 0.419654] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1247client # [ 0.446321] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1248worker1 # [ 0.420552] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1249lb # [ 0.454404] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1250worker2 # [ 0.444795] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1251client # [ 0.448148] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1252lb # [ 0.454731] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1253worker1 # [ 0.422250] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1254worker2 # [ 0.446052] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1255client # [ 0.449102] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1256lb # [ 0.455748] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1257worker2 # [ 0.446529] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1258client # [ 0.450760] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1259worker1 # [ 0.423798] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1260worker2 # [ 0.448015] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1261worker1 # [ 0.424654] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1262lb # [ 0.457314] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1263worker2 # [ 0.448532] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1264client # [ 0.452315] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1265worker1 # [ 0.425546] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1266lb # [ 0.458558] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1267worker2 # [ 0.450013] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1268worker1 # [ 0.425799] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1269client # [ 0.453251] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1270lb # [ 0.459630] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1271client # [ 0.454303] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1272worker1 # [ 0.427370] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1273lb # [ 0.461465] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1274client # [ 0.455928] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1275worker1 # [ 0.428641] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1276lb # [ 0.461931] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1277worker1 # [ 0.429549] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1278client # [ 0.457313] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1279lb # [ 0.463419] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1280client # [ 0.458252] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1281worker1 # [ 0.431279] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1282client # [ 0.459099] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1283worker1 # [ 0.432779] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1284client # [ 0.460863] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1285worker1 # [ 0.433817] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1286client # [ 0.461874] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1287worker1 # [ 0.435452] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1288worker2 # [ 0.450586] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1289client # [ 0.462495] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1290worker1 # [ 0.437606] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1291client # [ 0.463858] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1292worker1 # [ 0.438585] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1293client # [ 0.464516] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1294worker1 # [ 0.440242] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1295client # [ 0.465832] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1296worker2 # [ 0.453382] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1297worker1 # [ 0.441805] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1298lb # [ 0.464022] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1299worker2 # [ 0.454578] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1300worker1 # [ 0.442816] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1301worker2 # [ 0.455652] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1302worker1 # [ 0.444353] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1303lb # [ 0.467512] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1304worker2 # [ 0.456592] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1305lb # [ 0.468792] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1306worker1 # [ 0.445812] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1307worker1 # [ 0.446716] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1308lb # [ 0.470080] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1309worker2 # [ 0.459346] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1310worker1 # [ 0.447579] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1311worker2 # [ 0.460238] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1312lb # [ 0.471030] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1313worker1 # [ 0.449256] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1314worker2 # [ 0.461661] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1315lb # [ 0.473496] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1316worker2 # [ 0.463058] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1317client # [ 0.466536] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1318worker1 # [ 0.450799] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1319lb # [ 0.474408] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1320worker1 # [ 0.451716] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1321worker1 # [ 0.452555] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1322lb # [ 0.475860] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1323worker2 # [ 0.466379] ACPI: PCI: Interrupt link LNKA configured for IRQ 101324worker1 # [ 0.454280] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1325lb # [ 0.477650] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1326client # [ 0.469939] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1327worker2 # [ 0.467434] ACPI: PCI: Interrupt link LNKB configured for IRQ 101328worker1 # [ 0.455425] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1329worker2 # [ 0.468432] ACPI: PCI: Interrupt link LNKC configured for IRQ 111330client # [ 0.470595] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1331worker2 # [ 0.469646] ACPI: PCI: Interrupt link LNKD configured for IRQ 111332worker1 # [ 0.456013] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1333client # [ 0.471615] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1334worker2 # [ 0.470427] ACPI: PCI: Interrupt link LNKE configured for IRQ 101335lb # [ 0.481864] ACPI: PCI: Interrupt link LNKA configured for IRQ 101336worker2 # [ 0.471428] ACPI: PCI: Interrupt link LNKF configured for IRQ 101337worker1 # [ 0.457810] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1338client # [ 0.472569] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1339lb # [ 0.482834] ACPI: PCI: Interrupt link LNKB configured for IRQ 101340worker2 # [ 0.472430] ACPI: PCI: Interrupt link LNKG configured for IRQ 111341worker1 # [ 0.458875] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1342lb # [ 0.484081] ACPI: PCI: Interrupt link LNKC configured for IRQ 111343worker2 # [ 0.473457] ACPI: PCI: Interrupt link LNKH configured for IRQ 111344worker1 # [ 0.460327] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1345worker2 # [ 0.474364] ACPI: PCI: Interrupt link GSIA configured for IRQ 161346lb # [ 0.484825] ACPI: PCI: Interrupt link LNKD configured for IRQ 111347client # [ 0.475231] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1348worker2 # [ 0.475273] ACPI: PCI: Interrupt link GSIB configured for IRQ 171349lb # [ 0.485830] ACPI: PCI: Interrupt link LNKE configured for IRQ 101350client # [ 0.475960] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1351worker2 # [ 0.476076] ACPI: PCI: Interrupt link GSIC configured for IRQ 181352lb # [ 0.486825] ACPI: PCI: Interrupt link LNKF configured for IRQ 101353worker2 # [ 0.476345] ACPI: PCI: Interrupt link GSID configured for IRQ 191354client # [ 0.476623] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1355lb # [ 0.487831] ACPI: PCI: Interrupt link LNKG configured for IRQ 111356worker2 # [ 0.477316] ACPI: PCI: Interrupt link GSIE configured for IRQ 201357lb # [ 0.488840] ACPI: PCI: Interrupt link LNKH configured for IRQ 111358client # [ 0.477912] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1359worker2 # [ 0.478072] ACPI: PCI: Interrupt link GSIF configured for IRQ 211360lb # [ 0.489765] ACPI: PCI: Interrupt link GSIA configured for IRQ 161361worker2 # [ 0.479079] ACPI: PCI: Interrupt link GSIG configured for IRQ 221362worker2 # [ 0.479344] ACPI: PCI: Interrupt link GSIH configured for IRQ 231363lb # [ 0.490739] ACPI: PCI: Interrupt link GSIB configured for IRQ 171364lb # [ 0.491738] ACPI: PCI: Interrupt link GSIC configured for IRQ 181365worker2 # [ 0.481288] iommu: Default domain type: Translated1366client # [ 0.481613] ACPI: PCI: Interrupt link LNKA configured for IRQ 101367lb # [ 0.492739] ACPI: PCI: Interrupt link GSID configured for IRQ 191368worker2 # [ 0.481930] iommu: DMA domain TLB invalidation policy: lazy mode1369client # [ 0.482388] ACPI: PCI: Interrupt link LNKB configured for IRQ 101370lb # [ 0.493738] ACPI: PCI: Interrupt link GSIE configured for IRQ 201371worker2 # [ 0.482601] ACPI: bus type USB registered1372lb # [ 0.494735] ACPI: PCI: Interrupt link GSIF configured for IRQ 211373client # [ 0.483387] ACPI: PCI: Interrupt link LNKC configured for IRQ 111374worker2 # [ 0.483351] usbcore: registered new interface driver usbfs1375lb # [ 0.495734] ACPI: PCI: Interrupt link GSIG configured for IRQ 221376worker2 # [ 0.484238] usbcore: registered new interface driver hub1377client # [ 0.484380] ACPI: PCI: Interrupt link LNKD configured for IRQ 111378worker1 # [ 0.461020] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1379worker2 # [ 0.484997] usbcore: registered new device driver usb1380lb # [ 0.496734] ACPI: PCI: Interrupt link GSIH configured for IRQ 231381client # [ 0.485378] ACPI: PCI: Interrupt link LNKE configured for IRQ 101382worker1 # [ 0.463335] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1383worker2 # [ 0.486246] NetLabel: Initializing1384client # [ 0.486384] ACPI: PCI: Interrupt link LNKF configured for IRQ 101385lb # [ 0.498834] iommu: Default domain type: Translated1386worker2 # [ 0.486774] NetLabel: domain hash size = 1281387client # [ 0.487381] ACPI: PCI: Interrupt link LNKG configured for IRQ 111388lb # [ 0.499680] iommu: DMA domain TLB invalidation policy: lazy mode1389worker1 # [ 0.464099] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1390worker2 # [ 0.487330] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1391lb # [ 0.500751] ACPI: bus type USB registered1392worker2 # [ 0.488324] NetLabel: unlabeled traffic allowed by default1393worker1 # [ 0.465112] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1394client # [ 0.488379] ACPI: PCI: Interrupt link LNKH configured for IRQ 111395worker2 # [ 0.489248] PCI: Using ACPI for IRQ routing1396lb # [ 0.501535] usbcore: registered new interface driver usbfs1397client # [ 0.490150] ACPI: PCI: Interrupt link GSIA configured for IRQ 161398lb # [ 0.502655] usbcore: registered new interface driver hub1399worker1 # [ 0.466062] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1400client # [ 0.491100] ACPI: PCI: Interrupt link GSIB configured for IRQ 171401lb # [ 0.503500] usbcore: registered new device driver usb1402worker1 # [ 0.467640] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1403client # [ 0.492076] ACPI: PCI: Interrupt link GSIC configured for IRQ 181404lb # [ 0.505301] NetLabel: Initializing1405worker1 # [ 0.468507] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1406lb # [ 0.505724] NetLabel: domain hash size = 1281407client # [ 0.493296] ACPI: PCI: Interrupt link GSID configured for IRQ 191408client # [ 0.494293] ACPI: PCI: Interrupt link GSIE configured for IRQ 201409worker1 # [ 0.469124] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1410client # [ 0.495294] ACPI: PCI: Interrupt link GSIF configured for IRQ 211411lb # [ 0.506535] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1412lb # [ 0.507766] NetLabel: unlabeled traffic allowed by default1413client # [ 0.496291] ACPI: PCI: Interrupt link GSIG configured for IRQ 221414lb # [ 0.508736] PCI: Using ACPI for IRQ routing1415client # [ 0.497293] ACPI: PCI: Interrupt link GSIH configured for IRQ 231416worker1 # [ 0.472811] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1417client # [ 0.499379] iommu: Default domain type: Translated1418client # [ 0.500253] iommu: DMA domain TLB invalidation policy: lazy mode1419client # [ 0.501357] ACPI: bus type USB registered1420client # [ 0.502151] usbcore: registered new interface driver usbfs1421worker1 # [ 0.477301] ACPI: PCI: Interrupt link LNKA configured for IRQ 101422client # [ 0.503172] usbcore: registered new interface driver hub1423client # [ 0.504061] usbcore: registered new device driver usb1424worker1 # [ 0.477901] ACPI: PCI: Interrupt link LNKB configured for IRQ 101425worker1 # [ 0.478894] ACPI: PCI: Interrupt link LNKC configured for IRQ 111426client # [ 0.506119] NetLabel: Initializing1427worker1 # [ 0.479891] ACPI: PCI: Interrupt link LNKD configured for IRQ 111428client # [ 0.506762] NetLabel: domain hash size = 1281429client # [ 0.507278] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1430worker1 # [ 0.480902] ACPI: PCI: Interrupt link LNKE configured for IRQ 101431client # [ 0.508330] NetLabel: unlabeled traffic allowed by default1432worker1 # [ 0.481898] ACPI: PCI: Interrupt link LNKF configured for IRQ 101433client # [ 0.509292] PCI: Using ACPI for IRQ routing1434worker1 # [ 0.482898] ACPI: PCI: Interrupt link LNKG configured for IRQ 111435worker1 # [ 0.483895] ACPI: PCI: Interrupt link LNKH configured for IRQ 111436worker1 # [ 0.484832] ACPI: PCI: Interrupt link GSIA configured for IRQ 161437worker1 # [ 0.485806] ACPI: PCI: Interrupt link GSIB configured for IRQ 171438worker1 # [ 0.486806] ACPI: PCI: Interrupt link GSIC configured for IRQ 181439worker1 # [ 0.487806] ACPI: PCI: Interrupt link GSID configured for IRQ 191440worker1 # [ 0.488802] ACPI: PCI: Interrupt link GSIE configured for IRQ 201441worker1 # [ 0.489805] ACPI: PCI: Interrupt link GSIF configured for IRQ 211442worker1 # [ 0.490803] ACPI: PCI: Interrupt link GSIG configured for IRQ 221443worker1 # [ 0.491803] ACPI: PCI: Interrupt link GSIH configured for IRQ 231444worker1 # [ 0.493907] iommu: Default domain type: Translated1445worker1 # [ 0.494799] iommu: DMA domain TLB invalidation policy: lazy mode1446worker1 # [ 0.496104] ACPI: bus type USB registered1447worker1 # [ 0.496863] usbcore: registered new interface driver usbfs1448worker1 # [ 0.497804] usbcore: registered new interface driver hub1449worker1 # [ 0.498800] usbcore: registered new device driver usb1450worker1 # [ 0.500700] NetLabel: Initializing1451worker1 # [ 0.501240] NetLabel: domain hash size = 1281452worker1 # [ 0.501791] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453worker1 # [ 0.502827] NetLabel: unlabeled traffic allowed by default1454worker1 # [ 0.503859] PCI: Using ACPI for IRQ routing1455worker2 # [ 0.533609] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456worker2 # [ 0.534326] pci 0000:00:01.0: vgaarb: bridge control possible1457worker2 # [ 0.534326] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458worker2 # [ 0.534334] vgaarb: loaded1459worker2 # [ 0.535052] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460worker2 # [ 0.535331] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461worker2 # [ 0.538418] clocksource: Switched to clocksource kvm-clock1462worker2 # [ 0.541977] VFS: Disk quotas dquot_6.6.01463worker2 # [ 0.542700] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464worker2 # [ 0.544018] pnp: PnP ACPI init1465worker2 # [ 0.544904] ACPI: IRQ 4 override to edge(!), high(!)1466lb # [ 0.561466] pci 0000:00:01.0: vgaarb: setting as boot VGA device1467worker2 # [ 0.545936] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1468lb # [ 0.561719] pci 0000:00:01.0: vgaarb: bridge control possible1469worker2 # [ 0.547424] pnp: PnP ACPI: found 5 devices1470lb # [ 0.561719] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1471lb # [ 0.561727] vgaarb: loaded1472lb # [ 0.562607] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01473lb # [ 0.563411] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1474worker2 # [ 0.555030] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1475lb # [ 0.566820] clocksource: Switched to clocksource kvm-clock1476worker2 # [ 0.556630] clocksource: Switched to clocksource acpi_pm1477worker2 # [ 0.557641] NET: Registered PF_INET protocol family1478client # [ 0.560643] pci 0000:00:01.0: vgaarb: setting as boot VGA device1479client # [ 0.561275] pci 0000:00:01.0: vgaarb: bridge control possible1480worker2 # [ 0.558808] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1481lb # [ 0.570610] VFS: Disk quotas dquot_6.6.01482client # [ 0.561275] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1483lb # [ 0.571440] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1484client # [ 0.561284] vgaarb: loaded1485lb # [ 0.572922] pnp: PnP ACPI init1486client # [ 0.562060] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01487lb # [ 0.573777] ACPI: IRQ 4 override to edge(!), high(!)1488client # [ 0.562930] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1489lb # [ 0.574890] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1490lb # [ 0.576597] pnp: PnP ACPI: found 5 devices1491client # [ 0.565492] clocksource: Switched to clocksource kvm-clock1492client # [ 0.569581] VFS: Disk quotas dquot_6.6.01493client # [ 0.570324] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1494client # [ 0.571828] pnp: PnP ACPI init1495client # [ 0.572773] ACPI: IRQ 4 override to edge(!), high(!)1496lb # [ 0.584343] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1497lb # [ 0.585916] clocksource: Switched to clocksource acpi_pm1498client # [ 0.573879] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1499worker2 # [ 0.575464] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1500lb # [ 0.586990] NET: Registered PF_INET protocol family1501worker1 # [ 0.550837] pci 0000:00:01.0: vgaarb: setting as boot VGA device1502client # [ 0.575598] pnp: PnP ACPI: found 5 devices1503worker1 # [ 0.551787] pci 0000:00:01.0: vgaarb: bridge control possible1504worker2 # [ 0.577014] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1505lb # [ 0.588219] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1506worker1 # [ 0.551787] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1507worker2 # [ 0.578271] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1508worker1 # [ 0.551795] vgaarb: loaded1509worker2 # [ 0.579590] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1510worker1 # [ 0.552556] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01511worker2 # [ 0.580797] TCP: Hash tables configured (established 16384 bind 16384)1512worker1 # [ 0.552793] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1513worker2 # [ 0.581895] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1514worker2 # [ 0.583298] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1515client # [ 0.583376] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1516worker2 # [ 0.584587] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1517worker1 # [ 0.555880] clocksource: Switched to clocksource kvm-clock1518client # [ 0.584928] clocksource: Switched to clocksource acpi_pm1519worker2 # [ 0.585967] NET: Registered PF_UNIX/PF_LOCAL protocol family1520client # [ 0.586037] NET: Registered PF_INET protocol family1521worker2 # [ 0.587053] NET: Registered PF_XDP protocol family1522worker2 # [ 0.588066] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1523worker1 # [ 0.559429] VFS: Disk quotas dquot_6.6.01524client # [ 0.587342] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1525worker2 # [ 0.589217] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1526worker1 # [ 0.560218] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1527worker1 # [ 0.561636] pnp: PnP ACPI init1528worker2 # [ 0.590403] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1529worker1 # [ 0.562476] ACPI: IRQ 4 override to edge(!), high(!)1530worker2 # [ 0.591758] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1531worker1 # [ 0.563481] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1532worker2 # [ 0.592982] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1533worker1 # [ 0.565037] pnp: PnP ACPI: found 5 devices1534worker2 # [ 0.594235] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1535lb # [ 0.604998] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1536worker2 # [ 0.595971] ACPI: \_SB_.GSIA: Enabled at IRQ 161537lb # [ 0.606574] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1538lb # [ 0.607879] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1539worker2 # [ 0.598294] ACPI: \_SB_.GSIB: Enabled at IRQ 171540lb # [ 0.609216] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1541lb # [ 0.610446] TCP: Hash tables configured (established 16384 bind 16384)1542worker2 # [ 0.600730] ACPI: \_SB_.GSIC: Enabled at IRQ 181543lb # [ 0.611532] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1544lb # [ 0.612761] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1545worker1 # [ 0.572730] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1546worker2 # [ 0.602819] ACPI: \_SB_.GSID: Enabled at IRQ 191547lb # [ 0.613919] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1548worker1 # [ 0.574238] clocksource: Switched to clocksource acpi_pm1549lb # [ 0.615161] NET: Registered PF_UNIX/PF_LOCAL protocol family1550worker2 # [ 0.604581] PCI: CLS 0 bytes, default 641551worker1 # [ 0.575308] NET: Registered PF_INET protocol family1552lb # [ 0.616113] NET: Registered PF_XDP protocol family1553lb # [ 0.616969] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1554worker2 # [ 0.605488] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1555worker1 # [ 0.576560] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1556client # [ 0.604873] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1557worker2 # [ 0.607166] Trying to unpack rootfs image as initramfs...1558lb # [ 0.618029] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1559client # [ 0.606552] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1560lb # [ 0.619096] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1561client # [ 0.607970] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1562lb # [ 0.620292] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1563lb # [ 0.621476] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1564client # [ 0.609554] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1565lb # [ 0.622632] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1566client # [ 0.610917] TCP: Hash tables configured (established 16384 bind 16384)1567lb # [ 0.624405] ACPI: \_SB_.GSIA: Enabled at IRQ 161568client # [ 0.612227] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1569client # [ 0.613575] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1570lb # [ 0.626716] ACPI: \_SB_.GSIB: Enabled at IRQ 171571client # [ 0.614735] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1572client # [ 0.616082] NET: Registered PF_UNIX/PF_LOCAL protocol family1573lb # [ 0.628674] ACPI: \_SB_.GSIC: Enabled at IRQ 181574client # [ 0.617134] NET: Registered PF_XDP protocol family1575client # [ 0.618050] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1576lb # [ 0.630687] ACPI: \_SB_.GSID: Enabled at IRQ 191577client # [ 0.619167] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1578lb # [ 0.632406] PCI: CLS 0 bytes, default 641579client # [ 0.620225] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1580client # [ 0.621431] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1581lb # [ 0.633338] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1582client # [ 0.622631] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1583lb # [ 0.635108] Trying to unpack rootfs image as initramfs...1584worker1 # [ 0.594211] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1585client # [ 0.623805] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1586worker1 # [ 0.595794] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1587client # [ 0.625671] ACPI: \_SB_.GSIA: Enabled at IRQ 161588worker1 # [ 0.597177] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1589worker1 # [ 0.598581] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1590client # [ 0.628139] ACPI: \_SB_.GSIB: Enabled at IRQ 171591worker1 # [ 0.599914] TCP: Hash tables configured (established 16384 bind 16384)1592worker1 # [ 0.601035] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1593client # [ 0.630166] ACPI: \_SB_.GSIC: Enabled at IRQ 181594worker1 # [ 0.602304] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1595client # [ 0.632076] ACPI: \_SB_.GSID: Enabled at IRQ 191596worker1 # [ 0.603339] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1597worker1 # [ 0.604544] NET: Registered PF_UNIX/PF_LOCAL protocol family1598client # [ 0.633809] PCI: CLS 0 bytes, default 641599worker1 # [ 0.605569] NET: Registered PF_XDP protocol family1600worker1 # [ 0.606375] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1601client # [ 0.634810] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1602worker1 # [ 0.607370] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1603client # [ 0.636655] Trying to unpack rootfs image as initramfs...1604worker1 # [ 0.608383] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1605worker1 # [ 0.609529] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1606worker1 # [ 0.610646] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1607worker1 # [ 0.611776] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1608worker1 # [ 0.613600] ACPI: \_SB_.GSIA: Enabled at IRQ 161609worker1 # [ 0.615701] ACPI: \_SB_.GSIB: Enabled at IRQ 171610worker1 # [ 0.617695] ACPI: \_SB_.GSIC: Enabled at IRQ 181611worker1 # [ 0.619662] ACPI: \_SB_.GSID: Enabled at IRQ 191612worker1 # [ 0.621396] PCI: CLS 0 bytes, default 641613worker1 # [ 0.622429] Trying to unpack rootfs image as initramfs...1614worker2 # [ 0.654018] Initialise system trusted keyrings1615worker1 # [ 0.624433] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1616worker2 # [ 0.656653] workingset: timestamp_bits=40 max_order=19 bucket_order=01617lb # [ 0.679315] Initialise system trusted keyrings1618lb # [ 0.682198] workingset: timestamp_bits=40 max_order=19 bucket_order=01619worker2 # [ 0.681524] Key type asymmetric registered1620worker2 # [ 0.682374] Asymmetric key parser 'x509' registered1621client # [ 0.682740] Initialise system trusted keyrings1622worker2 # [ 0.685635] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1623worker2 # [ 0.687127] io scheduler mq-deadline registered1624client # [ 0.686223] workingset: timestamp_bits=40 max_order=19 bucket_order=01625worker2 # [ 0.688035] io scheduler kyber registered1626lb # [ 0.703495] Key type asymmetric registered1627worker2 # [ 0.692260] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1628lb # [ 0.704243] Asymmetric key parser 'x509' registered1629worker2 # [ 0.693577] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630lb # [ 0.705104] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1631worker1 # [ 0.665721] Initialise system trusted keyrings1632worker2 # [ 0.697252] Linux agpgart interface v0.1031633worker2 # [ 0.698049] ACPI: bus type drm_connector registered1634worker1 # [ 0.669358] workingset: timestamp_bits=40 max_order=19 bucket_order=01635lb # [ 0.710282] io scheduler mq-deadline registered1636lb # [ 0.711042] io scheduler kyber registered1637worker2 # [ 0.703075] usbcore: registered new interface driver usbserial_generic1638lb # [ 0.714745] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1639worker2 # [ 0.704143] usbserial: USB Serial support registered for generic1640lb # [ 0.716092] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1641worker2 # [ 0.705245] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1642worker2 # [ 0.705245] Please enable it if your BIOS has the CPPC option.1643worker2 # [ 0.707658] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1644client # [ 0.707702] Key type asymmetric registered1645lb # [ 0.719938] Linux agpgart interface v0.1031646lb # [ 0.720762] ACPI: bus type drm_connector registered1647client # [ 0.710181] Asymmetric key parser 'x509' registered1648client # [ 0.711096] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1649worker2 # [ 0.713748] drop_monitor: Initializing network drop monitor service1650lb # [ 0.725573] usbcore: registered new interface driver usbserial_generic1651worker2 # [ 0.714899] NET: Registered PF_INET6 protocol family1652client # [ 0.714307] io scheduler mq-deadline registered1653lb # [ 0.726687] usbserial: USB Serial support registered for generic1654client # [ 0.715091] io scheduler kyber registered1655worker2 # [ 0.718238] Segment Routing with IPv61656lb # [ 0.730156] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1657worker1 # [ 0.690800] Key type asymmetric registered1658client # [ 0.719391] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1659lb # [ 0.730156] Please enable it if your BIOS has the CPPC option.1660worker2 # [ 0.721599] In-situ OAM (IOAM) with IPv61661lb # [ 0.732493] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1662client # [ 0.720788] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1663worker2 # [ 0.722668] IPI shorthand broadcast: enabled1664worker1 # [ 0.693315] Asymmetric key parser 'x509' registered1665worker1 # [ 0.694180] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1666lb # [ 0.736285] drop_monitor: Initializing network drop monitor service1667lb # [ 0.737522] NET: Registered PF_INET6 protocol family1668worker1 # [ 0.697426] io scheduler mq-deadline registered1669client # [ 0.726367] Linux agpgart interface v0.1031670worker1 # [ 0.698193] io scheduler kyber registered1671client # [ 0.727172] ACPI: bus type drm_connector registered1672lb # [ 0.740603] Segment Routing with IPv61673worker1 # [ 0.700967] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1674client # [ 0.730611] usbcore: registered new interface driver usbserial_generic1675worker2 # [ 0.731349] sched_clock: Marking stable (598020458, 132545167)->(809289733, -78724108)1676client # [ 0.731852] usbserial: USB Serial support registered for generic1677lb # [ 0.744177] In-situ OAM (IOAM) with IPv61678worker1 # [ 0.704315] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1679lb # [ 0.745321] IPI shorthand broadcast: enabled1680worker2 # [ 0.734947] registered taskstats version 11681worker2 # [ 0.736095] Loading compiled-in X.509 certificates1682client # [ 0.735177] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1683worker1 # [ 0.707951] Linux agpgart interface v0.1031684client # [ 0.735177] Please enable it if your BIOS has the CPPC option.1685worker1 # [ 0.708745] ACPI: bus type drm_connector registered1686client # [ 0.737587] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1687worker1 # [ 0.711866] usbcore: registered new interface driver usbserial_generic1688client # [ 0.741357] drop_monitor: Initializing network drop monitor service1689worker1 # [ 0.713007] usbserial: USB Serial support registered for generic1690lb # [ 0.753724] sched_clock: Marking stable (608014288, 145121710)->(833257524, -80121526)1691client # [ 0.742635] NET: Registered PF_INET6 protocol family1692lb # [ 0.757226] registered taskstats version 11693worker1 # [ 0.716322] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1694worker1 # [ 0.716322] Please enable it if your BIOS has the CPPC option.1695lb # [ 0.758203] Loading compiled-in X.509 certificates1696worker1 # [ 0.718605] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1697client # [ 0.747775] Segment Routing with IPv61698client # [ 0.751214] In-situ OAM (IOAM) with IPv61699worker1 # [ 0.723472] drop_monitor: Initializing network drop monitor service1700client # [ 0.752383] IPI shorthand broadcast: enabled1701worker1 # [ 0.724664] NET: Registered PF_INET6 protocol family1702worker1 # [ 0.726777] Segment Routing with IPv61703worker2 # [ 0.756203] Demotion targets for Node 0: null1704worker2 # [ 0.758628] Key type .fscrypt registered1705worker1 # [ 0.729328] In-situ OAM (IOAM) with IPv61706worker2 # [ 0.759382] Key type fscrypt-provisioning registered1707worker1 # [ 0.730460] IPI shorthand broadcast: enabled1708worker2 # [ 0.760422] ima: No TPM chip found, activating TPM-bypass!1709client # [ 0.760781] sched_clock: Marking stable (618016795, 142143084)->(840591887, -80432008)1710worker2 # [ 0.765573] ima: Allocated hash algorithm: sha11711lb # [ 0.777153] Demotion targets for Node 0: null1712worker2 # [ 0.766329] ima: No architecture policies found1713lb # [ 0.778023] Key type .fscrypt registered1714client # [ 0.766318] registered taskstats version 11715client # [ 0.767360] Loading compiled-in X.509 certificates1716worker2 # [ 0.768571] PM: Magic number: 10:302:6671717worker1 # [ 0.738838] sched_clock: Marking stable (596014620, 142278353)->(813041135, -74748162)1718lb # [ 0.781142] Key type fscrypt-provisioning registered1719worker2 # [ 0.770197] RAS: Correctable Errors collector initialized.1720lb # [ 0.782073] ima: No TPM chip found, activating TPM-bypass!1721lb # [ 0.782988] ima: Allocated hash algorithm: sha11722worker1 # [ 0.744504] registered taskstats version 11723lb # [ 0.785173] ima: No architecture policies found1724worker1 # [ 0.745515] Loading compiled-in X.509 certificates1725lb # [ 0.786153] PM: Magic number: 10:302:6671726lb # [ 0.787689] RAS: Correctable Errors collector initialized.1727worker2 # [ 0.781961] clk: Disabling unused clocks1728worker2 # [ 0.782797] PM: genpd: Disabling unused power domains1729lb # [ 0.797838] clk: Disabling unused clocks1730client # [ 0.786310] Demotion targets for Node 0: null1731client # [ 0.787317] Key type .fscrypt registered1732client # [ 0.788024] Key type fscrypt-provisioning registered1733lb # [ 0.800163] PM: genpd: Disabling unused power domains1734client # [ 0.791309] ima: No TPM chip found, activating TPM-bypass!1735client # [ 0.792285] ima: Allocated hash algorithm: sha11736worker1 # [ 0.764308] Demotion targets for Node 0: null1737worker1 # [ 0.765207] Key type .fscrypt registered1738client # [ 0.795175] ima: No architecture policies found1739worker1 # [ 0.767304] Key type fscrypt-provisioning registered1740client # [ 0.796269] PM: Magic number: 10:302:6671741worker1 # [ 0.768271] ima: No TPM chip found, activating TPM-bypass!1742worker1 # [ 0.769223] ima: Allocated hash algorithm: sha11743client # [ 0.798067] RAS: Correctable Errors collector initialized.1744worker1 # [ 0.773332] ima: No architecture policies found1745worker1 # [ 0.774353] PM: Magic number: 10:302:6671746worker1 # [ 0.776054] RAS: Correctable Errors collector initialized.1747client # [ 0.809014] clk: Disabling unused clocks1748client # [ 0.811193] PM: genpd: Disabling unused power domains1749worker1 # [ 0.785092] clk: Disabling unused clocks1750worker1 # [ 0.788324] PM: genpd: Disabling unused power domains1751lb # [ 0.963623] Freeing initrd memory: 29064K1752lb # [ 0.966902] Freeing unused decrypted memory: 2028K1753lb # [ 0.970433] Freeing unused kernel image (initmem) memory: 3652K1754lb # [ 0.971664] Write protecting the kernel read-only data: 32768k1755lb # [ 0.973740] Freeing unused kernel image (text/rodata gap) memory: 1184K1756lb # [ 0.975403] Freeing unused kernel image (rodata/data gap) memory: 720K1757worker2 # [ 0.980940] Freeing initrd memory: 29060K1758client # [ 0.982346] Freeing initrd memory: 29056K1759worker2 # [ 0.984695] Freeing unused decrypted memory: 2028K1760client # [ 0.985870] Freeing unused decrypted memory: 2028K1761worker2 # [ 0.987477] Freeing unused kernel image (initmem) memory: 3652K1762worker2 # [ 0.988640] Write protecting the kernel read-only data: 32768k1763client # [ 0.988865] Freeing unused kernel image (initmem) memory: 3652K1764worker2 # [ 0.990719] Freeing unused kernel image (text/rodata gap) memory: 1184K1765client # [ 0.990014] Write protecting the kernel read-only data: 32768k1766worker1 # [ 0.962725] Freeing initrd memory: 29056K1767worker2 # [ 0.992267] Freeing unused kernel image (rodata/data gap) memory: 720K1768client # [ 0.991990] Freeing unused kernel image (text/rodata gap) memory: 1184K1769client # [ 0.993582] Freeing unused kernel image (rodata/data gap) memory: 720K1770worker1 # [ 0.965962] Freeing unused decrypted memory: 2028K1771worker1 # [ 0.968703] Freeing unused kernel image (initmem) memory: 3652K1772worker1 # [ 0.969803] Write protecting the kernel read-only data: 32768k1773worker1 # [ 0.971707] Freeing unused kernel image (text/rodata gap) memory: 1184K1774worker1 # [ 0.973222] Freeing unused kernel image (rodata/data gap) memory: 720K1775lb # [ 1.026670] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776lb # [ 1.027869] Run /init as init process1777lb # [ 1.040727] systemd[1]: Inserted module 'autofs4'1778worker2 # [ 1.045049] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779client # [ 1.044506] x86/mm: Checked W+X mappings: passed, no W+X pages found.1780worker2 # [ 1.046257] Run /init as init process1781client # [ 1.045648] Run /init as init process1782lb # [ 1.061616] fuse: init (API version 7.45)1783worker1 # [ 1.025268] x86/mm: Checked W+X mappings: passed, no W+X pages found.1784worker1 # [ 1.026409] Run /init as init process1785client # [ 1.057387] systemd[1]: Inserted module 'autofs4'1786lb # [ 1.069465] ACPI: \_SB_.GSIG: Enabled at IRQ 221787worker2 # [ 1.058661] systemd[1]: Inserted module 'autofs4'1788lb # [ 1.072395] ACPI: \_SB_.GSIH: Enabled at IRQ 231789lb # [ 1.075691] ACPI: \_SB_.GSIE: Enabled at IRQ 201790worker1 # [ 1.037523] systemd[1]: Inserted module 'autofs4'1791lb # [ 1.077938] ACPI: \_SB_.GSIF: Enabled at IRQ 211792lb # [ 1.083808] virtiofs virtio5: discovered new tag: nix-store1793lb # [ 1.085475] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1794client # [ 1.077514] fuse: init (API version 7.45)1795worker2 # [ 1.078643] fuse: init (API version 7.45)1796lb # [ 1.093540] virtiofs virtio6: discovered new tag: shared1797lb # [ 1.095342] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1798client # [ 1.084987] ACPI: \_SB_.GSIG: Enabled at IRQ 221799worker2 # [ 1.086092] ACPI: \_SB_.GSIG: Enabled at IRQ 221800worker1 # [ 1.057351] fuse: init (API version 7.45)1801client # [ 1.087658] ACPI: \_SB_.GSIH: Enabled at IRQ 231802worker2 # [ 1.088613] ACPI: \_SB_.GSIH: Enabled at IRQ 231803lb # [ 1.099517] virtiofs virtio7: discovered new tag: xchg1804lb # [ 1.101293] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1805worker2 # [ 1.092221] ACPI: \_SB_.GSIE: Enabled at IRQ 201806client # [ 1.091645] ACPI: \_SB_.GSIE: Enabled at IRQ 201807worker1 # [ 1.064737] ACPI: \_SB_.GSIG: Enabled at IRQ 221808worker2 # [ 1.094392] ACPI: \_SB_.GSIF: Enabled at IRQ 211809client # [ 1.093812] ACPI: \_SB_.GSIF: Enabled at IRQ 211810worker1 # [ 1.067184] ACPI: \_SB_.GSIH: Enabled at IRQ 231811worker2 # [ 1.099401] virtiofs virtio5: discovered new tag: nix-store1812worker1 # [ 1.070207] ACPI: \_SB_.GSIE: Enabled at IRQ 201813client # [ 1.099633] virtiofs virtio5: discovered new tag: nix-store1814worker2 # [ 1.101102] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1815worker1 # [ 1.072308] ACPI: \_SB_.GSIF: Enabled at IRQ 211816client # [ 1.101566] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1817worker1 # [ 1.077558] virtiofs virtio5: discovered new tag: nix-store1818worker2 # [ 1.108026] virtiofs virtio6: discovered new tag: shared1819worker1 # [ 1.079075] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1820worker2 # [ 1.109872] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1821client # [ 1.111419] virtiofs virtio6: discovered new tag: shared1822client # [ 1.113073] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1823worker2 # [ 1.114699] virtiofs virtio7: discovered new tag: xchg1824worker1 # [ 1.085408] virtiofs virtio6: discovered new tag: shared1825worker1 # [ 1.086871] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1826worker2 # [ 1.116319] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1827lb # [ 1.127484] systemd[1]: Successfully made /usr/ read-only.1828client # [ 1.118045] virtiofs virtio7: discovered new tag: xchg1829client # [ 1.119680] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1830worker1 # [ 1.091812] virtiofs virtio7: discovered new tag: xchg1831worker1 # [ 1.093452] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1832worker2 # [ 1.139346] systemd[1]: Successfully made /usr/ read-only.1833client # [ 1.142202] systemd[1]: Successfully made /usr/ read-only.1834worker1 # [ 1.115154] systemd[1]: Successfully made /usr/ read-only.1835lb # [ 1.464332] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836lb # [ 1.476412] systemd[1]: Detected virtualization kvm.1837lb # [ 1.478623] systemd[1]: Detected architecture x86-64.1838lb # [ 1.480781] systemd[1]: Running in initrd.1839lb # [ 1.483364] systemd[1]: Initializing machine ID from random generator.1840lb # [ 1.486281] systemd[1]: Hostname set to <lb>.1841worker2 # [ 1.475082] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842worker2 # [ 1.481034] systemd[1]: Detected virtualization kvm.1843worker2 # [ 1.482078] systemd[1]: Detected architecture x86-64.1844client # [ 1.477822] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1845worker2 # [ 1.483121] systemd[1]: Running in initrd.1846client # [ 1.482827] systemd[1]: Detected virtualization kvm.1847worker1 # [ 1.450698] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848client # [ 1.483721] systemd[1]: Detected architecture x86-64.1849worker1 # [ 1.455344] systemd[1]: Detected virtualization kvm.1850worker2 # [ 1.484436] systemd[1]: Initializing machine ID from random generator.1851client # [ 1.484612] systemd[1]: Running in initrd.1852worker1 # [ 1.456172] systemd[1]: Detected architecture x86-64.1853worker2 # [ 1.485908] systemd[1]: Hostname set to <worker2>.1854worker1 # [ 1.456999] systemd[1]: Running in initrd.1855client # [ 1.485711] systemd[1]: Initializing machine ID from random generator.1856worker1 # [ 1.458108] systemd[1]: Initializing machine ID from random generator.1857client # [ 1.486913] systemd[1]: Hostname set to <client>.1858worker1 # [ 1.459271] systemd[1]: Hostname set to <worker1>.1859client # [ 1.682775] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860worker1 # [ 1.656817] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861worker2 # [ 1.687056] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862lb # [ 1.710286] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863client # [ 1.721202] systemd[1]: Queued start job for default target Initrd Default Target.1864worker1 # [ 1.694661] systemd[1]: Queued start job for default target Initrd Default Target.1865client # [ 1.724486] systemd[1]: Created slice Slice /system/modprobe.1866worker2 # [ 1.725428] systemd[1]: Queued start job for default target Initrd Default Target.1867client # [ 1.725639] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1868client # [ 1.727018] systemd[1]: Expecting device /dev/disk/by-label/nixos...1869worker1 # [ 1.698639] systemd[1]: Created slice Slice /system/modprobe.1870client # [ 1.728081] systemd[1]: Reached target Path Units.1871worker1 # [ 1.699742] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1872client # [ 1.728918] systemd[1]: Reached target Slice Units.1873worker2 # [ 1.729844] systemd[1]: Created slice Slice /system/modprobe.1874client # [ 1.729823] systemd[1]: Reached target Swaps.1875worker1 # [ 1.701071] systemd[1]: Expecting device /dev/disk/by-label/nixos...1876client # [ 1.730608] systemd[1]: Reached target Timer Units.1877worker2 # [ 1.731022] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1878worker1 # [ 1.702113] systemd[1]: Reached target Path Units.1879worker1 # [ 1.702942] systemd[1]: Reached target Slice Units.1880worker2 # [ 1.732383] systemd[1]: Expecting device /dev/disk/by-label/nixos...1881client # [ 1.731544] systemd[1]: Listening on D-Bus System Message Bus Socket.1882worker1 # [ 1.703767] systemd[1]: Reached target Swaps.1883worker2 # [ 1.733448] systemd[1]: Reached target Path Units.1884client # [ 1.732702] systemd[1]: Listening on Journal Socket (/dev/log).1885worker1 # [ 1.704524] systemd[1]: Reached target Timer Units.1886worker2 # [ 1.734304] systemd[1]: Reached target Slice Units.1887client # [ 1.733821] systemd[1]: Listening on Journal Sockets.1888worker1 # [ 1.705442] systemd[1]: Listening on D-Bus System Message Bus Socket.1889worker2 # [ 1.735174] systemd[1]: Reached target Swaps.1890client # [ 1.734788] systemd[1]: Listening on udev Control Socket.1891worker2 # [ 1.735979] systemd[1]: Reached target Timer Units.1892worker1 # [ 1.706575] systemd[1]: Listening on Journal Socket (/dev/log).1893client # [ 1.735767] systemd[1]: Listening on udev Kernel Socket.1894worker1 # [ 1.707697] systemd[1]: Listening on Journal Sockets.1895worker2 # [ 1.736948] systemd[1]: Listening on D-Bus System Message Bus Socket.1896client # [ 1.736688] systemd[1]: Reached target Socket Units.1897worker1 # [ 1.708644] systemd[1]: Listening on udev Control Socket.1898worker2 # [ 1.738154] systemd[1]: Listening on Journal Socket (/dev/log).1899worker1 # [ 1.709599] systemd[1]: Listening on udev Kernel Socket.1900worker2 # [ 1.739316] systemd[1]: Listening on Journal Sockets.1901client # [ 1.738433] systemd[1]: Starting Create List of Static Device Nodes...1902worker1 # [ 1.710503] systemd[1]: Reached target Socket Units.1903worker2 # [ 1.740286] systemd[1]: Listening on udev Control Socket.1904worker2 # [ 1.741343] systemd[1]: Listening on udev Kernel Socket.1905worker1 # [ 1.712172] systemd[1]: Starting Create List of Static Device Nodes...1906worker2 # [ 1.742289] systemd[1]: Reached target Socket Units.1907client # [ 1.743228] systemd[1]: Starting Load Kernel Module configfs...1908worker2 # [ 1.744138] systemd[1]: Starting Create List of Static Device Nodes...1909worker1 # [ 1.716364] systemd[1]: Starting Load Kernel Module configfs...1910worker2 # [ 1.748630] systemd[1]: Starting Load Kernel Module configfs...1911lb # [ 1.763785] systemd[1]: Queued start job for default target Initrd Default Target.1912lb # [ 1.768545] systemd[1]: Created slice Slice /system/modprobe.1913lb # [ 1.769931] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1914lb # [ 1.771462] systemd[1]: Expecting device /dev/disk/by-label/nixos...1915lb # [ 1.772672] systemd[1]: Reached target Path Units.1916lb # [ 1.773663] systemd[1]: Reached target Slice Units.1917client # [ 1.751230] systemd[1]: Starting Journal Service...1918lb # [ 1.774623] systemd[1]: Reached target Swaps.1919lb # [ 1.775500] systemd[1]: Reached target Timer Units.1920worker1 # [ 1.724903] systemd[1]: Starting Journal Service...1921lb # [ 1.776580] systemd[1]: Listening on D-Bus System Message Bus Socket.1922lb # [ 1.777867] systemd[1]: Listening on Journal Socket (/dev/log).1923lb # [ 1.779071] systemd[1]: Listening on Journal Sockets.1924lb # [ 1.780061] systemd[1]: Listening on udev Control Socket.1925lb # [ 1.781158] systemd[1]: Listening on udev Kernel Socket.1926worker2 # [ 1.759632] systemd[1]: Starting Journal Service...1927lb # [ 1.782161] systemd[1]: Reached target Socket Units.1928lb # [ 1.784053] systemd[1]: Starting Create List of Static Device Nodes...1929lb # [ 1.788024] systemd[1]: Starting Load Kernel Module configfs...1930client # [ 1.776804] systemd[1]: Starting Load Kernel Modules...1931worker1 # [ 1.750491] systemd[1]: Starting Load Kernel Modules...1932worker2 # [ 1.782155] systemd[1]: Starting Load Kernel Modules...1933client # [ 1.781288] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1934worker1 # [ 1.754438] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1935worker2 # [ 1.786692] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1936client # [ 1.789230] systemd[1]: Starting Coldplug All udev Devices...1937worker1 # [ 1.762483] systemd[1]: Starting Coldplug All udev Devices...1938worker2 # [ 1.793741] systemd[1]: Starting Coldplug All udev Devices...1939lb # [ 1.797859] systemd[1]: Starting Journal Service...1940client # [ 1.801589] systemd-journald[66]: Collecting audit messages is disabled.1941worker1 # [ 1.773829] systemd-journald[66]: Collecting audit messages is disabled.1942client # [ 1.803271] systemd[1]: Finished Create List of Static Device Nodes.1943worker1 # [ 1.775637] systemd[1]: Finished Create List of Static Device Nodes.1944worker2 # [ 1.807110] systemd-journald[66]: Collecting audit messages is disabled.1945worker2 # [ 1.808979] systemd[1]: Finished Create List of Static Device Nodes.1946worker1 # [ 1.780890] systemd[1]: modprobe@configfs.service: Deactivated successfully.1947client # [ 1.809656] systemd[1]: modprobe@configfs.service: Deactivated successfully.1948lb # [ 1.823775] systemd[1]: Starting Load Kernel Modules...1949worker2 # [ 1.814059] systemd[1]: modprobe@configfs.service: Deactivated successfully.1950worker1 # [ 1.787690] systemd[1]: Finished Load Kernel Module configfs.1951client # [ 1.816636] systemd[1]: Finished Load Kernel Module configfs.1952lb # [ 1.828278] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1953worker2 # [ 1.820970] systemd[1]: Finished Load Kernel Module configfs.1954worker1 # [ 1.793724] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1955client # [ 1.822603] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1956lb # [ 1.837332] systemd[1]: Starting Coldplug All udev Devices...1957worker2 # [ 1.827009] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1958worker1 # [ 1.804378] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1959client # [ 1.834243] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1960lb # [ 1.848603] systemd-journald[66]: Collecting audit messages is disabled.1961worker2 # [ 1.839698] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1962lb # [ 1.852206] systemd[1]: Finished Create List of Static Device Nodes.1963client # [ 1.842510] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1964worker1 # [ 1.815697] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1965lb # [ 1.857800] systemd[1]: modprobe@configfs.service: Deactivated successfully.1966worker2 # [ 1.849777] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1967worker1 # [ 1.822326] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1968lb # [ 1.863678] systemd[1]: Finished Load Kernel Module configfs.1969client # [ 1.853395] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1970worker2 # [ 1.857632] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1971lb # [ 1.870556] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1972lb # [ 1.882200] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1973worker1 # [ 1.844859] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1974lb # [ 1.887953] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1975client # [ 1.881250] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1976worker2 # [ 1.882190] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1977worker1 # [ 1.853490] systemd[1]: Starting Create Static Device Nodes in /dev...1978lb # [ 1.900334] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1979worker2 # [ 1.892092] systemd[1]: Starting Create Static Device Nodes in /dev...1980client # [ 1.891363] systemd[1]: Starting Create Static Device Nodes in /dev...1981worker1 # [ 1.865395] systemd[1]: Finished Load Kernel Modules.1982client # [ 1.901227] systemd[1]: Finished Load Kernel Modules.1983worker2 # [ 1.902131] systemd[1]: Finished Load Kernel Modules.1984worker1 # [ 1.873381] systemd[1]: Starting Apply Kernel Variables...1985client # [ 1.908364] systemd[1]: Starting Apply Kernel Variables...1986worker2 # [ 1.910632] systemd[1]: Starting Apply Kernel Variables...1987lb # [ 1.923851] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1988worker1 # [ 1.892465] systemd[1]: Finished Create Static Device Nodes in /dev.1989client # [ 1.923725] systemd[1]: Started Journal Service.1990lb # [ 1.936452] systemd[1]: Starting Create Static Device Nodes in /dev...1991client # [ 1.780671] systemd-modules-load[67]: Inserted module 'dm_mod'1992worker1 # [ 1.897493] systemd[1]: Started Journal Service.1993worker1 # [ 1.752918] systemd-modules-load[68]: Inserted module 'dm_mod'1994client # [ 1.784596] systemd-modules-load[67]: Inserted module 'virtio_balloon'1995worker1 # [ 1.756946] systemd-modules-load[68]: Inserted module 'virtio_balloon'1996client # [ 1.785607] systemd-modules-load[67]: Inserted module 'virtio_gpu'1997worker2 # [ 1.796212] systemd-modules-load[67]: Inserted module 'dm_mod'[ 1.930758] systemd[1]: Started Journal Service.1998worker1 # [ 1.759449] systemd-modules-load[68]: Inserted module 'virtio_gpu'1999worker2 # 2000worker2 # [ 1.800332] systemd-modules-load[67]: Inserted module 'virtio_balloon'
loading 3,049 hidden lines…
5050worker1 # [ 52.829616] nix-grpc-daemon[779]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465051worker1 # [ 52.838378] nix-grpc-daemon[873]: time=2026-09-23T04:42:24.089Z level=INFO msg="All 1 paths already cached"5052client # copying 1 paths to 'lb:50051'...5053client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...5054worker1 # [ 52.846224] nix-grpc-daemon[779]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409465055worker1 # [ 52.856209] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5056worker1 # [ 52.947469] nix-daemon[847]: accepted connection from pid 915, user nix-grpc-daemon (trusted)5057worker1 # [ 52.961503] nix-daemon[847]: reaped child process 918, status = succeeded5058lb # [ 53.005466] niks3-server[1046]: 2026/09/23 04:42:25 INFO Received uploads request method=POST path=/api/pending_closures5059worker1 # [ 52.976079] nix-grpc-daemon[873]: time=2026-09-23T04:42:24.227Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5060worker1 # [ 52.977591] nix-grpc-daemon[873]: time=2026-09-23T04:42:24.229Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"5061lb # [ 53.039175] niks3-server[1046]: 2026/09/23 04:42:25 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst5062lb # [ 53.043416] niks3-server[1046]: 2026/09/23 04:42:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign5063lb # [ 53.048785] niks3-server[1046]: 2026/09/23 04:42:25 INFO Signed narinfos id=11 count=15064worker1 # [ 53.014358] nix-grpc-daemon[873]: time=2026-09-23T04:42:24.265Z level=INFO msg="Uploading 1 narinfos"5065lb # [ 53.060142] niks3-server[1046]: 2026/09/23 04:42:25 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls5066lb # [ 53.071249] niks3-server[1046]: 2026/09/23 04:42:25 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete5067lb # [ 53.077548] niks3-server[1046]: 2026/09/23 04:42:25 INFO Completed upload id=115068lb # [ 53.081112] niks3-server[1046]: 2026/09/23 04:42:25 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo5069worker1 # [ 53.043345] nix-grpc-daemon[873]: time=2026-09-23T04:42:24.294Z level=INFO msg="Upload complete. (177ms)"5070worker1 # [ 53.047570] nix-grpc-daemon[779]: ts=2026-09-23T04:42:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_in=6245071worker1 # [ 53.053401] nix-grpc-daemon[779]: ts=2026-09-23T04:42:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465072worker1 # [ 53.063695] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5073worker1 # [ 53.072657] nix-daemon[847]: reaped child process 913, status = succeeded5074worker1 # [ 53.077483] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5075client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...5076client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv5077client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5078client # farm-b> LOG-farm-b-sig5079worker1 # [ 53.919690] nix-daemon[847]: reaped child process 922, status = succeeded5080worker1 # [ 53.994714] nix-daemon[847]: accepted connection from pid 938, user nix-grpc-daemon (trusted)5081worker1 # [ 54.007773] nix-daemon[847]: reaped child process 941, status = succeeded5082lb # [ 54.053653] niks3-server[1046]: 2026/09/23 04:42:26 INFO Received uploads request method=POST path=/api/pending_closures5083worker1 # [ 54.030754] nix-grpc-daemon[873]: time=2026-09-23T04:42:25.282Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5084worker1 # [ 54.032397] nix-grpc-daemon[873]: time=2026-09-23T04:42:25.283Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"5085lb # [ 54.097679] niks3-server[1046]: 2026/09/23 04:42:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5086lb # [ 54.102104] niks3-server[1046]: 2026/09/23 04:42:26 INFO Signed narinfos id=12 count=15087worker1 # [ 54.067451] nix-grpc-daemon[873]: time=2026-09-23T04:42:25.318Z level=INFO msg="Uploading 1 narinfos"5088lb # [ 54.114640] niks3-server[1046]: 2026/09/23 04:42:26 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv5089lb # [ 54.120760] niks3-server[1046]: 2026/09/23 04:42:26 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls5090lb # [ 54.124627] niks3-server[1046]: 2026/09/23 04:42:26 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst5091lb # [ 54.128940] niks3-server[1046]: 2026/09/23 04:42:26 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5092lb # [ 54.132228] niks3-server[1046]: 2026/09/23 04:42:26 INFO Completed upload id=125093worker1 # [ 54.094930] nix-grpc-daemon[873]: time=2026-09-23T04:42:25.346Z level=INFO msg="Upload complete. (210ms)"5094lb # [ 54.134365] niks3-server[1046]: 2026/09/23 04:42:26 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo5095worker1 # [ 54.099885] nix-grpc-daemon[779]: ts=2026-09-23T04:42:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=15096worker2 # [ 54.146817] nix-grpc-daemon[783]: ts=2026-09-23T04:42:26Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=05097worker1 # [ 54.113206] nix-daemon[847]: reaped child process 920, status = succeeded5098worker2 # [ 54.151154] nix-grpc-daemon[783]: ts=2026-09-23T04:42:26Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=15099worker1 # [ 54.151407] nix-grpc-daemon[779]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465100worker1 # [ 54.164204] nix-grpc-daemon[779]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465101worker1 # [ 54.178171] nix-grpc-daemon[873]: time=2026-09-23T04:42:25.429Z level=INFO msg="All 2 paths already cached"5102client # copying 1 paths to 'lb:50051'...5103client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...5104worker1 # [ 54.187302] nix-grpc-daemon[779]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409465105worker1 # [ 54.197182] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5106worker1 # [ 54.293966] nix-daemon[847]: accepted connection from pid 945, user nix-grpc-daemon (trusted)5107worker1 # [ 54.307436] nix-daemon[847]: reaped child process 948, status = succeeded5108lb # [ 54.351429] niks3-server[1046]: 2026/09/23 04:42:26 INFO Received uploads request method=POST path=/api/pending_closures5109worker1 # [ 54.322336] nix-grpc-daemon[873]: time=2026-09-23T04:42:25.573Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5110worker1 # [ 54.323806] nix-grpc-daemon[873]: time=2026-09-23T04:42:25.575Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"5111lb # [ 54.382572] niks3-server[1046]: 2026/09/23 04:42:26 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst5112lb # [ 54.392746] niks3-server[1046]: 2026/09/23 04:42:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5113worker1 # [ 54.360778] nix-grpc-daemon[873]: time=2026-09-23T04:42:25.612Z level=INFO msg="Uploading 1 narinfos"5114lb # [ 54.398300] niks3-server[1046]: 2026/09/23 04:42:26 INFO Signed narinfos id=13 count=15115lb # [ 54.405736] niks3-server[1046]: 2026/09/23 04:42:26 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls5116lb # [ 54.415507] niks3-server[1046]: 2026/09/23 04:42:26 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5117lb # [ 54.419664] niks3-server[1046]: 2026/09/23 04:42:26 INFO Completed upload id=135118worker1 # [ 54.382336] nix-grpc-daemon[873]: time=2026-09-23T04:42:25.633Z level=INFO msg="Upload complete. (173ms)"5119lb # [ 54.421233] niks3-server[1046]: 2026/09/23 04:42:26 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo5120worker1 # [ 54.386341] nix-grpc-daemon[779]: ts=2026-09-23T04:42:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_in=7525121worker1 # [ 54.391155] nix-grpc-daemon[779]: ts=2026-09-23T04:42:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465122worker1 # [ 54.400689] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5123worker1 # [ 54.406532] nix-daemon[847]: reaped child process 943, status = succeeded5124worker1 # [ 54.411280] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5125client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...5126client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5127client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5128client # farm-top> LOG-farm-top-sig5129worker1 # [ 55.626101] nix-daemon[847]: reaped child process 952, status = succeeded5130worker1 # [ 55.701253] nix-daemon[847]: accepted connection from pid 968, user nix-grpc-daemon (trusted)5131worker1 # [ 55.714791] nix-daemon[847]: reaped child process 971, status = succeeded5132lb # [ 55.760507] niks3-server[1046]: 2026/09/23 04:42:27 INFO Received uploads request method=POST path=/api/pending_closures5133worker1 # [ 55.730178] nix-grpc-daemon[873]: time=2026-09-23T04:42:26.981Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5134worker1 # [ 55.731723] nix-grpc-daemon[873]: time=2026-09-23T04:42:26.983Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5135lb # [ 55.802530] niks3-server[1046]: 2026/09/23 04:42:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5136lb # [ 55.809267] niks3-server[1046]: 2026/09/23 04:42:27 INFO Signed narinfos id=14 count=15137worker1 # [ 55.775133] nix-grpc-daemon[873]: time=2026-09-23T04:42:27.026Z level=INFO msg="Uploading 1 narinfos"5138lb # [ 55.821287] niks3-server[1046]: 2026/09/23 04:42:27 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5139lb # [ 55.829699] niks3-server[1046]: 2026/09/23 04:42:27 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5140lb # [ 55.834323] niks3-server[1046]: 2026/09/23 04:42:27 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5141lb # [ 55.840643] niks3-server[1046]: 2026/09/23 04:42:27 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5142lb # [ 55.845687] niks3-server[1046]: 2026/09/23 04:42:27 INFO Completed upload id=145143worker1 # [ 55.808441] nix-grpc-daemon[873]: time=2026-09-23T04:42:27.059Z level=INFO msg="Upload complete. (190ms)"5144lb # [ 55.849957] niks3-server[1046]: 2026/09/23 04:42:27 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5145worker1 # [ 55.814567] nix-grpc-daemon[779]: ts=2026-09-23T04:42:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15146worker1 # [ 55.829090] nix-daemon[847]: reaped child process 950, status = succeeded5147client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.78 seconds)5148client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5149client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.11 seconds)5150client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5151worker1 # [ 56.053980] nix-grpc-daemon[779]: ts=2026-09-23T04:42:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:401645152worker1 # [ 56.095783] nix-grpc-daemon[779]: ts=2026-09-23T04:42:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:401645153client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.21 seconds)5154client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5155worker2 # [ 56.315073] nix-grpc-daemon[783]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:576885156client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5157client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5158client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5159client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5160worker2 # [ 56.347722] nix-grpc-daemon[783]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:576885161worker2 # [ 56.354718] nix-grpc-daemon[783]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5162worker2 # [ 56.368875] nix-grpc-daemon[783]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5163worker2 # [ 56.381527] nix-grpc-daemon[783]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5164client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.22 seconds)5165client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5166client # finding garbage collector roots...5167client # removing stale temporary roots file "/nix/var/nix/temproots/809"5168client # removing stale temporary roots file "/nix/var/nix/temproots/878"5169client # removing stale temporary roots file "/nix/var/nix/temproots/901"5170client # removing stale temporary roots file "/nix/var/nix/temproots/886"5171client # removing stale temporary roots file "/nix/var/nix/temproots/845"5172client # removing stale temporary roots file "/nix/var/nix/temproots/865"5173client # removing stale temporary roots file "/nix/var/nix/temproots/824"5174client # removing stale temporary roots file "/nix/var/nix/temproots/857"5175client # deleting specified paths...5176client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5177client # deleting unused links...5178client # note: hard linking is currently saving 0.0 KiB5179worker2 # [ 56.698779] nix-grpc-daemon[783]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461085180worker2 # [ 56.706271] nix-grpc-daemon[783]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461085181worker2 # [ 56.804186] nix-grpc-daemon[889]: time=2026-09-23T04:42:28.887Z level=INFO msg="All 1 paths already cached"5182worker1 # [ 56.773628] nix-grpc-daemon[779]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:409465183worker1 # [ 56.781111] nix-grpc-daemon[779]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:409465184client # copying 3 paths...5185worker1 # [ 56.791727] nix-grpc-daemon[779]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:409465186client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5187worker2 # [ 56.848779] nix-grpc-daemon[783]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:461085188worker2 # [ 56.854228] nix-grpc-daemon[783]: ts=2026-09-23T04:42:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=0 paths=1 nar_bytes_out=2325189worker1 # [ 56.817414] nix-grpc-daemon[779]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409465190worker1 # [ 56.823179] nix-grpc-daemon[779]: ts=2026-09-23T04:42:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_out=1765191worker1 # [ 56.829564] nix-grpc-daemon[779]: ts=2026-09-23T04:42:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409465192worker1 # [ 56.833290] nix-grpc-daemon[779]: ts=2026-09-23T04:42:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_out=1285193client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5194client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5195client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.54 seconds)5196(finished: subtest: outputs read back through any worker carry the cache signature, in 5.85 seconds)5197subtest: two clients wanting the same drv build it once5198client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5199client # [ 56.973970] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5200client # Running as unit: dedup1.service; invocation ID: cca64235fee44b469333edeb47b446ac5201client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.04 seconds)5202client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5203worker2 # [ 57.111215] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461085204client # [ 57.107208] nix[953]: don't know how to build these paths:5205client # [ 57.108145] nix[953]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5206worker2 # [ 57.125876] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05207worker2 # [ 57.131268] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=15208worker2 # [ 57.170934] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085209worker2 # [ 57.184376] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461085210client # [ 57.184392] nix[953]: copying 1 paths to 'lb:50051'...5211worker2 # [ 57.195947] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461085212client # [ 57.186623] nix[953]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5213worker2 # [ 57.208847] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5214worker2 # [ 57.316405] nix-daemon[874]: accepted connection from pid 1067, user nix-grpc-daemon (trusted)5215worker2 # [ 57.331545] nix-daemon[874]: reaped child process 1070, status = succeeded5216lb # [ 57.338382] niks3-server[1046]: 2026/09/23 04:42:29 INFO Received uploads request method=POST path=/api/pending_closures5217lb # [ 57.407230] envoy[926]: conn peer=[2001:db8:1::1]:35560 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=445218lb # [ 57.410993] envoy[926]: conn peer=[2001:db8:1::1]:35564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4165 tx=1879 ms=21575219lb # [ 57.414661] envoy[926]: conn peer=[2001:db8:1::1]:35570 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9494 tx=4379 ms=46255220lb # [ 57.418303] envoy[926]: conn peer=[2001:db8:1::1]:48154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2887 ms=2385221lb # [ 57.422571] envoy[926]: conn peer=[2001:db8:1::1]:48168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=885222lb # [ 57.426119] envoy[926]: conn peer=[2001:db8:1::1]:48184 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=815223lb # [ 57.429572] envoy[926]: conn peer=[2001:db8:1::1]:48192 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=745224worker2 # [ 57.644167] nix-grpc-daemon[889]: time=2026-09-23T04:42:29.728Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5225worker2 # [ 57.649166] nix-grpc-daemon[889]: time=2026-09-23T04:42:29.728Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5226lb # [ 57.668797] niks3-server[1046]: 2026/09/23 04:42:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5227worker2 # [ 57.679932] nix-grpc-daemon[889]: time=2026-09-23T04:42:29.764Z level=INFO msg="Uploading 1 narinfos"5228lb # [ 57.679140] niks3-server[1046]: 2026/09/23 04:42:29 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5229lb # [ 57.683435] niks3-server[1046]: 2026/09/23 04:42:29 INFO Signed narinfos id=15 count=15230lb # [ 57.695159] niks3-server[1046]: 2026/09/23 04:42:29 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5231lb # [ 57.702604] niks3-server[1046]: 2026/09/23 04:42:29 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5232lb # [ 57.708529] niks3-server[1046]: 2026/09/23 04:42:29 INFO Completed upload id=155233worker2 # [ 57.710527] nix-grpc-daemon[889]: time=2026-09-23T04:42:29.794Z level=INFO msg="Upload complete. (490ms)"5234worker2 # [ 57.718959] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=0 paths=1 nar_bytes_in=4325235worker2 # [ 57.724093] nix-grpc-daemon[783]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085236lb # [ 57.723445] niks3-server[1046]: 2026/09/23 04:42:29 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5237worker2 # [ 57.737488] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5238worker2 # [ 57.747303] nix-daemon[874]: reaped child process 1065, status = succeeded5239client # [ 57.746221] nix[953]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5240worker2 # [ 57.758208] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5241client # [ 58.017656] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5242client # Running as unit: dedup2.service; invocation ID: c01c6e55fbfa44729d95dd672ec904305243client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.04 seconds)5244worker1 # [ 58.126585] nix-grpc-daemon[779]: ts=2026-09-23T04:42:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409465245client # [ 58.160848] nix[969]: don't know how to build these paths:5246client # [ 58.162418] nix[969]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5247worker2 # [ 58.179692] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05248worker2 # [ 58.189171] nix-grpc-daemon[783]: ts=2026-09-23T04:42:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085249client # [ 58.184127] nix[969]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5250client: must succeed: sleep 25251client: (finished: must succeed: sleep 2, in 2.02 seconds)5252client: waiting for success: ! systemctl is-active dedup1 dedup25253worker2 # [ 61.380173] nix-daemon[874]: reaped child process 1074, status = succeeded5254worker2 # [ 61.454830] nix-daemon[874]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5255worker2 # [ 61.468162] nix-daemon[874]: reaped child process 1109, status = succeeded5256lb # [ 61.473953] niks3-server[1046]: 2026/09/23 04:42:33 INFO Received uploads request method=POST path=/api/pending_closures5257worker2 # [ 61.486983] nix-grpc-daemon[889]: time=2026-09-23T04:42:33.571Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5258worker2 # [ 61.490059] nix-grpc-daemon[889]: time=2026-09-23T04:42:33.573Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5259lb # [ 61.524312] niks3-server[1046]: 2026/09/23 04:42:33 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5260lb # [ 61.528751] niks3-server[1046]: 2026/09/23 04:42:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5261lb # [ 61.533636] niks3-server[1046]: 2026/09/23 04:42:33 INFO Signed narinfos id=16 count=15262worker2 # [ 61.538867] nix-grpc-daemon[889]: time=2026-09-23T04:42:33.623Z level=INFO msg="Uploading 1 narinfos"5263lb # [ 61.541375] niks3-server[1046]: 2026/09/23 04:42:33 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5264lb # [ 61.549045] niks3-server[1046]: 2026/09/23 04:42:33 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5265lb # [ 61.558883] niks3-server[1046]: 2026/09/23 04:42:33 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5266lb # [ 61.562148] niks3-server[1046]: 2026/09/23 04:42:33 INFO Completed upload id=165267worker2 # [ 61.564004] nix-grpc-daemon[889]: time=2026-09-23T04:42:33.648Z level=INFO msg="Upload complete. (191ms)"5268lb # [ 61.564277] niks3-server[1046]: 2026/09/23 04:42:33 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5269worker2 # [ 61.570705] nix-grpc-daemon[783]: ts=2026-09-23T04:42:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15270worker2 # [ 61.580301] nix-grpc-daemon[783]: ts=2026-09-23T04:42:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15271worker2 # [ 61.592229] nix-daemon[874]: reaped child process 1072, status = succeeded5272client # [ 61.610400] systemd[1]: dedup1.service: Deactivated successfully.5273client # [ 61.614169] systemd[1]: dedup1.service: Consumed 117ms CPU time over 4.639s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.6K outgoing IP traffic.5274client # [ 61.620716] systemd[1]: dedup2.service: Deactivated successfully.5275client # [ 61.623631] systemd[1]: dedup2.service: Consumed 128ms CPU time over 3.605s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5276client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)5277client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5278client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5279worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5280worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5281worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5282worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5283worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5284worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5285worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5286worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5287(finished: subtest: two clients wanting the same drv build it once, in 5.71 seconds)5288subtest: --timeout and --max-silent-time reach the builder5289worker2 # [ 62.811805] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461085290worker2 # [ 62.824737] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05291worker2 # [ 62.829343] nix-grpc-daemon[783]: ts=2026-09-23T04:42:34Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=15292worker1 # [ 62.793870] nix-grpc-daemon[779]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465293worker1 # [ 62.807917] nix-grpc-daemon[779]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465294worker1 # [ 62.820134] nix-grpc-daemon[779]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409465295worker1 # [ 62.831599] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5296worker1 # [ 62.935649] nix-daemon[847]: accepted connection from pid 996, user nix-grpc-daemon (trusted)5297worker1 # [ 62.950712] nix-daemon[847]: reaped child process 999, status = succeeded5298lb # [ 62.993121] niks3-server[1046]: 2026/09/23 04:42:34 INFO Received uploads request method=POST path=/api/pending_closures5299worker1 # [ 62.960161] nix-grpc-daemon[873]: time=2026-09-23T04:42:34.211Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5300worker1 # [ 62.962186] nix-grpc-daemon[873]: time=2026-09-23T04:42:34.213Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5301lb # [ 63.008940] niks3-server[1046]: 2026/09/23 04:42:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5302lb # [ 63.013257] niks3-server[1046]: 2026/09/23 04:42:35 INFO Signed narinfos id=17 count=15303worker1 # [ 62.975821] nix-grpc-daemon[873]: time=2026-09-23T04:42:34.227Z level=INFO msg="Uploading 1 narinfos"5304lb # [ 63.015754] niks3-server[1046]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5305lb # [ 63.018518] niks3-server[1046]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5306lb # [ 63.023775] niks3-server[1046]: 2026/09/23 04:42:35 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5307lb # [ 63.028123] niks3-server[1046]: 2026/09/23 04:42:35 INFO Completed upload id=175308worker1 # [ 62.990753] nix-grpc-daemon[873]: time=2026-09-23T04:42:34.242Z level=INFO msg="Upload complete. (147ms)"5309lb # [ 63.030505] niks3-server[1046]: 2026/09/23 04:42:35 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5310worker1 # [ 62.993487] nix-grpc-daemon[779]: ts=2026-09-23T04:42:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_in=4245311worker1 # [ 62.998225] nix-grpc-daemon[779]: ts=2026-09-23T04:42:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465312worker1 # [ 63.004876] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5313worker1 # [ 63.007374] nix-daemon[847]: reaped child process 994, status = succeeded5314worker1 # [ 63.013718] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5315lb # [ 67.407808] envoy[926]: conn peer=[2001:db8:1::1]:48194 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2998 tx=1563 ms=44775316lb # [ 67.411708] envoy[926]: conn peer=[2001:db8:1::1]:48208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=34255317worker1 # [ 68.824429] nix-grpc-daemon[779]: ts=2026-09-23T04:42:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=5 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05318worker1 # [ 68.844426] nix-daemon[847]: reaped child process 1001, status = succeeded5319worker1 # [ 68.845784] nix-daemon[847]: reaped child process 1003, status = succeeded5320worker1 # [ 69.031277] nix-grpc-daemon[779]: ts=2026-09-23T04:42:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409465321worker2 # [ 69.079898] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05322worker2 # [ 69.086140] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=15323worker2 # [ 69.092925] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085324worker2 # [ 69.104103] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461085325worker2 # [ 69.114822] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461085326worker2 # [ 69.125814] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5327worker2 # [ 69.229638] nix-daemon[874]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)5328worker2 # [ 69.245573] nix-daemon[874]: reaped child process 1129, status = succeeded5329lb # [ 69.250790] niks3-server[1046]: 2026/09/23 04:42:41 INFO Received uploads request method=POST path=/api/pending_closures5330worker2 # [ 69.262452] nix-grpc-daemon[889]: time=2026-09-23T04:42:41.346Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5331worker2 # [ 69.263843] nix-grpc-daemon[889]: time=2026-09-23T04:42:41.346Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5332lb # [ 69.284242] niks3-server[1046]: 2026/09/23 04:42:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5333lb # [ 69.292982] niks3-server[1046]: 2026/09/23 04:42:41 INFO Signed narinfos id=18 count=15334worker2 # [ 69.298673] nix-grpc-daemon[889]: time=2026-09-23T04:42:41.383Z level=INFO msg="Uploading 1 narinfos"5335lb # [ 69.300123] niks3-server[1046]: 2026/09/23 04:42:41 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5336lb # [ 69.307667] niks3-server[1046]: 2026/09/23 04:42:41 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5337lb # [ 69.319266] niks3-server[1046]: 2026/09/23 04:42:41 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5338lb # [ 69.327654] niks3-server[1046]: 2026/09/23 04:42:41 INFO Completed upload id=185339lb # [ 69.329479] niks3-server[1046]: 2026/09/23 04:42:41 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5340worker2 # [ 69.331084] nix-grpc-daemon[889]: time=2026-09-23T04:42:41.415Z level=INFO msg="Upload complete. (193ms)"5341worker2 # [ 69.337337] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=0 paths=1 nar_bytes_in=4245342worker2 # [ 69.342401] nix-grpc-daemon[783]: ts=2026-09-23T04:42:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085343worker2 # [ 69.354114] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5344worker2 # [ 69.362226] nix-daemon[874]: reaped child process 1124, status = succeeded5345worker2 # [ 69.367377] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5346worker2 # [ 75.379321] nix-grpc-daemon[783]: ts=2026-09-23T04:42:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05347worker2 # [ 75.391071] nix-daemon[874]: reaped child process 1131, status = succeeded5348worker2 # [ 75.392702] nix-daemon[874]: reaped child process 1133, status = succeeded5349(finished: subtest: --timeout and --max-silent-time reach the builder, in 12.84 seconds)5350subtest: the first client leaving does not fail the second5351client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5352client # [ 75.541904] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5353client # Running as unit: share1.service; invocation ID: 5b1eef9e0c654260983fd82bec86e88e5354client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5355client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5356worker2 # [ 75.695660] nix-grpc-daemon[783]: ts=2026-09-23T04:42:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461085357client # [ 75.692141] nix[1043]: don't know how to build these paths:5358client # [ 75.693035] nix[1043]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5359worker2 # [ 75.710370] nix-grpc-daemon[783]: ts=2026-09-23T04:42:47Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05360worker2 # [ 75.714985] nix-grpc-daemon[783]: ts=2026-09-23T04:42:47Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=15361worker1 # [ 75.680414] nix-grpc-daemon[779]: ts=2026-09-23T04:42:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465362worker1 # [ 75.692414] nix-grpc-daemon[779]: ts=2026-09-23T04:42:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465363client # [ 75.731621] nix[1043]: copying 1 paths to 'lb:50051'...5364client # [ 75.732578] nix[1043]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5365worker1 # [ 75.704370] nix-grpc-daemon[779]: ts=2026-09-23T04:42:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409465366worker1 # [ 75.715232] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5367worker1 # [ 75.817343] nix-daemon[847]: accepted connection from pid 1026, user nix-grpc-daemon (trusted)5368worker1 # [ 75.830092] nix-daemon[847]: reaped child process 1029, status = succeeded5369lb # [ 75.875499] niks3-server[1046]: 2026/09/23 04:42:47 INFO Received uploads request method=POST path=/api/pending_closures5370worker1 # [ 75.889667] nix-grpc-daemon[873]: time=2026-09-23T04:42:47.140Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5371worker1 # [ 75.893100] nix-grpc-daemon[873]: time=2026-09-23T04:42:47.143Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5372lb # [ 75.957580] niks3-server[1046]: 2026/09/23 04:42:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5373lb # [ 75.965358] niks3-server[1046]: 2026/09/23 04:42:47 INFO Signed narinfos id=19 count=15374worker1 # [ 75.932295] nix-grpc-daemon[873]: time=2026-09-23T04:42:47.183Z level=INFO msg="Uploading 1 narinfos"5375lb # [ 75.974102] niks3-server[1046]: 2026/09/23 04:42:47 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5376lb # [ 75.983822] niks3-server[1046]: 2026/09/23 04:42:47 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5377lb # [ 75.992617] niks3-server[1046]: 2026/09/23 04:42:47 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5378lb # [ 75.998478] niks3-server[1046]: 2026/09/23 04:42:48 INFO Completed upload id=195379worker1 # [ 75.961297] nix-grpc-daemon[873]: time=2026-09-23T04:42:47.212Z level=INFO msg="Upload complete. (234ms)"5380lb # [ 76.004856] niks3-server[1046]: 2026/09/23 04:42:48 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5381worker1 # [ 75.966511] nix-grpc-daemon[779]: ts=2026-09-23T04:42:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_in=4325382worker1 # [ 75.972978] nix-grpc-daemon[779]: ts=2026-09-23T04:42:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465383worker1 # [ 75.987205] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5384worker1 # [ 75.995264] nix-daemon[847]: reaped child process 1024, status = succeeded5385client # [ 76.032266] nix[1043]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5386worker1 # [ 76.006130] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5387client # [ 76.611882] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5388client # Running as unit: share2.service; invocation ID: 6466d0ca403e4d00baed16553b314d275389client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5390worker2 # [ 76.763311] nix-grpc-daemon[783]: ts=2026-09-23T04:42:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461085391client # [ 76.758842] nix[1059]: don't know how to build these paths:5392client # [ 76.759723] nix[1059]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5393worker2 # [ 76.775949] nix-grpc-daemon[783]: ts=2026-09-23T04:42:48Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05394worker1 # [ 76.740351] nix-grpc-daemon[779]: ts=2026-09-23T04:42:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465395client # [ 76.774139] nix[1059]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5396lb # [ 77.407408] envoy[926]: conn peer=[2001:db8:1::1]:50610 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=935 ms=61085397lb # [ 77.411512] envoy[926]: conn peer=[2001:db8:1::1]:50612 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=935 ms=63265398client: must succeed: sleep 2; systemctl kill -s INT share15399client # [ 79.863862] systemd[1]: share1.service: Sent signal SIGINT to main process 1043 (nix) on client request.5400client # [ 79.870843] nix[1043]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5401client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5402client: waiting for success: ! systemctl is-active share15403client # [ 79.878664] nix[1043]: error: interrupted by the user5404client # [ 79.920387] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5405client # [ 79.921844] systemd[1]: share1.service: Failed with result 'exit-code'.5406client # [ 79.923819] systemd[1]: share1.service: Consumed 118ms CPU time over 4.381s wall clock time, 41.5M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.5407client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5408client: waiting for success: ! systemctl is-active share25409worker1 # [ 81.256469] nix-daemon[847]: reaped child process 1033, status = succeeded5410worker1 # [ 81.331183] nix-daemon[847]: accepted connection from pid 1068, user nix-grpc-daemon (trusted)5411worker1 # [ 81.343693] nix-daemon[847]: reaped child process 1071, status = succeeded5412lb # [ 81.389191] niks3-server[1046]: 2026/09/23 04:42:53 INFO Received uploads request method=POST path=/api/pending_closures5413worker1 # [ 81.360687] nix-grpc-daemon[873]: time=2026-09-23T04:42:52.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5414worker1 # [ 81.362250] nix-grpc-daemon[873]: time=2026-09-23T04:42:52.613Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5415lb # [ 81.432074] niks3-server[1046]: 2026/09/23 04:42:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5416worker1 # [ 81.401798] nix-grpc-daemon[873]: time=2026-09-23T04:42:52.653Z level=INFO msg="Uploading 1 narinfos"5417lb # [ 81.438819] niks3-server[1046]: 2026/09/23 04:42:53 INFO Signed narinfos id=20 count=15418lb # [ 81.453521] niks3-server[1046]: 2026/09/23 04:42:53 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5419lb # [ 81.457161] niks3-server[1046]: 2026/09/23 04:42:53 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5420lb # [ 81.467783] niks3-server[1046]: 2026/09/23 04:42:53 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5421lb # [ 81.472165] niks3-server[1046]: 2026/09/23 04:42:53 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5422lb # [ 81.474633] niks3-server[1046]: 2026/09/23 04:42:53 INFO Completed upload id=205423worker1 # [ 81.438693] nix-grpc-daemon[873]: time=2026-09-23T04:42:52.690Z level=INFO msg="Upload complete. (217ms)"5424lb # [ 81.480871] niks3-server[1046]: 2026/09/23 04:42:53 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5425worker1 # [ 81.445833] nix-grpc-daemon[779]: ts=2026-09-23T04:42:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15426worker1 # [ 81.453311] nix-grpc-daemon[779]: ts=2026-09-23T04:42:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15427worker1 # [ 81.465553] nix-daemon[847]: reaped child process 1031, status = succeeded5428client # [ 81.548492] systemd[1]: share2.service: Deactivated successfully.5429client # [ 81.552749] systemd[1]: share2.service: Consumed 119ms CPU time over 4.940s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5430client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5431client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5432client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5433(finished: subtest: the first client leaving does not fail the second, in 6.75 seconds)5434subtest: build hook: nix-daemon with builders = grpc://lb5435client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5436client # [ 82.340092] systemd[1]: Started Nix Daemon.5437client # [ 82.429775] nix-daemon[1110]: accepted connection from pid 1107, user root (trusted)5438worker2 # [ 82.558638] nix-grpc-daemon[783]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:461085439worker2 # [ 82.563923] nix-grpc-daemon[783]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461085440worker2 # [ 82.574213] nix-grpc-daemon[783]: ts=2026-09-23T04:42:54Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05441worker2 # [ 82.579585] nix-grpc-daemon[783]: ts=2026-09-23T04:42:54Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15442worker2 # [ 82.585380] nix-grpc-daemon[783]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085443worker2 # [ 82.597230] nix-grpc-daemon[783]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461085444worker2 # [ 82.608523] nix-grpc-daemon[783]: ts=2026-09-23T04:42:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461085445worker2 # [ 82.617899] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5446worker2 # [ 82.716841] nix-daemon[874]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5447worker2 # [ 82.730217] nix-daemon[874]: reaped child process 1181, status = succeeded5448lb # [ 82.735539] niks3-server[1046]: 2026/09/23 04:42:54 INFO Received uploads request method=POST path=/api/pending_closures5449worker2 # [ 82.745116] nix-grpc-daemon[889]: time=2026-09-23T04:42:54.829Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5450worker2 # [ 82.746552] nix-grpc-daemon[889]: time=2026-09-23T04:42:54.829Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5451lb # [ 82.778061] niks3-server[1046]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5452worker2 # [ 82.791001] nix-grpc-daemon[889]: time=2026-09-23T04:42:54.875Z level=INFO msg="Uploading 1 narinfos"5453lb # [ 82.789062] niks3-server[1046]: 2026/09/23 04:42:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5454lb # [ 82.792224] niks3-server[1046]: 2026/09/23 04:42:54 INFO Signed narinfos id=21 count=15455lb # [ 82.804665] niks3-server[1046]: 2026/09/23 04:42:54 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5456lb # [ 82.942235] niks3-server[1046]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5457lb # [ 82.955272] niks3-server[1046]: 2026/09/23 04:42:54 INFO Completed upload id=215458worker2 # [ 82.956902] nix-grpc-daemon[889]: time=2026-09-23T04:42:55.041Z level=INFO msg="Upload complete. (325ms)"5459worker2 # [ 82.961886] nix-grpc-daemon[783]: ts=2026-09-23T04:42:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=0 paths=1 nar_bytes_in=5125460lb # [ 82.961864] niks3-server[1046]: 2026/09/23 04:42:54 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5461worker2 # [ 82.970187] nix-grpc-daemon[783]: ts=2026-09-23T04:42:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085462worker2 # [ 82.982130] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5463worker2 # [ 82.990754] nix-daemon[874]: reaped child process 1176, status = succeeded5464worker2 # [ 83.001754] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5465worker2 # [ 84.042865] nix-daemon[874]: reaped child process 1185, status = succeeded5466worker2 # [ 84.128903] nix-daemon[874]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5467worker2 # [ 84.141366] nix-daemon[874]: reaped child process 1204, status = succeeded5468lb # [ 84.147920] niks3-server[1046]: 2026/09/23 04:42:56 INFO Received uploads request method=POST path=/api/pending_closures5469worker2 # [ 84.158092] nix-grpc-daemon[889]: time=2026-09-23T04:42:56.242Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5470worker2 # [ 84.159565] nix-grpc-daemon[889]: time=2026-09-23T04:42:56.244Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5471lb # [ 84.190862] niks3-server[1046]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5472lb # [ 84.197566] niks3-server[1046]: 2026/09/23 04:42:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5473worker2 # [ 84.204424] nix-grpc-daemon[889]: time=2026-09-23T04:42:56.288Z level=INFO msg="Uploading 1 narinfos"5474lb # [ 84.202426] niks3-server[1046]: 2026/09/23 04:42:56 INFO Signed narinfos id=22 count=15475lb # [ 84.210892] niks3-server[1046]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5476lb # [ 84.223177] niks3-server[1046]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5477lb # [ 84.228883] niks3-server[1046]: 2026/09/23 04:42:56 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5478lb # [ 84.234570] niks3-server[1046]: 2026/09/23 04:42:56 INFO Completed upload id=225479worker2 # [ 84.237914] nix-grpc-daemon[889]: time=2026-09-23T04:42:56.322Z level=INFO msg="Upload complete. (223ms)"5480lb # [ 84.238403] niks3-server[1046]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5481worker2 # [ 84.244753] nix-grpc-daemon[783]: ts=2026-09-23T04:42:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15482worker2 # [ 84.259565] nix-daemon[874]: reaped child process 1183, status = succeeded5483worker1 # [ 84.253564] nix-grpc-daemon[779]: ts=2026-09-23T04:42:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:409465484worker1 # [ 84.265251] nix-grpc-daemon[779]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5485worker1 # [ 84.294373] nix-grpc-daemon[779]: ts=2026-09-23T04:42:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409465486worker1 # [ 84.299525] nix-grpc-daemon[779]: ts=2026-09-23T04:42:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_out=1285487worker1 # [ 84.511383] nix-grpc-daemon[779]: ts=2026-09-23T04:42:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:409465488worker1 # [ 84.518101] nix-grpc-daemon[779]: ts=2026-09-23T04:42:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465489worker1 # [ 84.523607] nix-grpc-daemon[873]: time=2026-09-23T04:42:55.773Z level=INFO msg="All 1 paths already cached"5490worker2 # [ 84.572694] nix-grpc-daemon[783]: ts=2026-09-23T04:42:56Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05491worker2 # [ 84.578721] nix-grpc-daemon[783]: ts=2026-09-23T04:42:56Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15492worker2 # [ 84.584700] nix-grpc-daemon[783]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085493worker2 # [ 84.597277] nix-grpc-daemon[783]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461085494worker2 # [ 84.609664] nix-grpc-daemon[889]: time=2026-09-23T04:42:56.694Z level=INFO msg="All 1 paths already cached"5495worker2 # [ 84.617406] nix-grpc-daemon[783]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461085496worker2 # [ 84.628135] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5497worker2 # [ 84.731496] nix-daemon[874]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)5498worker2 # [ 84.745908] nix-daemon[874]: reaped child process 1213, status = succeeded5499lb # [ 84.750103] niks3-server[1046]: 2026/09/23 04:42:56 INFO Received uploads request method=POST path=/api/pending_closures5500worker2 # [ 84.759775] nix-grpc-daemon[889]: time=2026-09-23T04:42:56.844Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5501worker2 # [ 84.761314] nix-grpc-daemon[889]: time=2026-09-23T04:42:56.845Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5502lb # [ 84.780947] niks3-server[1046]: 2026/09/23 04:42:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5503lb # [ 84.790353] niks3-server[1046]: 2026/09/23 04:42:56 INFO Signed narinfos id=23 count=15504worker2 # [ 84.795161] nix-grpc-daemon[889]: time=2026-09-23T04:42:56.879Z level=INFO msg="Uploading 1 narinfos"5505lb # [ 84.795715] niks3-server[1046]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5506lb # [ 84.801216] niks3-server[1046]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5507lb # [ 84.815124] niks3-server[1046]: 2026/09/23 04:42:56 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5508lb # [ 84.823248] niks3-server[1046]: 2026/09/23 04:42:56 INFO Completed upload id=235509worker2 # [ 84.825173] nix-grpc-daemon[889]: time=2026-09-23T04:42:56.909Z level=INFO msg="Upload complete. (184ms)"5510lb # [ 84.829083] niks3-server[1046]: 2026/09/23 04:42:56 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5511worker2 # [ 84.829315] nix-grpc-daemon[783]: ts=2026-09-23T04:42:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=0 paths=1 nar_bytes_in=6245512worker2 # [ 84.835641] nix-grpc-daemon[783]: ts=2026-09-23T04:42:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085513worker2 # [ 84.847142] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5514worker2 # [ 84.854689] nix-daemon[874]: reaped child process 1208, status = succeeded5515worker2 # [ 84.859601] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5516worker2 # [ 85.901671] nix-daemon[874]: reaped child process 1217, status = succeeded5517worker2 # [ 85.979688] nix-daemon[874]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)5518worker2 # [ 85.991199] nix-daemon[874]: reaped child process 1236, status = succeeded5519lb # [ 85.997533] niks3-server[1046]: 2026/09/23 04:42:58 INFO Received uploads request method=POST path=/api/pending_closures5520worker2 # [ 86.009036] nix-grpc-daemon[889]: time=2026-09-23T04:42:58.093Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5521worker2 # [ 86.010540] nix-grpc-daemon[889]: time=2026-09-23T04:42:58.095Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5522lb # [ 86.046456] niks3-server[1046]: 2026/09/23 04:42:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5523lb # [ 86.055825] niks3-server[1046]: 2026/09/23 04:42:58 INFO Signed narinfos id=24 count=15524worker2 # [ 86.060757] nix-grpc-daemon[889]: time=2026-09-23T04:42:58.145Z level=INFO msg="Uploading 1 narinfos"5525lb # [ 86.063836] niks3-server[1046]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5526lb # [ 86.068095] niks3-server[1046]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5527lb # [ 86.076452] niks3-server[1046]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5528lb # [ 86.086063] niks3-server[1046]: 2026/09/23 04:42:58 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5529lb # [ 86.089859] niks3-server[1046]: 2026/09/23 04:42:58 INFO Completed upload id=245530worker2 # [ 86.091590] nix-grpc-daemon[889]: time=2026-09-23T04:42:58.175Z level=INFO msg="Upload complete. (224ms)"5531lb # [ 86.091862] niks3-server[1046]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5532worker2 # [ 86.098798] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15533worker2 # [ 86.112747] nix-daemon[874]: reaped child process 1215, status = succeeded5534worker2 # [ 86.144144] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461085535worker1 # [ 86.124532] nix-grpc-daemon[779]: ts=2026-09-23T04:42:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409465536worker1 # [ 86.136175] nix-grpc-daemon[779]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5537worker1 # [ 86.149542] nix-grpc-daemon[779]: ts=2026-09-23T04:42:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_out=1765538worker2 # [ 86.398779] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:461085539worker1 # [ 86.367383] nix-grpc-daemon[779]: ts=2026-09-23T04:42:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465540worker1 # [ 86.378204] nix-grpc-daemon[873]: time=2026-09-23T04:42:57.629Z level=INFO msg="All 2 paths already cached"5541worker2 # [ 86.431181] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05542worker2 # [ 86.436587] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15543worker2 # [ 86.442897] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085544worker2 # [ 86.455681] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461085545worker2 # [ 86.468196] nix-grpc-daemon[889]: time=2026-09-23T04:42:58.552Z level=INFO msg="All 2 paths already cached"5546worker2 # [ 86.476551] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461085547worker2 # [ 86.486168] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5548worker2 # [ 86.583786] nix-daemon[874]: accepted connection from pid 1242, user nix-grpc-daemon (trusted)5549worker2 # [ 86.597508] nix-daemon[874]: reaped child process 1245, status = succeeded5550lb # [ 86.602619] niks3-server[1046]: 2026/09/23 04:42:58 INFO Received uploads request method=POST path=/api/pending_closures5551worker2 # [ 86.612723] nix-grpc-daemon[889]: time=2026-09-23T04:42:58.697Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5552worker2 # [ 86.614813] nix-grpc-daemon[889]: time=2026-09-23T04:42:58.698Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5553lb # [ 86.635514] niks3-server[1046]: 2026/09/23 04:42:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5554lb # [ 86.641750] niks3-server[1046]: 2026/09/23 04:42:58 INFO Signed narinfos id=25 count=15555worker2 # [ 86.646682] nix-grpc-daemon[889]: time=2026-09-23T04:42:58.731Z level=INFO msg="Uploading 1 narinfos"5556lb # [ 86.647488] niks3-server[1046]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5557lb # [ 86.660179] niks3-server[1046]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5558lb # [ 86.669770] niks3-server[1046]: 2026/09/23 04:42:58 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5559lb # [ 86.677301] niks3-server[1046]: 2026/09/23 04:42:58 INFO Completed upload id=255560lb # [ 86.680377] niks3-server[1046]: 2026/09/23 04:42:58 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5561worker2 # [ 86.681510] nix-grpc-daemon[889]: time=2026-09-23T04:42:58.765Z level=INFO msg="Upload complete. (182ms)"5562worker2 # [ 86.685956] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=0 paths=1 nar_bytes_in=7525563worker2 # [ 86.692596] nix-grpc-daemon[783]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085564worker2 # [ 86.703495] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5565worker2 # [ 86.713384] nix-daemon[874]: reaped child process 1240, status = succeeded5566worker2 # [ 86.718431] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5567lb # [ 87.407576] envoy[926]: conn peer=[2001:db8:1::1]:34024 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=771 ms=42015568lb # [ 87.412501] envoy[926]: conn peer=[2001:db8:1::1]:34028 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1194 ms=47745569lb # [ 87.416458] envoy[926]: conn peer=[2001:db8:1::1]:47188 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=585570lb # [ 87.420299] envoy[926]: conn peer=[2001:db8:1::1]:34038 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3426 tx=1939 ms=18345571lb # [ 87.424460] envoy[926]: conn peer=[2001:db8:1::1]:47192 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2140 ms=16905572lb # [ 87.428968] envoy[926]: conn peer=[2001:db8:1::1]:47208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=795573worker2 # [ 87.535188] nix-daemon[874]: reaped child process 1249, status = succeeded5574worker2 # [ 87.609379] nix-daemon[874]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)5575worker2 # [ 87.623611] nix-daemon[874]: reaped child process 1268, status = succeeded5576lb # [ 87.630395] niks3-server[1046]: 2026/09/23 04:42:59 INFO Received uploads request method=POST path=/api/pending_closures5577worker2 # [ 87.640054] nix-grpc-daemon[889]: time=2026-09-23T04:42:59.724Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5578worker2 # [ 87.641739] nix-grpc-daemon[889]: time=2026-09-23T04:42:59.726Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5579lb # [ 87.675804] niks3-server[1046]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5580lb # [ 87.679638] niks3-server[1046]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5581lb # [ 87.685315] niks3-server[1046]: 2026/09/23 04:42:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5582lb # [ 87.691090] niks3-server[1046]: 2026/09/23 04:42:59 INFO Signed narinfos id=26 count=15583worker2 # [ 87.695467] nix-grpc-daemon[889]: time=2026-09-23T04:42:59.779Z level=INFO msg="Uploading 1 narinfos"5584lb # [ 87.696447] niks3-server[1046]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5585lb # [ 87.706339] niks3-server[1046]: 2026/09/23 04:42:59 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5586lb # [ 87.710913] niks3-server[1046]: 2026/09/23 04:42:59 INFO Completed upload id=265587worker2 # [ 87.712885] nix-grpc-daemon[889]: time=2026-09-23T04:42:59.797Z level=INFO msg="Upload complete. (188ms)"5588lb # [ 87.712803] niks3-server[1046]: 2026/09/23 04:42:59 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5589worker2 # [ 87.719328] nix-grpc-daemon[783]: ts=2026-09-23T04:42:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15590worker1 # [ 87.689846] nix-grpc-daemon[779]: ts=2026-09-23T04:42:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:409465591worker2 # [ 87.732174] nix-daemon[874]: reaped child process 1247, status = succeeded5592worker1 # [ 87.699749] nix-grpc-daemon[779]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5593worker2 # [ 87.762858] nix-grpc-daemon[783]: ts=2026-09-23T04:42:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:461085594worker2 # [ 87.768336] nix-grpc-daemon[783]: ts=2026-09-23T04:42:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=0 paths=1 nar_bytes_out=2325595client # [ 87.827972] nix-daemon[1110]: reaped child process 1112, status = succeeded5596client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.59 seconds)5597client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5598client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5599client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5600client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5601client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5602client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5603(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.65 seconds)5604subtest: an input only one worker has still reaches the builder5605client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5606client # this derivation will be built:5607client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5608client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5609client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.24 seconds)5610client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5611client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5612worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5613worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5614worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5615worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5616client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25617client # [ 88.506199] nix-daemon[1110]: accepted connection from pid 1193, user root (trusted)5618client # this derivation will be built:5619client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5620worker1 # [ 88.602807] nix-grpc-daemon[779]: ts=2026-09-23T04:42:59Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:409465621client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5622worker1 # [ 88.608892] nix-grpc-daemon[779]: ts=2026-09-23T04:42:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465623worker1 # [ 88.713042] nix-daemon[847]: accepted connection from pid 1081, user nix-grpc-daemon (trusted)5624worker1 # [ 88.724892] nix-daemon[847]: reaped child process 1084, status = succeeded5625lb # [ 88.769345] niks3-server[1046]: 2026/09/23 04:43:00 INFO Received uploads request method=POST path=/api/pending_closures5626worker1 # [ 88.742040] nix-grpc-daemon[873]: time=2026-09-23T04:42:59.993Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5627worker1 # [ 88.743496] nix-grpc-daemon[873]: time=2026-09-23T04:42:59.993Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5628lb # [ 88.798483] niks3-server[1046]: 2026/09/23 04:43:00 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5629lb # [ 88.805806] niks3-server[1046]: 2026/09/23 04:43:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5630lb # [ 88.811828] niks3-server[1046]: 2026/09/23 04:43:00 INFO Signed narinfos id=27 count=15631worker1 # [ 88.777184] nix-grpc-daemon[873]: time=2026-09-23T04:43:00.028Z level=INFO msg="Uploading 1 narinfos"5632lb # [ 88.819505] niks3-server[1046]: 2026/09/23 04:43:00 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5633lb # [ 88.830331] niks3-server[1046]: 2026/09/23 04:43:00 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5634lb # [ 88.833241] niks3-server[1046]: 2026/09/23 04:43:00 INFO Completed upload id=275635worker1 # [ 88.797168] nix-grpc-daemon[873]: time=2026-09-23T04:43:00.048Z level=INFO msg="Upload complete. (189ms)"5636lb # [ 88.835911] niks3-server[1046]: 2026/09/23 04:43:00 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5637worker2 # [ 88.854807] nix-grpc-daemon[783]: ts=2026-09-23T04:43:00Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05638worker2 # [ 88.859474] nix-grpc-daemon[783]: ts=2026-09-23T04:43:00Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15639worker1 # [ 88.860578] nix-grpc-daemon[779]: ts=2026-09-23T04:43:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465640worker1 # [ 88.873589] nix-grpc-daemon[779]: ts=2026-09-23T04:43:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465641client # copying 2 paths to 'lb:50051'...5642client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5643worker1 # [ 88.890085] nix-grpc-daemon[779]: ts=2026-09-23T04:43:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409465644client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5645worker1 # [ 88.905159] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5646worker1 # [ 89.010769] nix-daemon[847]: accepted connection from pid 1088, user nix-grpc-daemon (trusted)5647worker1 # [ 89.024747] nix-daemon[847]: reaped child process 1091, status = succeeded5648lb # [ 89.068432] niks3-server[1046]: 2026/09/23 04:43:01 INFO Received uploads request method=POST path=/api/pending_closures5649lb # [ 89.077380] niks3-server[1046]: 2026/09/23 04:43:01 INFO Received uploads request method=POST path=/api/pending_closures5650worker1 # [ 89.049099] nix-grpc-daemon[873]: time=2026-09-23T04:43:00.300Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5651worker1 # [ 89.050623] nix-grpc-daemon[873]: time=2026-09-23T04:43:00.302Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5652worker1 # [ 89.052910] nix-grpc-daemon[873]: time=2026-09-23T04:43:00.303Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5653lb # [ 89.111428] niks3-server[1046]: 2026/09/23 04:43:01 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5654lb # [ 89.122153] niks3-server[1046]: 2026/09/23 04:43:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5655lb # [ 89.124484] niks3-server[1046]: 2026/09/23 04:43:01 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5656lb # [ 89.128385] niks3-server[1046]: 2026/09/23 04:43:01 INFO Signed narinfos id=28 count=15657lb # [ 89.129738] niks3-server[1046]: 2026/09/23 04:43:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5658lb # [ 89.132100] niks3-server[1046]: 2026/09/23 04:43:01 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5659lb # [ 89.135055] niks3-server[1046]: 2026/09/23 04:43:01 INFO Signed narinfos id=29 count=25660lb # [ 89.137286] niks3-server[1046]: 2026/09/23 04:43:01 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5661worker1 # [ 89.100090] nix-grpc-daemon[873]: time=2026-09-23T04:43:00.350Z level=INFO msg="Uploading 3 narinfos"5662lb # [ 89.151938] niks3-server[1046]: 2026/09/23 04:43:01 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5663lb # [ 89.153592] niks3-server[1046]: 2026/09/23 04:43:01 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5664lb # [ 89.157051] niks3-server[1046]: 2026/09/23 04:43:01 INFO Completed upload id=285665lb # [ 89.158599] niks3-server[1046]: 2026/09/23 04:43:01 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5666lb # [ 89.160960] niks3-server[1046]: 2026/09/23 04:43:01 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5667lb # [ 89.163019] niks3-server[1046]: 2026/09/23 04:43:01 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5668lb # [ 89.164759] niks3-server[1046]: 2026/09/23 04:43:01 INFO Completed upload id=295669worker1 # [ 89.129117] nix-grpc-daemon[873]: time=2026-09-23T04:43:00.380Z level=INFO msg="Upload complete. (209ms)"5670worker1 # [ 89.134453] nix-grpc-daemon[779]: ts=2026-09-23T04:43:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=2 nar_bytes_in=9765671worker1 # [ 89.140128] nix-grpc-daemon[779]: ts=2026-09-23T04:43:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465672worker1 # [ 89.152166] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5673worker1 # [ 89.161338] nix-daemon[847]: reaped child process 1086, status = succeeded5674worker1 # [ 89.166533] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5675client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5676client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5677client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5678worker1 # [ 89.999898] nix-daemon[847]: reaped child process 1095, status = succeeded5679worker1 # [ 90.086471] nix-daemon[847]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5680worker1 # [ 90.097479] nix-daemon[847]: reaped child process 1114, status = succeeded5681lb # [ 90.143901] niks3-server[1046]: 2026/09/23 04:43:02 INFO Received uploads request method=POST path=/api/pending_closures5682worker1 # [ 90.116824] nix-grpc-daemon[873]: time=2026-09-23T04:43:01.368Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5683lb # [ 90.176515] niks3-server[1046]: 2026/09/23 04:43:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5684worker1 # [ 90.146933] nix-grpc-daemon[873]: time=2026-09-23T04:43:01.398Z level=INFO msg="Uploading 1 narinfos"5685lb # [ 90.184199] niks3-server[1046]: 2026/09/23 04:43:02 INFO Signed narinfos id=30 count=15686lb # [ 90.192584] niks3-server[1046]: 2026/09/23 04:43:02 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5687lb # [ 90.200104] niks3-server[1046]: 2026/09/23 04:43:02 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5688lb # [ 90.210528] niks3-server[1046]: 2026/09/23 04:43:02 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5689lb # [ 90.216574] niks3-server[1046]: 2026/09/23 04:43:02 INFO Completed upload id=305690worker1 # [ 90.179501] nix-grpc-daemon[873]: time=2026-09-23T04:43:01.430Z level=INFO msg="Upload complete. (205ms)"5691lb # [ 90.221408] niks3-server[1046]: 2026/09/23 04:43:02 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5692worker1 # [ 90.185435] nix-grpc-daemon[779]: ts=2026-09-23T04:43:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15693client # copying 1 paths...5694worker2 # [ 90.233903] nix-grpc-daemon[783]: ts=2026-09-23T04:43:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461085695worker1 # [ 90.199654] nix-daemon[847]: reaped child process 1093, status = succeeded5696worker2 # [ 90.242837] nix-grpc-daemon[783]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5697client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5698worker1 # [ 90.229826] nix-grpc-daemon[779]: ts=2026-09-23T04:43:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409465699worker1 # [ 90.234968] nix-grpc-daemon[779]: ts=2026-09-23T04:43:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_out=1205700client # [ 90.334439] nix-daemon[1110]: reaped child process 1196, status = succeeded5701client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 1.95 seconds)5702client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25703client # [ 90.429942] nix-daemon[1110]: accepted connection from pid 1215, user root (trusted)5704client # this derivation will be built:5705client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5706worker2 # [ 90.559499] nix-grpc-daemon[783]: ts=2026-09-23T04:43:02Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:461085707client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5708worker1 # [ 90.528911] nix-grpc-daemon[779]: ts=2026-09-23T04:43:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465709worker1 # [ 90.537988] nix-grpc-daemon[873]: time=2026-09-23T04:43:01.789Z level=INFO msg="All 1 paths already cached"5710worker2 # [ 90.592107] nix-grpc-daemon[783]: ts=2026-09-23T04:43:02Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05711worker2 # [ 90.597883] nix-grpc-daemon[783]: ts=2026-09-23T04:43:02Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15712worker2 # [ 90.603804] nix-grpc-daemon[783]: ts=2026-09-23T04:43:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085713worker2 # [ 90.615870] nix-grpc-daemon[783]: ts=2026-09-23T04:43:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461085714client # copying 1 paths to 'lb:50051'...5715client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5716worker2 # [ 90.633410] nix-grpc-daemon[783]: ts=2026-09-23T04:43:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461085717worker2 # [ 90.644381] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5718worker2 # [ 90.660511] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5)5719worker2 # [ 90.713946] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 2/5)5720worker2 # [ 90.881289] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 3/5)5721worker2 # [ 90.952515] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 751 ms (attempt 4/5)5722worker2 # [ 91.713084] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5723worker2 # [ 91.716457] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5724worker2 # [ 91.721759] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5725worker2 # [ 91.845496] nix-daemon[874]: accepted connection from pid 1285, user nix-grpc-daemon (trusted)5726worker2 # [ 91.860095] nix-daemon[874]: reaped child process 1288, status = succeeded5727lb # [ 91.866937] niks3-server[1046]: 2026/09/23 04:43:03 INFO Received uploads request method=POST path=/api/pending_closures5728worker2 # [ 91.878525] nix-grpc-daemon[889]: time=2026-09-23T04:43:03.962Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5729worker2 # [ 91.880441] nix-grpc-daemon[889]: time=2026-09-23T04:43:03.964Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5730lb # [ 91.899098] niks3-server[1046]: 2026/09/23 04:43:03 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5731lb # [ 91.904940] niks3-server[1046]: 2026/09/23 04:43:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5732lb # [ 91.912938] niks3-server[1046]: 2026/09/23 04:43:03 INFO Signed narinfos id=31 count=15733worker2 # [ 91.918356] nix-grpc-daemon[889]: time=2026-09-23T04:43:04.002Z level=INFO msg="Uploading 1 narinfos"5734lb # [ 91.921075] niks3-server[1046]: 2026/09/23 04:43:03 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5735lb # [ 91.932713] niks3-server[1046]: 2026/09/23 04:43:03 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5736lb # [ 91.942174] niks3-server[1046]: 2026/09/23 04:43:03 INFO Completed upload id=315737worker2 # [ 91.944166] nix-grpc-daemon[889]: time=2026-09-23T04:43:04.028Z level=INFO msg="Upload complete. (204ms)"5738lb # [ 91.945056] niks3-server[1046]: 2026/09/23 04:43:03 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5739worker2 # [ 91.948494] nix-grpc-daemon[783]: ts=2026-09-23T04:43:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=1 paths=1 nar_bytes_in=5605740worker2 # [ 91.954610] nix-grpc-daemon[783]: ts=2026-09-23T04:43:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461085741worker2 # [ 91.962460] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5742worker2 # [ 91.977235] nix-daemon[874]: reaped child process 1278, status = succeeded5743worker2 # [ 91.981635] nix-grpc-daemon[783]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5744worker2 # [ 91.989724] nix-daemon[874]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5745client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5746client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5747client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5748worker2 # [ 92.616549] nix-daemon[874]: reaped child process 1293, status = succeeded5749worker2 # [ 92.709686] nix-daemon[874]: accepted connection from pid 1309, user nix-grpc-daemon (trusted)5750worker2 # [ 92.721169] nix-daemon[874]: reaped child process 1312, status = succeeded5751lb # [ 92.727346] niks3-server[1046]: 2026/09/23 04:43:04 INFO Received uploads request method=POST path=/api/pending_closures5752worker2 # [ 92.833271] nix-grpc-daemon[889]: time=2026-09-23T04:43:04.917Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5753lb # [ 92.854746] niks3-server[1046]: 2026/09/23 04:43:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5754lb # [ 92.863381] niks3-server[1046]: 2026/09/23 04:43:04 INFO Signed narinfos id=32 count=15755worker2 # [ 92.869583] nix-grpc-daemon[889]: time=2026-09-23T04:43:04.953Z level=INFO msg="Uploading 1 narinfos"5756lb # [ 92.877287] niks3-server[1046]: 2026/09/23 04:43:04 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5757lb # [ 92.890325] niks3-server[1046]: 2026/09/23 04:43:04 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5758lb # [ 92.902128] niks3-server[1046]: 2026/09/23 04:43:04 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5759lb # [ 92.915599] niks3-server[1046]: 2026/09/23 04:43:04 INFO Completed upload id=325760worker2 # [ 92.918318] nix-grpc-daemon[889]: time=2026-09-23T04:43:05.001Z level=INFO msg="Upload complete. (319ms)"5761lb # [ 92.922767] niks3-server[1046]: 2026/09/23 04:43:04 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5762client # copying 1 paths...5763worker2 # [ 92.925493] nix-grpc-daemon[783]: ts=2026-09-23T04:43:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46108 duration_s=0 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15764worker2 # [ 92.934989] nix-grpc-daemon[783]: ts=2026-09-23T04:43:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461085765client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5766worker2 # [ 92.946734] nix-daemon[874]: reaped child process 1290, status = succeeded5767worker1 # [ 92.919556] nix-grpc-daemon[779]: ts=2026-09-23T04:43:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409465768worker1 # [ 92.931555] nix-grpc-daemon[779]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5769worker1 # [ 92.945161] nix-grpc-daemon[779]: ts=2026-09-23T04:43:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_out=1205770client # [ 93.085381] nix-daemon[1110]: reaped child process 1218, status = succeeded5771client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.75 seconds)5772client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25773client # [ 93.193777] nix-daemon[1110]: accepted connection from pid 1236, user root (trusted)5774client # this derivation will be built:5775client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5776worker2 # [ 93.325961] nix-grpc-daemon[783]: ts=2026-09-23T04:43:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:461085777client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5778worker1 # [ 93.290409] nix-grpc-daemon[779]: ts=2026-09-23T04:43:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465779worker1 # [ 93.300820] nix-grpc-daemon[873]: time=2026-09-23T04:43:04.551Z level=INFO msg="All 1 paths already cached"5780worker2 # [ 93.354559] nix-grpc-daemon[783]: ts=2026-09-23T04:43:05Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05781worker2 # [ 93.359216] nix-grpc-daemon[783]: ts=2026-09-23T04:43:05Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15782worker1 # [ 93.359957] nix-grpc-daemon[779]: ts=2026-09-23T04:43:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465783worker1 # [ 93.372574] nix-grpc-daemon[779]: ts=2026-09-23T04:43:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409465784worker1 # [ 93.385542] nix-grpc-daemon[873]: time=2026-09-23T04:43:04.637Z level=INFO msg="All 1 paths already cached"5785client # copying 1 paths to 'lb:50051'...5786client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5787worker1 # [ 93.395248] nix-grpc-daemon[779]: ts=2026-09-23T04:43:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409465788worker1 # [ 93.406583] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5789worker1 # [ 93.509168] nix-daemon[847]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5790worker1 # [ 93.522390] nix-daemon[847]: reaped child process 1122, status = succeeded5791lb # [ 93.565941] niks3-server[1046]: 2026/09/23 04:43:05 INFO Received uploads request method=POST path=/api/pending_closures5792worker1 # [ 93.578590] nix-grpc-daemon[873]: time=2026-09-23T04:43:04.830Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5793worker1 # [ 93.581095] nix-grpc-daemon[873]: time=2026-09-23T04:43:04.831Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5794worker1 # [ 93.604260] nix-grpc-daemon[873]: time=2026-09-23T04:43:04.855Z level=INFO msg="Uploading 1 narinfos"5795lb # [ 93.641169] niks3-server[1046]: 2026/09/23 04:43:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5796lb # [ 93.645089] niks3-server[1046]: 2026/09/23 04:43:05 INFO Signed narinfos id=33 count=15797lb # [ 93.652604] niks3-server[1046]: 2026/09/23 04:43:05 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5798lb # [ 93.663603] niks3-server[1046]: 2026/09/23 04:43:05 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5799lb # [ 93.670763] niks3-server[1046]: 2026/09/23 04:43:05 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5800lb # [ 93.676128] niks3-server[1046]: 2026/09/23 04:43:05 INFO Completed upload id=335801worker1 # [ 93.638748] nix-grpc-daemon[873]: time=2026-09-23T04:43:04.890Z level=INFO msg="Upload complete. (220ms)"5802worker1 # [ 93.643430] nix-grpc-daemon[779]: ts=2026-09-23T04:43:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_in=5605803lb # [ 93.683794] niks3-server[1046]: 2026/09/23 04:43:05 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5804worker1 # [ 93.651365] nix-grpc-daemon[779]: ts=2026-09-23T04:43:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409465805worker1 # [ 93.662119] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5806worker1 # [ 93.674287] nix-daemon[847]: reaped child process 1117, status = succeeded5807client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5808worker1 # [ 93.681897] nix-daemon[847]: accepted connection from pid 779, user nix-grpc-daemon (trusted)5809client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5810client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5811worker1 # [ 94.522396] nix-daemon[847]: reaped child process 1126, status = succeeded5812worker1 # [ 94.606908] nix-daemon[847]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)5813worker1 # [ 94.619222] nix-daemon[847]: reaped child process 1145, status = succeeded5814lb # [ 94.663764] niks3-server[1046]: 2026/09/23 04:43:06 INFO Received uploads request method=POST path=/api/pending_closures5815worker1 # [ 94.842552] nix-grpc-daemon[873]: time=2026-09-23T04:43:06.093Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5816lb # [ 94.903359] niks3-server[1046]: 2026/09/23 04:43:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5817lb # [ 94.910075] niks3-server[1046]: 2026/09/23 04:43:06 INFO Signed narinfos id=34 count=15818worker1 # [ 94.872415] nix-grpc-daemon[873]: time=2026-09-23T04:43:06.123Z level=INFO msg="Uploading 1 narinfos"5819lb # [ 94.929322] niks3-server[1046]: 2026/09/23 04:43:06 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5820lb # [ 94.947481] niks3-server[1046]: 2026/09/23 04:43:06 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5821lb # [ 94.953552] niks3-server[1046]: 2026/09/23 04:43:06 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5822lb # [ 94.967592] niks3-server[1046]: 2026/09/23 04:43:06 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5823lb # [ 94.975864] niks3-server[1046]: 2026/09/23 04:43:06 INFO Completed upload id=345824worker1 # [ 94.938854] nix-grpc-daemon[873]: time=2026-09-23T04:43:06.190Z level=INFO msg="Upload complete. (447ms)"5825client # copying 1 paths...5826worker1 # [ 94.946376] nix-grpc-daemon[779]: ts=2026-09-23T04:43:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15827worker2 # [ 94.992601] nix-grpc-daemon[783]: ts=2026-09-23T04:43:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461085828worker1 # [ 94.960971] nix-daemon[847]: reaped child process 1124, status = succeeded5829worker2 # [ 95.004361] nix-grpc-daemon[783]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5830client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5831worker1 # [ 94.996587] nix-grpc-daemon[779]: ts=2026-09-23T04:43:06Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:409465832worker1 # [ 95.001626] nix-grpc-daemon[779]: ts=2026-09-23T04:43:06Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40946 duration_s=0 paths=1 nar_bytes_out=1205833client # [ 95.099816] nix-daemon[1110]: reaped child process 1239, status = succeeded5834client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.01 seconds)5835(finished: subtest: an input only one worker has still reaches the builder, in 7.21 seconds)5836subtest: upload whose reference the worker lacks is completed from the cache5837client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5838client # this derivation will be built:5839client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5840client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5841client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.25 seconds)5842client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5843client # this derivation will be built:5844client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5845client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5846client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.27 seconds)5847client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5848client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5849worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5850worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5851worker2 # [ 95.905937] systemd[1]: Stopping Nix worker-protocol over gRPC...5852worker2 # [ 96.651914] nix-grpc-daemon[783]: ts=2026-09-23T04:43:08Z level=info event=signal_exit5853lb # [ 97.408588] envoy[926]: conn peer=[2001:db8:1::1]:47212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4324 tx=2394 ms=13805854lb # [ 97.412719] envoy[926]: conn peer=[2001:db8:1::1]:47216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=185855lb # [ 97.416357] envoy[926]: conn peer=[2001:db8:1::1]:47224 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3852 tx=2043 ms=16445856lb # [ 97.419930] envoy[926]: conn peer=[2001:db8:1::1]:47228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=165857lb # [ 97.423406] envoy[926]: conn peer=[2001:db8:1::1]:47230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=2103 ms=24775858lb # [ 97.426877] envoy[926]: conn peer=[2001:db8:1::1]:42190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=805859lb # [ 97.430871] envoy[926]: conn peer=[2001:db8:1::1]:42206 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3731 tx=2125 ms=17275860lb # [ 97.434355] envoy[926]: conn peer=[2001:db8:1::1]:42208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=205861lb # [ 97.535818] niks3-server[1046]: 2026/09/23 04:43:09 INFO lead: released remote=192.168.1.4:563565862lb # [ 97.540755] niks3-server[1046]: 2026/09/23 04:43:09 INFO lead: acquired remote=192.168.1.3:606025863worker1 # [ 97.504775] nix-grpc-daemon[779]: ts=2026-09-23T04:43:08Z level=info event=scheduler_take_over5864worker1 # [ 98.499223] nix-grpc-daemon[779]: ts=2026-09-23T04:43:09Z level=info event=scheduler_restarting addr=lb:500515865worker2 # [ 98.536856] nix-grpc-daemon[783]: ts=2026-09-23T04:43:10Z level=info event=scheduler_restarting5866worker2 # [ 98.546526] nix-grpc-daemon[783]: ts=2026-09-23T04:43:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:461045867worker2 # [ 98.550604] nix-grpc-daemon[783]: ts=2026-09-23T04:43:10Z level=info event=worker_gone addr=192.168.1.3:500515868worker2 # [ 98.553617] nix-grpc-daemon[783]: ts=2026-09-23T04:43:10Z level=info event=scheduler_restarting addr=lb:500515869worker2 # [ 98.556709] nix-grpc-daemon[783]: ts=2026-09-23T04:43:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:461045870worker2 # [ 98.560265] nix-grpc-daemon[783]: ts=2026-09-23T04:43:10Z level=info event=worker_gone addr=192.168.1.4:500515871worker2 # [ 98.578226] nix-daemon[874]: reaped child process 877, status = succeeded5872worker1 # [ 98.701314] nix-grpc-daemon[779]: ts=2026-09-23T04:43:09Z level=info event=scheduler_connected addr=lb:500515873worker1 # [ 98.705545] nix-grpc-daemon[779]: ts=2026-09-23T04:43:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5874worker2 # [ 98.759285] nix-grpc-daemon[783]: ts=2026-09-23T04:43:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5875worker2 # [ 98.798134] nix-grpc-daemon[783]: E0000 00:00:1790138590.882122 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5876worker2 # [ 98.803511] nix-grpc-daemon[783]: E0000 00:00:1790138590.882372 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5877worker1 # [ 98.905978] nix-grpc-daemon[779]: ts=2026-09-23T04:43:10Z level=info event=scheduler_connected addr=lb:500515878worker1 # [ 98.918993] nix-grpc-daemon[779]: ts=2026-09-23T04:43:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:362725879worker1 # [ 98.923889] nix-grpc-daemon[779]: ts=2026-09-23T04:43:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05880worker2 # [ 100.730469] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5881worker2 # [ 100.734497] systemd[1]: Stopped Nix worker-protocol over gRPC.5882worker2 # [ 100.738729] systemd[1]: nix-grpc-daemon.service: Consumed 2.585s CPU time over 1min 23.268s wall clock time, 72.1M memory peak, 117.1K incoming IP traffic, 321.3K outgoing IP traffic.5883worker2 # [ 100.750164] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5884worker2 # [ 100.754477] systemd[1]: Closed Nix worker-protocol over gRPC.5885worker2 # [ 100.757959] systemd[1]: nix-grpc-daemon.socket: Consumed 335us CPU time over 1min 33.157s wall clock time, 256K memory peak, 120.5K incoming IP traffic, 100.6K outgoing IP traffic.5886lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5887lb: must succeed: curl -sf localhost:9901/clusters5888lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5889lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5890client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25891worker1 # [ 100.919604] nix-grpc-daemon[779]: ts=2026-09-23T04:43:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409465892client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5893worker1 # [ 100.924950] nix-grpc-daemon[779]: ts=2026-09-23T04:43:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:409465894client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds)5895worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5896worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5897worker2 # [ 101.044896] systemd[1]: Listening on Nix worker-protocol over gRPC.5898worker1 # [ 101.053442] systemd[1]: Stopping Nix worker-protocol over gRPC...5899worker1 # [ 101.570662] nix-grpc-daemon[779]: ts=2026-09-23T04:43:12Z level=info event=signal_exit5900worker2 # [ 101.727182] systemd[1]: Starting scheduler-token.service...5901worker2 # [ 101.788783] systemd[1]: scheduler-token.service: Deactivated successfully.5902worker2 # [ 101.791840] systemd[1]: Finished scheduler-token.service.5903worker2 # [ 101.794354] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 64ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5904worker2 # [ 101.802785] systemd[1]: Starting Nix worker-protocol over gRPC...5905worker2 # [ 101.881370] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5906worker2 # [ 101.883441] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5907worker2 # [ 101.886624] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15908worker2 # [ 101.891029] nix-grpc-daemon[1339]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5909worker2 # [ 101.892386] nix-grpc-daemon[1339]: E0000 00:00:1790138593.975148 1339 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5910worker2 # [ 101.898169] nix-grpc-daemon[1339]: E0000 00:00:1790138593.980649 1339 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5911worker2 # [ 101.901757] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5912worker2 # [ 101.904720] systemd[1]: Started Nix worker-protocol over gRPC.5913worker2 # [ 101.910063] nix-grpc-daemon[1339]: E0000 00:00:1790138593.993984 1380 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5914worker2 # [ 101.923573] nix-grpc-daemon[1339]: E0000 00:00:1790138594.008028 1380 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5915worker2 # [ 101.928107] nix-grpc-daemon[1339]: E0000 00:00:1790138594.010380 1380 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5916worker2 # [ 101.937819] nix-grpc-daemon[1339]: E0000 00:00:1790138594.022282 1380 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5917worker2 # [ 101.947358] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_connected addr=lb:500515918worker2 # [ 101.948925] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5919worker2 # [ 102.150333] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_connected addr=lb:500515920worker2 # [ 102.154412] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5921worker2 # [ 102.355349] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_connected addr=lb:500515922worker2 # [ 102.359302] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5923lb # [ 102.537369] niks3-server[1046]: 2026/09/23 04:43:14 INFO lead: released remote=192.168.1.3:606025924worker2 # [ 102.560515] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_connected addr=lb:500515925worker2 # [ 102.564372] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5926worker2 # [ 102.765505] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_connected addr=lb:500515927worker2 # [ 102.769370] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5928worker2 # [ 102.970858] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_connected addr=lb:500515929worker2 # [ 102.974766] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5930worker2 # [ 103.175961] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_connected addr=lb:500515931worker2 # [ 103.179880] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5932worker2 # [ 103.381303] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_connected addr=lb:500515933worker2 # [ 103.385336] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5934worker1 # [ 103.504908] nix-grpc-daemon[779]: ts=2026-09-23T04:43:14Z level=info event=scheduler_restarting5935worker1 # [ 103.508955] nix-grpc-daemon[779]: ts=2026-09-23T04:43:14Z level=info event=scheduler_restarting addr=lb:500515936worker1 # [ 103.512841] nix-grpc-daemon[779]: ts=2026-09-23T04:43:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:362725937worker1 # [ 103.516482] nix-grpc-daemon[779]: ts=2026-09-23T04:43:14Z level=info event=worker_gone addr=192.168.1.3:500515938worker2 # [ 103.585813] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_connected addr=lb:500515939worker2 # [ 103.587674] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5940worker1 # [ 103.557862] nix-daemon[847]: reaped child process 849, status = succeeded5941worker1 # [ 103.715808] nix-grpc-daemon[779]: ts=2026-09-23T04:43:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5942worker1 # [ 103.723987] nix-grpc-daemon[779]: E0000 00:00:1790138594.975248 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5943worker1 # [ 103.728745] nix-grpc-daemon[779]: E0000 00:00:1790138594.975453 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5944worker2 # [ 103.788446] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_connected addr=lb:500515945worker2 # [ 103.790298] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5946worker2 # [ 103.991331] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_connected addr=lb:500515947worker2 # [ 103.993098] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5948worker2 # [ 104.194205] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_connected addr=lb:500515949worker2 # [ 104.198232] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5950worker2 # [ 104.399234] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_connected addr=lb:500515951worker2 # [ 104.403289] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5952worker2 # [ 104.604650] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_connected addr=lb:500515953worker2 # [ 104.608616] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5954worker1 # [ 104.650772] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5955worker1 # [ 104.653826] systemd[1]: Stopped Nix worker-protocol over gRPC.5956worker1 # [ 104.658279] systemd[1]: nix-grpc-daemon.service: Consumed 2.064s CPU time over 1min 27.282s wall clock time, 61.6M memory peak, 92.5K incoming IP traffic, 170.7K outgoing IP traffic.5957worker1 # [ 104.669860] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5958worker1 # [ 104.674353] systemd[1]: Closed Nix worker-protocol over gRPC.5959worker1 # [ 104.677513] systemd[1]: nix-grpc-daemon.socket: Consumed 316us CPU time over 1min 36.200s wall clock time, 256K memory peak, 99.2K incoming IP traffic, 89.1K outgoing IP traffic.5960lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5961lb: must succeed: curl -sf localhost:9901/clusters5962lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5963lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5964client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25965worker2 # [ 104.809635] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_connected addr=lb:500515966worker2 # [ 104.813610] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5967worker2 # [ 104.924773] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551105968worker2 # [ 104.936400] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5969worker2 # [ 104.952726] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 1/5)5970worker2 # [ 105.009114] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 126 ms (attempt 2/5)5971worker2 # [ 105.015529] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_connected addr=lb:500515972worker2 # [ 105.019280] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5973worker2 # [ 105.136610] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 266 ms (attempt 3/5)5974worker2 # [ 105.220538] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_connected addr=lb:500515975worker2 # [ 105.224474] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5976worker2 # [ 105.404736] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 761 ms (attempt 4/5)5977worker2 # [ 105.425322] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_connected addr=lb:500515978worker2 # [ 105.429126] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5979worker2 # [ 105.630395] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_connected addr=lb:500515980worker2 # [ 105.634335] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5981worker2 # [ 105.835452] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_connected addr=lb:500515982worker2 # [ 105.839343] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5983worker2 # [ 106.040924] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500515984worker2 # [ 106.044889] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5985worker2 # [ 106.168321] nix-grpc-daemon[1339]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5986worker2 # [ 106.171918] nix-grpc-daemon[1339]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5987client # copying 2 paths...5988client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5989worker2 # [ 106.195657] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551105990client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5991worker2 # [ 106.205821] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5992worker2 # [ 106.230428] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=niks3_push_started pid=14005993worker2 # [ 106.247366] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500515994worker2 # [ 106.251987] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5995worker2 # [ 106.341278] nix-grpc-daemon[1400]: time=2026-09-23T04:43:18.423Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5996worker2 # [ 106.427078] nix-daemon[874]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5997worker2 # [ 106.438905] nix-daemon[874]: reaped child process 1415, status = succeeded5998lb # [ 106.445277] niks3-server[1046]: 2026/09/23 04:43:18 INFO Received uploads request method=POST path=/api/pending_closures5999worker2 # [ 106.451398] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500516000worker2 # [ 106.453918] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6001lb # [ 106.458900] niks3-server[1046]: 2026/09/23 04:43:18 INFO Received uploads request method=POST path=/api/pending_closures6002worker2 # [ 106.468353] nix-grpc-daemon[1400]: time=2026-09-23T04:43:18.552Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"6003worker2 # [ 106.471059] nix-grpc-daemon[1400]: time=2026-09-23T04:43:18.554Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"6004worker2 # [ 106.472660] nix-grpc-daemon[1400]: time=2026-09-23T04:43:18.554Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"6005lb # [ 106.516707] niks3-server[1046]: 2026/09/23 04:43:18 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst6006lb # [ 106.519542] niks3-server[1046]: 2026/09/23 04:43:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign6007lb # [ 106.521877] niks3-server[1046]: 2026/09/23 04:43:18 INFO Signed narinfos id=35 count=16008lb # [ 106.523973] niks3-server[1046]: 2026/09/23 04:43:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6009lb # [ 106.527542] niks3-server[1046]: 2026/09/23 04:43:18 INFO Signed narinfos id=36 count=26010worker2 # [ 106.529075] nix-grpc-daemon[1400]: time=2026-09-23T04:43:18.612Z level=INFO msg="Uploading 3 narinfos"6011lb # [ 106.528582] niks3-server[1046]: 2026/09/23 04:43:18 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls6012lb # [ 106.543094] niks3-server[1046]: 2026/09/23 04:43:18 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst6013lb # [ 106.547911] niks3-server[1046]: 2026/09/23 04:43:18 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls6014lb # [ 106.556059] niks3-server[1046]: 2026/09/23 04:43:18 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete6015lb # [ 106.560418] niks3-server[1046]: 2026/09/23 04:43:18 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo6016lb # [ 106.562975] niks3-server[1046]: 2026/09/23 04:43:18 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo6017lb # [ 106.567412] niks3-server[1046]: 2026/09/23 04:43:18 INFO Completed upload id=356018lb # [ 106.570334] niks3-server[1046]: 2026/09/23 04:43:18 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6019lb # [ 106.572697] niks3-server[1046]: 2026/09/23 04:43:18 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo6020lb # [ 106.575557] niks3-server[1046]: 2026/09/23 04:43:18 INFO Completed upload id=366021worker2 # [ 106.577583] nix-grpc-daemon[1400]: time=2026-09-23T04:43:18.661Z level=INFO msg="Upload complete. (237ms)"6022worker2 # [ 106.582787] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55110 duration_s=0 paths=2 nar_bytes_in=3126023worker2 # [ 106.596711] nix-daemon[874]: reaped child process 1398, status = succeeded6024client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.84 seconds)6025worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache6026worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)6027worker2 # [ 106.656756] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500516028worker2 # [ 106.660671] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6029worker1 # [ 106.642832] systemd[1]: Listening on Nix worker-protocol over gRPC.6030worker1 # [ 106.652689] systemd[1]: Starting Nix worker-protocol over gRPC...6031worker1 # [ 106.737400] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:17Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6032worker1 # [ 106.741742] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6033worker1 # [ 106.745318] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16034worker1 # [ 106.747316] nix-grpc-daemon[1168]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6035worker1 # [ 106.748642] nix-grpc-daemon[1168]: E0000 00:00:1790138597.996127 1168 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6036worker1 # [ 106.757369] nix-grpc-daemon[1168]: E0000 00:00:1790138598.003488 1168 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6037worker1 # [ 106.760127] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6038lb # [ 106.797300] niks3-server[1046]: 2026/09/23 04:43:18 INFO lead: acquired remote=192.168.1.3:406826039worker1 # [ 106.764147] systemd[1]: Started Nix worker-protocol over gRPC.6040worker1 # [ 106.764912] nix-grpc-daemon[1168]: E0000 00:00:1790138598.013992 1210 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6041worker1 # [ 106.771101] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_take_over6042worker1 # [ 106.779187] nix-grpc-daemon[1168]: E0000 00:00:1790138598.029508 1210 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6043worker1 # [ 106.783128] nix-grpc-daemon[1168]: E0000 00:00:1790138598.029755 1210 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6044lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6045lb: must succeed: curl -sf localhost:9901/clusters6046worker1 # [ 106.792926] nix-grpc-daemon[1168]: E0000 00:00:1790138598.044494 1210 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6047worker1 # [ 106.805403] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500516048worker1 # [ 106.807076] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6049worker2 # [ 106.861870] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500516050worker2 # [ 106.865795] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6051lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6052worker1 # [ 107.008206] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500516053worker1 # [ 107.012177] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6054worker2 # [ 107.067802] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:19Z level=info event=scheduler_connected addr=lb:500516055worker2 # [ 107.071695] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6056worker1 # [ 107.212920] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500516057worker1 # [ 107.216975] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6058worker2 # [ 107.272878] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:19Z level=info event=scheduler_connected addr=lb:500516059worker2 # [ 107.276892] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6060lb # [ 107.409429] envoy[926]: conn peer=[2001:db8:1::4]:42470 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=84117 tx=4489 ms=810836061lb # [ 107.413983] envoy[926]: conn peer=[2001:db8:1::1]:42212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=166062lb # [ 107.417533] envoy[926]: conn peer=[2001:db8:1::3]:39768 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=22239 tx=4335 ms=861096063lb # [ 107.421500] envoy[926]: conn peer=[2001:db8:1::1]:35180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1008 tx=305 ms=16716064worker1 # [ 107.417859] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500516065worker1 # [ 107.421943] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6066worker2 # [ 107.478092] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:19Z level=info event=scheduler_connected addr=lb:500516067worker2 # [ 107.482126] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6068worker1 # [ 107.622856] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_connected addr=lb:500516069worker1 # [ 107.626944] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6070worker2 # [ 107.683141] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:19Z level=info event=scheduler_connected addr=lb:500516071worker2 # [ 107.687228] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6072worker1 # [ 107.828636] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=info event=scheduler_connected addr=lb:500516073worker1 # [ 107.835466] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:454146074lb: must succeed: curl -sf localhost:9901/clusters6075worker1 # [ 107.840074] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06076worker2 # [ 107.888205] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:19Z level=info event=scheduler_connected addr=lb:500516077worker1 # [ 107.857154] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=info event=oidc_jwks provider=mock keys=16078worker1 # [ 107.861387] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:454146079worker1 # [ 107.865377] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06080lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6081lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)6082(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.82 seconds)6083subtest: a niks3 restart does not cost the leader its role6084lb: must succeed: curl -sf localhost:9901/clusters6085lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6086lb: must succeed: curl -sf localhost:9901/clusters6087lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6088lb # [ 108.010588] niks3-server[1046]: 2026/09/23 04:43:20 INFO Shutdown signal received, draining in-flight requests timeout=10s6089worker1 # [ 107.973307] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=info event=scheduler_lock_lost status=2006090worker2 # [ 108.012539] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:20Z level=info event=scheduler_lock_lost status=2006091lb # [ 108.013254] niks3-server[1046]: 2026/09/23 04:43:20 INFO lead: released remote=192.168.1.3:406826092lb # [ 108.014680] systemd[1]: Stopping niks3 server...6093lb # [ 108.027215] systemd[1]: niks3.service: Deactivated successfully.6094lb # [ 108.028814] systemd[1]: Stopped niks3 server.6095lb # [ 108.029620] systemd[1]: niks3.service: Consumed 771ms CPU time over 1min 20.713s wall clock time, 28.7M memory peak, 98.5K incoming IP traffic, 141K outgoing IP traffic.6096lb # [ 108.037351] systemd[1]: Starting niks3 server...6097lb # [ 108.087698] niks3-server[1185]: 2026/09/23 04:43:20 goose: no migrations to run. current version: 202609200000006098lb # [ 108.093914] niks3-server[1185]: 2026/09/23 04:43:20 OK 1_commit_pending_closure.sql (4.36ms)6099lb # [ 108.097766] niks3-server[1185]: 2026/09/23 04:43:20 OK 2_object_stats_trigger.sql (2.19ms)6100lb # [ 108.098864] niks3-server[1185]: 2026/09/23 04:43:20 goose: up to current file version: 26101lb # [ 108.100530] niks3-server[1185]: 2026/09/23 04:43:20 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key6102lb # [ 108.111493] niks3-server[1185]: 2026/09/23 04:43:20 INFO Read proxy enabled — serving cache objects from S36103lb # [ 108.112861] niks3-server[1185]: 2026/09/23 04:43:20 INFO Using socket-activated listener address=0.0.0.0:57516104lb # [ 108.115217] systemd[1]: Started niks3 server.6105lb # [ 108.116096] niks3-server[1185]: 2026/09/23 04:43:20 INFO Starting HTTP server address=0.0.0.0:57516106lb # [ 108.119076] niks3-server[1185]: 2026/09/23 04:43:20 INFO systemd watchdog enabled interval=15s6107lb: waiting for unit niks3.service6108lb: (finished: waiting for unit niks3.service, in 0.02 seconds)6109lb: must succeed: curl -sf localhost:9901/clusters6110lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6111worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6112worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6113lb: must succeed: curl -sf localhost:9901/clusters6114lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6115lb: must succeed: curl -sf localhost:9901/clusters6116lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6117worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield6118worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)6119client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&26120worker1 # [ 108.409172] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:454166121worker1 # [ 108.420897] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6122client # don't know how to build these paths:6123client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6124worker2 # [ 108.476831] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551106125worker1 # [ 108.464899] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=06126worker1 # [ 108.469725] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.3:50051 clients=16127worker1 # [ 108.513925] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454166128worker1 # [ 108.526089] nix-grpc-daemon[1168]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5)6129worker1 # [ 108.615993] nix-grpc-daemon[1168]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 2/5)6130worker1 # [ 108.705268] nix-grpc-daemon[1168]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 369 ms (attempt 3/5)6131lb # [ 109.018408] niks3-server[1185]: 2026/09/23 04:43:21 INFO lead: acquired remote=192.168.1.3:406906132worker1 # [ 109.082978] nix-grpc-daemon[1168]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 389 ms (attempt 4/5)6133worker1 # [ 109.473716] nix-grpc-daemon[1168]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6134worker1 # [ 109.477537] nix-grpc-daemon[1168]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6135worker1 # [ 109.494777] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454166136client # copying 1 paths to 'lb:50051'...6137client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...6138worker1 # [ 109.508342] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454166139worker1 # [ 109.519364] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6140worker1 # [ 109.539813] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:20Z level=info event=niks3_push_started pid=12416141worker1 # [ 109.643646] nix-grpc-daemon[1241]: time=2026-09-23T04:43:20.893Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6142worker1 # [ 109.740662] nix-daemon[847]: accepted connection from pid 1254, user nix-grpc-daemon (trusted)6143worker1 # [ 109.753421] nix-daemon[847]: reaped child process 1257, status = succeeded6144lb # [ 109.798863] niks3-server[1185]: 2026/09/23 04:43:21 INFO Received uploads request method=POST path=/api/pending_closures6145worker1 # [ 109.774040] nix-grpc-daemon[1241]: time=2026-09-23T04:43:21.025Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6146worker1 # [ 109.775627] nix-grpc-daemon[1241]: time=2026-09-23T04:43:21.027Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"6147lb # [ 109.840096] niks3-server[1185]: 2026/09/23 04:43:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6148lb # [ 109.869835] niks3-server[1185]: 2026/09/23 04:43:21 INFO Signed narinfos id=37 count=16149lb # [ 109.871595] niks3-server[1185]: 2026/09/23 04:43:21 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst6150worker1 # [ 109.835095] nix-grpc-daemon[1241]: time=2026-09-23T04:43:21.085Z level=INFO msg="Uploading 1 narinfos"6151lb # [ 109.879348] niks3-server[1185]: 2026/09/23 04:43:21 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls6152lb # [ 109.881641] niks3-server[1185]: 2026/09/23 04:43:21 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6153lb # [ 109.887457] niks3-server[1185]: 2026/09/23 04:43:21 INFO Completed upload id=376154worker1 # [ 109.850324] nix-grpc-daemon[1241]: time=2026-09-23T04:43:21.101Z level=INFO msg="Upload complete. (207ms)"6155lb # [ 109.889556] niks3-server[1185]: 2026/09/23 04:43:21 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo6156worker1 # [ 109.855718] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45416 duration_s=0 paths=1 nar_bytes_in=5366157worker1 # [ 109.861898] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454166158worker1 # [ 109.874113] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6159worker1 # [ 109.880437] nix-daemon[847]: reaped child process 1239, status = succeeded6160worker1 # [ 109.885825] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6161client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...6162client # farm-a> node-a: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6163client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6164client # farm-a> LOG-farm-a-after-niks3-restart6165worker1 # [ 110.728827] nix-daemon[847]: reaped child process 1261, status = succeeded6166worker1 # [ 110.803193] nix-daemon[847]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)6167worker1 # [ 110.815691] nix-daemon[847]: reaped child process 1280, status = succeeded6168lb # [ 110.862520] niks3-server[1185]: 2026/09/23 04:43:22 INFO Received uploads request method=POST path=/api/pending_closures6169worker1 # [ 110.836159] nix-grpc-daemon[1241]: time=2026-09-23T04:43:22.087Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6170worker1 # [ 110.837942] nix-grpc-daemon[1241]: time=2026-09-23T04:43:22.089Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"6171lb # [ 110.911791] niks3-server[1185]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6172lb # [ 110.919321] niks3-server[1185]: 2026/09/23 04:43:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6173lb # [ 110.926927] niks3-server[1185]: 2026/09/23 04:43:22 INFO Signed narinfos id=38 count=16174worker1 # [ 110.892542] nix-grpc-daemon[1241]: time=2026-09-23T04:43:22.143Z level=INFO msg="Uploading 1 narinfos"6175lb # [ 110.934866] niks3-server[1185]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls6176lb # [ 110.939801] niks3-server[1185]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst6177lb # [ 110.947494] niks3-server[1185]: 2026/09/23 04:43:22 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6178lb # [ 110.953582] niks3-server[1185]: 2026/09/23 04:43:22 INFO Completed upload id=386179worker1 # [ 110.916539] nix-grpc-daemon[1241]: time=2026-09-23T04:43:22.168Z level=INFO msg="Upload complete. (223ms)"6180lb # [ 110.956043] niks3-server[1185]: 2026/09/23 04:43:22 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6181worker1 # [ 110.923841] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45416 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16182worker1 # [ 110.937937] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:22Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06183worker1 # [ 110.943204] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:22Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.3:50051 clients=16184worker1 # [ 110.952454] nix-daemon[847]: reaped child process 1259, status = succeeded6185worker1 # [ 110.983168] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454166186worker1 # [ 110.996806] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454166187worker1 # [ 111.010524] nix-grpc-daemon[1241]: time=2026-09-23T04:43:22.261Z level=INFO msg="All 1 paths already cached"6188client # copying 1 paths to 'lb:50051'...6189client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6190worker1 # [ 111.019499] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454166191worker1 # [ 111.027809] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6192worker1 # [ 111.123904] nix-daemon[847]: accepted connection from pid 1286, user nix-grpc-daemon (trusted)6193worker1 # [ 111.136559] nix-daemon[847]: reaped child process 1289, status = succeeded6194lb # [ 111.181505] niks3-server[1185]: 2026/09/23 04:43:23 INFO Received uploads request method=POST path=/api/pending_closures6195worker1 # [ 111.153481] nix-grpc-daemon[1241]: time=2026-09-23T04:43:22.405Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6196worker1 # [ 111.154963] nix-grpc-daemon[1241]: time=2026-09-23T04:43:22.406Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6197lb # [ 111.214180] niks3-server[1185]: 2026/09/23 04:43:23 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6198lb # [ 111.220322] niks3-server[1185]: 2026/09/23 04:43:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6199worker1 # [ 111.187689] nix-grpc-daemon[1241]: time=2026-09-23T04:43:22.439Z level=INFO msg="Uploading 1 narinfos"6200lb # [ 111.224638] niks3-server[1185]: 2026/09/23 04:43:23 INFO Signed narinfos id=39 count=16201lb # [ 111.235195] niks3-server[1185]: 2026/09/23 04:43:23 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6202lb # [ 111.238846] niks3-server[1185]: 2026/09/23 04:43:23 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6203lb # [ 111.242926] niks3-server[1185]: 2026/09/23 04:43:23 INFO Completed upload id=396204worker1 # [ 111.205790] nix-grpc-daemon[1241]: time=2026-09-23T04:43:22.457Z level=INFO msg="Upload complete. (165ms)"6205lb # [ 111.245412] niks3-server[1185]: 2026/09/23 04:43:23 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6206worker1 # [ 111.209771] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45416 duration_s=0 paths=1 nar_bytes_in=6566207worker1 # [ 111.214700] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454166208worker1 # [ 111.223124] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6209worker1 # [ 111.229213] nix-daemon[847]: reaped child process 1284, status = succeeded6210worker1 # [ 111.232457] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6211client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6212client # farm-b> node-a: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6213client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6214client # farm-b> LOG-farm-b-after-niks3-restart6215worker1 # [ 112.247700] nix-daemon[847]: reaped child process 1293, status = succeeded6216worker1 # [ 112.322534] nix-daemon[847]: accepted connection from pid 1309, user nix-grpc-daemon (trusted)6217worker1 # [ 112.334101] nix-daemon[847]: reaped child process 1312, status = succeeded6218lb # [ 112.381691] niks3-server[1185]: 2026/09/23 04:43:24 INFO Received uploads request method=POST path=/api/pending_closures6219worker1 # [ 112.352811] nix-grpc-daemon[1241]: time=2026-09-23T04:43:23.604Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6220worker1 # [ 112.354514] nix-grpc-daemon[1241]: time=2026-09-23T04:43:23.605Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6221lb # [ 112.418081] niks3-server[1185]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6222lb # [ 112.429303] niks3-server[1185]: 2026/09/23 04:43:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6223lb # [ 112.433833] niks3-server[1185]: 2026/09/23 04:43:24 INFO Signed narinfos id=40 count=16224worker1 # [ 112.399049] nix-grpc-daemon[1241]: time=2026-09-23T04:43:23.650Z level=INFO msg="Uploading 1 narinfos"6225lb # [ 112.440585] niks3-server[1185]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6226lb # [ 112.450103] niks3-server[1185]: 2026/09/23 04:43:24 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6227lb # [ 112.453310] niks3-server[1185]: 2026/09/23 04:43:24 INFO Completed upload id=406228worker1 # [ 112.416183] nix-grpc-daemon[1241]: time=2026-09-23T04:43:23.667Z level=INFO msg="Upload complete. (176ms)"6229lb # [ 112.457073] niks3-server[1185]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6230lb # [ 112.460128] niks3-server[1185]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6231worker1 # [ 112.421061] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45416 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16232worker1 # [ 112.433373] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:23Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06233worker1 # [ 112.436945] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:23Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.3:50051 clients=16234worker1 # [ 112.443272] nix-daemon[847]: reaped child process 1291, status = succeeded6235worker1 # [ 112.477597] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454166236worker1 # [ 112.490607] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454166237worker1 # [ 112.505172] nix-grpc-daemon[1241]: time=2026-09-23T04:43:23.756Z level=INFO msg="All 2 paths already cached"6238client # copying 1 paths to 'lb:50051'...6239client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6240worker1 # [ 112.514355] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454166241worker1 # [ 112.522694] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6242worker1 # [ 112.616981] nix-daemon[847]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)6243worker1 # [ 112.630232] nix-daemon[847]: reaped child process 1321, status = succeeded6244lb # [ 112.673967] niks3-server[1185]: 2026/09/23 04:43:24 INFO Received uploads request method=POST path=/api/pending_closures6245worker1 # [ 112.646169] nix-grpc-daemon[1241]: time=2026-09-23T04:43:23.897Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6246worker1 # [ 112.647690] nix-grpc-daemon[1241]: time=2026-09-23T04:43:23.899Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6247lb # [ 112.710427] niks3-server[1185]: 2026/09/23 04:43:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6248lb # [ 112.713351] niks3-server[1185]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6249lb # [ 112.719829] niks3-server[1185]: 2026/09/23 04:43:24 INFO Signed narinfos id=41 count=16250worker1 # [ 112.684986] nix-grpc-daemon[1241]: time=2026-09-23T04:43:23.936Z level=INFO msg="Uploading 1 narinfos"6251lb # [ 112.725348] niks3-server[1185]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6252lb # [ 112.736110] niks3-server[1185]: 2026/09/23 04:43:24 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6253lb # [ 112.740146] niks3-server[1185]: 2026/09/23 04:43:24 INFO Completed upload id=416254worker1 # [ 112.702879] nix-grpc-daemon[1241]: time=2026-09-23T04:43:23.954Z level=INFO msg="Upload complete. (167ms)"6255lb # [ 112.741445] niks3-server[1185]: 2026/09/23 04:43:24 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6256worker1 # [ 112.706859] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45416 duration_s=0 paths=1 nar_bytes_in=7846257worker1 # [ 112.711619] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454166258worker1 # [ 112.723626] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6259worker1 # [ 112.726736] nix-daemon[847]: reaped child process 1316, status = succeeded6260worker1 # [ 112.730926] nix-daemon[847]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)6261client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6262client # farm-top> node-a: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6263client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6264client # farm-top> LOG-farm-top-after-niks3-restart6265worker1 # [ 113.768665] nix-daemon[847]: reaped child process 1325, status = succeeded6266worker1 # [ 113.854076] nix-daemon[847]: accepted connection from pid 1341, user nix-grpc-daemon (trusted)6267worker1 # [ 113.866147] nix-daemon[847]: reaped child process 1344, status = succeeded6268lb # [ 113.911912] niks3-server[1185]: 2026/09/23 04:43:25 INFO Received uploads request method=POST path=/api/pending_closures6269worker1 # [ 113.883940] nix-grpc-daemon[1241]: time=2026-09-23T04:43:25.135Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6270worker1 # [ 113.887510] nix-grpc-daemon[1241]: time=2026-09-23T04:43:25.137Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6271lb # [ 113.951171] niks3-server[1185]: 2026/09/23 04:43:25 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6272lb # [ 113.965161] niks3-server[1185]: 2026/09/23 04:43:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6273worker1 # [ 113.934716] nix-grpc-daemon[1241]: time=2026-09-23T04:43:25.186Z level=INFO msg="Uploading 1 narinfos"6274lb # [ 113.971912] niks3-server[1185]: 2026/09/23 04:43:25 INFO Signed narinfos id=42 count=16275lb # [ 113.975774] niks3-server[1185]: 2026/09/23 04:43:25 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6276lb # [ 113.982547] niks3-server[1185]: 2026/09/23 04:43:25 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6277lb # [ 113.990446] niks3-server[1185]: 2026/09/23 04:43:25 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6278lb # [ 113.994235] niks3-server[1185]: 2026/09/23 04:43:25 INFO Completed upload id=426279worker1 # [ 113.958224] nix-grpc-daemon[1241]: time=2026-09-23T04:43:25.209Z level=INFO msg="Upload complete. (219ms)"6280lb # [ 113.997728] niks3-server[1185]: 2026/09/23 04:43:26 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6281worker1 # [ 113.964725] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45416 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16282worker1 # [ 113.980590] nix-daemon[847]: reaped child process 1323, status = succeeded6283client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.74 seconds)6284client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6285client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)6286(finished: subtest: a niks3 restart does not cost the leader its role, in 6.23 seconds)6287subtest: scheduler down: the other node takes the lock and keeps it6288lb: must succeed: curl -sf localhost:9901/clusters6289lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6290lb: must succeed: curl -sf localhost:9901/clusters6291lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6292lb: must succeed: curl -sf localhost:9901/clusters6293lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6294lb: must succeed: curl -sf localhost:9901/clusters6295lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6296worker1 # [ 114.288302] systemd[1]: Stopping Nix worker-protocol over gRPC...6297worker1 # [ 114.756444] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:26Z level=info event=signal_exit6298lb # [ 115.021957] niks3-server[1185]: 2026/09/23 04:43:27 INFO lead: released remote=192.168.1.3:406906299worker2 # [ 116.024846] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:28Z level=info event=scheduler_restarting addr=lb:500516300worker1 # [ 115.989255] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:27Z level=info event=scheduler_restarting6301worker1 # [ 115.992736] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:27Z level=info event=scheduler_restarting addr=lb:500516302worker1 # [ 115.996234] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:454146303worker1 # [ 115.999917] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:27Z level=info event=worker_gone addr=192.168.1.3:500516304worker1 # [ 116.003308] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:454146305worker1 # [ 116.006915] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:27Z level=info event=worker_gone addr=192.168.1.4:500516306worker1 # [ 116.024283] nix-daemon[847]: reaped child process 1229, status = succeeded6307worker2 # [ 116.225200] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:28Z level=info event=scheduler_connected addr=lb:500516308worker2 # [ 116.229464] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6309worker1 # [ 116.197474] nix-grpc-daemon[1168]: ts=2026-09-23T04:43:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6310worker2 # [ 116.430706] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:28Z level=info event=scheduler_connected addr=lb:500516311worker2 # [ 116.434647] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6312worker2 # [ 116.635771] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:28Z level=info event=scheduler_connected addr=lb:500516313worker2 # [ 116.639653] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6314worker1 # [ 116.676420] nix-grpc-daemon[1168]: E0000 00:00:1790138607.927730 1214 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true6315worker1 # [ 116.681366] nix-grpc-daemon[1168]: E0000 00:00:1790138607.927894 1214 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6316worker2 # [ 116.840960] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:28Z level=info event=scheduler_connected addr=lb:500516317worker2 # [ 116.845094] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6318worker2 # [ 117.046802] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_connected addr=lb:500516319worker2 # [ 117.050650] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6320worker2 # [ 117.251425] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_connected addr=lb:500516321worker2 # [ 117.255302] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6322lb # [ 117.409877] envoy[926]: conn peer=[2001:db8:1::1]:35182 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9709 tx=4439 ms=55796323lb # [ 117.413726] envoy[926]: conn peer=[2001:db8:1::3]:40958 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2124 tx=648 ms=93906324worker2 # [ 117.456286] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_connected addr=lb:500516325worker2 # [ 117.460223] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6326worker2 # [ 117.661070] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_connected addr=lb:500516327worker2 # [ 117.664881] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6328worker2 # [ 117.865913] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_connected addr=lb:500516329worker2 # [ 117.869910] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6330worker2 # [ 118.071796] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_connected addr=lb:500516331worker2 # [ 118.075628] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6332worker1 # [ 118.213312] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6333worker1 # [ 118.217240] systemd[1]: Stopped Nix worker-protocol over gRPC.6334worker1 # [ 118.221331] systemd[1]: nix-grpc-daemon.service: Consumed 797ms CPU time over 11.566s wall clock time, 37.1M memory peak, 36.3K incoming IP traffic, 61.2K outgoing IP traffic.6335worker1 # [ 118.232741] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6336worker1 # [ 118.237168] systemd[1]: Closed Nix worker-protocol over gRPC.6337worker2 # [ 118.276731] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_connected addr=lb:500516338worker1 # [ 118.240650] systemd[1]: nix-grpc-daemon.socket: Consumed 557us CPU time over 11.600s wall clock time, 256K memory peak, 32.8K incoming IP traffic, 23.3K outgoing IP traffic.6339worker2 # [ 118.280770] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6340worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6341worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)6342worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6343worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6344worker2 # [ 118.481909] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_connected addr=lb:500516345worker2 # [ 118.485902] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6346worker2 # [ 118.686906] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_connected addr=lb:500516347worker2 # [ 118.690859] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6348worker2 # [ 118.891890] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_connected addr=lb:500516349worker2 # [ 118.896242] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6350worker2 # [ 119.022104] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:31Z level=info event=scheduler_take_over6351lb # [ 119.020835] niks3-server[1185]: 2026/09/23 04:43:31 INFO lead: acquired remote=192.168.1.4:517546352worker2 # [ 119.098295] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:31Z level=info event=scheduler_connected addr=lb:500516353worker2 # [ 119.102439] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6354worker2 # [ 119.303956] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:31Z level=info event=scheduler_connected addr=lb:500516355worker2 # [ 119.308129] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6356worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6357worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6358worker2 # [ 119.509428] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:31Z level=info event=scheduler_connected addr=lb:500516359worker2 # [ 119.513593] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6360worker2 # [ 119.714814] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:31Z level=info event=scheduler_connected addr=lb:500516361worker2 # [ 119.718798] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6362worker2 # [ 119.919729] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=info event=scheduler_connected addr=lb:500516363worker2 # [ 119.923905] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6364worker2 # [ 120.125886] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=info event=scheduler_connected addr=lb:500516365worker2 # [ 120.143151] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=info event=oidc_jwks provider=mock keys=16366worker2 # [ 120.147261] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:376746367worker2 # [ 120.151231] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06368worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6369worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6370client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26371worker2 # [ 120.599231] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:551106372client # don't know how to build these paths:6373client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6374worker2 # [ 120.608114] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551106375worker2 # [ 120.631472] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06376worker2 # [ 120.636678] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=16377worker2 # [ 120.675818] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551106378worker2 # [ 120.688788] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551106379client # copying 1 paths to 'lb:50051'...6380client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6381worker2 # [ 120.701468] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551106382worker2 # [ 120.712309] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6383worker2 # [ 120.814673] nix-daemon[874]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)6384worker2 # [ 120.827718] nix-daemon[874]: reaped child process 1444, status = succeeded6385lb # [ 120.833396] niks3-server[1185]: 2026/09/23 04:43:32 INFO Received uploads request method=POST path=/api/pending_closures6386worker2 # [ 120.843952] nix-grpc-daemon[1400]: time=2026-09-23T04:43:32.928Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6387worker2 # [ 120.846170] nix-grpc-daemon[1400]: time=2026-09-23T04:43:32.930Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6388lb # [ 120.864625] niks3-server[1185]: 2026/09/23 04:43:32 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6389lb # [ 120.871436] niks3-server[1185]: 2026/09/23 04:43:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6390worker2 # [ 120.880793] nix-grpc-daemon[1400]: time=2026-09-23T04:43:32.964Z level=INFO msg="Uploading 1 narinfos"6391lb # [ 120.878644] niks3-server[1185]: 2026/09/23 04:43:32 INFO Signed narinfos id=43 count=16392lb # [ 120.899350] niks3-server[1185]: 2026/09/23 04:43:32 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6393lb # [ 120.901040] niks3-server[1185]: 2026/09/23 04:43:32 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6394lb # [ 120.904886] niks3-server[1185]: 2026/09/23 04:43:32 INFO Completed upload id=436395worker2 # [ 120.906656] nix-grpc-daemon[1400]: time=2026-09-23T04:43:32.991Z level=INFO msg="Upload complete. (182ms)"6396lb # [ 120.907770] niks3-server[1185]: 2026/09/23 04:43:32 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6397worker2 # [ 120.910319] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55110 duration_s=0 paths=1 nar_bytes_in=5206398worker2 # [ 120.915474] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551106399worker2 # [ 120.923590] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6400worker2 # [ 120.929176] nix-daemon[874]: reaped child process 1439, status = succeeded6401client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6402worker2 # [ 120.934163] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6403client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6404client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6405client # farm-a> LOG-farm-a-on-standby6406worker2 # [ 121.367915] nix-daemon[874]: reaped child process 1448, status = succeeded6407worker2 # [ 121.442863] nix-daemon[874]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)6408worker2 # [ 121.456453] nix-daemon[874]: reaped child process 1467, status = succeeded6409lb # [ 121.461323] niks3-server[1185]: 2026/09/23 04:43:33 INFO Received uploads request method=POST path=/api/pending_closures6410worker2 # [ 121.471843] nix-grpc-daemon[1400]: time=2026-09-23T04:43:33.556Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6411worker2 # [ 121.473380] nix-grpc-daemon[1400]: time=2026-09-23T04:43:33.557Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6412lb # [ 121.502775] niks3-server[1185]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6413lb # [ 121.509343] niks3-server[1185]: 2026/09/23 04:43:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6414worker2 # [ 121.520400] nix-grpc-daemon[1400]: time=2026-09-23T04:43:33.604Z level=INFO msg="Uploading 1 narinfos"6415lb # [ 121.518699] niks3-server[1185]: 2026/09/23 04:43:33 INFO Signed narinfos id=44 count=16416lb # [ 121.523354] niks3-server[1185]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6417lb # [ 121.527954] niks3-server[1185]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6418lb # [ 121.537986] niks3-server[1185]: 2026/09/23 04:43:33 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6419lb # [ 121.541303] niks3-server[1185]: 2026/09/23 04:43:33 INFO Completed upload id=446420worker2 # [ 121.544408] nix-grpc-daemon[1400]: time=2026-09-23T04:43:33.628Z level=INFO msg="Upload complete. (204ms)"6421lb # [ 121.544582] niks3-server[1185]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6422worker2 # [ 121.552222] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55110 duration_s=0 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16423worker2 # [ 121.564104] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:33Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06424worker2 # [ 121.568217] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:33Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16425worker2 # [ 121.577704] nix-daemon[874]: reaped child process 1446, status = succeeded6426worker2 # [ 121.608387] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551106427worker2 # [ 121.621665] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551106428worker2 # [ 121.634485] nix-grpc-daemon[1400]: time=2026-09-23T04:43:33.718Z level=INFO msg="All 1 paths already cached"6429client # copying 1 paths to 'lb:50051'...6430client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6431worker2 # [ 121.643123] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551106432worker2 # [ 121.651398] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6433worker2 # [ 121.745747] nix-daemon[874]: accepted connection from pid 1474, user nix-grpc-daemon (trusted)6434worker2 # [ 121.758642] nix-daemon[874]: reaped child process 1477, status = succeeded6435lb # [ 121.763805] niks3-server[1185]: 2026/09/23 04:43:33 INFO Received uploads request method=POST path=/api/pending_closures6436worker2 # [ 121.773228] nix-grpc-daemon[1400]: time=2026-09-23T04:43:33.857Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6437worker2 # [ 121.774682] nix-grpc-daemon[1400]: time=2026-09-23T04:43:33.859Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6438lb # [ 121.793792] niks3-server[1185]: 2026/09/23 04:43:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6439lb # [ 121.799660] niks3-server[1185]: 2026/09/23 04:43:33 INFO Signed narinfos id=45 count=16440worker2 # [ 121.804280] nix-grpc-daemon[1400]: time=2026-09-23T04:43:33.888Z level=INFO msg="Uploading 1 narinfos"6441lb # [ 121.810892] niks3-server[1185]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6442lb # [ 121.820201] niks3-server[1185]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6443lb # [ 121.828149] niks3-server[1185]: 2026/09/23 04:43:33 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6444lb # [ 121.840190] niks3-server[1185]: 2026/09/23 04:43:33 INFO Completed upload id=456445worker2 # [ 121.841926] nix-grpc-daemon[1400]: time=2026-09-23T04:43:33.926Z level=INFO msg="Upload complete. (178ms)"6446lb # [ 121.842096] niks3-server[1185]: 2026/09/23 04:43:33 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6447worker2 # [ 121.846880] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55110 duration_s=0 paths=1 nar_bytes_in=6406448worker2 # [ 121.852784] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551106449worker2 # [ 121.862667] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6450worker2 # [ 121.872778] nix-daemon[874]: reaped child process 1471, status = succeeded6451worker2 # [ 121.877498] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6452client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6453client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6454client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6455client # farm-b> LOG-farm-b-on-standby6456worker2 # [ 123.120842] nix-daemon[874]: reaped child process 1481, status = succeeded6457worker2 # [ 123.199470] nix-daemon[874]: accepted connection from pid 1497, user nix-grpc-daemon (trusted)6458worker2 # [ 123.212190] nix-daemon[874]: reaped child process 1500, status = succeeded6459lb # [ 123.218539] niks3-server[1185]: 2026/09/23 04:43:35 INFO Received uploads request method=POST path=/api/pending_closures6460worker2 # [ 123.230968] nix-grpc-daemon[1400]: time=2026-09-23T04:43:35.315Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6461worker2 # [ 123.232439] nix-grpc-daemon[1400]: time=2026-09-23T04:43:35.316Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6462lb # [ 123.262136] niks3-server[1185]: 2026/09/23 04:43:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6463lb # [ 123.271778] niks3-server[1185]: 2026/09/23 04:43:35 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6464worker2 # [ 123.280770] nix-grpc-daemon[1400]: time=2026-09-23T04:43:35.365Z level=INFO msg="Uploading 1 narinfos"6465lb # [ 123.280667] niks3-server[1185]: 2026/09/23 04:43:35 INFO Signed narinfos id=46 count=16466lb # [ 123.282650] niks3-server[1185]: 2026/09/23 04:43:35 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6467lb # [ 123.293749] niks3-server[1185]: 2026/09/23 04:43:35 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6468lb # [ 123.301226] niks3-server[1185]: 2026/09/23 04:43:35 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6469lb # [ 123.304078] niks3-server[1185]: 2026/09/23 04:43:35 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6470lb # [ 123.308110] niks3-server[1185]: 2026/09/23 04:43:35 INFO Completed upload id=466471worker2 # [ 123.311433] nix-grpc-daemon[1400]: time=2026-09-23T04:43:35.395Z level=INFO msg="Upload complete. (226ms)"6472worker2 # [ 123.318205] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55110 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16473worker2 # [ 123.330865] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:35Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06474worker2 # [ 123.334353] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:35Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16475worker2 # [ 123.343643] nix-daemon[874]: reaped child process 1479, status = succeeded6476worker2 # [ 123.374678] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551106477worker2 # [ 123.387471] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551106478worker2 # [ 123.402232] nix-grpc-daemon[1400]: time=2026-09-23T04:43:35.486Z level=INFO msg="All 2 paths already cached"6479client # copying 1 paths to 'lb:50051'...6480client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6481worker2 # [ 123.411490] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551106482worker2 # [ 123.419874] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6483worker2 # [ 123.516298] nix-daemon[874]: accepted connection from pid 1506, user nix-grpc-daemon (trusted)6484worker2 # [ 123.529675] nix-daemon[874]: reaped child process 1509, status = succeeded6485lb # [ 123.534706] niks3-server[1185]: 2026/09/23 04:43:35 INFO Received uploads request method=POST path=/api/pending_closures6486worker2 # [ 123.548698] nix-grpc-daemon[1400]: time=2026-09-23T04:43:35.633Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6487worker2 # [ 123.550231] nix-grpc-daemon[1400]: time=2026-09-23T04:43:35.634Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6488lb # [ 123.571206] niks3-server[1185]: 2026/09/23 04:43:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6489lb # [ 123.575215] niks3-server[1185]: 2026/09/23 04:43:35 INFO Signed narinfos id=47 count=16490worker2 # [ 123.579761] nix-grpc-daemon[1400]: time=2026-09-23T04:43:35.664Z level=INFO msg="Uploading 1 narinfos"6491lb # [ 123.585087] niks3-server[1185]: 2026/09/23 04:43:35 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6492lb # [ 123.589708] niks3-server[1185]: 2026/09/23 04:43:35 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6493lb # [ 123.598389] niks3-server[1185]: 2026/09/23 04:43:35 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6494lb # [ 123.602959] niks3-server[1185]: 2026/09/23 04:43:35 INFO Completed upload id=476495worker2 # [ 123.605000] nix-grpc-daemon[1400]: time=2026-09-23T04:43:35.689Z level=INFO msg="Upload complete. (171ms)"6496lb # [ 123.605403] niks3-server[1185]: 2026/09/23 04:43:35 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6497worker2 # [ 123.609648] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55110 duration_s=0 paths=1 nar_bytes_in=7686498worker2 # [ 123.614380] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551106499worker2 # [ 123.625933] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6500worker2 # [ 123.629819] nix-daemon[874]: reaped child process 1504, status = succeeded6501worker2 # [ 123.634685] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6502client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6503client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6504client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6505client # farm-top> LOG-farm-top-on-standby6506worker2 # [ 124.248890] nix-daemon[874]: reaped child process 1513, status = succeeded6507worker2 # [ 124.323024] nix-daemon[874]: accepted connection from pid 1529, user nix-grpc-daemon (trusted)6508worker2 # [ 124.338176] nix-daemon[874]: reaped child process 1532, status = succeeded6509lb # [ 124.340977] niks3-server[1185]: 2026/09/23 04:43:36 INFO Received uploads request method=POST path=/api/pending_closures6510worker2 # [ 124.351886] nix-grpc-daemon[1400]: time=2026-09-23T04:43:36.435Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6511worker2 # [ 124.353295] nix-grpc-daemon[1400]: time=2026-09-23T04:43:36.435Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6512lb # [ 124.381948] niks3-server[1185]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6513lb # [ 124.387930] niks3-server[1185]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6514lb # [ 124.391247] niks3-server[1185]: 2026/09/23 04:43:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6515worker2 # [ 124.397644] nix-grpc-daemon[1400]: time=2026-09-23T04:43:36.481Z level=INFO msg="Uploading 1 narinfos"6516lb # [ 124.395734] niks3-server[1185]: 2026/09/23 04:43:36 INFO Signed narinfos id=48 count=16517lb # [ 124.404855] niks3-server[1185]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6518lb # [ 124.410063] niks3-server[1185]: 2026/09/23 04:43:36 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6519lb # [ 124.413913] niks3-server[1185]: 2026/09/23 04:43:36 INFO Completed upload id=486520lb # [ 124.416289] niks3-server[1185]: 2026/09/23 04:43:36 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6521worker2 # [ 124.417341] nix-grpc-daemon[1400]: time=2026-09-23T04:43:36.501Z level=INFO msg="Upload complete. (176ms)"6522worker2 # [ 124.425560] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55110 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16523worker2 # [ 124.437843] nix-daemon[874]: reaped child process 1511, status = succeeded6524client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 4.01 seconds)6525client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6526client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)6527worker1 # [ 124.595896] systemd[1]: Listening on Nix worker-protocol over gRPC.6528worker1 # [ 124.604165] systemd[1]: Starting Nix worker-protocol over gRPC...6529worker1 # [ 124.694124] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6530worker1 # [ 124.696123] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6531worker1 # [ 124.699858] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16532worker1 # [ 124.704058] nix-grpc-daemon[1362]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6533worker1 # [ 124.705423] nix-grpc-daemon[1362]: E0000 00:00:1790138615.955115 1362 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6534worker1 # [ 124.711201] nix-grpc-daemon[1362]: E0000 00:00:1790138615.960312 1362 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6535worker1 # [ 124.715106] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6536worker1 # [ 124.716673] systemd[1]: Started Nix worker-protocol over gRPC.6537worker1 # [ 124.722109] nix-grpc-daemon[1362]: E0000 00:00:1790138615.972760 1404 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6538lb: must succeed: curl -sf localhost:9901/clusters6539worker1 # [ 124.734171] nix-grpc-daemon[1362]: E0000 00:00:1790138615.985728 1404 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6540worker1 # [ 124.738150] nix-grpc-daemon[1362]: E0000 00:00:1790138615.987981 1404 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6541worker1 # [ 124.748335] nix-grpc-daemon[1362]: E0000 00:00:1790138615.999912 1404 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6542worker1 # [ 124.758524] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:36Z level=info event=scheduler_connected addr=lb:500516543worker2 # [ 124.799751] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:376746544worker2 # [ 124.804077] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06545lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6546worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6547worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6548lb: must succeed: curl -sf localhost:9901/clusters6549lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6550lb: must succeed: curl -sf localhost:9901/clusters6551lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6552worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6553worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6554(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.81 seconds)6555subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6556client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6557client # [ 125.016597] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6558client # Running as unit: rs.service; invocation ID: 964fc14fe0f04a548e5b139789d0a1c36559client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6560client # [ 125.162455] nix[1363]: don't know how to build these paths:6561client # [ 125.163337] nix[1363]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6562worker2 # [ 125.171556] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:551106563worker2 # [ 125.181884] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:37Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06564worker2 # [ 125.186695] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:37Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16565worker2 # [ 125.226285] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551106566worker2 # [ 125.238574] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551106567client # [ 125.238189] nix[1363]: copying 1 paths to 'lb:50051'...6568worker2 # [ 125.250272] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551106569client # [ 125.241147] nix[1363]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6570worker2 # [ 125.262959] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6571worker2 # [ 125.363383] nix-daemon[874]: accepted connection from pid 1545, user nix-grpc-daemon (trusted)6572worker2 # [ 125.376272] nix-daemon[874]: reaped child process 1548, status = succeeded6573lb # [ 125.382480] niks3-server[1185]: 2026/09/23 04:43:37 INFO Received uploads request method=POST path=/api/pending_closures6574worker2 # [ 125.393375] nix-grpc-daemon[1400]: time=2026-09-23T04:43:37.477Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6575worker2 # [ 125.395336] nix-grpc-daemon[1400]: time=2026-09-23T04:43:37.479Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6576lb # [ 125.411819] niks3-server[1185]: 2026/09/23 04:43:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6577lb # [ 125.422307] niks3-server[1185]: 2026/09/23 04:43:37 INFO Signed narinfos id=49 count=16578worker2 # [ 125.427187] nix-grpc-daemon[1400]: time=2026-09-23T04:43:37.511Z level=INFO msg="Uploading 1 narinfos"6579lb # [ 125.427389] niks3-server[1185]: 2026/09/23 04:43:37 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6580lb # [ 125.435819] niks3-server[1185]: 2026/09/23 04:43:37 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6581lb # [ 125.445110] niks3-server[1185]: 2026/09/23 04:43:37 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6582lb # [ 125.452506] niks3-server[1185]: 2026/09/23 04:43:37 INFO Completed upload id=496583worker2 # [ 125.454447] nix-grpc-daemon[1400]: time=2026-09-23T04:43:37.538Z level=INFO msg="Upload complete. (179ms)"6584lb # [ 125.454821] niks3-server[1185]: 2026/09/23 04:43:37 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6585worker2 # [ 125.458845] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55110 duration_s=0 paths=1 nar_bytes_in=4166586worker2 # [ 125.464280] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551106587worker2 # [ 125.475107] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6588worker2 # [ 125.481075] nix-daemon[874]: reaped child process 1543, status = succeeded6589worker2 # [ 125.484052] nix-daemon[874]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6590client # [ 125.473397] nix[1363]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6591lb: must succeed: curl -sf localhost:9901/clusters6592lb # [ 127.411197] envoy[926]: conn peer=[2001:db8:1::1]:40056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9543 tx=4359 ms=38366593lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6594lb: must succeed: curl -sf localhost:9901/clusters6595lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6596lb: must succeed: curl -sf localhost:9901/clusters6597lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6598lb: must succeed: curl -sf localhost:9901/clusters6599lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6600worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6601worker1 # [ 127.655625] systemd-journald[282]: Received client request to rotate journal, rotating.6602worker1 # [ 127.675311] systemd-journald[282]: Vacuuming done, freed 0B of archived journals from /var/log/journal/d01b1d4f78524b4fa22d224a2f3d8216.6603worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6604worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6605worker2 # [ 127.616105] systemd[1]: Stopping Nix worker-protocol over gRPC...6606worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)6607client: must succeed: sleep 5.06608worker2 # [ 127.906405] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:39Z level=info event=draining6609client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6610worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6611worker2 # [ 132.907994] nix-daemon[874]: reaped child process 1552, status = succeeded6612worker2 # [ 132.986090] nix-daemon[874]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6613worker2 # [ 132.999953] nix-daemon[874]: reaped child process 1596, status = succeeded6614lb # [ 133.003363] niks3-server[1185]: 2026/09/23 04:43:45 INFO Received uploads request method=POST path=/api/pending_closures6615worker2 # [ 133.013931] nix-grpc-daemon[1400]: time=2026-09-23T04:43:45.098Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6616worker2 # [ 133.017061] nix-grpc-daemon[1400]: time=2026-09-23T04:43:45.100Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6617lb # [ 133.045138] niks3-server[1185]: 2026/09/23 04:43:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6618lb # [ 133.056170] niks3-server[1185]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6619worker2 # [ 133.062608] nix-grpc-daemon[1400]: time=2026-09-23T04:43:45.146Z level=INFO msg="Uploading 1 narinfos"6620lb # [ 133.060712] niks3-server[1185]: 2026/09/23 04:43:45 INFO Signed narinfos id=50 count=16621lb # [ 133.064518] niks3-server[1185]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6622lb # [ 133.068906] niks3-server[1185]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6623lb # [ 133.079706] niks3-server[1185]: 2026/09/23 04:43:45 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6624lb # [ 133.084373] niks3-server[1185]: 2026/09/23 04:43:45 INFO Completed upload id=506625worker2 # [ 133.086201] nix-grpc-daemon[1400]: time=2026-09-23T04:43:45.170Z level=INFO msg="Upload complete. (189ms)"6626lb # [ 133.086444] niks3-server[1185]: 2026/09/23 04:43:45 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6627worker2 # [ 133.093917] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55110 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16628worker2 # [ 133.109383] nix-daemon[874]: reaped child process 1550, status = succeeded6629client # [ 133.122630] systemd[1]: rs.service: Deactivated successfully.6630client # [ 133.126849] systemd[1]: rs.service: Consumed 133ms CPU time over 8.109s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6631worker2 # [ 133.908307] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:45Z level=info event=signal_exit6632lb # [ 134.021873] niks3-server[1185]: 2026/09/23 04:43:46 INFO lead: released remote=192.168.1.4:517546633lb # [ 134.760765] niks3-server[1185]: 2026/09/23 04:43:46 INFO lead: acquired remote=192.168.1.3:550006634worker1 # [ 134.724572] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:45Z level=info event=scheduler_take_over6635worker1 # [ 134.984495] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:46Z level=info event=scheduler_restarting addr=lb:500516636worker2 # [ 135.028313] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:47Z level=info event=scheduler_restarting6637worker2 # [ 135.032360] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:47Z level=info event=scheduler_restarting addr=lb:500516638worker2 # [ 135.036500] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:376746639worker2 # [ 135.040420] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:47Z level=info event=worker_gone addr=192.168.1.3:500516640worker2 # [ 135.043863] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:376746641worker2 # [ 135.047526] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:47Z level=info event=worker_gone addr=192.168.1.4:500516642worker2 # [ 135.066704] nix-daemon[874]: reaped child process 1392, status = succeeded6643worker1 # [ 135.185545] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:46Z level=info event=scheduler_connected addr=lb:500516644worker1 # [ 135.189949] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6645worker2 # [ 135.238805] nix-grpc-daemon[1339]: ts=2026-09-23T04:43:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6646worker1 # [ 135.390829] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:46Z level=info event=scheduler_connected addr=lb:500516647worker1 # [ 135.397685] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:427566648worker1 # [ 135.402595] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06649worker2 # [ 135.979425] nix-grpc-daemon[1339]: E0000 00:00:1790138628.063647 1385 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6650worker2 # [ 135.984749] nix-grpc-daemon[1339]: E0000 00:00:1790138628.063807 1385 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6651worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.17 seconds)6652client: waiting for success: ! systemctl is-active rs6653client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6654client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6655client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6656client: must fail: journalctl -u rs | grep -q 'reconnecting'6657client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)6658lb: must succeed: curl -sf localhost:9901/clusters6659lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6660worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6661worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6662worker2 # [ 136.908869] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6663worker2 # [ 136.912870] systemd[1]: Stopped Nix worker-protocol over gRPC.6664worker2 # [ 136.915124] systemd[1]: nix-grpc-daemon.service: Consumed 1.220s CPU time over 35.105s wall clock time, 38.4M memory peak, 64.9K incoming IP traffic, 174.6K outgoing IP traffic.6665worker2 # [ 136.930708] systemd[1]: Starting scheduler-token.service...6666worker2 # [ 136.977735] systemd[1]: scheduler-token.service: Deactivated successfully.6667worker2 # [ 136.979372] systemd[1]: Finished scheduler-token.service.6668worker2 # [ 136.980659] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 48ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6669worker2 # [ 136.985221] systemd[1]: Starting Nix worker-protocol over gRPC...6670worker2 # [ 137.065157] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6671worker2 # [ 137.070092] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6672worker2 # [ 137.073179] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16673worker2 # [ 137.075232] nix-grpc-daemon[1606]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6674worker2 # [ 137.076624] nix-grpc-daemon[1606]: E0000 00:00:1790138629.156961 1606 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6675worker2 # [ 137.084164] nix-grpc-daemon[1606]: E0000 00:00:1790138629.164502 1606 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6676worker2 # [ 137.087155] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6677worker2 # [ 137.090062] systemd[1]: Started Nix worker-protocol over gRPC.6678worker2 # [ 137.090870] nix-grpc-daemon[1606]: E0000 00:00:1790138629.173877 1648 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6679worker2 # [ 137.102854] nix-grpc-daemon[1606]: E0000 00:00:1790138629.187301 1648 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6680worker2 # [ 137.106210] nix-grpc-daemon[1606]: E0000 00:00:1790138629.189491 1648 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6681worker2 # [ 137.117513] nix-grpc-daemon[1606]: E0000 00:00:1790138629.201979 1648 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6682worker2 # [ 137.127206] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:49Z level=info event=scheduler_connected addr=lb:500516683worker1 # [ 137.096827] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:48Z level=info event=oidc_jwks provider=mock keys=16684worker1 # [ 137.101297] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:427566685worker1 # [ 137.105274] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06686lb: must succeed: curl -sf localhost:9901/clusters6687lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6688worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6689worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6690(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.31 seconds)6691subtest: scheduler restart mid-build: build finishes, no double build6692client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6693client # [ 137.319606] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6694client # Running as unit: mid.service; invocation ID: e4b49f13c90647f0a9c5033d1dd01f116695client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6696lb # [ 137.411130] envoy[926]: conn peer=[2001:db8:1::1]:54770 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1499 ms=79426697lb # [ 137.415122] envoy[926]: conn peer=[2001:db8:1::4]:42492 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=42525 tx=2265 ms=332846698worker1 # [ 137.436166] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:427586699worker1 # [ 137.447710] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6700client # [ 137.489053] nix[1399]: don't know how to build these paths:6701client # [ 137.490259] nix[1399]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6702worker1 # [ 137.467797] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:48Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06703worker1 # [ 137.472726] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:48Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16704worker1 # [ 137.518767] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427586705worker1 # [ 137.530425] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)6706worker1 # [ 137.554873] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 2/5)6707worker1 # [ 137.618953] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 200 ms (attempt 3/5)6708worker1 # [ 137.821230] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 313 ms (attempt 4/5)6709worker1 # [ 138.143877] nix-grpc-daemon[1362]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6710worker1 # [ 138.147407] nix-grpc-daemon[1362]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6711worker1 # [ 138.165817] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:427586712client # [ 138.205285] nix[1399]: copying 1 paths to 'lb:50051'...6713worker1 # [ 138.177899] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:427586714client # [ 138.207834] nix[1399]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6715worker1 # [ 138.189179] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6716worker1 # [ 138.211411] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:49Z level=info event=niks3_push_started pid=14916717worker1 # [ 138.317726] nix-grpc-daemon[1491]: time=2026-09-23T04:43:49.567Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6718worker1 # [ 138.401446] nix-daemon[847]: accepted connection from pid 1503, user nix-grpc-daemon (trusted)6719worker1 # [ 138.418748] nix-daemon[847]: reaped child process 1506, status = succeeded6720lb # [ 138.459851] niks3-server[1185]: 2026/09/23 04:43:50 INFO Received uploads request method=POST path=/api/pending_closures6721worker1 # [ 138.432828] nix-grpc-daemon[1491]: time=2026-09-23T04:43:49.684Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6722worker1 # [ 138.436594] nix-grpc-daemon[1491]: time=2026-09-23T04:43:49.686Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6723lb # [ 138.502438] niks3-server[1185]: 2026/09/23 04:43:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6724lb # [ 138.510145] niks3-server[1185]: 2026/09/23 04:43:50 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6725lb # [ 138.525178] niks3-server[1185]: 2026/09/23 04:43:50 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6726worker1 # [ 138.495782] nix-grpc-daemon[1491]: time=2026-09-23T04:43:49.747Z level=INFO msg="Uploading 1 narinfos"6727lb # [ 138.533163] niks3-server[1185]: 2026/09/23 04:43:50 INFO Signed narinfos id=51 count=16728lb # [ 138.541753] niks3-server[1185]: 2026/09/23 04:43:50 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6729lb # [ 138.546786] niks3-server[1185]: 2026/09/23 04:43:50 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6730lb # [ 138.553556] niks3-server[1185]: 2026/09/23 04:43:50 INFO Completed upload id=516731worker1 # [ 138.516451] nix-grpc-daemon[1491]: time=2026-09-23T04:43:49.767Z level=INFO msg="Upload complete. (199ms)"6732worker1 # [ 138.520676] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42758 duration_s=0 paths=1 nar_bytes_in=4246733worker1 # [ 138.526281] nix-grpc-daemon[1362]: ts=2026-09-23T04:43:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:427586734worker1 # [ 138.534543] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6735worker1 # [ 138.541773] nix-daemon[847]: reaped child process 1489, status = succeeded6736worker1 # [ 138.544705] nix-daemon[847]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6737client # [ 138.573671] nix[1399]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6738lb: must succeed: curl -sf localhost:9901/clusters6739lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6740lb: must succeed: curl -sf localhost:9901/clusters6741lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6742worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6743worker1 # [ 140.890364] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1362 (nix-grpc-daemon) on client request.6744worker1 # [ 140.894397] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6745lb # [ 140.937916] niks3-server[1185]: 2026/09/23 04:43:52 INFO lead: released remote=192.168.1.3:550006746worker1 # [ 140.911084] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6747client # [ 140.937518] nix[1399]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6748worker1 # [ 140.912534] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6749worker2 # [ 140.947595] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6750worker1 # [ 140.913766] systemd[1]: nix-grpc-daemon.service: Consumed 378ms CPU time over 16.302s wall clock time, 35.6M memory peak, 15.4K incoming IP traffic, 27.4K outgoing IP traffic.6751client # [ 140.942272] nix[1399]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6752worker1 # [ 140.919331] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6753worker1 # [ 140.922548] systemd[1]: Starting Nix worker-protocol over gRPC...6754worker1 # [ 140.957181] nix-daemon[847]: reaped child process 1512, status = succeeded6755worker1 # [ 140.958883] nix-daemon[847]: reaped child process 1481, status = succeeded6756worker1 # [ 140.965529] nix-daemon[1514]: unexpected Nix daemon error: error: interrupted by the user6757worker1 # [ 140.974198] nix-daemon[847]: reaped child process 1514, status = failed with exit code 16758worker1 # [ 141.014254] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6759worker1 # [ 141.016204] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6760worker1 # [ 141.019561] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16761worker1 # [ 141.023784] nix-grpc-daemon[1550]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6762worker1 # [ 141.025203] nix-grpc-daemon[1550]: E0000 00:00:1790138632.275109 1550 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6763worker1 # [ 141.033276] nix-grpc-daemon[1550]: E0000 00:00:1790138632.280531 1550 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6764worker1 # [ 141.037111] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6765worker1 # [ 141.038768] systemd[1]: Started Nix worker-protocol over gRPC.6766lb # [ 141.076446] niks3-server[1185]: 2026/09/23 04:43:53 INFO lead: acquired remote=192.168.1.3:437766767worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.18 seconds)6768client: must succeed: sleep 5.06769worker1 # [ 141.044248] nix-grpc-daemon[1550]: E0000 00:00:1790138632.292199 1593 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6770worker1 # [ 141.047114] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=scheduler_take_over6771worker1 # [ 141.054623] nix-grpc-daemon[1550]: E0000 00:00:1790138632.306198 1593 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6772worker1 # [ 141.059056] nix-grpc-daemon[1550]: E0000 00:00:1790138632.308561 1593 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6773worker1 # [ 141.068731] nix-grpc-daemon[1550]: E0000 00:00:1790138632.320270 1593 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6774worker1 # [ 141.078254] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=scheduler_connected addr=lb:500516775worker1 # [ 141.084649] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:427746776worker1 # [ 141.086450] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06777worker2 # [ 141.148078] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:53Z level=info event=scheduler_connected addr=lb:500516778worker1 # [ 141.113726] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=oidc_jwks provider=mock keys=16779worker1 # [ 141.115186] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:427746780worker1 # [ 141.116741] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06781worker1 # [ 141.423582] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06782worker1 # [ 141.428809] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16783worker1 # [ 141.472899] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428026784worker1 # [ 141.483176] nix-daemon[847]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6785worker1 # [ 141.497647] nix-daemon[847]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6786client # [ 141.539766] nix[1399]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6787worker1 # [ 141.512466] nix-daemon[847]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6788client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6789client: waiting for success: ! systemctl is-active mid6790worker1 # [ 146.335612] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:57Z level=info event=niks3_push_started pid=16356791worker1 # [ 146.366771] nix-daemon[847]: reaped child process 1610, status = succeeded6792worker1 # [ 146.444648] nix-grpc-daemon[1635]: time=2026-09-23T04:43:57.695Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6793worker1 # [ 146.529461] nix-daemon[847]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6794worker1 # [ 146.542654] nix-daemon[847]: reaped child process 1651, status = succeeded6795lb # [ 146.589856] niks3-server[1185]: 2026/09/23 04:43:58 INFO Received uploads request method=POST path=/api/pending_closures6796worker1 # [ 146.560094] nix-grpc-daemon[1635]: time=2026-09-23T04:43:57.811Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6797worker1 # [ 146.561680] nix-grpc-daemon[1635]: time=2026-09-23T04:43:57.813Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6798lb # [ 146.636308] niks3-server[1185]: 2026/09/23 04:43:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6799worker1 # [ 146.607224] nix-grpc-daemon[1635]: time=2026-09-23T04:43:57.858Z level=INFO msg="Uploading 1 narinfos"6800lb # [ 146.644978] niks3-server[1185]: 2026/09/23 04:43:58 INFO Signed narinfos id=52 count=16801lb # [ 146.658450] niks3-server[1185]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6802lb # [ 146.660952] niks3-server[1185]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6803lb # [ 146.665981] niks3-server[1185]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6804lb # [ 146.671465] niks3-server[1185]: 2026/09/23 04:43:58 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6805lb # [ 146.675322] niks3-server[1185]: 2026/09/23 04:43:58 INFO Completed upload id=526806worker1 # [ 146.638063] nix-grpc-daemon[1635]: time=2026-09-23T04:43:57.889Z level=INFO msg="Upload complete. (190ms)"6807lb # [ 146.677950] niks3-server[1185]: 2026/09/23 04:43:58 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6808worker1 # [ 146.645126] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42802 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16809worker1 # [ 146.658419] nix-daemon[847]: reaped child process 1608, status = succeeded6810client # [ 146.710724] systemd[1]: mid.service: Deactivated successfully.6811client # [ 146.714533] systemd[1]: mid.service: Consumed 146ms CPU time over 9.394s wall clock time, 43.5M memory peak, 8.7K incoming IP traffic, 10.3K outgoing IP traffic.6812client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6813client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6814client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6815(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.13 seconds)6816subtest: interrupting the client stops the build on the worker6817client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6818lb # [ 147.412187] envoy[926]: conn peer=[2001:db8:1::3]:37008 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1010 tx=393 ms=161536819lb # [ 147.416610] envoy[926]: conn peer=[2001:db8:1::1]:58130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3891 tx=1923 ms=92306820client # [ 147.421432] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6821client # Running as unit: intr.service; invocation ID: d5502ae8dd584a35a017f537b380e2756822client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6823worker2 # [ 147.565174] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587366824worker2 # [ 147.575593] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6825client # [ 147.577934] nix[1435]: don't know how to build these paths:6826client # [ 147.578941] nix[1435]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6827worker1 # [ 147.558915] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:58Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06828worker1 # [ 147.563574] nix-grpc-daemon[1550]: ts=2026-09-23T04:43:58Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16829worker2 # [ 147.647195] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587366830worker2 # [ 147.658555] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5)6831worker2 # [ 147.726858] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 2/5)6832worker2 # [ 147.745609] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 3/5)6833worker2 # [ 147.782364] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 4/5)6834worker2 # [ 147.846907] nix-grpc-daemon[1606]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6835worker2 # [ 147.850632] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6836worker2 # [ 147.867465] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587366837client # [ 147.867199] nix[1435]: copying 1 paths to 'lb:50051'...6838client # [ 147.870313] nix[1435]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6839worker2 # [ 147.881510] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587366840worker2 # [ 147.890332] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6841worker2 # [ 147.912754] nix-grpc-daemon[1606]: ts=2026-09-23T04:43:59Z level=info event=niks3_push_started pid=16996842worker2 # [ 148.018487] nix-grpc-daemon[1699]: time=2026-09-23T04:44:00.102Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6843worker2 # [ 148.106106] nix-daemon[874]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6844worker2 # [ 148.118164] nix-daemon[874]: reaped child process 1715, status = succeeded6845lb # [ 148.126570] niks3-server[1185]: 2026/09/23 04:44:00 INFO Received uploads request method=POST path=/api/pending_closures6846worker2 # [ 148.135527] nix-grpc-daemon[1699]: time=2026-09-23T04:44:00.220Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6847worker2 # [ 148.137028] nix-grpc-daemon[1699]: time=2026-09-23T04:44:00.221Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6848lb # [ 148.165226] niks3-server[1185]: 2026/09/23 04:44:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6849lb # [ 148.170869] niks3-server[1185]: 2026/09/23 04:44:00 INFO Signed narinfos id=53 count=16850worker2 # [ 148.176450] nix-grpc-daemon[1699]: time=2026-09-23T04:44:00.260Z level=INFO msg="Uploading 1 narinfos"6851lb # [ 148.177381] niks3-server[1185]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6852lb # [ 148.192247] niks3-server[1185]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6853lb # [ 148.198136] niks3-server[1185]: 2026/09/23 04:44:00 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6854lb # [ 148.204588] niks3-server[1185]: 2026/09/23 04:44:00 INFO Completed upload id=536855worker2 # [ 148.206243] nix-grpc-daemon[1699]: time=2026-09-23T04:44:00.290Z level=INFO msg="Upload complete. (184ms)"6856lb # [ 148.208553] niks3-server[1185]: 2026/09/23 04:44:00 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6857worker2 # [ 148.210749] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=0 paths=1 nar_bytes_in=4246858worker2 # [ 148.215677] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587366859worker2 # [ 148.227061] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6860worker2 # [ 148.231036] nix-daemon[874]: reaped child process 1697, status = succeeded6861worker2 # [ 148.235849] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6862client # [ 148.225089] nix[1435]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6863client: must succeed: systemctl kill -s INT intr6864worker2 # [ 149.761227] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:01Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6865client # [ 149.747889] systemd[1]: intr.service: Sent signal SIGINT to main process 1435 (nix) on client request.6866client # [ 149.755799] nix[1435]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6867client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6868client # [ 149.760964] nix[1435]: error: interrupted by the user6869client # [ 149.790900] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6870client # [ 149.793970] systemd[1]: intr.service: Failed with result 'exit-code'.6871client # [ 149.797787] systemd[1]: intr.service: Consumed 119ms CPU time over 2.372s wall clock time, 41.3M memory peak, 6.4K incoming IP traffic, 8.4K outgoing IP traffic.6872(finished: subtest: interrupting the client stops the build on the worker, in 2.45 seconds)6873subtest: a deploy mid-build drains: the build finishes, the new generation takes over6874client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6875worker2 # [ 149.863528] nix-daemon[874]: reaped child process 1717, status = succeeded6876worker2 # [ 149.865628] nix-daemon[1719]: unexpected Nix daemon error: error: interrupted by the user6877worker2 # [ 149.870393] nix-daemon[874]: reaped child process 1719, status = failed with exit code 16878client # [ 149.887449] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6879client # Running as unit: sw.service; invocation ID: b26128f4581c4ca49654d8f7345e64c26880client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.05 seconds)6881client # [ 150.011233] nix[1456]: don't know how to build these paths:6882client # [ 150.012146] nix[1456]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6883worker2 # [ 150.019965] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587366884worker1 # [ 149.990908] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:01Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06885worker1 # [ 149.996818] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:01Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16886worker1 # [ 150.040100] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428026887worker1 # [ 150.051589] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 1/5)6888worker1 # [ 150.090835] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 2/5)6889worker1 # [ 150.123883] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 211 ms (attempt 3/5)6890worker1 # [ 150.336997] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 585 ms (attempt 4/5)6891worker1 # [ 150.924254] nix-grpc-daemon[1550]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6892worker1 # [ 150.928155] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6893worker1 # [ 150.945514] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428026894client # [ 150.987303] nix[1456]: copying 1 paths to 'lb:50051'...6895worker1 # [ 150.959690] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428026896client # [ 150.990945] nix[1456]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6897worker1 # [ 150.975844] nix-daemon[847]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6898worker1 # [ 151.081536] nix-daemon[847]: accepted connection from pid 1687, user nix-grpc-daemon (trusted)6899worker1 # [ 151.095154] nix-daemon[847]: reaped child process 1690, status = succeeded6900lb # [ 151.139268] niks3-server[1185]: 2026/09/23 04:44:03 INFO Received uploads request method=POST path=/api/pending_closures6901worker1 # [ 151.112511] nix-grpc-daemon[1635]: time=2026-09-23T04:44:02.363Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6902worker1 # [ 151.113992] nix-grpc-daemon[1635]: time=2026-09-23T04:44:02.363Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6903lb # [ 151.172807] niks3-server[1185]: 2026/09/23 04:44:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6904lb # [ 151.179281] niks3-server[1185]: 2026/09/23 04:44:03 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6905lb # [ 151.191756] niks3-server[1185]: 2026/09/23 04:44:03 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6906lb # [ 151.195282] niks3-server[1185]: 2026/09/23 04:44:03 INFO Signed narinfos id=54 count=16907worker1 # [ 151.159070] nix-grpc-daemon[1635]: time=2026-09-23T04:44:02.409Z level=INFO msg="Uploading 1 narinfos"6908lb # [ 151.200979] niks3-server[1185]: 2026/09/23 04:44:03 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6909lb # [ 151.207580] niks3-server[1185]: 2026/09/23 04:44:03 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6910lb # [ 151.211518] niks3-server[1185]: 2026/09/23 04:44:03 INFO Completed upload id=546911worker1 # [ 151.174491] nix-grpc-daemon[1635]: time=2026-09-23T04:44:02.425Z level=INFO msg="Upload complete. (183ms)"6912worker1 # [ 151.178678] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42802 duration_s=0 paths=1 nar_bytes_in=4166913worker1 # [ 151.184379] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428026914worker1 # [ 151.194061] nix-daemon[847]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6915worker1 # [ 151.198131] nix-daemon[847]: reaped child process 1684, status = succeeded6916worker1 # [ 151.202700] nix-daemon[847]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6917client # [ 151.231773] nix[1456]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6918worker1: must succeed: systemctl reload nix-grpc-daemon.service6919worker1 # [ 153.379595] systemd[1]: Reloading Nix worker-protocol over gRPC...6920worker1 # [ 153.409498] systemd[1]: Reloaded Nix worker-protocol over gRPC.6921worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6922worker1: must succeed: pgrep -f 'read -t [9]0 x'6923worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6924client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26925worker2 # [ 153.642945] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587366926client # don't know how to build these paths:6927client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6928worker1 # [ 153.613607] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428026929worker1 # [ 153.635699] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:04Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06930worker1 # [ 153.637782] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:04Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16931worker2 # [ 153.725593] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587366932worker2 # [ 153.739679] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587366933client # copying 1 paths to 'lb:50051'...6934worker2 # [ 153.754243] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587366935client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6936worker2 # [ 153.775821] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6937worker2 # [ 153.887432] nix-daemon[874]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6938worker2 # [ 153.901172] nix-daemon[874]: reaped child process 1773, status = succeeded6939lb # [ 153.907549] niks3-server[1185]: 2026/09/23 04:44:05 INFO Received uploads request method=POST path=/api/pending_closures6940worker2 # [ 153.922726] nix-grpc-daemon[1699]: time=2026-09-23T04:44:06.006Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6941worker2 # [ 153.924233] nix-grpc-daemon[1699]: time=2026-09-23T04:44:06.006Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6942lb # [ 153.941403] niks3-server[1185]: 2026/09/23 04:44:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6943worker2 # [ 153.952608] nix-grpc-daemon[1699]: time=2026-09-23T04:44:06.036Z level=INFO msg="Uploading 1 narinfos"6944lb # [ 153.950909] niks3-server[1185]: 2026/09/23 04:44:05 INFO Signed narinfos id=55 count=16945lb # [ 153.958234] niks3-server[1185]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6946lb # [ 153.963354] niks3-server[1185]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6947lb # [ 153.969868] niks3-server[1185]: 2026/09/23 04:44:05 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6948lb # [ 153.973784] niks3-server[1185]: 2026/09/23 04:44:05 INFO Completed upload id=556949worker2 # [ 153.975758] nix-grpc-daemon[1699]: time=2026-09-23T04:44:06.060Z level=INFO msg="Upload complete. (184ms)"6950lb # [ 153.975944] niks3-server[1185]: 2026/09/23 04:44:05 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6951worker2 # [ 153.979887] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=0 paths=1 nar_bytes_in=5286952worker2 # [ 153.984871] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587366953worker2 # [ 153.996221] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6954worker2 # [ 154.000088] nix-daemon[874]: reaped child process 1768, status = succeeded6955worker2 # [ 154.004222] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6956client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6957client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6958worker1 # [ 154.035414] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:05Z level=info event=draining6959client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6960client # farm-a> LOG-farm-a-during-drain6961worker2 # [ 155.447380] nix-daemon[874]: reaped child process 1777, status = succeeded6962worker2 # [ 155.527948] nix-daemon[874]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6963worker2 # [ 155.541640] nix-daemon[874]: reaped child process 1796, status = succeeded6964lb # [ 155.549986] niks3-server[1185]: 2026/09/23 04:44:07 INFO Received uploads request method=POST path=/api/pending_closures6965worker2 # [ 155.559183] nix-grpc-daemon[1699]: time=2026-09-23T04:44:07.643Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6966worker2 # [ 155.560799] nix-grpc-daemon[1699]: time=2026-09-23T04:44:07.645Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6967lb # [ 155.591984] niks3-server[1185]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6968lb # [ 155.596712] niks3-server[1185]: 2026/09/23 04:44:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6969lb # [ 155.604191] niks3-server[1185]: 2026/09/23 04:44:07 INFO Signed narinfos id=56 count=16970worker2 # [ 155.609071] nix-grpc-daemon[1699]: time=2026-09-23T04:44:07.693Z level=INFO msg="Uploading 1 narinfos"6971lb # [ 155.612453] niks3-server[1185]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6972lb # [ 155.623210] niks3-server[1185]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6973lb # [ 155.630441] niks3-server[1185]: 2026/09/23 04:44:07 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6974lb # [ 155.634651] niks3-server[1185]: 2026/09/23 04:44:07 INFO Completed upload id=566975worker2 # [ 155.637926] nix-grpc-daemon[1699]: time=2026-09-23T04:44:07.722Z level=INFO msg="Upload complete. (225ms)"6976worker2 # [ 155.644722] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16977worker1 # [ 155.611124] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:06Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06978worker1 # [ 155.612958] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:06Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16979worker2 # [ 155.662052] nix-daemon[874]: reaped child process 1775, status = succeeded6980worker2 # [ 155.694778] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587366981worker2 # [ 155.708878] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587366982worker2 # [ 155.722285] nix-grpc-daemon[1699]: time=2026-09-23T04:44:07.806Z level=INFO msg="All 1 paths already cached"6983client # copying 1 paths to 'lb:50051'...6984worker2 # [ 155.731306] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587366985client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6986worker2 # [ 155.751176] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6987worker2 # [ 155.856719] nix-daemon[874]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6988worker2 # [ 155.870925] nix-daemon[874]: reaped child process 1803, status = succeeded6989lb # [ 155.876115] niks3-server[1185]: 2026/09/23 04:44:07 INFO Received uploads request method=POST path=/api/pending_closures6990worker2 # [ 155.886688] nix-grpc-daemon[1699]: time=2026-09-23T04:44:07.971Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6991worker2 # [ 155.888293] nix-grpc-daemon[1699]: time=2026-09-23T04:44:07.972Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6992lb # [ 155.913647] niks3-server[1185]: 2026/09/23 04:44:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6993lb # [ 155.919831] niks3-server[1185]: 2026/09/23 04:44:07 INFO Signed narinfos id=57 count=16994worker2 # [ 155.924927] nix-grpc-daemon[1699]: time=2026-09-23T04:44:08.009Z level=INFO msg="Uploading 1 narinfos"6995lb # [ 155.927717] niks3-server[1185]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6996lb # [ 155.934187] niks3-server[1185]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6997lb # [ 155.946117] niks3-server[1185]: 2026/09/23 04:44:07 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6998lb # [ 155.952135] niks3-server[1185]: 2026/09/23 04:44:07 INFO Completed upload id=576999lb # [ 155.956156] niks3-server[1185]: 2026/09/23 04:44:07 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo7000worker2 # [ 155.956547] nix-grpc-daemon[1699]: time=2026-09-23T04:44:08.040Z level=INFO msg="Upload complete. (191ms)"7001worker2 # [ 155.961474] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=0 paths=1 nar_bytes_in=6407002worker2 # [ 155.967761] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367003worker2 # [ 155.981114] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7004worker2 # [ 155.989468] nix-daemon[874]: reaped child process 1798, status = succeeded7005worker2 # [ 155.993892] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7006client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...7007client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv7008client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7009client # farm-b> LOG-farm-b-during-drain7010worker2 # [ 157.223574] nix-daemon[874]: reaped child process 1807, status = succeeded7011worker2 # [ 157.298155] nix-daemon[874]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)7012worker2 # [ 157.311832] nix-daemon[874]: reaped child process 1826, status = succeeded7013lb # [ 157.318371] niks3-server[1185]: 2026/09/23 04:44:09 INFO Received uploads request method=POST path=/api/pending_closures7014worker2 # [ 157.331374] nix-grpc-daemon[1699]: time=2026-09-23T04:44:09.415Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7015worker2 # [ 157.332886] nix-grpc-daemon[1699]: time=2026-09-23T04:44:09.417Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"7016lb # [ 157.362983] niks3-server[1185]: 2026/09/23 04:44:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign7017worker2 # [ 157.369469] nix-grpc-daemon[1699]: time=2026-09-23T04:44:09.453Z level=INFO msg="Uploading 1 narinfos"7018lb # [ 157.368247] niks3-server[1185]: 2026/09/23 04:44:09 INFO Signed narinfos id=58 count=17019lb # [ 157.380629] niks3-server[1185]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv7020lb # [ 157.384744] niks3-server[1185]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls7021lb # [ 157.395660] niks3-server[1185]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst7022lb # [ 157.399059] niks3-server[1185]: 2026/09/23 04:44:09 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete7023lb # [ 157.403449] niks3-server[1185]: 2026/09/23 04:44:09 INFO Completed upload id=587024lb # [ 157.405804] niks3-server[1185]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo7025worker2 # [ 157.406952] nix-grpc-daemon[1699]: time=2026-09-23T04:44:09.491Z level=INFO msg="Upload complete. (205ms)"7026lb # [ 157.412351] envoy[926]: conn peer=[2001:db8:1::1]:52616 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=828 ms=22287027worker2 # [ 157.413776] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=17028worker1 # [ 157.382162] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:08Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=07029worker1 # [ 157.384154] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:08Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=17030worker2 # [ 157.430805] nix-daemon[874]: reaped child process 1805, status = succeeded7031worker2 # [ 157.471822] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367032worker2 # [ 157.485661] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587367033worker2 # [ 157.501086] nix-grpc-daemon[1699]: time=2026-09-23T04:44:09.584Z level=INFO msg="All 2 paths already cached"7034client # copying 1 paths to 'lb:50051'...7035worker2 # [ 157.510680] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587367036client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...7037worker2 # [ 157.528586] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7038worker2 # [ 157.637164] nix-daemon[874]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)7039worker2 # [ 157.651963] nix-daemon[874]: reaped child process 1833, status = succeeded7040lb # [ 157.655812] niks3-server[1185]: 2026/09/23 04:44:09 INFO Received uploads request method=POST path=/api/pending_closures7041worker2 # [ 157.666683] nix-grpc-daemon[1699]: time=2026-09-23T04:44:09.751Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7042worker2 # [ 157.668307] nix-grpc-daemon[1699]: time=2026-09-23T04:44:09.752Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"7043lb # [ 157.692259] niks3-server[1185]: 2026/09/23 04:44:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign7044lb # [ 157.697553] niks3-server[1185]: 2026/09/23 04:44:09 INFO Signed narinfos id=59 count=17045worker2 # [ 157.702621] nix-grpc-daemon[1699]: time=2026-09-23T04:44:09.787Z level=INFO msg="Uploading 1 narinfos"7046lb # [ 157.702493] niks3-server[1185]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls7047lb # [ 157.711523] niks3-server[1185]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst7048lb # [ 157.722916] niks3-server[1185]: 2026/09/23 04:44:09 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete7049lb # [ 157.730067] niks3-server[1185]: 2026/09/23 04:44:09 INFO Completed upload id=597050worker2 # [ 157.731869] nix-grpc-daemon[1699]: time=2026-09-23T04:44:09.816Z level=INFO msg="Upload complete. (187ms)"7051lb # [ 157.733286] niks3-server[1185]: 2026/09/23 04:44:09 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo7052worker2 # [ 157.736945] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=0 paths=1 nar_bytes_in=7687053worker2 # [ 157.742892] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367054worker2 # [ 157.755115] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7055worker2 # [ 157.762072] nix-daemon[874]: reaped child process 1828, status = succeeded7056worker2 # [ 157.767425] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7057client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...7058client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv7059client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7060client # farm-top> LOG-farm-top-during-drain7061worker2 # [ 158.982850] nix-daemon[874]: reaped child process 1837, status = succeeded7062worker2 # [ 159.062413] nix-daemon[874]: accepted connection from pid 1853, user nix-grpc-daemon (trusted)7063worker2 # [ 159.074721] nix-daemon[874]: reaped child process 1856, status = succeeded7064lb # [ 159.081373] niks3-server[1185]: 2026/09/23 04:44:11 INFO Received uploads request method=POST path=/api/pending_closures7065worker2 # [ 159.091990] nix-grpc-daemon[1699]: time=2026-09-23T04:44:11.176Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7066worker2 # [ 159.093653] nix-grpc-daemon[1699]: time=2026-09-23T04:44:11.178Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"7067lb # [ 159.122636] niks3-server[1185]: 2026/09/23 04:44:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign7068lb # [ 159.129645] niks3-server[1185]: 2026/09/23 04:44:11 INFO Signed narinfos id=60 count=17069worker2 # [ 159.134973] nix-grpc-daemon[1699]: time=2026-09-23T04:44:11.219Z level=INFO msg="Uploading 1 narinfos"7070lb # [ 159.136799] niks3-server[1185]: 2026/09/23 04:44:11 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls7071lb # [ 159.143368] niks3-server[1185]: 2026/09/23 04:44:11 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv7072lb # [ 159.149860] niks3-server[1185]: 2026/09/23 04:44:11 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst7073lb # [ 159.159695] niks3-server[1185]: 2026/09/23 04:44:11 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete7074lb # [ 159.164324] niks3-server[1185]: 2026/09/23 04:44:11 INFO Completed upload id=607075worker2 # [ 159.166354] nix-grpc-daemon[1699]: time=2026-09-23T04:44:11.250Z level=INFO msg="Upload complete. (191ms)"7076lb # [ 159.166644] niks3-server[1185]: 2026/09/23 04:44:11 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo7077worker2 # [ 159.174129] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=17078worker2 # [ 159.188477] nix-daemon[874]: reaped child process 1835, status = succeeded7079client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.72 seconds)7080client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath7081client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)7082worker1: must succeed: pkill -f 'read -t [9]0 x'7083worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)7084client: waiting for success: ! systemctl is-active sw7085worker1 # [ 159.437935] nix-daemon[847]: reaped child process 1694, status = succeeded7086worker1 # [ 159.514574] nix-daemon[847]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)7087worker1 # [ 159.527719] nix-daemon[847]: reaped child process 1745, status = succeeded7088lb # [ 159.572403] niks3-server[1185]: 2026/09/23 04:44:11 INFO Received uploads request method=POST path=/api/pending_closures7089worker1 # [ 159.543842] nix-grpc-daemon[1635]: time=2026-09-23T04:44:10.795Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7090worker1 # [ 159.545537] nix-grpc-daemon[1635]: time=2026-09-23T04:44:10.796Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"7091lb # [ 159.613930] niks3-server[1185]: 2026/09/23 04:44:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign7092lb # [ 159.618927] niks3-server[1185]: 2026/09/23 04:44:11 INFO Signed narinfos id=61 count=17093worker1 # [ 159.584720] nix-grpc-daemon[1635]: time=2026-09-23T04:44:10.836Z level=INFO msg="Uploading 1 narinfos"7094lb # [ 159.630556] niks3-server[1185]: 2026/09/23 04:44:11 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls7095lb # [ 159.635604] niks3-server[1185]: 2026/09/23 04:44:11 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv7096lb # [ 159.641615] niks3-server[1185]: 2026/09/23 04:44:11 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst7097lb # [ 159.651128] niks3-server[1185]: 2026/09/23 04:44:11 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete7098lb # [ 159.654962] niks3-server[1185]: 2026/09/23 04:44:11 INFO Completed upload id=617099lb # [ 159.657306] niks3-server[1185]: 2026/09/23 04:44:11 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo7100worker1 # [ 159.619425] nix-grpc-daemon[1635]: time=2026-09-23T04:44:10.870Z level=INFO msg="Upload complete. (202ms)"7101worker1 # [ 159.627920] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42802 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=17102worker1 # [ 159.642118] nix-daemon[847]: reaped child process 1692, status = succeeded7103client # [ 159.686682] systemd[1]: sw.service: Deactivated successfully.7104client # [ 159.690533] systemd[1]: sw.service: Consumed 127ms CPU time over 9.802s wall clock time, 43.4M memory peak, 7.4K incoming IP traffic, 8.4K outgoing IP traffic.7105worker1 # [ 160.037251] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:11Z level=info event=signal_exit7106lb # [ 160.083886] niks3-server[1185]: 2026/09/23 04:44:12 INFO lead: released remote=192.168.1.3:437767107client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)7108client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }7109client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)7110worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service7111worker2 # [ 161.086764] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:13Z level=info event=scheduler_restarting addr=lb:500517112worker1 # [ 161.051535] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:12Z level=info event=scheduler_restarting7113worker1 # [ 161.055609] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:427747114worker1 # [ 161.059465] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:12Z level=info event=worker_gone addr=192.168.1.4:500517115worker1 # [ 161.063543] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:12Z level=info event=scheduler_restarting addr=lb:500517116worker1 # [ 161.066797] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:427747117worker1 # [ 161.070480] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:12Z level=info event=worker_gone addr=192.168.1.3:500517118worker1 # [ 161.088887] nix-daemon[847]: reaped child process 1606, status = succeeded7119worker2 # [ 161.287301] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:13Z level=info event=scheduler_connected addr=lb:500517120worker2 # [ 161.291771] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7121worker1 # [ 161.261290] nix-grpc-daemon[1550]: ts=2026-09-23T04:44:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"7122worker1 # [ 161.306076] nix-grpc-daemon[1550]: E0000 00:00:1790138652.557279 1597 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true7123worker1 # [ 161.310983] nix-grpc-daemon[1550]: E0000 00:00:1790138652.557431 1597 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true7124worker2 # [ 161.492883] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:13Z level=info event=scheduler_connected addr=lb:500517125worker2 # [ 161.497222] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7126worker2 # [ 161.698311] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:13Z level=info event=scheduler_connected addr=lb:500517127worker2 # [ 161.702469] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7128worker2 # [ 161.903663] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:13Z level=info event=scheduler_connected addr=lb:500517129worker2 # [ 161.907844] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7130lb # [ 162.085724] niks3-server[1185]: 2026/09/23 04:44:14 INFO lead: acquired remote=192.168.1.4:595547131worker2 # [ 162.091634] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:14Z level=info event=scheduler_take_over7132worker2 # [ 162.108757] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:14Z level=info event=scheduler_connected addr=lb:500517133worker2 # [ 162.112874] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7134worker2 # [ 162.314429] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:14Z level=info event=scheduler_connected addr=lb:500517135worker2 # [ 162.318653] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7136worker2 # [ 162.519771] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:14Z level=info event=scheduler_connected addr=lb:500517137worker2 # [ 162.523942] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7138worker2 # [ 162.725157] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:14Z level=info event=scheduler_connected addr=lb:500517139worker2 # [ 162.729317] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7140worker2 # [ 162.930337] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:15Z level=info event=scheduler_connected addr=lb:500517141worker2 # [ 162.935504] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7142worker1 # [ 163.065343] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.7143worker1 # [ 163.069376] systemd[1]: nix-grpc-daemon.service: Consumed 573ms CPU time over 22.162s wall clock time, 35.8M memory peak, 24.9K incoming IP traffic, 32.8K outgoing IP traffic.7144worker1 # [ 163.085286] systemd[1]: Starting Nix worker-protocol over gRPC...7145worker2 # [ 163.135995] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:15Z level=info event=scheduler_connected addr=lb:500517146worker2 # [ 163.153988] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:15Z level=info event=oidc_jwks provider=mock keys=17147worker2 # [ 163.157270] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:15Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:519847148worker2 # [ 163.161872] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:15Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07149worker1 # [ 163.177673] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:14Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"7150worker1 # [ 163.182131] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc7151worker1 # [ 163.185498] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=17152worker1 # [ 163.187494] nix-grpc-daemon[1759]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR7153worker1 # [ 163.188817] nix-grpc-daemon[1759]: E0000 00:00:1790138654.436236 1759 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7154worker1 # [ 163.197707] nix-grpc-daemon[1759]: E0000 00:00:1790138654.443824 1759 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7155worker1 # [ 163.203076] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:14Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket7156worker1 # [ 163.204751] systemd[1]: Started Nix worker-protocol over gRPC.7157worker1 # [ 163.205664] nix-grpc-daemon[1759]: E0000 00:00:1790138654.452745 1801 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7158worker1 # [ 163.216978] nix-grpc-daemon[1759]: E0000 00:00:1790138654.468551 1801 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7159worker1 # [ 163.222215] nix-grpc-daemon[1759]: E0000 00:00:1790138654.468778 1801 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7160worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.81 seconds)7161lb: waiting until exactly no endpoints of x86_64-linux are unhealthy7162lb: must succeed: curl -sf localhost:9901/clusters7163worker1 # [ 163.232508] nix-grpc-daemon[1759]: E0000 00:00:1790138654.484031 1801 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7164worker1 # [ 163.245968] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:14Z level=info event=scheduler_connected addr=lb:500517165worker2 # [ 163.287172] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:15Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:519847166worker2 # [ 163.291919] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:15Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07167lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7168lb: must succeed: curl -sf localhost:9901/clusters7169lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7170lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)7171lb: must succeed: curl -sf localhost:9901/clusters7172lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7173worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7174worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7175(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.59 seconds)7176subtest: queue gauges show a burst while every build blocks7177lb: must succeed: curl -sf localhost:9901/clusters7178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7179lb: must succeed: curl -sf localhost:9901/clusters7180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7181worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7182worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7183client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'7184client # [ 164.590930] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".7185client # Running as unit: burst.service; invocation ID: 6c048fec59f548ba8cb35d555107cd5d7186client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.07 seconds)7187lb: must succeed: curl -sf localhost:9901/clusters7188lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7189lb: must succeed: curl -sf localhost:9901/clusters7190lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7191worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7192worker2 # [ 164.744807] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587367193client # [ 164.740339] nix[1512]: don't know how to build these paths:7194client # [ 164.741397] nix[1512]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7195client # [ 164.742567] nix[1512]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7196client # [ 164.743627] nix[1512]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7197client # [ 164.744802] nix[1512]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7198worker2 # [ 164.756881] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=want client=1 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=07199worker2 # [ 164.761274] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=17200worker2 # [ 164.765761] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=want client=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=07201worker2 # [ 164.770746] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=17202worker2 # [ 164.773344] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=want client=1 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=07203worker2 # [ 164.775116] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=want client=1 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=07204worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7205worker2 # [ 164.806505] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367206worker1 # [ 164.771808] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327207worker2 # [ 164.819602] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587367208worker1 # [ 164.783591] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7209client # [ 164.820313] nix[1512]: copying 1 paths to 'lb:50051'...7210worker2 # [ 164.831892] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587367211client # [ 164.823887] nix[1512]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...7212worker1 # [ 164.800344] nix-grpc-daemon[1759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)7213worker2 # [ 164.843697] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7214worker1 # [ 164.844156] nix-grpc-daemon[1759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5)7215worker2 # [ 164.942467] nix-daemon[874]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)7216worker2 # [ 164.956620] nix-daemon[874]: reaped child process 1874, status = succeeded7217lb # [ 164.962783] niks3-server[1185]: 2026/09/23 04:44:16 INFO Received uploads request method=POST path=/api/pending_closures7218worker2 # [ 164.974666] nix-grpc-daemon[1699]: time=2026-09-23T04:44:17.058Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7219worker2 # [ 164.976506] nix-grpc-daemon[1699]: time=2026-09-23T04:44:17.060Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"7220lb # [ 164.994535] niks3-server[1185]: 2026/09/23 04:44:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7221lb # [ 165.004722] niks3-server[1185]: 2026/09/23 04:44:17 INFO Signed narinfos id=62 count=17222worker2 # [ 165.009730] nix-grpc-daemon[1699]: time=2026-09-23T04:44:17.094Z level=INFO msg="Uploading 1 narinfos"7223lb # [ 165.008540] niks3-server[1185]: 2026/09/23 04:44:17 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls7224lb # [ 165.015950] niks3-server[1185]: 2026/09/23 04:44:17 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst7225lb # [ 165.024607] niks3-server[1185]: 2026/09/23 04:44:17 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7226lb # [ 165.033377] niks3-server[1185]: 2026/09/23 04:44:17 INFO Completed upload id=627227worker2 # [ 165.035293] nix-grpc-daemon[1699]: time=2026-09-23T04:44:17.119Z level=INFO msg="Upload complete. (181ms)"7228lb # [ 165.035874] niks3-server[1185]: 2026/09/23 04:44:17 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo7229worker2 # [ 165.039668] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=0 paths=1 nar_bytes_in=4167230worker2 # [ 165.044654] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367231worker2 # [ 165.055572] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7232worker2 # [ 165.060233] nix-daemon[874]: reaped child process 1869, status = succeeded7233worker2 # [ 165.064668] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7234worker1 # [ 165.022365] nix-grpc-daemon[1759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 3/5)7235client # [ 165.053842] nix[1512]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...7236worker1 # [ 165.089095] nix-grpc-daemon[1759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 586 ms (attempt 4/5)7237worker1 # [ 165.677258] nix-grpc-daemon[1759]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7238worker1 # [ 165.680585] nix-grpc-daemon[1759]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7239worker1 # [ 165.699479] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327240client # [ 165.738239] nix[1512]: copying 1 paths to 'lb:50051'...7241client # [ 165.742748] nix[1512]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...7242worker1 # [ 165.713897] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327243worker1 # [ 165.724203] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7244lb: must succeed: curl -sf localhost:9901/clusters7245worker1 # [ 165.744745] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:16Z level=info event=niks3_push_started pid=18207246lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7247lb: must succeed: curl -sf localhost:9901/clusters7248lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7249worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7250worker1 # [ 165.849851] nix-grpc-daemon[1820]: time=2026-09-23T04:44:17.101Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""7251worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7252worker1 # [ 165.951171] nix-daemon[847]: accepted connection from pid 1833, user nix-grpc-daemon (trusted)7253worker1 # [ 165.963299] nix-daemon[847]: reaped child process 1840, status = succeeded7254lb # [ 166.008578] niks3-server[1185]: 2026/09/23 04:44:18 INFO Received uploads request method=POST path=/api/pending_closures7255worker1 # [ 165.979421] nix-grpc-daemon[1820]: time=2026-09-23T04:44:17.230Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7256worker1 # [ 165.981419] nix-grpc-daemon[1820]: time=2026-09-23T04:44:17.232Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"7257lb # [ 166.051923] niks3-server[1185]: 2026/09/23 04:44:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7258lb # [ 166.058148] niks3-server[1185]: 2026/09/23 04:44:18 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst7259worker1 # [ 166.030323] nix-grpc-daemon[1820]: time=2026-09-23T04:44:17.281Z level=INFO msg="Uploading 1 narinfos"7260lb # [ 166.067124] niks3-server[1185]: 2026/09/23 04:44:18 INFO Signed narinfos id=63 count=17261lb # [ 166.082074] niks3-server[1185]: 2026/09/23 04:44:18 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7262lb # [ 166.088282] niks3-server[1185]: 2026/09/23 04:44:18 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls7263lb # [ 166.091248] niks3-server[1185]: 2026/09/23 04:44:18 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo7264lb # [ 166.095607] niks3-server[1185]: 2026/09/23 04:44:18 INFO Completed upload id=637265worker1 # [ 166.058482] nix-grpc-daemon[1820]: time=2026-09-23T04:44:17.309Z level=INFO msg="Upload complete. (205ms)"7266worker1 # [ 166.063829] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=4167267worker1 # [ 166.073117] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327268worker1 # [ 166.081845] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7269worker1 # [ 166.089522] nix-daemon[847]: reaped child process 1818, status = succeeded7270worker1 # [ 166.094490] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7271client # [ 166.123096] nix[1512]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...7272worker1 # [ 167.108180] nix-daemon[847]: reaped child process 1844, status = succeeded7273worker1 # [ 167.184626] nix-daemon[847]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)7274worker1 # [ 167.195813] nix-daemon[847]: reaped child process 1868, status = succeeded7275lb # [ 167.242650] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received uploads request method=POST path=/api/pending_closures7276worker1 # [ 167.216583] nix-grpc-daemon[1820]: time=2026-09-23T04:44:18.467Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7277worker1 # [ 167.218116] nix-grpc-daemon[1820]: time=2026-09-23T04:44:18.469Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"7278worker2 # [ 167.282545] nix-daemon[874]: reaped child process 1878, status = succeeded7279lb # [ 167.285796] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst7280lb # [ 167.296281] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7281lb # [ 167.299766] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv7282worker1 # [ 167.269402] nix-grpc-daemon[1820]: time=2026-09-23T04:44:18.520Z level=INFO msg="Uploading 1 narinfos"7283lb # [ 167.306387] niks3-server[1185]: 2026/09/23 04:44:19 INFO Signed narinfos id=64 count=17284lb # [ 167.316463] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls7285lb # [ 167.317918] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7286lb # [ 167.323340] niks3-server[1185]: 2026/09/23 04:44:19 INFO Completed upload id=647287worker1 # [ 167.285980] nix-grpc-daemon[1820]: time=2026-09-23T04:44:18.537Z level=INFO msg="Upload complete. (185ms)"7288lb # [ 167.324740] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo7289worker2 # [ 167.331300] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:19Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.3:50051 clients=17290worker1 # [ 167.291188] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=2 outputs=17291worker1 # [ 167.303261] nix-daemon[847]: reaped child process 1842, status = succeeded7292worker2 # [ 167.368122] nix-daemon[874]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)7293worker1 # [ 167.336922] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327294worker2 # [ 167.381858] nix-daemon[874]: reaped child process 1909, status = succeeded7295lb # [ 167.385369] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received uploads request method=POST path=/api/pending_closures7296worker1 # [ 167.353168] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327297worker2 # [ 167.396881] nix-grpc-daemon[1699]: time=2026-09-23T04:44:19.481Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7298worker2 # [ 167.399889] nix-grpc-daemon[1699]: time=2026-09-23T04:44:19.482Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"7299client # [ 167.392982] nix[1512]: copying 1 paths to 'lb:50051'...7300worker1 # [ 167.364933] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327301client # [ 167.396246] nix[1512]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...7302worker1 # [ 167.373248] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7303lb # [ 167.413598] envoy[926]: conn peer=[2001:db8:1::1]:34910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9581 tx=4359 ms=55517304lb # [ 167.417470] envoy[926]: conn peer=[2001:db8:1::1]:52628 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=1540 ms=96587305lb # [ 167.421179] envoy[926]: conn peer=[2001:db8:1::3]:46970 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=975 tx=361 ms=201787306lb # [ 167.432817] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv7307lb # [ 167.435187] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7308lb # [ 167.438119] niks3-server[1185]: 2026/09/23 04:44:19 INFO Signed narinfos id=65 count=17309worker2 # [ 167.441322] nix-grpc-daemon[1699]: time=2026-09-23T04:44:19.525Z level=INFO msg="Uploading 1 narinfos"7310lb # [ 167.441588] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls7311lb # [ 167.446842] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7312lb # [ 167.452263] niks3-server[1185]: 2026/09/23 04:44:19 INFO Completed upload id=657313worker2 # [ 167.454051] nix-grpc-daemon[1699]: time=2026-09-23T04:44:19.538Z level=INFO msg="Upload complete. (180ms)"7314lb # [ 167.453804] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst7315worker2 # [ 167.457323] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=1 outputs=17316lb # [ 167.457689] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo7317worker2 # [ 167.460690] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:19Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.4:50051 clients=17318worker2 # [ 167.464964] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367319worker2 # [ 167.471793] nix-daemon[874]: reaped child process 1876, status = succeeded7320worker2 # [ 167.473284] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587367321client # [ 167.468220] nix[1512]: copying 1 paths to 'lb:50051'...7322worker2 # [ 167.478939] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587367323client # [ 167.470328] nix[1512]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...7324worker2 # [ 167.488072] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7325worker1 # [ 167.471307] nix-daemon[847]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)7326worker1 # [ 167.483923] nix-daemon[847]: reaped child process 1875, status = succeeded7327lb # [ 167.528826] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received uploads request method=POST path=/api/pending_closures7328worker1 # [ 167.499519] nix-grpc-daemon[1820]: time=2026-09-23T04:44:18.751Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7329worker1 # [ 167.501433] nix-grpc-daemon[1820]: time=2026-09-23T04:44:18.752Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"7330lb # [ 167.557148] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7331lb # [ 167.563365] niks3-server[1185]: 2026/09/23 04:44:19 INFO Signed narinfos id=66 count=17332worker1 # [ 167.528791] nix-grpc-daemon[1820]: time=2026-09-23T04:44:18.780Z level=INFO msg="Uploading 1 narinfos"7333lb # [ 167.568316] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst7334lb # [ 167.573410] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls7335worker2 # [ 167.576513] nix-daemon[874]: accepted connection from pid 1913, user nix-grpc-daemon (trusted)7336lb # [ 167.579912] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7337lb # [ 167.582906] niks3-server[1185]: 2026/09/23 04:44:19 INFO Completed upload id=667338worker1 # [ 167.545621] nix-grpc-daemon[1820]: time=2026-09-23T04:44:18.797Z level=INFO msg="Upload complete. (159ms)"7339lb # [ 167.584642] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo7340worker1 # [ 167.547908] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=4167341worker2 # [ 167.590546] nix-daemon[874]: reaped child process 1916, status = succeeded7342worker1 # [ 167.551681] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327343worker1 # [ 167.558110] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7344lb # [ 167.595508] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received uploads request method=POST path=/api/pending_closures7345worker1 # [ 167.560387] nix-daemon[847]: reaped child process 1870, status = succeeded7346worker2 # [ 167.600654] nix-grpc-daemon[1699]: time=2026-09-23T04:44:19.685Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7347worker2 # [ 167.602641] nix-grpc-daemon[1699]: time=2026-09-23T04:44:19.686Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"7348worker1 # [ 167.565471] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7349client # [ 167.593984] nix[1512]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...7350lb # [ 167.608751] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7351lb # [ 167.611559] niks3-server[1185]: 2026/09/23 04:44:19 INFO Signed narinfos id=67 count=17352worker2 # [ 167.613685] nix-grpc-daemon[1699]: time=2026-09-23T04:44:19.698Z level=INFO msg="Uploading 1 narinfos"7353lb # [ 167.615273] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst7354lb # [ 167.619633] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls7355lb # [ 167.623160] niks3-server[1185]: 2026/09/23 04:44:19 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7356lb # [ 167.626764] niks3-server[1185]: 2026/09/23 04:44:19 INFO Completed upload id=677357worker2 # [ 167.628493] nix-grpc-daemon[1699]: time=2026-09-23T04:44:19.712Z level=INFO msg="Upload complete. (135ms)"7358lb # [ 167.628597] niks3-server[1185]: 2026/09/23 04:44:19 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo7359worker2 # [ 167.630512] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=0 paths=1 nar_bytes_in=4167360worker2 # [ 167.634478] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367361worker2 # [ 167.641721] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7362worker2 # [ 167.643132] nix-daemon[874]: reaped child process 1911, status = succeeded7363worker2 # [ 167.648681] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7364client # [ 167.637958] nix[1512]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...7365worker1 # [ 169.380263] nix-daemon[847]: reaped child process 1879, status = succeeded7366worker2 # [ 169.465504] nix-daemon[874]: reaped child process 1920, status = succeeded7367worker1 # [ 169.458330] nix-daemon[847]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)7368worker1 # [ 169.470128] nix-daemon[847]: reaped child process 1907, status = succeeded7369lb # [ 169.515946] niks3-server[1185]: 2026/09/23 04:44:21 INFO Received uploads request method=POST path=/api/pending_closures7370worker1 # [ 169.487397] nix-grpc-daemon[1820]: time=2026-09-23T04:44:20.738Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7371worker1 # [ 169.489165] nix-grpc-daemon[1820]: time=2026-09-23T04:44:20.740Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"7372worker2 # [ 169.543661] nix-daemon[874]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)7373worker2 # [ 169.556612] nix-daemon[874]: reaped child process 1948, status = succeeded7374lb # [ 169.558950] niks3-server[1185]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv7375lb # [ 169.563751] niks3-server[1185]: 2026/09/23 04:44:21 INFO Received uploads request method=POST path=/api/pending_closures7376lb # [ 169.570285] niks3-server[1185]: 2026/09/23 04:44:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7377worker1 # [ 169.543695] nix-grpc-daemon[1820]: time=2026-09-23T04:44:20.795Z level=INFO msg="Uploading 1 narinfos"7378lb # [ 169.580697] niks3-server[1185]: 2026/09/23 04:44:21 INFO Signed narinfos id=68 count=17379lb # [ 169.585557] niks3-server[1185]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls7380worker2 # [ 169.593234] nix-grpc-daemon[1699]: time=2026-09-23T04:44:21.677Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7381lb # [ 169.592267] niks3-server[1185]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst7382worker2 # [ 169.596452] nix-grpc-daemon[1699]: time=2026-09-23T04:44:21.679Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"7383lb # [ 169.601494] niks3-server[1185]: 2026/09/23 04:44:21 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7384lb # [ 169.605191] niks3-server[1185]: 2026/09/23 04:44:21 INFO Completed upload id=687385worker1 # [ 169.568300] nix-grpc-daemon[1820]: time=2026-09-23T04:44:20.819Z level=INFO msg="Upload complete. (195ms)"7386lb # [ 169.609893] niks3-server[1185]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo7387worker1 # [ 169.574681] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=3 outputs=17388lb # [ 169.619546] niks3-server[1185]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv7389lb # [ 169.623455] niks3-server[1185]: 2026/09/23 04:44:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7390lb # [ 169.625817] niks3-server[1185]: 2026/09/23 04:44:21 INFO Signed narinfos id=69 count=17391worker2 # [ 169.629028] nix-grpc-daemon[1699]: time=2026-09-23T04:44:21.713Z level=INFO msg="Uploading 1 narinfos"7392worker1 # [ 169.589697] nix-daemon[847]: reaped child process 1877, status = succeeded7393lb # [ 169.629157] niks3-server[1185]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst7394lb # [ 169.634297] niks3-server[1185]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls7395lb # [ 169.636829] niks3-server[1185]: 2026/09/23 04:44:21 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7396lb # [ 169.640587] niks3-server[1185]: 2026/09/23 04:44:21 INFO Completed upload id=697397worker2 # [ 169.642489] nix-grpc-daemon[1699]: time=2026-09-23T04:44:21.726Z level=INFO msg="Upload complete. (186ms)"7398lb # [ 169.642732] niks3-server[1185]: 2026/09/23 04:44:21 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo7399worker2 # [ 169.647973] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=4 outputs=17400worker2 # [ 169.659372] nix-daemon[874]: reaped child process 1918, status = succeeded7401client # [ 169.673298] systemd[1]: burst.service: Deactivated successfully.7402client # [ 169.677116] systemd[1]: burst.service: Consumed 166ms CPU time over 5.086s wall clock time, 55.9M memory peak, 14.1K incoming IP traffic, 18.5K outgoing IP traffic.7403client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }7404client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)7405lb: must succeed: curl -sf localhost:9901/clusters7406lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7407lb: must succeed: curl -sf localhost:9901/clusters7408lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7409worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7410worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7411(finished: subtest: queue gauges show a burst while every build blocks, in 6.33 seconds)7412subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7413client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17414worker2 # [ 170.926175] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587367415worker2 # [ 170.932672] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587367416worker2 # [ 170.974894] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:23Z level=debug event=want client=2 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07417worker2 # [ 170.980287] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:23Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17418worker2 # [ 170.985090] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:23Z level=debug event=want client=2 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07419worker2 # [ 170.990943] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:23Z level=debug event=want client=2 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07420worker1 # [ 170.987728] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327421worker1 # [ 171.001668] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327422worker1 # [ 171.013550] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327423worker1 # [ 171.024154] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7424worker1 # [ 171.127755] nix-daemon[847]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)7425worker1 # [ 171.140460] nix-daemon[847]: reaped child process 1918, status = succeeded7426lb # [ 171.188371] niks3-server[1185]: 2026/09/23 04:44:23 INFO Received uploads request method=POST path=/api/pending_closures7427worker1 # [ 171.158592] nix-grpc-daemon[1820]: time=2026-09-23T04:44:22.409Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7428worker1 # [ 171.160658] nix-grpc-daemon[1820]: time=2026-09-23T04:44:22.411Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7429lb # [ 171.217593] niks3-server[1185]: 2026/09/23 04:44:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7430worker1 # [ 171.192527] nix-grpc-daemon[1820]: time=2026-09-23T04:44:22.443Z level=INFO msg="Uploading 1 narinfos"7431lb # [ 171.229712] niks3-server[1185]: 2026/09/23 04:44:23 INFO Signed narinfos id=70 count=17432lb # [ 171.237230] niks3-server[1185]: 2026/09/23 04:44:23 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7433lb # [ 171.242118] niks3-server[1185]: 2026/09/23 04:44:23 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7434lb # [ 171.255679] niks3-server[1185]: 2026/09/23 04:44:23 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7435lb # [ 171.259856] niks3-server[1185]: 2026/09/23 04:44:23 INFO Completed upload id=707436worker1 # [ 171.223877] nix-grpc-daemon[1820]: time=2026-09-23T04:44:22.475Z level=INFO msg="Upload complete. (188ms)"7437lb # [ 171.262880] niks3-server[1185]: 2026/09/23 04:44:23 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7438worker1 # [ 171.227809] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=5047439worker1 # [ 171.232632] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327440worker1 # [ 171.244295] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7441worker1 # [ 171.252471] nix-daemon[847]: reaped child process 1913, status = succeeded7442worker1 # [ 171.256514] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7443worker1 # [ 172.286590] nix-daemon[847]: reaped child process 1922, status = succeeded7444worker1 # [ 172.362156] nix-daemon[847]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)7445worker1 # [ 172.375441] nix-daemon[847]: reaped child process 1941, status = succeeded7446lb # [ 172.420906] niks3-server[1185]: 2026/09/23 04:44:24 INFO Received uploads request method=POST path=/api/pending_closures7447worker1 # [ 172.392852] nix-grpc-daemon[1820]: time=2026-09-23T04:44:23.644Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7448worker1 # [ 172.394433] nix-grpc-daemon[1820]: time=2026-09-23T04:44:23.646Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7449lb # [ 172.466302] niks3-server[1185]: 2026/09/23 04:44:24 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7450lb # [ 172.474067] niks3-server[1185]: 2026/09/23 04:44:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7451worker1 # [ 172.443047] nix-grpc-daemon[1820]: time=2026-09-23T04:44:23.694Z level=INFO msg="Uploading 1 narinfos"7452lb # [ 172.480211] niks3-server[1185]: 2026/09/23 04:44:24 INFO Signed narinfos id=71 count=17453lb # [ 172.488663] niks3-server[1185]: 2026/09/23 04:44:24 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7454lb # [ 172.496337] niks3-server[1185]: 2026/09/23 04:44:24 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7455lb # [ 172.505127] niks3-server[1185]: 2026/09/23 04:44:24 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7456lb # [ 172.511431] niks3-server[1185]: 2026/09/23 04:44:24 INFO Completed upload id=717457worker1 # [ 172.473957] nix-grpc-daemon[1820]: time=2026-09-23T04:44:23.725Z level=INFO msg="Upload complete. (210ms)"7458lb # [ 172.517598] niks3-server[1185]: 2026/09/23 04:44:24 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7459worker1 # [ 172.479728] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=5 outputs=17460worker2 # [ 172.523074] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:24Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17461worker1 # [ 172.496262] nix-daemon[847]: reaped child process 1920, status = succeeded7462worker2 # [ 172.533873] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:24Z level=debug event=want client=2 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07463worker1 # [ 172.530441] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327464worker1 # [ 172.542780] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327465worker1 # [ 172.554483] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327466worker1 # [ 172.562683] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7467worker1 # [ 172.663978] nix-daemon[847]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)7468worker1 # [ 172.676269] nix-daemon[847]: reaped child process 1948, status = succeeded7469lb # [ 172.722256] niks3-server[1185]: 2026/09/23 04:44:24 INFO Received uploads request method=POST path=/api/pending_closures7470worker1 # [ 172.692448] nix-grpc-daemon[1820]: time=2026-09-23T04:44:23.944Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7471worker1 # [ 172.693897] nix-grpc-daemon[1820]: time=2026-09-23T04:44:23.945Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7472lb # [ 172.754349] niks3-server[1185]: 2026/09/23 04:44:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7473worker1 # [ 172.722728] nix-grpc-daemon[1820]: time=2026-09-23T04:44:23.974Z level=INFO msg="Uploading 1 narinfos"7474lb # [ 172.760552] niks3-server[1185]: 2026/09/23 04:44:24 INFO Signed narinfos id=72 count=17475lb # [ 172.768920] niks3-server[1185]: 2026/09/23 04:44:24 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7476lb # [ 172.783660] niks3-server[1185]: 2026/09/23 04:44:24 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7477lb # [ 172.791923] niks3-server[1185]: 2026/09/23 04:44:24 INFO Completed upload id=727478worker1 # [ 172.754432] nix-grpc-daemon[1820]: time=2026-09-23T04:44:24.005Z level=INFO msg="Upload complete. (178ms)"7479lb # [ 172.795335] niks3-server[1185]: 2026/09/23 04:44:24 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7480worker1 # [ 172.758354] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=5047481lb # [ 172.800290] niks3-server[1185]: 2026/09/23 04:44:24 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7482worker1 # [ 172.765683] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327483worker1 # [ 172.775789] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7484worker1 # [ 172.778212] nix-daemon[847]: reaped child process 1943, status = succeeded7485worker1 # [ 172.783464] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7486worker1 # [ 174.218521] nix-daemon[847]: reaped child process 1952, status = succeeded7487worker1 # [ 174.305333] nix-daemon[847]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)7488worker1 # [ 174.317878] nix-daemon[847]: reaped child process 1971, status = succeeded7489lb # [ 174.365709] niks3-server[1185]: 2026/09/23 04:44:26 INFO Received uploads request method=POST path=/api/pending_closures7490worker1 # [ 174.335494] nix-grpc-daemon[1820]: time=2026-09-23T04:44:25.586Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7491worker1 # [ 174.337096] nix-grpc-daemon[1820]: time=2026-09-23T04:44:25.588Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7492lb # [ 174.407755] niks3-server[1185]: 2026/09/23 04:44:26 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7493lb # [ 174.416268] niks3-server[1185]: 2026/09/23 04:44:26 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7494lb # [ 174.424211] niks3-server[1185]: 2026/09/23 04:44:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7495worker1 # [ 174.393607] nix-grpc-daemon[1820]: time=2026-09-23T04:44:25.644Z level=INFO msg="Uploading 1 narinfos"7496lb # [ 174.430131] niks3-server[1185]: 2026/09/23 04:44:26 INFO Signed narinfos id=73 count=17497lb # [ 174.436451] niks3-server[1185]: 2026/09/23 04:44:26 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7498lb # [ 174.446457] niks3-server[1185]: 2026/09/23 04:44:26 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7499lb # [ 174.449486] niks3-server[1185]: 2026/09/23 04:44:26 INFO Completed upload id=737500worker1 # [ 174.412452] nix-grpc-daemon[1820]: time=2026-09-23T04:44:25.663Z level=INFO msg="Upload complete. (219ms)"7501lb # [ 174.452365] niks3-server[1185]: 2026/09/23 04:44:26 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7502worker1 # [ 174.418909] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=6 outputs=17503worker2 # [ 174.458877] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:26Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17504worker2 # [ 174.469382] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:26Z level=debug event=want client=2 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07505worker1 # [ 174.433551] nix-daemon[847]: reaped child process 1950, status = succeeded7506worker1 # [ 174.471271] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327507worker1 # [ 174.483471] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327508worker1 # [ 174.494873] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327509worker1 # [ 174.502993] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7510worker1 # [ 174.603219] nix-daemon[847]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)7511worker1 # [ 174.616702] nix-daemon[847]: reaped child process 1978, status = succeeded7512lb # [ 174.661576] niks3-server[1185]: 2026/09/23 04:44:26 INFO Received uploads request method=POST path=/api/pending_closures7513worker1 # [ 174.632654] nix-grpc-daemon[1820]: time=2026-09-23T04:44:25.884Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7514worker1 # [ 174.634221] nix-grpc-daemon[1820]: time=2026-09-23T04:44:25.885Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7515lb # [ 174.692231] niks3-server[1185]: 2026/09/23 04:44:26 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7516lb # [ 174.699237] niks3-server[1185]: 2026/09/23 04:44:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7517lb # [ 174.703549] niks3-server[1185]: 2026/09/23 04:44:26 INFO Signed narinfos id=74 count=17518worker1 # [ 174.669176] nix-grpc-daemon[1820]: time=2026-09-23T04:44:25.920Z level=INFO msg="Uploading 1 narinfos"7519lb # [ 174.718778] niks3-server[1185]: 2026/09/23 04:44:26 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7520lb # [ 174.728521] niks3-server[1185]: 2026/09/23 04:44:26 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7521lb # [ 174.734705] niks3-server[1185]: 2026/09/23 04:44:26 INFO Completed upload id=747522worker1 # [ 174.697526] nix-grpc-daemon[1820]: time=2026-09-23T04:44:25.949Z level=INFO msg="Upload complete. (181ms)"7523lb # [ 174.737813] niks3-server[1185]: 2026/09/23 04:44:26 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7524worker1 # [ 174.701262] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=5047525worker1 # [ 174.705847] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327526worker1 # [ 174.714698] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7527worker1 # [ 174.723594] nix-daemon[847]: reaped child process 1973, status = succeeded7528worker1 # [ 174.728307] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7529worker1 # [ 175.742280] nix-daemon[847]: reaped child process 1982, status = succeeded7530worker1 # [ 175.818520] nix-daemon[847]: accepted connection from pid 1998, user nix-grpc-daemon (trusted)7531worker1 # [ 175.831873] nix-daemon[847]: reaped child process 2001, status = succeeded7532lb # [ 175.877644] niks3-server[1185]: 2026/09/23 04:44:27 INFO Received uploads request method=POST path=/api/pending_closures7533worker1 # [ 175.850293] nix-grpc-daemon[1820]: time=2026-09-23T04:44:27.100Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7534worker1 # [ 175.851661] nix-grpc-daemon[1820]: time=2026-09-23T04:44:27.100Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7535lb # [ 175.923153] niks3-server[1185]: 2026/09/23 04:44:27 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7536lb # [ 175.929938] niks3-server[1185]: 2026/09/23 04:44:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7537lb # [ 175.938364] niks3-server[1185]: 2026/09/23 04:44:27 INFO Signed narinfos id=75 count=17538worker1 # [ 175.904634] nix-grpc-daemon[1820]: time=2026-09-23T04:44:27.156Z level=INFO msg="Uploading 1 narinfos"7539lb # [ 175.946522] niks3-server[1185]: 2026/09/23 04:44:27 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7540lb # [ 175.951179] niks3-server[1185]: 2026/09/23 04:44:27 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7541lb # [ 175.961084] niks3-server[1185]: 2026/09/23 04:44:27 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7542lb # [ 175.965932] niks3-server[1185]: 2026/09/23 04:44:27 INFO Completed upload id=757543worker1 # [ 175.928723] nix-grpc-daemon[1820]: time=2026-09-23T04:44:27.180Z level=INFO msg="Upload complete. (193ms)"7544lb # [ 175.967680] niks3-server[1185]: 2026/09/23 04:44:27 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7545worker1 # [ 175.934863] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=7 outputs=17546worker2 # [ 175.975713] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:28Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17547worker1 # [ 175.943774] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327548worker2 # [ 175.985948] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:28Z level=debug event=want client=2 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07549worker1 # [ 175.954507] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327550worker1 # [ 175.960393] nix-daemon[847]: reaped child process 1980, status = succeeded7551worker1 # [ 175.965790] nix-grpc-daemon[1820]: time=2026-09-23T04:44:27.217Z level=INFO msg="All 1 paths already cached"7552worker1 # [ 175.972192] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327553worker1 # [ 175.979961] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7554worker1 # [ 176.064296] nix-daemon[847]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)7555worker1 # [ 176.078595] nix-daemon[847]: reaped child process 2008, status = succeeded7556lb # [ 176.121566] niks3-server[1185]: 2026/09/23 04:44:28 INFO Received uploads request method=POST path=/api/pending_closures7557worker1 # [ 176.092487] nix-grpc-daemon[1820]: time=2026-09-23T04:44:27.344Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7558worker1 # [ 176.094461] nix-grpc-daemon[1820]: time=2026-09-23T04:44:27.345Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7559lb # [ 176.151369] niks3-server[1185]: 2026/09/23 04:44:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7560lb # [ 176.158342] niks3-server[1185]: 2026/09/23 04:44:28 INFO Signed narinfos id=76 count=17561worker1 # [ 176.124377] nix-grpc-daemon[1820]: time=2026-09-23T04:44:27.375Z level=INFO msg="Uploading 1 narinfos"7562lb # [ 176.168134] niks3-server[1185]: 2026/09/23 04:44:28 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7563lb # [ 176.173182] niks3-server[1185]: 2026/09/23 04:44:28 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7564lb # [ 176.185114] niks3-server[1185]: 2026/09/23 04:44:28 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7565lb # [ 176.190692] niks3-server[1185]: 2026/09/23 04:44:28 INFO Completed upload id=767566worker1 # [ 176.153564] nix-grpc-daemon[1820]: time=2026-09-23T04:44:27.405Z level=INFO msg="Upload complete. (168ms)"7567lb # [ 176.193158] niks3-server[1185]: 2026/09/23 04:44:28 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7568worker1 # [ 176.158093] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=6247569worker1 # [ 176.164197] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327570worker1 # [ 176.172729] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7571worker1 # [ 176.180204] nix-daemon[847]: reaped child process 2003, status = succeeded7572worker1 # [ 176.185331] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7573worker1 # [ 177.223424] nix-daemon[847]: reaped child process 2012, status = succeeded7574worker1 # [ 177.306668] nix-daemon[847]: accepted connection from pid 2028, user nix-grpc-daemon (trusted)7575worker1 # [ 177.319525] nix-daemon[847]: reaped child process 2031, status = succeeded7576lb # [ 177.365829] niks3-server[1185]: 2026/09/23 04:44:29 INFO Received uploads request method=POST path=/api/pending_closures7577worker1 # [ 177.339776] nix-grpc-daemon[1820]: time=2026-09-23T04:44:28.591Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7578worker1 # [ 177.341348] nix-grpc-daemon[1820]: time=2026-09-23T04:44:28.592Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7579lb # [ 177.405803] niks3-server[1185]: 2026/09/23 04:44:29 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7580lb # [ 177.414722] envoy[926]: conn peer=[2001:db8:1::1]:51070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9591 tx=5163 ms=49207581lb # [ 177.420871] niks3-server[1185]: 2026/09/23 04:44:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7582lb # [ 177.426514] niks3-server[1185]: 2026/09/23 04:44:29 INFO Signed narinfos id=77 count=17583worker1 # [ 177.391845] nix-grpc-daemon[1820]: time=2026-09-23T04:44:28.643Z level=INFO msg="Uploading 1 narinfos"7584lb # [ 177.433970] niks3-server[1185]: 2026/09/23 04:44:29 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7585lb # [ 177.439950] niks3-server[1185]: 2026/09/23 04:44:29 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7586lb # [ 177.446847] niks3-server[1185]: 2026/09/23 04:44:29 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7587lb # [ 177.451556] niks3-server[1185]: 2026/09/23 04:44:29 INFO Completed upload id=777588worker1 # [ 177.414322] nix-grpc-daemon[1820]: time=2026-09-23T04:44:28.665Z level=INFO msg="Upload complete. (222ms)"7589lb # [ 177.453593] niks3-server[1185]: 2026/09/23 04:44:29 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7590worker1 # [ 177.420870] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=8 outputs=17591worker2 # [ 177.461586] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:29Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17592worker1 # [ 177.430148] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327593worker2 # [ 177.471699] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:29Z level=debug event=want client=2 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07594worker1 # [ 177.442533] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327595worker1 # [ 177.449516] nix-daemon[847]: reaped child process 2010, status = succeeded7596worker1 # [ 177.452232] nix-grpc-daemon[1820]: time=2026-09-23T04:44:28.703Z level=INFO msg="All 1 paths already cached"7597worker1 # [ 177.458222] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327598worker1 # [ 177.466495] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7599worker1 # [ 177.551769] nix-daemon[847]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)7600worker1 # [ 177.565796] nix-daemon[847]: reaped child process 2038, status = succeeded7601lb # [ 177.608645] niks3-server[1185]: 2026/09/23 04:44:29 INFO Received uploads request method=POST path=/api/pending_closures7602worker1 # [ 177.579408] nix-grpc-daemon[1820]: time=2026-09-23T04:44:28.830Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7603worker1 # [ 177.580846] nix-grpc-daemon[1820]: time=2026-09-23T04:44:28.832Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7604lb # [ 177.640593] niks3-server[1185]: 2026/09/23 04:44:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7605worker1 # [ 177.609525] nix-grpc-daemon[1820]: time=2026-09-23T04:44:28.860Z level=INFO msg="Uploading 1 narinfos"7606lb # [ 177.646484] niks3-server[1185]: 2026/09/23 04:44:29 INFO Signed narinfos id=78 count=17607lb # [ 177.653925] niks3-server[1185]: 2026/09/23 04:44:29 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7608lb # [ 177.666138] niks3-server[1185]: 2026/09/23 04:44:29 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7609lb # [ 177.672946] niks3-server[1185]: 2026/09/23 04:44:29 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7610lb # [ 177.678978] niks3-server[1185]: 2026/09/23 04:44:29 INFO Completed upload id=787611worker1 # [ 177.641958] nix-grpc-daemon[1820]: time=2026-09-23T04:44:28.893Z level=INFO msg="Upload complete. (170ms)"7612lb # [ 177.681787] niks3-server[1185]: 2026/09/23 04:44:29 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7613worker1 # [ 177.645984] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=6247614worker1 # [ 177.650929] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327615worker1 # [ 177.658550] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7616worker1 # [ 177.667256] nix-daemon[847]: reaped child process 2033, status = succeeded7617worker1 # [ 177.672254] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7618worker1 # [ 178.914714] nix-daemon[847]: reaped child process 2042, status = succeeded7619worker1 # [ 178.989770] nix-daemon[847]: accepted connection from pid 2058, user nix-grpc-daemon (trusted)7620worker1 # [ 179.002548] nix-daemon[847]: reaped child process 2061, status = succeeded7621lb # [ 179.049528] niks3-server[1185]: 2026/09/23 04:44:31 INFO Received uploads request method=POST path=/api/pending_closures7622worker1 # [ 179.019167] nix-grpc-daemon[1820]: time=2026-09-23T04:44:30.270Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7623worker1 # [ 179.020877] nix-grpc-daemon[1820]: time=2026-09-23T04:44:30.272Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7624lb # [ 179.092252] niks3-server[1185]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7625lb # [ 179.099271] niks3-server[1185]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7626lb # [ 179.105514] niks3-server[1185]: 2026/09/23 04:44:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7627worker1 # [ 179.074207] nix-grpc-daemon[1820]: time=2026-09-23T04:44:30.325Z level=INFO msg="Uploading 1 narinfos"7628lb # [ 179.111443] niks3-server[1185]: 2026/09/23 04:44:31 INFO Signed narinfos id=79 count=17629lb # [ 179.117662] niks3-server[1185]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7630lb # [ 179.125093] niks3-server[1185]: 2026/09/23 04:44:31 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7631lb # [ 179.127971] niks3-server[1185]: 2026/09/23 04:44:31 INFO Completed upload id=797632worker1 # [ 179.090690] nix-grpc-daemon[1820]: time=2026-09-23T04:44:30.342Z level=INFO msg="Upload complete. (210ms)"7633lb # [ 179.130614] niks3-server[1185]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7634worker2 # [ 179.137680] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:31Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17635worker1 # [ 179.096921] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=9 outputs=17636worker1 # [ 179.107474] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327637worker2 # [ 179.148352] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:31Z level=debug event=want client=2 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07638worker1 # [ 179.116716] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327639worker1 # [ 179.125324] nix-daemon[847]: reaped child process 2040, status = succeeded7640worker1 # [ 179.127267] nix-grpc-daemon[1820]: time=2026-09-23T04:44:30.378Z level=INFO msg="All 1 paths already cached"7641worker1 # [ 179.131378] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327642worker1 # [ 179.136196] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7643worker1 # [ 179.222354] nix-daemon[847]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)7644worker1 # [ 179.236221] nix-daemon[847]: reaped child process 2068, status = succeeded7645lb # [ 179.280465] niks3-server[1185]: 2026/09/23 04:44:31 INFO Received uploads request method=POST path=/api/pending_closures7646worker1 # [ 179.251388] nix-grpc-daemon[1820]: time=2026-09-23T04:44:30.502Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7647worker1 # [ 179.253656] nix-grpc-daemon[1820]: time=2026-09-23T04:44:30.504Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7648lb # [ 179.314445] niks3-server[1185]: 2026/09/23 04:44:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7649worker1 # [ 179.281972] nix-grpc-daemon[1820]: time=2026-09-23T04:44:30.533Z level=INFO msg="Uploading 1 narinfos"7650lb # [ 179.320622] niks3-server[1185]: 2026/09/23 04:44:31 INFO Signed narinfos id=80 count=17651lb # [ 179.326979] niks3-server[1185]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7652lb # [ 179.331602] niks3-server[1185]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7653lb # [ 179.345130] niks3-server[1185]: 2026/09/23 04:44:31 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7654lb # [ 179.349978] niks3-server[1185]: 2026/09/23 04:44:31 INFO Completed upload id=807655worker1 # [ 179.314797] nix-grpc-daemon[1820]: time=2026-09-23T04:44:30.566Z level=INFO msg="Upload complete. (173ms)"7656lb # [ 179.356093] niks3-server[1185]: 2026/09/23 04:44:31 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7657worker1 # [ 179.319172] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=6247658worker1 # [ 179.323935] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327659worker1 # [ 179.335375] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7660worker1 # [ 179.340771] nix-daemon[847]: reaped child process 2063, status = succeeded7661worker1 # [ 179.343692] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7662worker1 # [ 180.783947] nix-daemon[847]: reaped child process 2072, status = succeeded7663worker1 # [ 180.867471] nix-daemon[847]: accepted connection from pid 2088, user nix-grpc-daemon (trusted)7664worker1 # [ 180.878678] nix-daemon[847]: reaped child process 2091, status = succeeded7665lb # [ 180.925379] niks3-server[1185]: 2026/09/23 04:44:32 INFO Received uploads request method=POST path=/api/pending_closures7666worker1 # [ 180.896917] nix-grpc-daemon[1820]: time=2026-09-23T04:44:32.148Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7667worker1 # [ 180.898593] nix-grpc-daemon[1820]: time=2026-09-23T04:44:32.150Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7668lb # [ 180.969089] niks3-server[1185]: 2026/09/23 04:44:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7669lb # [ 180.975343] niks3-server[1185]: 2026/09/23 04:44:32 INFO Signed narinfos id=81 count=17670worker1 # [ 180.940993] nix-grpc-daemon[1820]: time=2026-09-23T04:44:32.192Z level=INFO msg="Uploading 1 narinfos"7671lb # [ 180.980749] niks3-server[1185]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7672lb # [ 180.988932] niks3-server[1185]: 2026/09/23 04:44:32 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7673lb # [ 180.997053] niks3-server[1185]: 2026/09/23 04:44:33 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7674lb # [ 181.005251] niks3-server[1185]: 2026/09/23 04:44:33 INFO Completed upload id=817675worker1 # [ 180.967837] nix-grpc-daemon[1820]: time=2026-09-23T04:44:32.219Z level=INFO msg="Upload complete. (215ms)"7676lb # [ 181.007575] niks3-server[1185]: 2026/09/23 04:44:33 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7677lb # [ 181.010654] niks3-server[1185]: 2026/09/23 04:44:33 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7678worker1 # [ 180.973786] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=10 outputs=17679worker2 # [ 181.017721] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:33Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17680worker1 # [ 180.983445] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327681worker2 # [ 181.026849] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:33Z level=debug event=want client=2 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07682worker1 # [ 180.993372] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327683worker1 # [ 181.002173] nix-daemon[847]: reaped child process 2070, status = succeeded7684worker1 # [ 181.004331] nix-grpc-daemon[1820]: time=2026-09-23T04:44:32.255Z level=INFO msg="All 2 paths already cached"7685worker1 # [ 181.009027] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327686worker1 # [ 181.014391] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7687worker1 # [ 181.098075] nix-daemon[847]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)7688worker1 # [ 181.112163] nix-daemon[847]: reaped child process 2098, status = succeeded7689lb # [ 181.157335] niks3-server[1185]: 2026/09/23 04:44:33 INFO Received uploads request method=POST path=/api/pending_closures7690worker1 # [ 181.128317] nix-grpc-daemon[1820]: time=2026-09-23T04:44:32.379Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7691worker1 # [ 181.130236] nix-grpc-daemon[1820]: time=2026-09-23T04:44:32.381Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7692lb # [ 181.187352] niks3-server[1185]: 2026/09/23 04:44:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7693worker1 # [ 181.160170] nix-grpc-daemon[1820]: time=2026-09-23T04:44:32.411Z level=INFO msg="Uploading 1 narinfos"7694lb # [ 181.198410] niks3-server[1185]: 2026/09/23 04:44:33 INFO Signed narinfos id=82 count=17695lb # [ 181.204612] niks3-server[1185]: 2026/09/23 04:44:33 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7696lb # [ 181.210052] niks3-server[1185]: 2026/09/23 04:44:33 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7697lb # [ 181.222073] niks3-server[1185]: 2026/09/23 04:44:33 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7698lb # [ 181.227839] niks3-server[1185]: 2026/09/23 04:44:33 INFO Completed upload id=827699worker1 # [ 181.190576] nix-grpc-daemon[1820]: time=2026-09-23T04:44:32.442Z level=INFO msg="Upload complete. (171ms)"7700lb # [ 181.231425] niks3-server[1185]: 2026/09/23 04:44:33 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7701worker1 # [ 181.193066] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=7527702worker1 # [ 181.198326] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327703worker1 # [ 181.210245] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7704worker1 # [ 181.213391] nix-daemon[847]: reaped child process 2093, status = succeeded7705worker1 # [ 181.219356] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7706worker1 # [ 182.033099] nix-daemon[847]: reaped child process 2102, status = succeeded7707worker1 # [ 182.109324] nix-daemon[847]: accepted connection from pid 2118, user nix-grpc-daemon (trusted)7708worker1 # [ 182.124320] nix-daemon[847]: reaped child process 2121, status = succeeded7709lb # [ 182.168431] niks3-server[1185]: 2026/09/23 04:44:34 INFO Received uploads request method=POST path=/api/pending_closures7710worker1 # [ 182.139499] nix-grpc-daemon[1820]: time=2026-09-23T04:44:33.390Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7711worker1 # [ 182.141044] nix-grpc-daemon[1820]: time=2026-09-23T04:44:33.392Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7712lb # [ 182.212102] niks3-server[1185]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7713lb # [ 182.216568] niks3-server[1185]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7714lb # [ 182.224402] niks3-server[1185]: 2026/09/23 04:44:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7715worker1 # [ 182.192281] nix-grpc-daemon[1820]: time=2026-09-23T04:44:33.443Z level=INFO msg="Uploading 1 narinfos"7716lb # [ 182.229548] niks3-server[1185]: 2026/09/23 04:44:34 INFO Signed narinfos id=83 count=17717lb # [ 182.233661] niks3-server[1185]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7718lb # [ 182.242876] niks3-server[1185]: 2026/09/23 04:44:34 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7719lb # [ 182.245571] niks3-server[1185]: 2026/09/23 04:44:34 INFO Completed upload id=837720lb # [ 182.248548] niks3-server[1185]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7721worker1 # [ 182.209748] nix-grpc-daemon[1820]: time=2026-09-23T04:44:33.461Z level=INFO msg="Upload complete. (184ms)"7722worker1 # [ 182.215988] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=11 outputs=17723worker2 # [ 182.256664] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:34Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17724worker1 # [ 182.229922] nix-daemon[847]: reaped child process 2100, status = succeeded7725worker1 # [ 182.268410] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327726worker1 # [ 182.280966] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327727worker1 # [ 182.294544] nix-grpc-daemon[1820]: time=2026-09-23T04:44:33.545Z level=INFO msg="All 2 paths already cached"7728worker1 # [ 182.302967] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327729worker1 # [ 182.311289] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7730worker1 # [ 182.408910] nix-daemon[847]: accepted connection from pid 2125, user nix-grpc-daemon (trusted)7731worker1 # [ 182.421503] nix-daemon[847]: reaped child process 2128, status = succeeded7732lb # [ 182.467722] niks3-server[1185]: 2026/09/23 04:44:34 INFO Received uploads request method=POST path=/api/pending_closures7733worker1 # [ 182.438853] nix-grpc-daemon[1820]: time=2026-09-23T04:44:33.690Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7734worker1 # [ 182.440820] nix-grpc-daemon[1820]: time=2026-09-23T04:44:33.691Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7735lb # [ 182.500150] niks3-server[1185]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7736lb # [ 182.503663] niks3-server[1185]: 2026/09/23 04:44:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7737lb # [ 182.509260] niks3-server[1185]: 2026/09/23 04:44:34 INFO Signed narinfos id=84 count=17738worker1 # [ 182.474947] nix-grpc-daemon[1820]: time=2026-09-23T04:44:33.726Z level=INFO msg="Uploading 1 narinfos"7739lb # [ 182.522292] niks3-server[1185]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7740lb # [ 182.529218] niks3-server[1185]: 2026/09/23 04:44:34 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7741worker1 # [ 182.497309] nix-grpc-daemon[1820]: time=2026-09-23T04:44:33.748Z level=INFO msg="Upload complete. (172ms)"7742lb # [ 182.534726] niks3-server[1185]: 2026/09/23 04:44:34 INFO Completed upload id=847743worker1 # [ 182.500652] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=7527744lb # [ 182.542134] niks3-server[1185]: 2026/09/23 04:44:34 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7745worker1 # [ 182.506213] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327746worker1 # [ 182.514770] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7747worker1 # [ 182.524123] nix-daemon[847]: reaped child process 2123, status = succeeded7748worker1 # [ 182.527826] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7749worker1 # [ 183.967241] nix-daemon[847]: reaped child process 2132, status = succeeded7750worker1 # [ 184.051309] nix-daemon[847]: accepted connection from pid 2148, user nix-grpc-daemon (trusted)7751worker1 # [ 184.063404] nix-daemon[847]: reaped child process 2151, status = succeeded7752lb # [ 184.108835] niks3-server[1185]: 2026/09/23 04:44:36 INFO Received uploads request method=POST path=/api/pending_closures7753worker1 # [ 184.084282] nix-grpc-daemon[1820]: time=2026-09-23T04:44:35.335Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7754worker1 # [ 184.086032] nix-grpc-daemon[1820]: time=2026-09-23T04:44:35.337Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7755lb # [ 184.154179] niks3-server[1185]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7756lb # [ 184.161361] niks3-server[1185]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7757lb # [ 184.167637] niks3-server[1185]: 2026/09/23 04:44:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7758lb # [ 184.171493] niks3-server[1185]: 2026/09/23 04:44:36 INFO Signed narinfos id=85 count=17759worker1 # [ 184.134351] nix-grpc-daemon[1820]: time=2026-09-23T04:44:35.385Z level=INFO msg="Uploading 1 narinfos"7760lb # [ 184.177516] niks3-server[1185]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7761lb # [ 184.182355] niks3-server[1185]: 2026/09/23 04:44:36 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7762lb # [ 184.186626] niks3-server[1185]: 2026/09/23 04:44:36 INFO Completed upload id=857763worker1 # [ 184.149204] nix-grpc-daemon[1820]: time=2026-09-23T04:44:35.400Z level=INFO msg="Upload complete. (213ms)"7764lb # [ 184.188546] niks3-server[1185]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7765worker1 # [ 184.155069] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=12 outputs=17766worker2 # [ 184.195556] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:36Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17767worker1 # [ 184.169759] nix-daemon[847]: reaped child process 2130, status = succeeded7768worker1 # [ 184.206875] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327769worker1 # [ 184.218856] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550327770worker1 # [ 184.231431] nix-grpc-daemon[1820]: time=2026-09-23T04:44:35.482Z level=INFO msg="All 2 paths already cached"7771worker1 # [ 184.239260] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550327772worker1 # [ 184.249179] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7773worker1 # [ 184.344681] nix-daemon[847]: accepted connection from pid 2155, user nix-grpc-daemon (trusted)7774worker1 # [ 184.358293] nix-daemon[847]: reaped child process 2158, status = succeeded7775lb # [ 184.402303] niks3-server[1185]: 2026/09/23 04:44:36 INFO Received uploads request method=POST path=/api/pending_closures7776worker1 # [ 184.373559] nix-grpc-daemon[1820]: time=2026-09-23T04:44:35.625Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7777worker1 # [ 184.375047] nix-grpc-daemon[1820]: time=2026-09-23T04:44:35.626Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7778lb # [ 184.437417] niks3-server[1185]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7779lb # [ 184.444129] niks3-server[1185]: 2026/09/23 04:44:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7780lb # [ 184.447988] niks3-server[1185]: 2026/09/23 04:44:36 INFO Signed narinfos id=86 count=17781worker1 # [ 184.413175] nix-grpc-daemon[1820]: time=2026-09-23T04:44:35.664Z level=INFO msg="Uploading 1 narinfos"7782lb # [ 184.454157] niks3-server[1185]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7783lb # [ 184.462881] niks3-server[1185]: 2026/09/23 04:44:36 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7784lb # [ 184.466217] niks3-server[1185]: 2026/09/23 04:44:36 INFO Completed upload id=867785worker1 # [ 184.428919] nix-grpc-daemon[1820]: time=2026-09-23T04:44:35.680Z level=INFO msg="Upload complete. (168ms)"7786lb # [ 184.469060] niks3-server[1185]: 2026/09/23 04:44:36 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7787worker1 # [ 184.432240] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=0 paths=1 nar_bytes_in=7527788worker1 # [ 184.437129] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550327789worker1 # [ 184.446370] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7790worker1 # [ 184.451749] nix-daemon[847]: reaped child process 2153, status = succeeded7791worker1 # [ 184.456446] nix-daemon[847]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)7792worker1 # [ 185.271162] nix-daemon[847]: reaped child process 2162, status = succeeded7793worker1 # [ 185.347076] nix-daemon[847]: accepted connection from pid 2178, user nix-grpc-daemon (trusted)7794worker1 # [ 185.360330] nix-daemon[847]: reaped child process 2181, status = succeeded7795lb # [ 185.405665] niks3-server[1185]: 2026/09/23 04:44:37 INFO Received uploads request method=POST path=/api/pending_closures7796worker1 # [ 185.377215] nix-grpc-daemon[1820]: time=2026-09-23T04:44:36.628Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7797worker1 # [ 185.378947] nix-grpc-daemon[1820]: time=2026-09-23T04:44:36.630Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7798lb # [ 185.446796] niks3-server[1185]: 2026/09/23 04:44:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7799lb # [ 185.454581] niks3-server[1185]: 2026/09/23 04:44:37 INFO Signed narinfos id=87 count=17800worker1 # [ 185.420202] nix-grpc-daemon[1820]: time=2026-09-23T04:44:36.671Z level=INFO msg="Uploading 1 narinfos"7801lb # [ 185.459228] niks3-server[1185]: 2026/09/23 04:44:37 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7802lb # [ 185.465144] niks3-server[1185]: 2026/09/23 04:44:37 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7803lb # [ 185.469537] niks3-server[1185]: 2026/09/23 04:44:37 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7804lb # [ 185.475974] niks3-server[1185]: 2026/09/23 04:44:37 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7805lb # [ 185.479566] niks3-server[1185]: 2026/09/23 04:44:37 INFO Completed upload id=877806lb # [ 185.481935] niks3-server[1185]: 2026/09/23 04:44:37 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7807worker1 # [ 185.443936] nix-grpc-daemon[1820]: time=2026-09-23T04:44:36.695Z level=INFO msg="Upload complete. (180ms)"7808worker1 # [ 185.450347] nix-grpc-daemon[1759]: ts=2026-09-23T04:44:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55032 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=13 outputs=17809worker1 # [ 185.462598] nix-daemon[847]: reaped child process 2160, status = succeeded7810client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.76 seconds)7811worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7812worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7813client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7814client # [ 185.610953] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7815client # Running as unit: gpu.service; invocation ID: 8ebd390a9e1e450da0ae6f48f597a14b7816client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7817lb: must succeed: curl -sf localhost:9901/clusters7818lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7819lb: must succeed: curl -sf localhost:9901/clusters7820lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7821worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7822worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7823client # [ 185.750468] nix[1582]: don't know how to build these paths:7824client # [ 185.751413] nix[1582]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7825worker2 # [ 185.759922] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587367826worker2 # [ 185.765638] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587367827client # [ 185.776871] nix[1582]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7828worker2 # [ 185.786725] nix-grpc-daemon[1606]: ts=2026-09-23T04:44:37Z level=debug event=want client=3 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07829lb: must succeed: curl -sf localhost:9901/clusters7830lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7831lb: must succeed: curl -sf localhost:9901/clusters7832lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7833worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7834worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7835client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7836lb # [ 187.413635] envoy[926]: conn peer=[2001:db8:1::1]:51080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27146 tx=12321 ms=145837837client # [ 215.779597] nix[1582]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7838client # [ 215.787841] nix[1582]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7839client # [ 215.820355] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7840client # [ 215.824340] systemd[1]: gpu.service: Failed with result 'exit-code'.7841client # [ 215.829187] systemd[1]: gpu.service: Consumed 147ms CPU time over 30.213s wall clock time, 35.6M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7842client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.48 seconds)7843client: must succeed: journalctl -u gpu -o cat7844client: (finished: must succeed: journalctl -u gpu -o cat, in 0.01 seconds)7845lb: must succeed: curl -sf localhost:9901/clusters7846lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7847lb: must succeed: curl -sf localhost:9901/clusters7848lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7849worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7850worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7851(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.72 seconds)7852subtest: low disk drains a worker and builds go to the other7853worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7854lb # [ 217.416992] envoy[926]: conn peer=[2001:db8:1::1]:56886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300537855worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.01 seconds)7856worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7857worker1 # [ 218.215986] nix-grpc-daemon[1759]: ts=2026-09-23T04:45:09Z level=info event=unhealthy reason=min_free7858worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.07 seconds)7859client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27860worker1 # [ 218.694681] nix-grpc-daemon[1759]: ts=2026-09-23T04:45:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550327861client # don't know how to build these paths:7862client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7863worker2 # [ 218.744556] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587367864worker2 # [ 218.766841] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:10Z level=debug event=want client=4 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07865worker2 # [ 218.772842] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:10Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17866worker2 # [ 218.812171] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367867worker2 # [ 218.825715] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587367868client # copying 1 paths to 'lb:50051'...7869client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7870worker2 # [ 218.839338] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587367871worker2 # [ 218.851111] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7872worker2 # [ 218.949934] nix-daemon[874]: accepted connection from pid 1986, user nix-grpc-daemon (trusted)7873worker2 # [ 218.963654] nix-daemon[874]: reaped child process 1989, status = succeeded7874lb # [ 218.970448] niks3-server[1185]: 2026/09/23 04:45:10 INFO Received uploads request method=POST path=/api/pending_closures7875worker2 # [ 218.982262] nix-grpc-daemon[1699]: time=2026-09-23T04:45:11.066Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7876worker2 # [ 218.984534] nix-grpc-daemon[1699]: time=2026-09-23T04:45:11.068Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7877lb # [ 219.006587] niks3-server[1185]: 2026/09/23 04:45:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7878worker2 # [ 219.018052] nix-grpc-daemon[1699]: time=2026-09-23T04:45:11.102Z level=INFO msg="Uploading 1 narinfos"7879lb # [ 219.015945] niks3-server[1185]: 2026/09/23 04:45:11 INFO Signed narinfos id=88 count=17880lb # [ 219.025410] niks3-server[1185]: 2026/09/23 04:45:11 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7881lb # [ 219.029251] niks3-server[1185]: 2026/09/23 04:45:11 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7882lb # [ 219.041300] niks3-server[1185]: 2026/09/23 04:45:11 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7883lb # [ 219.044144] niks3-server[1185]: 2026/09/23 04:45:11 INFO Completed upload id=887884worker2 # [ 219.046209] nix-grpc-daemon[1699]: time=2026-09-23T04:45:11.130Z level=INFO msg="Upload complete. (182ms)"7885lb # [ 219.046962] niks3-server[1185]: 2026/09/23 04:45:11 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7886worker2 # [ 219.050677] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=0 paths=1 nar_bytes_in=5127887worker2 # [ 219.056306] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367888worker2 # [ 219.067588] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7889worker2 # [ 219.071799] nix-daemon[874]: reaped child process 1984, status = succeeded7890worker2 # [ 219.076625] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7891client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7892client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7893client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7894client # farm-a> LOG-farm-a-drain7895worker2 # [ 220.093267] nix-daemon[874]: reaped child process 1993, status = succeeded7896worker2 # [ 220.170827] nix-daemon[874]: accepted connection from pid 2009, user nix-grpc-daemon (trusted)7897worker2 # [ 220.183031] nix-daemon[874]: reaped child process 2012, status = succeeded7898lb # [ 220.189930] niks3-server[1185]: 2026/09/23 04:45:12 INFO Received uploads request method=POST path=/api/pending_closures7899worker2 # [ 220.207371] nix-grpc-daemon[1699]: time=2026-09-23T04:45:12.291Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7900worker2 # [ 220.209240] nix-grpc-daemon[1699]: time=2026-09-23T04:45:12.293Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7901lb # [ 220.240314] niks3-server[1185]: 2026/09/23 04:45:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7902lb # [ 220.245799] niks3-server[1185]: 2026/09/23 04:45:12 INFO Signed narinfos id=89 count=17903worker2 # [ 220.253535] nix-grpc-daemon[1699]: time=2026-09-23T04:45:12.337Z level=INFO msg="Uploading 1 narinfos"7904lb # [ 220.252502] niks3-server[1185]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7905lb # [ 220.256787] niks3-server[1185]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7906lb # [ 220.263196] niks3-server[1185]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7907lb # [ 220.268927] niks3-server[1185]: 2026/09/23 04:45:12 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7908lb # [ 220.273258] niks3-server[1185]: 2026/09/23 04:45:12 INFO Completed upload id=897909worker2 # [ 220.275108] nix-grpc-daemon[1699]: time=2026-09-23T04:45:12.359Z level=INFO msg="Upload complete. (191ms)"7910lb # [ 220.275151] niks3-server[1185]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7911worker2 # [ 220.282490] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=14 outputs=17912worker2 # [ 220.294975] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:12Z level=debug event=want client=4 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07913worker2 # [ 220.299205] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:12Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17914worker2 # [ 220.309211] nix-daemon[874]: reaped child process 1991, status = succeeded7915worker2 # [ 220.338548] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367916worker2 # [ 220.351657] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587367917worker2 # [ 220.366263] nix-grpc-daemon[1699]: time=2026-09-23T04:45:12.450Z level=INFO msg="All 1 paths already cached"7918client # copying 1 paths to 'lb:50051'...7919client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7920worker2 # [ 220.375358] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587367921worker2 # [ 220.385785] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7922worker2 # [ 220.484454] nix-daemon[874]: accepted connection from pid 2018, user nix-grpc-daemon (trusted)7923worker2 # [ 220.498636] nix-daemon[874]: reaped child process 2021, status = succeeded7924lb # [ 220.503160] niks3-server[1185]: 2026/09/23 04:45:12 INFO Received uploads request method=POST path=/api/pending_closures7925worker2 # [ 220.513845] nix-grpc-daemon[1699]: time=2026-09-23T04:45:12.598Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7926worker2 # [ 220.515789] nix-grpc-daemon[1699]: time=2026-09-23T04:45:12.599Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7927lb # [ 220.533225] niks3-server[1185]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7928lb # [ 220.540434] niks3-server[1185]: 2026/09/23 04:45:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7929lb # [ 220.547269] niks3-server[1185]: 2026/09/23 04:45:12 INFO Signed narinfos id=90 count=17930worker2 # [ 220.552100] nix-grpc-daemon[1699]: time=2026-09-23T04:45:12.636Z level=INFO msg="Uploading 1 narinfos"7931lb # [ 220.555303] niks3-server[1185]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7932lb # [ 220.567949] niks3-server[1185]: 2026/09/23 04:45:12 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7933lb # [ 220.573312] niks3-server[1185]: 2026/09/23 04:45:12 INFO Completed upload id=907934worker2 # [ 220.575325] nix-grpc-daemon[1699]: time=2026-09-23T04:45:12.659Z level=INFO msg="Upload complete. (176ms)"7935lb # [ 220.577800] niks3-server[1185]: 2026/09/23 04:45:12 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7936worker2 # [ 220.579947] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=0 paths=1 nar_bytes_in=6247937worker2 # [ 220.586453] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367938worker2 # [ 220.596714] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7939worker2 # [ 220.605970] nix-daemon[874]: reaped child process 2016, status = succeeded7940worker2 # [ 220.610213] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7941client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7942client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7943client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7944client # farm-b> LOG-farm-b-drain7945worker2 # [ 221.438551] nix-daemon[874]: reaped child process 2025, status = succeeded7946worker2 # [ 221.511677] nix-daemon[874]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)7947worker2 # [ 221.525350] nix-daemon[874]: reaped child process 2044, status = succeeded7948lb # [ 221.531376] niks3-server[1185]: 2026/09/23 04:45:13 INFO Received uploads request method=POST path=/api/pending_closures7949worker2 # [ 221.541942] nix-grpc-daemon[1699]: time=2026-09-23T04:45:13.626Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7950worker2 # [ 221.544058] nix-grpc-daemon[1699]: time=2026-09-23T04:45:13.628Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7951lb # [ 221.576075] niks3-server[1185]: 2026/09/23 04:45:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7952worker2 # [ 221.583752] nix-grpc-daemon[1699]: time=2026-09-23T04:45:13.668Z level=INFO msg="Uploading 1 narinfos"7953lb # [ 221.582200] niks3-server[1185]: 2026/09/23 04:45:13 INFO Signed narinfos id=91 count=17954lb # [ 221.593604] niks3-server[1185]: 2026/09/23 04:45:13 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7955lb # [ 221.597985] niks3-server[1185]: 2026/09/23 04:45:13 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7956lb # [ 221.602397] niks3-server[1185]: 2026/09/23 04:45:13 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7957lb # [ 221.611952] niks3-server[1185]: 2026/09/23 04:45:13 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7958lb # [ 221.616347] niks3-server[1185]: 2026/09/23 04:45:13 INFO Completed upload id=917959worker2 # [ 221.618512] nix-grpc-daemon[1699]: time=2026-09-23T04:45:13.702Z level=INFO msg="Upload complete. (201ms)"7960lb # [ 221.619181] niks3-server[1185]: 2026/09/23 04:45:13 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7961worker2 # [ 221.625618] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=15 outputs=17962worker2 # [ 221.638623] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:13Z level=debug event=want client=4 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07963worker2 # [ 221.642493] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:13Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17964worker2 # [ 221.652726] nix-daemon[874]: reaped child process 2023, status = succeeded7965worker2 # [ 221.682235] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367966worker2 # [ 221.695124] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587367967worker2 # [ 221.709305] nix-grpc-daemon[1699]: time=2026-09-23T04:45:13.793Z level=INFO msg="All 2 paths already cached"7968client # copying 1 paths to 'lb:50051'...7969client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7970worker2 # [ 221.720593] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587367971worker2 # [ 221.731178] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7972worker2 # [ 221.829562] nix-daemon[874]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)7973worker2 # [ 221.844117] nix-daemon[874]: reaped child process 2053, status = succeeded7974lb # [ 221.849075] niks3-server[1185]: 2026/09/23 04:45:13 INFO Received uploads request method=POST path=/api/pending_closures7975worker2 # [ 221.859508] nix-grpc-daemon[1699]: time=2026-09-23T04:45:13.943Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7976worker2 # [ 221.861026] nix-grpc-daemon[1699]: time=2026-09-23T04:45:13.945Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7977lb # [ 221.883920] niks3-server[1185]: 2026/09/23 04:45:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7978worker2 # [ 221.895519] nix-grpc-daemon[1699]: time=2026-09-23T04:45:13.979Z level=INFO msg="Uploading 1 narinfos"7979lb # [ 221.894254] niks3-server[1185]: 2026/09/23 04:45:13 INFO Signed narinfos id=92 count=17980lb # [ 221.898924] niks3-server[1185]: 2026/09/23 04:45:13 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7981lb # [ 221.903636] niks3-server[1185]: 2026/09/23 04:45:13 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7982lb # [ 221.915483] niks3-server[1185]: 2026/09/23 04:45:13 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7983lb # [ 221.921835] niks3-server[1185]: 2026/09/23 04:45:13 INFO Completed upload id=927984worker2 # [ 221.923056] nix-grpc-daemon[1699]: time=2026-09-23T04:45:14.007Z level=INFO msg="Upload complete. (180ms)"7985lb # [ 221.926714] niks3-server[1185]: 2026/09/23 04:45:13 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7986worker2 # [ 221.927631] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=0 paths=1 nar_bytes_in=7527987worker2 # [ 221.934315] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587367988worker2 # [ 221.947198] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7989worker2 # [ 221.951827] nix-daemon[874]: reaped child process 2048, status = succeeded7990worker2 # [ 221.957194] nix-daemon[874]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)7991client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7992client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7993client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7994client # farm-top> LOG-farm-top-drain7995worker2 # [ 222.803801] nix-daemon[874]: reaped child process 2057, status = succeeded7996worker2 # [ 222.885094] nix-daemon[874]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)7997worker2 # [ 222.899102] nix-daemon[874]: reaped child process 2077, status = succeeded7998lb # [ 222.903281] niks3-server[1185]: 2026/09/23 04:45:14 INFO Received uploads request method=POST path=/api/pending_closures7999worker2 # [ 222.914779] nix-grpc-daemon[1699]: time=2026-09-23T04:45:14.999Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"8000worker2 # [ 222.917722] nix-grpc-daemon[1699]: time=2026-09-23T04:45:15.000Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"8001lb # [ 222.945554] niks3-server[1185]: 2026/09/23 04:45:14 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv8002lb # [ 222.951122] niks3-server[1185]: 2026/09/23 04:45:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign8003worker2 # [ 222.958553] nix-grpc-daemon[1699]: time=2026-09-23T04:45:15.042Z level=INFO msg="Uploading 1 narinfos"8004lb # [ 222.956316] niks3-server[1185]: 2026/09/23 04:45:14 INFO Signed narinfos id=93 count=18005lb # [ 222.963465] niks3-server[1185]: 2026/09/23 04:45:14 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls8006lb # [ 222.974149] niks3-server[1185]: 2026/09/23 04:45:14 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete8007lb # [ 222.976376] niks3-server[1185]: 2026/09/23 04:45:14 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst8008lb # [ 222.980456] niks3-server[1185]: 2026/09/23 04:45:14 INFO Completed upload id=938009worker2 # [ 222.982267] nix-grpc-daemon[1699]: time=2026-09-23T04:45:15.066Z level=INFO msg="Upload complete. (216ms)"8010lb # [ 222.982964] niks3-server[1185]: 2026/09/23 04:45:14 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo8011worker2 # [ 222.989944] nix-grpc-daemon[1606]: ts=2026-09-23T04:45:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58736 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=16 outputs=18012worker2 # [ 223.005099] nix-daemon[874]: reaped child process 2055, status = succeeded8013client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.45 seconds)8014client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath8015client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)8016worker1: must succeed: rm /nix/.rw-store/fill8017worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)8018worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'8019worker1 # [ 224.218509] nix-grpc-daemon[1759]: ts=2026-09-23T04:45:15Z level=info event=healthy reason=min_free8020worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)8021(finished: subtest: low disk drains a worker and builds go to the other, in 7.90 seconds)8022(finished: run the VM test script, in 225.38 seconds)8023test script finished in 225.56s8024cleanup8025kill QemuMachine (pid 50)8026client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)8027client # [2026-09-23T04:45:17Z INFO virtiofsd] Client disconnected, shutting down8028client # [2026-09-23T04:45:17Z INFO virtiofsd] Client disconnected, shutting down8029client # [2026-09-23T04:45:17Z INFO virtiofsd] Client disconnected, shutting down8030kill QemuMachine (pid 49)8031lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)8032lb # [2026-09-23T04:45:17Z INFO virtiofsd] Client disconnected, shutting down8033lb # [2026-09-23T04:45:17Z INFO virtiofsd] Client disconnected, shutting down8034lb # [2026-09-23T04:45:17Z INFO virtiofsd] Client disconnected, shutting down8035kill QemuMachine (pid 51)8036worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)8037worker1 # [2026-09-23T04:45:17Z INFO virtiofsd] Client disconnected, shutting down8038worker1 # [2026-09-23T04:45:17Z INFO virtiofsd] Client disconnected, shutting down8039worker1 # [2026-09-23T04:45:17Z INFO virtiofsd] Client disconnected, shutting down8040kill QemuMachine (pid 52)8041worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)8042worker2 # [2026-09-23T04:45:18Z INFO virtiofsd] Client disconnected, shutting down8043worker2 # [2026-09-23T04:45:18Z INFO virtiofsd] Client disconnected, shutting down8044worker2 # [2026-09-23T04:45:18Z INFO virtiofsd] Client disconnected, shutting down8045(finished: cleanup, in 1.05 seconds)8046additionally exposed symbols:8047 client, lb, worker1, worker2,8048 vlan1,8049 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh