vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #509
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1: QEMU running (pid 50)14worker1 # Formatting '/build/vm-state-worker1/tmp.2X1EX33DPL', fmt=raw size=107374182415worker1 # mke2fs 1.47.4 (6-Mar-2025)16worker1 # Discarding device blocks: 0/262144 done17worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes18worker1 # Filesystem UUID: 3a6caa4c-790b-4a67-be89-148a89a204bb19worker1 # Superblock backups stored on blocks:20worker1 # 32768, 98304, 163840, 22937621worker1 # 22worker1 # Allocating group tables: 0/8 done23worker1 # Writing inode tables: 0/8 done24worker1 # Creating journal (8192 blocks): done25worker1 # Writing superblocks and filesystem accounting information: 0/8 done26worker1 # 27worker1 # Virtualisation disk image created.28worker1 # Starting virtiofs daemons...29worker1 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)30worker1 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether31worker1 # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...32worker1 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)33worker1 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether34worker1 # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...35worker1 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)36worker1 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether37worker1 # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...38worker1 # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests39worker1 # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests40worker1 # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests41client # Disk image does not exist, creating the virtualisation disk image...42client: QEMU running (pid 49)43client # Formatting '/build/vm-state-client/tmp.ScHHbpzcTU', fmt=raw size=107374182444client # mke2fs 1.47.4 (6-Mar-2025)45client # Discarding device blocks: 0/262144 done46client # Creating filesystem with 262144 4k blocks and 65536 inodes47client # Filesystem UUID: f6a9df58-7426-410f-8e3b-2351ab2cda0a48client # Superblock backups stored on blocks:49client # 32768, 98304, 163840, 22937650client # 51client # Allocating group tables: 0/8 done52client # Writing inode tables: 0/8 done53client # Creating journal (8192 blocks): done54client # Writing superblocks and filesystem accounting information: 0/8 done55client # 56client # Virtualisation disk image created.57client # Starting virtiofs daemons...58client # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)59client # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether60client # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)61client # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether62client # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...63client # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...64client # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)65client # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether66client # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...67client # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests68worker2 # Disk image does not exist, creating the virtualisation disk image...69client # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests70worker2 # Formatting '/build/vm-state-worker2/tmp.Vbj5EST1cc', fmt=raw size=107374182471client # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests72worker2 # mke2fs 1.47.4 (6-Mar-2025)73worker2: QEMU running (pid 52)74worker2 # Discarding device blocks: 0/262144 done75worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes76worker2 # Filesystem UUID: 7289fd16-8454-424a-8852-6402c3e1b0df77worker2 # Superblock backups stored on blocks:78worker2 # 32768, 98304, 163840, 22937679worker2 # 80worker2 # Allocating group tables: 0/8 done81worker2 # Writing inode tables: 0/8 done82worker2 # Creating journal (8192 blocks): done83worker2 # Writing superblocks and filesystem accounting information: 0/8 done84worker2 # 85worker2 # Virtualisation disk image created.86worker2 # Starting virtiofs daemons...87worker2 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88worker2 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...90worker2 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91worker2 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker2 # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker2 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94worker2 # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95worker2 # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...96worker2 # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests97worker2 # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests98worker2 # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests99lb: QEMU running (pid 51)100lb # Disk image does not exist, creating the virtualisation disk image...101lb # Formatting '/build/vm-state-lb/tmp.lo0Rm0gzOa', fmt=raw size=1073741824102lb # mke2fs 1.47.4 (6-Mar-2025)103lb # Discarding device blocks: 0/262144 done104lb # Creating filesystem with 262144 4k blocks and 65536 inodes105lb # Filesystem UUID: 95deba15-7376-41e2-870f-88e303c7f063106lb # Superblock backups stored on blocks:107lb # 32768, 98304, 163840, 229376108lb # 109lb # Allocating group tables: 0/8 done110lb # Writing inode tables: 0/8 done111lb # Creating journal (8192 blocks): done112lb # Writing superblocks and filesystem accounting information: 0/8 done113(finished: start all VMs, in 0.24 seconds)114lb # 115client: waiting for unit network-addresses-eth1.service116lb # Virtualisation disk image created.117client: waiting for the VM to finish booting118lb # Starting virtiofs daemons...119lb # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...122lb # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123lb # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124lb # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-22T22:20:13Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-22T22:20:13Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-22T22:20:13Z INFO virtiofsd] Client connected, servicing requests131client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135client # 136client # 137client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139client # 140client # 141client # 142client # 143client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00144client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...145client # 146client # 147worker2 # 148worker2 # 149lb # 150lb # 151worker1 # 152worker1 # 153worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00154lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00155worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00156worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...157worker2 # 158worker2 # 159worker2 # 160worker2 # 161lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...162lb # 163lb # 164lb # 165lb # 166worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...167worker1 # 168worker1 # 169worker1 # 170worker1 # 171worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00172lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00173worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00174worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...175worker2 # 176worker2 # 177lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178lb # 179lb # 180worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker1 # 182worker1 # 183client # Booting from ROM...184worker2 # Booting from ROM...185lb # Booting from ROM...186worker1 # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zpiq30cb38yj5xq9czkzgv1bvg949hyw-nixos-system-client-test/init regInfo=/nix/store/9lyia4mvv2i1s4lyj2244g6qjddq61k6-closure-info/registration console=ttyS0,115200n8 console=tty0189client # [ 0.000000] BIOS-provided physical RAM map:190client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200client # [ 0.000000] NX (Execute Disable) protection: active201client # [ 0.000000] APIC: Static calls initialized202client # [ 0.000000] SMBIOS 2.8 present.203client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014204client # [ 0.000000] DMI: Memory slots populated: 1/1205client # [ 0.000000] Hypervisor detected: KVM206client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000207client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00208client # [ 0.000001] kvm-clock: using sched offset of 477485373 cycles209client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns210client # [ 0.000005] tsc: Detected 2400.012 MHz processor211client # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000212client # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs213client # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT214client # [ 0.002722] found SMP MP-table at [mem 0x000f5450-0x000f545f]215client # [ 0.002733] Using GB pages for direct mapping216client # [ 0.002882] RAMDISK: [mem 0x5e370000-0x5ffcffff]217client # [ 0.002891] ACPI: Early table checksum verification disabled218client # [ 0.002894] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )219client # [ 0.002898] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)220client # [ 0.002902] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)221client # [ 0.002909] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)222client # [ 0.002911] ACPI: FACS 0x000000005FFE0000 000040223client # [ 0.002912] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)224client # [ 0.002914] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)225client # [ 0.002915] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)226client # [ 0.002917] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)227worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026228client # [ 0.002918] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]229lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026230client # [ 0.002919] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]231client # [ 0.002920] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]232client # [ 0.002920] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]233worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026234worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xnscg6z9qxa84gbw1h8qb1f0c0aarhsv-nixos-system-worker2-test/init regInfo=/nix/store/l9mgb5s68rv42f8xbdmqfv6fgq35z9b7-closure-info/registration console=ttyS0,115200n8 console=tty0235client # [ 0.002920] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]236worker2 # [ 0.000000] BIOS-provided physical RAM map:237client # [ 0.002921] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]238lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0239worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable240lb # [ 0.000000] BIOS-provided physical RAM map:241client # [ 0.002921] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]242worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved243client # [ 0.003160] No NUMA configuration found244lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable245worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved246client # [ 0.003161] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]247lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved248worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4idnay9kf4i79rgksq2a0cx0x5kksln8-nixos-system-worker1-test/init regInfo=/nix/store/hnxmbc46xwssrsa5c66xddlq152f9x0y-closure-info/registration console=ttyS0,115200n8 console=tty0249worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable250client # [ 0.003164] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]251worker1 # [ 0.000000] BIOS-provided physical RAM map:252lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved253client # [ 0.005553] Zone ranges:254worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved255client # [ 0.005554] DMA [mem 0x0000000000001000-0x0000000000ffffff]256worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable257lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable258worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved259client # [ 0.005555] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]260worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved261lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved262client # [ 0.005557] Normal empty263worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved264client # [ 0.005558] Device empty265worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved266lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved267client # [ 0.005558] Movable zone start for each node268worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved269client # [ 0.005559] Early memory node ranges270worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable271lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved272worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved273client # [ 0.005559] node 0: [mem 0x0000000000001000-0x000000000009efff]274worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved275lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved276client # [ 0.005561] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]277worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved278worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved279lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved280worker2 # [ 0.000000] NX (Execute Disable) protection: active281client # [ 0.005562] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]282worker2 # [ 0.000000] APIC: Static calls initialized283worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved284lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved285worker2 # [ 0.000000] SMBIOS 2.8 present.286client # [ 0.005586] On node 0, zone DMA: 1 pages in unavailable ranges287lb # [ 0.000000] NX (Execute Disable) protection: active288worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved289client # [ 0.005859] On node 0, zone DMA: 97 pages in unavailable ranges290lb # [ 0.000000] APIC: Static calls initialized291worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014292lb # [ 0.000000] SMBIOS 2.8 present.293worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved294client # [ 0.033872] On node 0, zone DMA32: 40 pages in unavailable ranges295worker2 # [ 0.000000] DMI: Memory slots populated: 1/1296client # [ 0.034337] ACPI: PM-Timer IO Port: 0x608297worker2 # [ 0.000000] Hypervisor detected: KVM298worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved299lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014300client # [ 0.034349] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])301worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000302worker1 # [ 0.000000] NX (Execute Disable) protection: active303lb # [ 0.000000] DMI: Memory slots populated: 1/1304worker1 # [ 0.000000] APIC: Static calls initialized305lb # [ 0.000000] Hypervisor detected: KVM306worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00307client # [ 0.034377] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23308worker1 # [ 0.000000] SMBIOS 2.8 present.309lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000310worker2 # [ 0.000001] kvm-clock: using sched offset of 514210300 cycles311client # [ 0.034380] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)312lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00313worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014314client # [ 0.034381] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)315worker1 # [ 0.000000] DMI: Memory slots populated: 1/1316worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns317lb # [ 0.000001] kvm-clock: using sched offset of 496687857 cycles318client # [ 0.034382] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)319worker1 # [ 0.000000] Hypervisor detected: KVM320worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor321worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000322worker2 # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000323client # [ 0.034383] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)324lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns325lb # [ 0.000004] tsc: Detected 2400.012 MHz processor326worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00327client # [ 0.034384] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)328worker2 # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs329worker1 # [ 0.000001] kvm-clock: using sched offset of 504887046 cycles330lb # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000331client # [ 0.034386] ACPI: Using ACPI (MADT) for SMP configuration information332worker2 # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT333client # [ 0.034387] ACPI: HPET id: 0x8086a201 base: 0xfed00000334lb # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs335worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns336worker2 # [ 0.002727] found SMP MP-table at [mem 0x000f5450-0x000f545f]337client # [ 0.034391] TSC deadline timer available338worker2 # [ 0.002738] Using GB pages for direct mapping339worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor340lb # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT341client # [ 0.034395] CPU topo: Max. logical packages: 1342worker2 # [ 0.002903] RAMDISK: [mem 0x5e36f000-0x5ffcffff]343worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000344client # [ 0.034396] CPU topo: Max. logical dies: 1345lb # [ 0.002724] found SMP MP-table at [mem 0x000f5450-0x000f545f]346worker2 # [ 0.002914] ACPI: Early table checksum verification disabled347lb # [ 0.002735] Using GB pages for direct mapping348client # [ 0.034396] CPU topo: Max. dies per package: 1349worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs350client # [ 0.034399] CPU topo: Max. threads per core: 1351lb # [ 0.002839] RAMDISK: [mem 0x5e36e000-0x5ffcffff]352worker2 # [ 0.002917] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )353client # [ 0.034400] CPU topo: Num. cores per package: 1354worker1 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT355lb # [ 0.002848] ACPI: Early table checksum verification disabled356worker2 # [ 0.002921] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)357client # [ 0.034400] CPU topo: Num. threads per package: 1358worker1 # [ 0.002722] found SMP MP-table at [mem 0x000f5450-0x000f545f]359lb # [ 0.002851] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )360worker1 # [ 0.002732] Using GB pages for direct mapping361client # [ 0.034400] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs362worker2 # [ 0.002925] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)363lb # [ 0.002855] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)364worker1 # [ 0.002828] RAMDISK: [mem 0x5e370000-0x5ffcffff]365client # [ 0.034419] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()366worker1 # [ 0.002838] ACPI: Early table checksum verification disabled367worker2 # [ 0.002932] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)368lb # [ 0.002859] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)369worker2 # [ 0.002934] ACPI: FACS 0x000000005FFE0000 000040370client # [ 0.034451] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]371worker1 # [ 0.002841] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )372lb # [ 0.002866] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)373client # [ 0.034453] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]374worker2 # [ 0.002936] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)375worker1 # [ 0.002845] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)376lb # [ 0.002868] ACPI: FACS 0x000000005FFE0000 000040377client # [ 0.034455] [mem 0x60000000-0xafffffff] available for PCI devices378worker2 # [ 0.002937] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)379worker1 # [ 0.002849] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)380client # [ 0.034456] Booting paravirtualized kernel on KVM381lb # [ 0.002870] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)382worker2 # [ 0.002939] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)383worker1 # [ 0.002856] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)384lb # [ 0.002871] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)385client # [ 0.034459] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns386worker1 # [ 0.002858] ACPI: FACS 0x000000005FFE0000 000040387worker2 # [ 0.002940] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)388client # [ 0.038930] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1389lb # [ 0.002873] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)390worker1 # [ 0.002859] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)391worker2 # [ 0.002942] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]392client # [ 0.041499] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152393lb # [ 0.002874] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)394worker1 # [ 0.002861] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)395worker2 # [ 0.002943] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]396client # [ 0.041545] kvm-guest: PV spinlocks disabled, single CPU397lb # [ 0.002876] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]398worker2 # [ 0.002943] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]399worker1 # [ 0.002863] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)400lb # [ 0.002877] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]401worker2 # [ 0.002944] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]402worker1 # [ 0.002864] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)403lb # [ 0.002877] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]404worker2 # [ 0.002944] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]405worker1 # [ 0.002865] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]406lb # [ 0.002878] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]407worker2 # [ 0.002945] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]408client # [ 0.041546] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zpiq30cb38yj5xq9czkzgv1bvg949hyw-nixos-system-client-test/init regInfo=/nix/store/9lyia4mvv2i1s4lyj2244g6qjddq61k6-closure-info/registration console=ttyS0,115200n8 console=tty0409worker1 # [ 0.002866] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]410lb # [ 0.002878] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]411worker2 # [ 0.002945] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]412worker1 # [ 0.002867] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]413lb # [ 0.002879] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]414worker2 # [ 0.003179] No NUMA configuration found415lb # [ 0.002879] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]416client # [ 0.041642] Unknown kernel command line parameters "regInfo=/nix/store/9lyia4mvv2i1s4lyj2244g6qjddq61k6-closure-info/registration", will be passed to user space.417worker2 # [ 0.003180] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]418worker1 # [ 0.002867] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]419client # [ 0.041654] random: crng init done420lb # [ 0.003109] No NUMA configuration found421worker2 # [ 0.003182] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]422worker1 # [ 0.002868] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]423worker2 # [ 0.005606] Zone ranges:424lb # [ 0.003110] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]425client # [ 0.041655] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes426worker1 # [ 0.002868] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]427worker2 # [ 0.005607] DMA [mem 0x0000000000001000-0x0000000000ffffff]428lb # [ 0.003113] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]429client # [ 0.043895] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)430lb # [ 0.005897] Zone ranges:431worker2 # [ 0.005609] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]432worker1 # [ 0.002869] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]433worker2 # [ 0.005610] Normal empty434lb # [ 0.005898] DMA [mem 0x0000000000001000-0x0000000000ffffff]435worker1 # [ 0.003100] No NUMA configuration found436client # [ 0.045024] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)437worker2 # [ 0.005611] Device empty438client # [ 0.045057] Fallback order for Node 0: 0439worker2 # [ 0.005611] Movable zone start for each node440worker1 # [ 0.003101] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]441lb # [ 0.005900] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]442worker2 # [ 0.005612] Early memory node ranges443lb # [ 0.005901] Normal empty444client # [ 0.045060] Built 1 zonelists, mobility grouping on. Total pages: 393078445lb # [ 0.005902] Device empty446worker1 # [ 0.003103] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]447worker1 # [ 0.005513] Zone ranges:448client # [ 0.045061] Policy zone: DMA32449worker2 # [ 0.005612] node 0: [mem 0x0000000000001000-0x000000000009efff]450lb # [ 0.005903] Movable zone start for each node451lb # [ 0.005904] Early memory node ranges452worker1 # [ 0.005514] DMA [mem 0x0000000000001000-0x0000000000ffffff]453worker2 # [ 0.005613] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]454client # [ 0.047733] mem auto-init: stack:all(zero), heap alloc:on, heap free:off455lb # [ 0.005904] node 0: [mem 0x0000000000001000-0x000000000009efff]456client # [ 0.051018] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1457worker1 # [ 0.005516] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]458worker2 # [ 0.005614] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]459worker1 # [ 0.005517] Normal empty460lb # [ 0.005905] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]461client # [ 0.054806] allocated 3145728 bytes of page_ext462worker1 # [ 0.005518] Device empty463worker2 # [ 0.005634] On node 0, zone DMA: 1 pages in unavailable ranges464client # [ 0.065119] ftrace: allocating 48787 entries in 192 pages465worker1 # [ 0.005519] Movable zone start for each node466lb # [ 0.005906] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]467worker2 # [ 0.005914] On node 0, zone DMA: 97 pages in unavailable ranges468worker1 # [ 0.005519] Early memory node ranges469client # [ 0.065121] ftrace: allocated 192 pages with 2 groups470lb # [ 0.005926] On node 0, zone DMA: 1 pages in unavailable ranges471client # [ 0.065997] Dynamic Preempt: lazy472worker2 # [ 0.033969] On node 0, zone DMA32: 40 pages in unavailable ranges473worker1 # [ 0.005520] node 0: [mem 0x0000000000001000-0x000000000009efff]474worker2 # [ 0.034424] ACPI: PM-Timer IO Port: 0x608475lb # [ 0.006204] On node 0, zone DMA: 97 pages in unavailable ranges476client # [ 0.066140] rcu: Preemptible hierarchical RCU implementation.477worker1 # [ 0.005521] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]478worker2 # [ 0.034436] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])479client # [ 0.066141] rcu: RCU event tracing is enabled.480lb # [ 0.034484] On node 0, zone DMA32: 40 pages in unavailable ranges481worker1 # [ 0.005522] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]482lb # [ 0.034938] ACPI: PM-Timer IO Port: 0x608483client # [ 0.066142] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.484worker2 # [ 0.034465] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23485lb # [ 0.034951] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])486worker1 # [ 0.005541] On node 0, zone DMA: 1 pages in unavailable ranges487client # [ 0.066143] Trampoline variant of Tasks RCU enabled.488worker2 # [ 0.034468] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)489client # [ 0.066144] Rude variant of Tasks RCU enabled.490lb # [ 0.034980] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23491worker1 # [ 0.005815] On node 0, zone DMA: 97 pages in unavailable ranges492worker2 # [ 0.034470] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)493client # [ 0.066144] Tracing variant of Tasks RCU enabled.494worker1 # [ 0.033868] On node 0, zone DMA32: 40 pages in unavailable ranges495lb # [ 0.034983] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)496worker2 # [ 0.034471] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)497worker1 # [ 0.034330] ACPI: PM-Timer IO Port: 0x608498client # [ 0.066145] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.499lb # [ 0.034984] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)500worker1 # [ 0.034341] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])501worker2 # [ 0.034472] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)502client # [ 0.066145] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1503lb # [ 0.034986] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)504worker1 # [ 0.034375] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23505worker2 # [ 0.034473] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)506lb # [ 0.034987] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)507client # [ 0.066206] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.508worker2 # [ 0.034475] ACPI: Using ACPI (MADT) for SMP configuration information509worker1 # [ 0.034377] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)510lb # [ 0.034987] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)511worker2 # [ 0.034476] ACPI: HPET id: 0x8086a201 base: 0xfed00000512worker1 # [ 0.034379] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)513client # [ 0.066208] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.514worker2 # [ 0.034480] TSC deadline timer available515lb # [ 0.034990] ACPI: Using ACPI (MADT) for SMP configuration information516worker1 # [ 0.034380] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)517worker2 # [ 0.034484] CPU topo: Max. logical packages: 1518lb # [ 0.034991] ACPI: HPET id: 0x8086a201 base: 0xfed00000519client # [ 0.066209] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.520worker2 # [ 0.034485] CPU topo: Max. logical dies: 1521worker1 # [ 0.034381] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)522lb # [ 0.034995] TSC deadline timer available523client # [ 0.070700] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16524worker2 # [ 0.034485] CPU topo: Max. dies per package: 1525lb # [ 0.034999] CPU topo: Max. logical packages: 1526worker1 # [ 0.034381] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)527worker2 # [ 0.034488] CPU topo: Max. threads per core: 1528lb # [ 0.034999] CPU topo: Max. logical dies: 1529client # [ 0.070993] rcu: srcu_init: Setting srcu_struct sizes based on contention.530worker1 # [ 0.034384] ACPI: Using ACPI (MADT) for SMP configuration information531worker2 # [ 0.034489] CPU topo: Num. cores per package: 1532lb # [ 0.035000] CPU topo: Max. dies per package: 1533worker2 # [ 0.034489] CPU topo: Num. threads per package: 1534client # [ 0.071000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns535worker1 # [ 0.034385] ACPI: HPET id: 0x8086a201 base: 0xfed00000536lb # [ 0.035003] CPU topo: Max. threads per core: 1537worker1 # [ 0.034389] TSC deadline timer available538lb # [ 0.035004] CPU topo: Num. cores per package: 1539worker2 # [ 0.034489] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs540worker1 # [ 0.034392] CPU topo: Max. logical packages: 1541client # [ 0.071111] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)542lb # [ 0.035004] CPU topo: Num. threads per package: 1543worker2 # [ 0.034508] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()544worker1 # [ 0.034393] CPU topo: Max. logical dies: 1545client # [ 0.074653] Console: colour VGA+ 80x25546lb # [ 0.035004] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs547worker1 # [ 0.034393] CPU topo: Max. dies per package: 1548client # [ 0.074656] printk: legacy console [tty0] enabled549worker2 # [ 0.034539] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]550worker1 # [ 0.034397] CPU topo: Max. threads per core: 1551lb # [ 0.035024] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()552client # [ 0.104089] printk: legacy console [ttyS0] enabled553worker1 # [ 0.034397] CPU topo: Num. cores per package: 1554worker2 # [ 0.034541] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]555client # [ 0.202545] ACPI: Core revision 20250807556lb # [ 0.035055] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]557worker1 # [ 0.034398] CPU topo: Num. threads per package: 1558worker2 # [ 0.034542] [mem 0x60000000-0xafffffff] available for PCI devices559lb # [ 0.035057] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]560worker1 # [ 0.034398] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs561worker2 # [ 0.034543] Booting paravirtualized kernel on KVM562client # [ 0.203426] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns563lb # [ 0.035059] [mem 0x60000000-0xafffffff] available for PCI devices564worker1 # [ 0.034416] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()565lb # [ 0.035060] Booting paravirtualized kernel on KVM566worker2 # [ 0.034546] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns567client # [ 0.204901] APIC: Switch to symmetric I/O mode setup568worker1 # [ 0.034447] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]569worker2 # [ 0.039023] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1570client # [ 0.205860] x2apic enabled571lb # [ 0.035063] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns572worker1 # [ 0.034449] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]573worker2 # [ 0.041296] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152574worker1 # [ 0.034450] [mem 0x60000000-0xafffffff] available for PCI devices575client # [ 0.206600] APIC: Switched APIC routing to: physical x2apic576lb # [ 0.039550] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1577worker2 # [ 0.041344] kvm-guest: PV spinlocks disabled, single CPU578worker1 # [ 0.034451] Booting paravirtualized kernel on KVM579lb # [ 0.041878] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152580lb # [ 0.041924] kvm-guest: PV spinlocks disabled, single CPU581worker1 # [ 0.034454] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns582client # [ 0.208384] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1583worker1 # [ 0.038948] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1584worker1 # [ 0.041225] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152585client # [ 0.209339] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns586worker2 # [ 0.041346] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/xnscg6z9qxa84gbw1h8qb1f0c0aarhsv-nixos-system-worker2-test/init regInfo=/nix/store/l9mgb5s68rv42f8xbdmqfv6fgq35z9b7-closure-info/registration console=ttyS0,115200n8 console=tty0587worker1 # [ 0.041274] kvm-guest: PV spinlocks disabled, single CPU588client # [ 0.210957] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)589lb # [ 0.041925] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ggsnp37phfgq4b11zirffk4vxgicg7h8-nixos-system-lb-test/init regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration console=ttyS0,115200n8 console=tty0590worker2 # [ 0.041440] Unknown kernel command line parameters "regInfo=/nix/store/l9mgb5s68rv42f8xbdmqfv6fgq35z9b7-closure-info/registration", will be passed to user space.591client # [ 0.212044] x86/cpu: User Mode Instruction Prevention (UMIP) activated592worker2 # [ 0.041452] random: crng init done593lb # [ 0.042020] Unknown kernel command line parameters "regInfo=/nix/store/94h72crvqzbw5sjsl7v7v31gd90khbjb-closure-info/registration", will be passed to user space.594worker2 # [ 0.041453] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes595client # [ 0.213086] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127596lb # [ 0.042033] random: crng init done597worker2 # [ 0.043684] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)598worker1 # [ 0.041276] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/4idnay9kf4i79rgksq2a0cx0x5kksln8-nixos-system-worker1-test/init regInfo=/nix/store/hnxmbc46xwssrsa5c66xddlq152f9x0y-closure-info/registration console=ttyS0,115200n8 console=tty0599lb # [ 0.042034] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes600client # [ 0.213953] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0601worker2 # [ 0.044800] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)602lb # [ 0.044294] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)603worker2 # [ 0.044832] Fallback order for Node 0: 0604worker1 # [ 0.041377] Unknown kernel command line parameters "regInfo=/nix/store/hnxmbc46xwssrsa5c66xddlq152f9x0y-closure-info/registration", will be passed to user space.605client # [ 0.214957] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto606lb # [ 0.045427] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)607worker2 # [ 0.044835] Built 1 zonelists, mobility grouping on. Total pages: 393078608worker1 # [ 0.041389] random: crng init done609worker2 # [ 0.044836] Policy zone: DMA32610lb # [ 0.045459] Fallback order for Node 0: 0611worker1 # [ 0.041389] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes612lb # [ 0.045463] Built 1 zonelists, mobility grouping on. Total pages: 393078613client # [ 0.215953] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl614worker2 # [ 0.047715] mem auto-init: stack:all(zero), heap alloc:on, heap free:off615lb # [ 0.045464] Policy zone: DMA32616worker1 # [ 0.043629] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)617worker2 # [ 0.051022] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1618client # [ 0.216954] Transient Scheduler Attacks: Vulnerable: No microcode619lb # [ 0.048411] mem auto-init: stack:all(zero), heap alloc:on, heap free:off620worker2 # [ 0.054514] allocated 3145728 bytes of page_ext621worker1 # [ 0.044777] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)622lb # [ 0.051716] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1623client # [ 0.217953] Spectre V2 : Mitigation: Enhanced / Automatic IBRS624worker1 # [ 0.044809] Fallback order for Node 0: 0625worker2 # [ 0.064701] ftrace: allocating 48787 entries in 192 pages626lb # [ 0.055237] allocated 3145728 bytes of page_ext627worker2 # [ 0.064704] ftrace: allocated 192 pages with 2 groups628worker1 # [ 0.044813] Built 1 zonelists, mobility grouping on. Total pages: 393078629client # [ 0.218953] Speculative Return Stack Overflow: Mitigation: Safe RET630worker2 # [ 0.065552] Dynamic Preempt: lazy631lb # [ 0.065521] ftrace: allocating 48787 entries in 192 pages632worker1 # [ 0.044814] Policy zone: DMA32633worker2 # [ 0.065701] rcu: Preemptible hierarchical RCU implementation.634lb # [ 0.065523] ftrace: allocated 192 pages with 2 groups635worker1 # [ 0.047770] mem auto-init: stack:all(zero), heap alloc:on, heap free:off636lb # [ 0.066389] Dynamic Preempt: lazy637worker2 # [ 0.065701] rcu: RCU event tracing is enabled.638worker1 # [ 0.051065] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1639lb # [ 0.066583] rcu: Preemptible hierarchical RCU implementation.640worker2 # [ 0.065702] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.641lb # [ 0.066584] rcu: RCU event tracing is enabled.642worker1 # [ 0.054537] allocated 3145728 bytes of page_ext643worker2 # [ 0.065703] Trampoline variant of Tasks RCU enabled.644worker1 # [ 0.064805] ftrace: allocating 48787 entries in 192 pages645worker2 # [ 0.065704] Rude variant of Tasks RCU enabled.646lb # [ 0.066584] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.647worker1 # [ 0.064808] ftrace: allocated 192 pages with 2 groups648worker2 # [ 0.065704] Tracing variant of Tasks RCU enabled.649lb # [ 0.066585] Trampoline variant of Tasks RCU enabled.650client # [ 0.219953] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization651worker1 # [ 0.065732] Dynamic Preempt: lazy652lb # [ 0.066586] Rude variant of Tasks RCU enabled.653worker2 # [ 0.065705] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.654client # [ 0.221961] Spectre V2 : Enabling IBPB for BPF655worker1 # [ 0.065906] rcu: Preemptible hierarchical RCU implementation.656lb # [ 0.066586] Tracing variant of Tasks RCU enabled.657worker1 # [ 0.065907] rcu: RCU event tracing is enabled.658worker2 # [ 0.065706] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1659lb # [ 0.066587] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.660client # [ 0.222657] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier661worker1 # [ 0.065907] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.662worker2 # [ 0.065716] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.663lb # [ 0.066588] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1664worker1 # [ 0.065909] Trampoline variant of Tasks RCU enabled.665client # [ 0.223954] active return thunk: srso_alias_return_thunk666worker1 # [ 0.065910] Rude variant of Tasks RCU enabled.667worker2 # [ 0.065718] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.668lb # [ 0.066615] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.669worker1 # [ 0.065910] Tracing variant of Tasks RCU enabled.670client # [ 0.224975] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'671worker2 # [ 0.065719] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.672lb # [ 0.066617] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.673worker1 # [ 0.065911] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.674client # [ 0.225953] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'675worker2 # [ 0.070210] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16676worker1 # [ 0.065911] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1677lb # [ 0.066617] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.678client # [ 0.226953] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'679worker2 # [ 0.070493] rcu: srcu_init: Setting srcu_struct sizes based on contention.680lb # [ 0.071109] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16681worker1 # [ 0.065922] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.682client # [ 0.227953] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'683lb # [ 0.071399] rcu: srcu_init: Setting srcu_struct sizes based on contention.684worker2 # [ 0.070501] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns685worker1 # [ 0.065923] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.686client # [ 0.228953] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'687lb # [ 0.071406] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns688worker2 # [ 0.070607] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)689worker1 # [ 0.065924] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.690worker2 # [ 0.074134] Console: colour VGA+ 80x25691client # [ 0.229953] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'692worker1 # [ 0.070329] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16693worker2 # [ 0.074137] printk: legacy console [tty0] enabled694lb # [ 0.071512] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)695lb # [ 0.075042] Console: colour VGA+ 80x25696worker2 # [ 0.103659] printk: legacy console [ttyS0] enabled697worker1 # [ 0.070621] rcu: srcu_init: Setting srcu_struct sizes based on contention.698client # [ 0.230953] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'699lb # [ 0.075045] printk: legacy console [tty0] enabled700worker2 # [ 0.198903] ACPI: Core revision 20250807701lb # [ 0.104512] printk: legacy console [ttyS0] enabled702client # [ 0.231953] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256703worker1 # [ 0.070628] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns704lb # [ 0.199482] ACPI: Core revision 20250807705client # [ 0.232953] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64706worker2 # [ 0.199766] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns707worker1 # [ 0.070732] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)708worker1 # [ 0.074267] Console: colour VGA+ 80x25709client # [ 0.233953] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512710worker2 # [ 0.201250] APIC: Switch to symmetric I/O mode setup711lb # [ 0.200383] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns712worker1 # [ 0.074270] printk: legacy console [tty0] enabled713worker2 # [ 0.202220] x2apic enabled714worker1 # [ 0.103721] printk: legacy console [ttyS0] enabled715client # [ 0.234953] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024716lb # [ 0.201861] APIC: Switch to symmetric I/O mode setup717worker1 # [ 0.199412] ACPI: Core revision 20250807718worker2 # [ 0.202952] APIC: Switched APIC routing to: physical x2apic719lb # [ 0.202846] x2apic enabled720client # [ 0.235953] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8721lb # [ 0.203617] APIC: Switched APIC routing to: physical x2apic722worker1 # [ 0.200310] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns723client # [ 0.236953] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.724worker2 # [ 0.204780] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1725worker1 # [ 0.201848] APIC: Switch to symmetric I/O mode setup726lb # [ 0.205427] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1727worker1 # [ 0.202824] x2apic enabled728worker2 # [ 0.205787] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns729worker1 # [ 0.203628] APIC: Switched APIC routing to: physical x2apic730lb # [ 0.206428] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns731worker2 # [ 0.207514] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)732lb # [ 0.208118] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)733worker1 # [ 0.205478] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1734worker2 # [ 0.208605] x86/cpu: User Mode Instruction Prevention (UMIP) activated735worker2 # [ 0.209638] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127736lb # [ 0.209204] x86/cpu: User Mode Instruction Prevention (UMIP) activated737worker1 # [ 0.206473] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns738worker2 # [ 0.211186] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0739lb # [ 0.210245] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127740worker1 # [ 0.208166] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)741lb # [ 0.211812] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0742worker2 # [ 0.211514] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto743worker1 # [ 0.209251] x86/cpu: User Mode Instruction Prevention (UMIP) activated744lb # [ 0.212119] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto745worker2 # [ 0.213510] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl746worker1 # [ 0.211217] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127747worker2 # [ 0.214511] Transient Scheduler Attacks: Vulnerable: No microcode748lb # [ 0.214115] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl749worker1 # [ 0.212162] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0750worker2 # [ 0.216245] Spectre V2 : Mitigation: Enhanced / Automatic IBRS751lb # [ 0.215116] Transient Scheduler Attacks: Vulnerable: No microcode752worker2 # [ 0.217193] Speculative Return Stack Overflow: Mitigation: Safe RET753lb # [ 0.216836] Spectre V2 : Mitigation: Enhanced / Automatic IBRS754worker1 # [ 0.213167] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto755lb # [ 0.217787] Speculative Return Stack Overflow: Mitigation: Safe RET756worker1 # [ 0.214163] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl757worker1 # [ 0.215163] Transient Scheduler Attacks: Vulnerable: No microcode758worker1 # [ 0.216162] Spectre V2 : Mitigation: Enhanced / Automatic IBRS759worker2 # [ 0.218236] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization760worker2 # [ 0.219517] Spectre V2 : Enabling IBPB for BPF761worker1 # [ 0.217162] Speculative Return Stack Overflow: Mitigation: Safe RET762lb # [ 0.218115] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization763lb # [ 0.220122] Spectre V2 : Enabling IBPB for BPF764worker2 # [ 0.220511] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier765worker2 # [ 0.221511] active return thunk: srso_alias_return_thunk766lb # [ 0.221116] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier767lb # [ 0.222115] active return thunk: srso_alias_return_thunk768worker2 # [ 0.223194] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'769worker1 # [ 0.218162] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization770lb # [ 0.223793] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'771worker2 # [ 0.224444] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'772worker1 # [ 0.220170] Spectre V2 : Enabling IBPB for BPF773lb # [ 0.224988] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'774worker2 # [ 0.225510] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'775worker1 # [ 0.221163] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier776lb # [ 0.225858] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'777worker2 # [ 0.226510] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'778worker1 # [ 0.222163] active return thunk: srso_alias_return_thunk779lb # [ 0.226841] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'780worker2 # [ 0.227480] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'781lb # [ 0.227872] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'782worker1 # [ 0.223184] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'783worker2 # [ 0.228510] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'784worker1 # [ 0.224162] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'785lb # [ 0.228874] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'786worker2 # [ 0.229510] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'787worker1 # [ 0.225945] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'788lb # [ 0.229904] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'789worker2 # [ 0.230511] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256790worker1 # [ 0.226913] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'791lb # [ 0.231115] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256792worker2 # [ 0.231510] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64793worker1 # [ 0.227925] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'794lb # [ 0.232115] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64795worker2 # [ 0.232510] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512796worker1 # [ 0.228921] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'797lb # [ 0.233115] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512798worker2 # [ 0.233510] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024799lb # [ 0.234115] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024800worker1 # [ 0.229958] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'801worker2 # [ 0.234510] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8802lb # [ 0.235115] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8803worker1 # [ 0.231163] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256804worker2 # [ 0.235510] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.805worker1 # [ 0.232162] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64806lb # [ 0.236115] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.807client # [ 0.272249] Freeing SMP alternatives memory: 44K808worker1 # [ 0.233162] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512809client # [ 0.272956] pid_max: default: 32768 minimum: 301810worker1 # [ 0.234116] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024811client # [ 0.273808] LSM: initializing lsm=capability,landlock,yama,bpf,ima812worker1 # [ 0.235162] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8813client # [ 0.275023] landlock: Up and running.814client # [ 0.275644] Yama: becoming mindful.815client # [ 0.276169] LSM support for eBPF active816worker1 # [ 0.236146] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.817client # [ 0.277068] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818client # [ 0.277994] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819client # [ 0.280952] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820client # [ 0.281530] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821client # [ 0.281958] ... version: 2822client # [ 0.282683] ... bit width: 48823client # [ 0.282955] ... generic counters: 6824client # [ 0.283663] ... generic bitmap: 000000000000003f825client # [ 0.283967] ... fixed-purpose counters: 0826client # [ 0.284691] ... fixed-purpose bitmap: 0000000000000000827client # [ 0.284955] ... value mask: 0000ffffffffffff828client # [ 0.285879] ... max period: 00007fffffffffff829client # [ 0.286666] ... global_ctrl mask: 000000000000003f830client # [ 0.287072] signal: max sigframe size: 3376831client # [ 0.287901] rcu: Hierarchical SRCU implementation.832client # [ 0.288577] rcu: Max phase no-delay instances is 400.833client # [ 0.293593] smp: Bringing up secondary CPUs ...834client # [ 0.293970] smp: Brought up 1 node, 1 CPU835client # [ 0.294656] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)836client # [ 0.295169] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)837client # [ 0.296230] devtmpfs: initialized838client # [ 0.297028] x86/mm: Memory block size: 128MB839worker2 # [ 0.270748] Freeing SMP alternatives memory: 44K840lb # [ 0.270804] Freeing SMP alternatives memory: 44K841worker2 # [ 0.271514] pid_max: default: 32768 minimum: 301842lb # [ 0.271119] pid_max: default: 32768 minimum: 301843client # [ 0.298855] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)844worker2 # [ 0.272340] LSM: initializing lsm=capability,landlock,yama,bpf,ima845lb # [ 0.272210] LSM: initializing lsm=capability,landlock,yama,bpf,ima846worker2 # [ 0.273342] landlock: Up and running.847client # [ 0.299796] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).848worker2 # [ 0.273934] Yama: becoming mindful.849lb # [ 0.273225] landlock: Up and running.850client # [ 0.301041] pinctrl core: initialized pinctrl subsystem851lb # [ 0.274115] Yama: becoming mindful.852worker2 # [ 0.274726] LSM support for eBPF active853lb # [ 0.275120] LSM support for eBPF active854client # [ 0.302115] PM: RTC time: 22:20:13, date: 2026-09-22855worker1 # [ 0.271515] Freeing SMP alternatives memory: 44K856worker2 # [ 0.275636] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)857lb # [ 0.275888] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)858worker1 # [ 0.272166] pid_max: default: 32768 minimum: 301859worker2 # [ 0.276559] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)860worker1 # [ 0.273227] LSM: initializing lsm=capability,landlock,yama,bpf,ima861lb # [ 0.277019] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)862worker1 # [ 0.274269] landlock: Up and running.863worker1 # [ 0.274913] Yama: becoming mindful.864client # [ 0.306004] NET: Registered PF_NETLINK/PF_ROUTE protocol family865worker1 # [ 0.275383] LSM support for eBPF active866worker2 # [ 0.279861] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)867lb # [ 0.279420] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)868client # [ 0.307460] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations869worker1 # [ 0.276288] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)870worker2 # [ 0.281085] Performance Events: Fam17h+ core perfctr, AMD PMU driver.871client # [ 0.307980] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations872lb # [ 0.281051] Performance Events: Fam17h+ core perfctr, AMD PMU driver.873worker1 # [ 0.277203] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)874worker2 # [ 0.281520] ... version: 2875lb # [ 0.281860] ... version: 2876worker2 # [ 0.282242] ... bit width: 48877lb # [ 0.282117] ... bit width: 48878client # [ 0.309246] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations879worker2 # [ 0.282513] ... generic counters: 6880lb # [ 0.282844] ... generic counters: 6881client # [ 0.309966] audit: initializing netlink subsys (disabled)882worker2 # [ 0.283224] ... generic bitmap: 000000000000003f883worker1 # [ 0.280212] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)884lb # [ 0.283127] ... generic bitmap: 000000000000003f885worker2 # [ 0.283513] ... fixed-purpose counters: 0886client # [ 0.311088] thermal_sys: Registered thermal governor 'fair_share'887lb # [ 0.283966] ... fixed-purpose counters: 0888worker2 # [ 0.284220] ... fixed-purpose bitmap: 0000000000000000889worker1 # [ 0.281736] Performance Events: Fam17h+ core perfctr, AMD PMU driver.890client # [ 0.311090] thermal_sys: Registered thermal governor 'bang_bang'891lb # [ 0.284642] ... fixed-purpose bitmap: 0000000000000000892worker1 # [ 0.282171] ... version: 2893worker2 # [ 0.284513] ... value mask: 0000ffffffffffff894client # [ 0.311957] thermal_sys: Registered thermal governor 'step_wise'895lb # [ 0.285117] ... value mask: 0000ffffffffffff896worker1 # [ 0.282844] ... bit width: 48897worker2 # [ 0.285364] ... max period: 00007fffffffffff898lb # [ 0.285992] ... max period: 00007fffffffffff899client # [ 0.312843] thermal_sys: Registered thermal governor 'user_space'900worker1 # [ 0.283165] ... generic counters: 6901worker2 # [ 0.286145] ... global_ctrl mask: 000000000000003f902lb # [ 0.286777] ... global_ctrl mask: 000000000000003f903worker1 # [ 0.283874] ... generic bitmap: 000000000000003f904client # [ 0.313648] audit: type=2000 audit(1790115614.504:1): state=initialized audit_enabled=0 res=1905worker2 # [ 0.286625] signal: max sigframe size: 3376906lb # [ 0.287230] signal: max sigframe size: 3376907worker1 # [ 0.284164] ... fixed-purpose counters: 0908client # [ 0.314958] thermal_sys: Registered thermal governor 'power_allocator'909worker2 # [ 0.287453] rcu: Hierarchical SRCU implementation.910lb # [ 0.288041] rcu: Hierarchical SRCU implementation.911worker1 # [ 0.284848] ... fixed-purpose bitmap: 0000000000000000912client # [ 0.314972] cpuidle: using governor menu913worker2 # [ 0.288098] rcu: Max phase no-delay instances is 400.914lb # [ 0.288661] rcu: Max phase no-delay instances is 400.915worker1 # [ 0.285165] ... value mask: 0000ffffffffffff916worker1 # [ 0.286052] ... max period: 00007fffffffffff917client # [ 0.316948] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5918worker1 # [ 0.286870] ... global_ctrl mask: 000000000000003f919client # [ 0.317970] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]920worker1 # [ 0.287279] signal: max sigframe size: 3376921worker1 # [ 0.288119] rcu: Hierarchical SRCU implementation.922client # [ 0.318957] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry923worker1 # [ 0.288788] rcu: Max phase no-delay instances is 400.924client # [ 0.319967] PCI: Using configuration type 1 for base access925worker2 # [ 0.293197] smp: Bringing up secondary CPUs ...926lb # [ 0.293776] smp: Bringing up secondary CPUs ...927worker2 # [ 0.293526] smp: Brought up 1 node, 1 CPU928lb # [ 0.294131] smp: Brought up 1 node, 1 CPU929client # [ 0.321084] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.930worker2 # [ 0.294204] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)931lb # [ 0.294784] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)932worker2 # [ 0.294692] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)933lb # [ 0.295270] Memory: 1454556K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)934worker1 # [ 0.293843] smp: Bringing up secondary CPUs ...935worker2 # [ 0.295787] devtmpfs: initialized936lb # [ 0.296383] devtmpfs: initialized937worker1 # [ 0.294179] smp: Brought up 1 node, 1 CPU938worker2 # [ 0.296573] x86/mm: Memory block size: 128MB939lb # [ 0.297177] x86/mm: Memory block size: 128MB940worker1 # [ 0.294874] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)941worker2 # [ 0.298361] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)942client # [ 0.328224] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages943lb # [ 0.298957] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)944worker1 # [ 0.295384] Memory: 1454616K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)945client # [ 0.328956] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page946worker2 # [ 0.299382] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).947lb # [ 0.299991] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).948worker1 # [ 0.296430] devtmpfs: initialized949client # [ 0.329931] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages950worker2 # [ 0.300604] pinctrl core: initialized pinctrl subsystem951lb # [ 0.301213] pinctrl core: initialized pinctrl subsystem952worker1 # [ 0.297228] x86/mm: Memory block size: 128MB953worker2 # [ 0.301698] PM: RTC time: 22:20:13, date: 2026-09-22954lb # [ 0.302341] PM: RTC time: 22:20:13, date: 2026-09-22955worker1 # [ 0.299060] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)956worker1 # [ 0.300142] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).957worker2 # [ 0.305406] NET: Registered PF_NETLINK/PF_ROUTE protocol family958client # [ 0.334957] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page959worker1 # [ 0.301251] pinctrl core: initialized pinctrl subsystem960lb # [ 0.306040] NET: Registered PF_NETLINK/PF_ROUTE protocol family961worker1 # [ 0.302401] PM: RTC time: 22:20:13, date: 2026-09-22962worker2 # [ 0.306771] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations963lb # [ 0.307367] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations964worker2 # [ 0.307541] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations965lb # [ 0.308146] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations966worker2 # [ 0.308808] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations967lb # [ 0.309417] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations968worker1 # [ 0.306087] NET: Registered PF_NETLINK/PF_ROUTE protocol family969worker2 # [ 0.309522] audit: initializing netlink subsys (disabled)970lb # [ 0.310128] audit: initializing netlink subsys (disabled)971worker1 # [ 0.307411] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations972worker2 # [ 0.310767] thermal_sys: Registered thermal governor 'fair_share'973lb # [ 0.311299] thermal_sys: Registered thermal governor 'fair_share'974worker2 # [ 0.310769] thermal_sys: Registered thermal governor 'bang_bang'975lb # [ 0.311302] thermal_sys: Registered thermal governor 'bang_bang'976worker1 # [ 0.308189] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations977client # [ 0.343388] ACPI: Added _OSI(Module Device)978lb # [ 0.312119] thermal_sys: Registered thermal governor 'step_wise'979client # [ 0.343958] ACPI: Added _OSI(Processor Device)980worker2 # [ 0.311517] audit: type=2000 audit(1790115614.544:1): state=initialized audit_enabled=0 res=1981worker1 # [ 0.309449] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations982client # [ 0.344656] ACPI: Added _OSI(Processor Aggregator Device)983lb # [ 0.313060] thermal_sys: Registered thermal governor 'user_space'984worker2 # [ 0.313514] thermal_sys: Registered thermal governor 'step_wise'985worker1 # [ 0.310175] audit: initializing netlink subsys (disabled)986worker2 # [ 0.313515] thermal_sys: Registered thermal governor 'user_space'987lb # [ 0.313805] audit: type=2000 audit(1790115614.544:1): state=initialized audit_enabled=0 res=1988worker1 # [ 0.311385] thermal_sys: Registered thermal governor 'fair_share'989worker2 # [ 0.314450] thermal_sys: Registered thermal governor 'power_allocator'990lb # [ 0.315121] thermal_sys: Registered thermal governor 'power_allocator'991worker1 # [ 0.311387] thermal_sys: Registered thermal governor 'bang_bang'992worker2 # [ 0.315287] cpuidle: using governor menu993lb # [ 0.315135] cpuidle: using governor menu994worker1 # [ 0.312166] thermal_sys: Registered thermal governor 'step_wise'995worker2 # [ 0.316734] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5996lb # [ 0.317337] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5997worker1 # [ 0.313086] thermal_sys: Registered thermal governor 'user_space'998worker1 # [ 0.313926] audit: type=2000 audit(1790115614.546:1): state=initialized audit_enabled=0 res=1999worker2 # [ 0.317781] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1000lb # [ 0.318413] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1001worker1 # [ 0.316168] thermal_sys: Registered thermal governor 'power_allocator'1002worker2 # [ 0.318516] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1003lb # [ 0.319121] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1004client # [ 0.351533] ACPI: 1 ACPI AML tables successfully acquired and loaded1005worker1 # [ 0.316192] cpuidle: using governor menu1006worker2 # [ 0.319525] PCI: Using configuration type 1 for base access1007lb # [ 0.320129] PCI: Using configuration type 1 for base access1008worker1 # [ 0.318370] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51009worker2 # [ 0.320676] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1010lb # [ 0.321222] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1011worker1 # [ 0.319429] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1012client # [ 0.355381] ACPI: Interpreter enabled1013client # [ 0.355971] ACPI: PM: (supports S0 S3 S4 S5)1014worker1 # [ 0.320168] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1015client # [ 0.356651] ACPI: Using IOAPIC for interrupt routing1016worker1 # [ 0.321177] PCI: Using configuration type 1 for base access1017worker1 # [ 0.322301] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1018worker2 # [ 0.325779] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1019worker2 # [ 0.326514] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1020client # [ 0.359023] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1021client # [ 0.359957] PCI: Using E820 reservations for host bridge windows1022lb # [ 0.330142] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1023lb # [ 0.331118] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1024worker1 # [ 0.327411] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1025client # [ 0.363089] ACPI: Enabled 2 GPEs in block 00 to 3F1026worker1 # [ 0.328166] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1027worker2 # [ 0.331514] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1028worker2 # [ 0.332513] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1029lb # [ 0.336119] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1030lb # [ 0.337117] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1031worker1 # [ 0.333166] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1032worker1 # [ 0.334166] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1033client # [ 0.371624] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1034client # [ 0.371962] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1035client # [ 0.373036] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1036worker2 # [ 0.342853] ACPI: Added _OSI(Module Device)1037client # [ 0.374078] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1038worker2 # [ 0.343515] ACPI: Added _OSI(Processor Device)1039lb # [ 0.344697] ACPI: Added _OSI(Module Device)1040client # [ 0.375299] PCI host bridge to bus 0000:001041worker2 # [ 0.344248] ACPI: Added _OSI(Processor Aggregator Device)1042lb # [ 0.345119] ACPI: Added _OSI(Processor Device)1043lb # [ 0.345882] ACPI: Added _OSI(Processor Aggregator Device)1044client # [ 0.375960] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1045client # [ 0.376956] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1046client # [ 0.377956] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1047worker1 # [ 0.344557] ACPI: Added _OSI(Module Device)1048worker1 # [ 0.345167] ACPI: Added _OSI(Processor Device)1049client # [ 0.378956] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1050worker1 # [ 0.345974] ACPI: Added _OSI(Processor Aggregator Device)1051client # [ 0.379956] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1052worker2 # [ 0.350456] ACPI: 1 ACPI AML tables successfully acquired and loaded1053client # [ 0.380956] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1054client # [ 0.381956] pci_bus 0000:00: root bus resource [bus 00-ff]1055lb # [ 0.352702] ACPI: 1 ACPI AML tables successfully acquired and loaded1056client # [ 0.383010] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1057client # [ 0.384394] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1058lb # [ 0.356066] ACPI: Interpreter enabled1059lb # [ 0.356689] ACPI: PM: (supports S0 S3 S4 S5)1060worker2 # [ 0.356927] ACPI: Interpreter enabled1061lb # [ 0.357119] ACPI: Using IOAPIC for interrupt routing1062worker2 # [ 0.357528] ACPI: PM: (supports S0 S3 S4 S5)1063worker2 # [ 0.358282] ACPI: Using IOAPIC for interrupt routing1064worker1 # [ 0.355738] ACPI: 1 ACPI AML tables successfully acquired and loaded1065lb # [ 0.358044] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1066worker2 # [ 0.358568] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1067worker2 # [ 0.359513] PCI: Using E820 reservations for host bridge windows1068worker1 # [ 0.359566] ACPI: Interpreter enabled1069client # [ 0.387022] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1070worker1 # [ 0.360181] ACPI: PM: (supports S0 S3 S4 S5)1071worker2 # [ 0.362649] ACPI: Enabled 2 GPEs in block 00 to 3F1072lb # [ 0.363118] PCI: Using E820 reservations for host bridge windows1073worker1 # [ 0.360886] ACPI: Using IOAPIC for interrupt routing1074client # [ 0.387968] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1075lb # [ 0.364194] ACPI: Enabled 2 GPEs in block 00 to 3F1076client # [ 0.388896] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1077worker1 # [ 0.361200] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1078worker1 # [ 0.362165] PCI: Using E820 reservations for host bridge windows1079client # [ 0.390061] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1080client # [ 0.391746] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1081worker1 # [ 0.365277] ACPI: Enabled 2 GPEs in block 00 to 3F1082client # [ 0.393964] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1083worker2 # [ 0.371277] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1084client # [ 0.394827] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1085lb # [ 0.372550] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1086worker2 # [ 0.371519] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1087client # [ 0.395674] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1088lb # [ 0.373126] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1089client # [ 0.395961] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1090worker2 # [ 0.373163] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1091lb # [ 0.374195] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1092worker2 # [ 0.373637] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1093lb # [ 0.375239] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1094client # [ 0.398005] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1095worker2 # [ 0.374851] PCI host bridge to bus 0000:001096lb # [ 0.376469] PCI host bridge to bus 0000:001097worker1 # [ 0.373975] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1098worker2 # [ 0.375517] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1099lb # [ 0.377123] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1100client # [ 0.399963] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1101worker1 # [ 0.374948] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1102worker2 # [ 0.376513] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1103lb # [ 0.378118] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1104client # [ 0.400821] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1105worker1 # [ 0.376244] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1106worker2 # [ 0.377512] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1107lb # [ 0.379117] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1108client # [ 0.401651] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1109worker1 # [ 0.377289] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1110worker2 # [ 0.378513] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1111lb # [ 0.380118] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1112client # [ 0.402520] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1113worker1 # [ 0.378504] PCI host bridge to bus 0000:001114worker2 # [ 0.379513] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1115lb # [ 0.381117] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1116worker1 # [ 0.379169] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1117worker2 # [ 0.380513] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1118lb # [ 0.382117] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1119worker1 # [ 0.380165] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1120worker2 # [ 0.381513] pci_bus 0000:00: root bus resource [bus 00-ff]1121lb # [ 0.383118] pci_bus 0000:00: root bus resource [bus 00-ff]1122worker1 # [ 0.381165] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1123worker2 # [ 0.382548] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1124lb # [ 0.384107] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1125client # [ 0.404964] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1126worker1 # [ 0.382165] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1127client # [ 0.405962] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1128worker2 # [ 0.383990] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1129lb # [ 0.385568] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1130worker1 # [ 0.383165] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1131client # [ 0.406866] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1132worker1 # [ 0.384166] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1133worker1 # [ 0.385166] pci_bus 0000:00: root bus resource [bus 00-ff]1134client # [ 0.408333] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1135worker1 # [ 0.386194] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1136client # [ 0.409963] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1137worker1 # [ 0.387625] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1138client # [ 0.410854] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1139client # [ 0.411691] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1140client # [ 0.411962] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1141worker2 # [ 0.386603] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1142lb # [ 0.388304] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1143worker2 # [ 0.387598] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1144lb # [ 0.389132] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1145client # [ 0.413506] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1146lb # [ 0.390078] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1147worker2 # [ 0.388576] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1148client # [ 0.414777] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1149client # [ 0.415675] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1150worker2 # [ 0.389837] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1151lb # [ 0.391477] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1152client # [ 0.416510] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1153worker2 # [ 0.391224] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1154lb # [ 0.392832] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1155worker1 # [ 0.390311] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1156worker1 # [ 0.391177] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1157client # [ 0.417975] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1158worker1 # [ 0.392316] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1159client # [ 0.418947] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1160lb # [ 0.395151] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1161worker2 # [ 0.394550] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1162lb # [ 0.395990] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1163worker1 # [ 0.393495] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1164worker2 # [ 0.395426] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1165client # [ 0.420830] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1166lb # [ 0.396834] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1167worker2 # [ 0.396242] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1168worker1 # [ 0.394909] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1169lb # [ 0.397124] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1170client # [ 0.422671] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1171worker2 # [ 0.396520] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1172client # [ 0.422979] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1173lb # [ 0.398719] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1174worker2 # [ 0.398086] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1175worker1 # [ 0.398215] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1176client # [ 0.424483] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1177worker1 # [ 0.399091] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1178lb # [ 0.400154] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1179client # [ 0.425969] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1180worker2 # [ 0.400555] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1181worker1 # [ 0.399972] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1182lb # [ 0.401026] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1183worker2 # [ 0.401405] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1184client # [ 0.426881] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1185lb # [ 0.401855] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1186worker1 # [ 0.401197] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1187worker2 # [ 0.402239] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1188client # [ 0.428318] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1189lb # [ 0.402666] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1190worker1 # [ 0.402707] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1191worker2 # [ 0.403086] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1192client # [ 0.430982] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1193client # [ 0.431850] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1194worker1 # [ 0.405202] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1195client # [ 0.432717] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1196lb # [ 0.405156] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1197worker1 # [ 0.406078] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1198lb # [ 0.406083] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1199worker2 # [ 0.406550] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1200worker1 # [ 0.406922] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1201client # [ 0.433510] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1202worker2 # [ 0.407414] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1203lb # [ 0.406892] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1204worker2 # [ 0.408255] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1205worker1 # [ 0.407733] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1206lb # [ 0.408579] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1207client # [ 0.434964] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1208worker2 # [ 0.409083] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1209client # [ 0.435799] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1210client # [ 0.436979] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1211worker1 # [ 0.410207] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1212lb # [ 0.411155] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1213worker1 # [ 0.411079] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1214worker2 # [ 0.411551] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1215lb # [ 0.412009] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1216client # [ 0.438520] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1217worker2 # [ 0.412394] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1218worker1 # [ 0.411920] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1219lb # [ 0.412870] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1220client # [ 0.439475] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1221lb # [ 0.413124] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1222worker2 # [ 0.413253] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1223worker1 # [ 0.412763] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1224client # [ 0.440160] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1225worker2 # [ 0.414362] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1226lb # [ 0.414637] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1227client # [ 0.441479] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1228worker2 # [ 0.415854] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1229client # [ 0.442194] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1230lb # [ 0.416162] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1231worker1 # [ 0.415203] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1232lb # [ 0.417140] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1233worker2 # [ 0.417526] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1234client # [ 0.443520] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1235worker1 # [ 0.416797] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1236worker2 # [ 0.418485] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1237worker1 # [ 0.417187] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1238lb # [ 0.418685] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1239worker1 # [ 0.418172] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1240worker2 # [ 0.419945] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1241worker1 # [ 0.419739] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1242lb # [ 0.420506] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1243lb # [ 0.421918] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1244worker2 # [ 0.422513] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1245worker1 # [ 0.421209] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1246worker2 # [ 0.423475] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1247lb # [ 0.423586] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1248worker1 # [ 0.422187] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1249worker2 # [ 0.424940] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1250lb # [ 0.425161] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1251worker1 # [ 0.423799] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1252lb # [ 0.426140] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1253worker2 # [ 0.426555] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1254client # [ 0.444172] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1255worker2 # [ 0.427534] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1256lb # [ 0.427672] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1257worker1 # [ 0.426178] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1258worker1 # [ 0.427157] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1259worker2 # [ 0.429063] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1260client # [ 0.447534] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1261lb # [ 0.429131] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1262worker1 # [ 0.428640] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1263client # [ 0.448249] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1264lb # [ 0.430140] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1265worker2 # [ 0.431228] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1266client # [ 0.449249] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1267worker1 # [ 0.430207] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1268lb # [ 0.431665] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1269worker2 # [ 0.431536] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1270worker1 # [ 0.431181] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1271client # [ 0.450200] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1272worker2 # [ 0.433087] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1273lb # [ 0.433161] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1274lb # [ 0.434033] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1275worker1 # [ 0.433577] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1276client # [ 0.452840] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1277lb # [ 0.434870] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1278worker2 # [ 0.435340] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1279client # [ 0.453593] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1280worker2 # [ 0.436176] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1281worker1 # [ 0.435209] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1282client # [ 0.454516] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1283worker2 # [ 0.436535] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1284lb # [ 0.435692] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1285worker1 # [ 0.436187] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1286client # [ 0.455567] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1287worker1 # [ 0.437709] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1288lb # [ 0.437125] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1289worker2 # [ 0.438053] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1290lb # [ 0.437947] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1291lb # [ 0.438967] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1292client # [ 0.458921] ACPI: PCI: Interrupt link LNKA configured for IRQ 101293worker2 # [ 0.441287] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1294client # [ 0.459774] ACPI: PCI: Interrupt link LNKB configured for IRQ 101295worker2 # [ 0.441520] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1296worker1 # [ 0.440150] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1297lb # [ 0.440587] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1298client # [ 0.460750] ACPI: PCI: Interrupt link LNKC configured for IRQ 111299worker1 # [ 0.441172] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1300worker2 # [ 0.442482] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1301lb # [ 0.441837] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1302client # [ 0.461758] ACPI: PCI: Interrupt link LNKD configured for IRQ 111303worker1 # [ 0.442142] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1304worker2 # [ 0.443933] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1305client # [ 0.462759] ACPI: PCI: Interrupt link LNKE configured for IRQ 101306lb # [ 0.442320] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1307worker1 # [ 0.443598] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1308client # [ 0.463751] ACPI: PCI: Interrupt link LNKF configured for IRQ 101309worker2 # [ 0.445290] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1310client # [ 0.464776] ACPI: PCI: Interrupt link LNKG configured for IRQ 111311lb # [ 0.444534] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1312client # [ 0.465811] ACPI: PCI: Interrupt link LNKH configured for IRQ 111313worker1 # [ 0.445975] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1314worker2 # [ 0.445722] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1315client # [ 0.466728] ACPI: PCI: Interrupt link GSIA configured for IRQ 161316lb # [ 0.445316] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1317worker1 # [ 0.446812] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1318client # [ 0.466970] ACPI: PCI: Interrupt link GSIB configured for IRQ 171319worker2 # [ 0.448336] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1320worker1 # [ 0.447187] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1321lb # [ 0.446816] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1322client # [ 0.467951] ACPI: PCI: Interrupt link GSIC configured for IRQ 181323client # [ 0.468882] ACPI: PCI: Interrupt link GSID configured for IRQ 191324worker2 # [ 0.449445] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1325client # [ 0.469706] ACPI: PCI: Interrupt link GSIE configured for IRQ 201326worker1 # [ 0.448780] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1327worker2 # [ 0.451206] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1328client # [ 0.469967] ACPI: PCI: Interrupt link GSIF configured for IRQ 211329client # [ 0.470892] ACPI: PCI: Interrupt link GSIG configured for IRQ 221330worker1 # [ 0.450022] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1331client # [ 0.471823] ACPI: PCI: Interrupt link GSIH configured for IRQ 231332worker1 # [ 0.451051] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1333client # [ 0.473775] iommu: Default domain type: Translated1334worker1 # [ 0.452896] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1335client # [ 0.474592] iommu: DMA domain TLB invalidation policy: lazy mode1336client # [ 0.475216] ACPI: bus type USB registered1337worker1 # [ 0.453365] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1338client # [ 0.475982] usbcore: registered new interface driver usbfs1339client # [ 0.476904] usbcore: registered new interface driver hub1340worker1 # [ 0.454918] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1341client # [ 0.477625] usbcore: registered new device driver usb1342lb # [ 0.447345] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1343client # [ 0.478851] NetLabel: Initializing1344client # [ 0.479380] NetLabel: domain hash size = 1281345lb # [ 0.449739] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1346client # [ 0.479956] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1347client # [ 0.480993] NetLabel: unlabeled traffic allowed by default1348worker2 # [ 0.451724] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1349lb # [ 0.450415] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1350client # [ 0.481861] PCI: Using ACPI for IRQ routing1351worker2 # [ 0.454266] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1352lb # [ 0.451574] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1353lb # [ 0.452379] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1354worker2 # [ 0.454971] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1355worker2 # [ 0.455808] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1356lb # [ 0.454177] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1357lb # [ 0.455089] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1358worker2 # [ 0.456771] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1359lb # [ 0.456213] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1360worker1 # [ 0.455423] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1361worker2 # [ 0.458541] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1362lb # [ 0.457879] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1363worker2 # [ 0.459417] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1364worker2 # [ 0.460635] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1365worker2 # [ 0.462513] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1366worker1 # [ 0.458885] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1367lb # [ 0.461717] ACPI: PCI: Interrupt link LNKA configured for IRQ 101368lb # [ 0.462224] ACPI: PCI: Interrupt link LNKB configured for IRQ 101369worker1 # [ 0.459434] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1370lb # [ 0.463218] ACPI: PCI: Interrupt link LNKC configured for IRQ 111371worker1 # [ 0.460492] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1372lb # [ 0.464218] ACPI: PCI: Interrupt link LNKD configured for IRQ 111373worker2 # [ 0.466416] ACPI: PCI: Interrupt link LNKA configured for IRQ 101374lb # [ 0.465214] ACPI: PCI: Interrupt link LNKE configured for IRQ 101375worker2 # [ 0.467330] ACPI: PCI: Interrupt link LNKB configured for IRQ 101376worker1 # [ 0.461436] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1377lb # [ 0.466217] ACPI: PCI: Interrupt link LNKF configured for IRQ 101378worker2 # [ 0.468350] ACPI: PCI: Interrupt link LNKC configured for IRQ 111379worker2 # [ 0.469327] ACPI: PCI: Interrupt link LNKD configured for IRQ 111380lb # [ 0.467218] ACPI: PCI: Interrupt link LNKG configured for IRQ 111381worker1 # [ 0.464138] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1382worker2 # [ 0.470447] ACPI: PCI: Interrupt link LNKE configured for IRQ 101383lb # [ 0.468248] ACPI: PCI: Interrupt link LNKH configured for IRQ 111384lb # [ 0.469164] ACPI: PCI: Interrupt link GSIA configured for IRQ 161385worker1 # [ 0.464820] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1386worker2 # [ 0.471352] ACPI: PCI: Interrupt link LNKF configured for IRQ 101387lb # [ 0.470132] ACPI: PCI: Interrupt link GSIB configured for IRQ 171388worker2 # [ 0.472621] ACPI: PCI: Interrupt link LNKG configured for IRQ 111389worker1 # [ 0.465494] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1390lb # [ 0.471135] ACPI: PCI: Interrupt link GSIC configured for IRQ 181391worker2 # [ 0.473617] ACPI: PCI: Interrupt link LNKH configured for IRQ 111392lb # [ 0.472133] ACPI: PCI: Interrupt link GSID configured for IRQ 191393worker2 # [ 0.474553] ACPI: PCI: Interrupt link GSIA configured for IRQ 161394worker1 # [ 0.467096] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1395lb # [ 0.473132] ACPI: PCI: Interrupt link GSIE configured for IRQ 201396worker2 # [ 0.475506] ACPI: PCI: Interrupt link GSIB configured for IRQ 171397lb # [ 0.474128] ACPI: PCI: Interrupt link GSIF configured for IRQ 211398worker2 # [ 0.476495] ACPI: PCI: Interrupt link GSIC configured for IRQ 181399lb # [ 0.475130] ACPI: PCI: Interrupt link GSIG configured for IRQ 221400worker2 # [ 0.477260] ACPI: PCI: Interrupt link GSID configured for IRQ 191401lb # [ 0.476133] ACPI: PCI: Interrupt link GSIH configured for IRQ 231402worker2 # [ 0.478266] ACPI: PCI: Interrupt link GSIE configured for IRQ 201403worker1 # [ 0.470962] ACPI: PCI: Interrupt link LNKA configured for IRQ 101404worker2 # [ 0.478525] ACPI: PCI: Interrupt link GSIF configured for IRQ 211405worker1 # [ 0.471994] ACPI: PCI: Interrupt link LNKB configured for IRQ 101406lb # [ 0.478170] iommu: Default domain type: Translated1407worker2 # [ 0.479528] ACPI: PCI: Interrupt link GSIG configured for IRQ 221408worker1 # [ 0.473013] ACPI: PCI: Interrupt link LNKC configured for IRQ 111409lb # [ 0.478992] iommu: DMA domain TLB invalidation policy: lazy mode1410worker2 # [ 0.480497] ACPI: PCI: Interrupt link GSIH configured for IRQ 231411lb # [ 0.480162] ACPI: bus type USB registered1412worker1 # [ 0.474012] ACPI: PCI: Interrupt link LNKD configured for IRQ 111413lb # [ 0.480937] usbcore: registered new interface driver usbfs1414worker2 # [ 0.482348] iommu: Default domain type: Translated1415worker1 # [ 0.475097] ACPI: PCI: Interrupt link LNKE configured for IRQ 101416lb # [ 0.481855] usbcore: registered new interface driver hub1417worker2 # [ 0.483101] iommu: DMA domain TLB invalidation policy: lazy mode1418worker1 # [ 0.476070] ACPI: PCI: Interrupt link LNKF configured for IRQ 101419lb # [ 0.482127] usbcore: registered new device driver usb1420worker2 # [ 0.483791] ACPI: bus type USB registered1421worker1 # [ 0.477010] ACPI: PCI: Interrupt link LNKG configured for IRQ 111422worker1 # [ 0.478016] ACPI: PCI: Interrupt link LNKH configured for IRQ 111423lb # [ 0.484476] NetLabel: Initializing1424worker1 # [ 0.478976] ACPI: PCI: Interrupt link GSIA configured for IRQ 161425worker2 # [ 0.484564] usbcore: registered new interface driver usbfs1426lb # [ 0.485118] NetLabel: domain hash size = 1281427worker1 # [ 0.479896] ACPI: PCI: Interrupt link GSIB configured for IRQ 171428worker2 # [ 0.485533] usbcore: registered new interface driver hub1429lb # [ 0.485860] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1430worker2 # [ 0.486413] usbcore: registered new device driver usb1431lb # [ 0.486192] NetLabel: unlabeled traffic allowed by default1432lb # [ 0.487129] PCI: Using ACPI for IRQ routing1433worker2 # [ 0.487980] NetLabel: Initializing1434worker2 # [ 0.488514] NetLabel: domain hash size = 1281435client # [ 0.524582] pci 0000:00:01.0: vgaarb: setting as boot VGA device1436worker2 # [ 0.489262] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1437client # [ 0.524952] pci 0000:00:01.0: vgaarb: bridge control possible1438worker2 # [ 0.489550] NetLabel: unlabeled traffic allowed by default1439worker1 # [ 0.480176] ACPI: PCI: Interrupt link GSIC configured for IRQ 181440client # [ 0.524952] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1441worker2 # [ 0.490455] PCI: Using ACPI for IRQ routing1442client # [ 0.524960] vgaarb: loaded1443worker1 # [ 0.481187] ACPI: PCI: Interrupt link GSID configured for IRQ 191444client # [ 0.525697] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01445worker1 # [ 0.482184] ACPI: PCI: Interrupt link GSIE configured for IRQ 201446client # [ 0.525958] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1447worker1 # [ 0.483177] ACPI: PCI: Interrupt link GSIF configured for IRQ 211448worker1 # [ 0.484177] ACPI: PCI: Interrupt link GSIG configured for IRQ 221449worker1 # [ 0.485187] ACPI: PCI: Interrupt link GSIH configured for IRQ 231450client # [ 0.528041] clocksource: Switched to clocksource kvm-clock1451worker1 # [ 0.487255] iommu: Default domain type: Translated1452client # [ 0.529772] VFS: Disk quotas dquot_6.6.01453worker1 # [ 0.488131] iommu: DMA domain TLB invalidation policy: lazy mode1454worker1 # [ 0.489259] ACPI: bus type USB registered1455client # [ 0.532304] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1456worker1 # [ 0.490055] usbcore: registered new interface driver usbfs1457client # [ 0.533689] pnp: PnP ACPI init1458worker1 # [ 0.491184] usbcore: registered new interface driver hub1459client # [ 0.534507] ACPI: IRQ 4 override to edge(!), high(!)1460worker1 # [ 0.492178] usbcore: registered new device driver usb1461client # [ 0.535463] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1462worker1 # [ 0.494134] NetLabel: Initializing1463client # [ 0.537039] pnp: PnP ACPI: found 5 devices1464worker1 # [ 0.494679] NetLabel: domain hash size = 1281465worker1 # [ 0.495165] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1466worker1 # [ 0.496205] NetLabel: unlabeled traffic allowed by default1467worker1 # [ 0.497177] PCI: Using ACPI for IRQ routing1468client # [ 0.544642] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1469client # [ 0.546127] clocksource: Switched to clocksource acpi_pm1470client # [ 0.547150] NET: Registered PF_INET protocol family1471client # [ 0.548543] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1472lb # [ 0.530882] pci 0000:00:01.0: vgaarb: setting as boot VGA device1473lb # [ 0.531114] pci 0000:00:01.0: vgaarb: bridge control possible1474lb # [ 0.531114] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1475client # [ 0.565265] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1476lb # [ 0.531124] vgaarb: loaded1477worker2 # [ 0.534126] pci 0000:00:01.0: vgaarb: setting as boot VGA device1478lb # [ 0.531906] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01479client # [ 0.566871] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1480worker2 # [ 0.534509] pci 0000:00:01.0: vgaarb: bridge control possible1481lb # [ 0.532120] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1482client # [ 0.568234] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1483worker2 # [ 0.534509] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1484worker2 # [ 0.534518] vgaarb: loaded1485client # [ 0.569664] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1486worker2 # [ 0.535269] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01487client # [ 0.570943] TCP: Hash tables configured (established 16384 bind 16384)1488worker2 # [ 0.535515] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1489lb # [ 0.535202] clocksource: Switched to clocksource kvm-clock1490client # [ 0.572095] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1491client # [ 0.573359] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1492client # [ 0.574473] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1493client # [ 0.575729] NET: Registered PF_UNIX/PF_LOCAL protocol family1494lb # [ 0.538990] VFS: Disk quotas dquot_6.6.01495client # [ 0.576724] NET: Registered PF_XDP protocol family1496worker2 # [ 0.539639] clocksource: Switched to clocksource kvm-clock1497lb # [ 0.539691] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1498client # [ 0.577562] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1499lb # [ 0.541013] pnp: PnP ACPI init1500client # [ 0.578620] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1501lb # [ 0.541819] ACPI: IRQ 4 override to edge(!), high(!)1502client # [ 0.579665] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1503worker2 # [ 0.543046] VFS: Disk quotas dquot_6.6.01504lb # [ 0.542812] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1505client # [ 0.580783] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1506worker2 # [ 0.543761] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1507lb # [ 0.544335] pnp: PnP ACPI: found 5 devices1508worker2 # [ 0.545104] pnp: PnP ACPI init1509client # [ 0.581892] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1510worker2 # [ 0.545921] ACPI: IRQ 4 override to edge(!), high(!)1511client # [ 0.583016] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1512worker2 # [ 0.546892] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1513client # [ 0.584733] ACPI: \_SB_.GSIA: Enabled at IRQ 161514worker1 # [ 0.541014] pci 0000:00:01.0: vgaarb: setting as boot VGA device1515worker2 # [ 0.548431] pnp: PnP ACPI: found 5 devices1516worker1 # [ 0.541161] pci 0000:00:01.0: vgaarb: bridge control possible1517client # [ 0.587058] ACPI: \_SB_.GSIB: Enabled at IRQ 171518worker1 # [ 0.541161] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1519worker1 # [ 0.541170] vgaarb: loaded1520worker1 # [ 0.541958] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01521client # [ 0.589357] ACPI: \_SB_.GSIC: Enabled at IRQ 181522lb # [ 0.551979] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1523worker1 # [ 0.542784] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1524lb # [ 0.553424] clocksource: Switched to clocksource acpi_pm1525client # [ 0.591356] ACPI: \_SB_.GSID: Enabled at IRQ 191526lb # [ 0.554436] NET: Registered PF_INET protocol family1527client # [ 0.593043] PCI: CLS 0 bytes, default 641528lb # [ 0.555661] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1529worker2 # [ 0.556043] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1530worker1 # [ 0.547254] clocksource: Switched to clocksource kvm-clock1531client # [ 0.593898] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1532worker2 # [ 0.557486] clocksource: Switched to clocksource acpi_pm1533worker2 # [ 0.558472] NET: Registered PF_INET protocol family1534client # [ 0.595642] Trying to unpack rootfs image as initramfs...1535worker2 # [ 0.559623] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1536worker1 # [ 0.551083] VFS: Disk quotas dquot_6.6.01537worker1 # [ 0.551834] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1538worker1 # [ 0.553243] pnp: PnP ACPI init1539worker1 # [ 0.554082] ACPI: IRQ 4 override to edge(!), high(!)1540worker1 # [ 0.555063] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1541worker1 # [ 0.556660] pnp: PnP ACPI: found 5 devices1542lb # [ 0.572134] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1543lb # [ 0.573753] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1544worker1 # [ 0.564341] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1545lb # [ 0.575119] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1546worker1 # [ 0.565853] clocksource: Switched to clocksource acpi_pm1547lb # [ 0.576546] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1548worker2 # [ 0.576509] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1549worker1 # [ 0.566844] NET: Registered PF_INET protocol family1550lb # [ 0.577811] TCP: Hash tables configured (established 16384 bind 16384)1551worker2 # [ 0.578092] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1552worker1 # [ 0.568058] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1553lb # [ 0.579035] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1554worker2 # [ 0.579410] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1555lb # [ 0.580419] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1556worker2 # [ 0.580818] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1557lb # [ 0.581602] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1558worker2 # [ 0.582113] TCP: Hash tables configured (established 16384 bind 16384)1559lb # [ 0.582865] NET: Registered PF_UNIX/PF_LOCAL protocol family1560worker2 # [ 0.583256] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1561lb # [ 0.583873] NET: Registered PF_XDP protocol family1562worker2 # [ 0.584538] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1563lb # [ 0.584745] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1564lb # [ 0.585778] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1565worker2 # [ 0.585639] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1566lb # [ 0.586800] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1567worker2 # [ 0.586891] NET: Registered PF_UNIX/PF_LOCAL protocol family1568worker2 # [ 0.587884] NET: Registered PF_XDP protocol family1569lb # [ 0.587933] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1570worker2 # [ 0.588752] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1571lb # [ 0.589057] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1572worker2 # [ 0.589795] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1573lb # [ 0.590205] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1574worker2 # [ 0.590832] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1575lb # [ 0.591935] ACPI: \_SB_.GSIA: Enabled at IRQ 161576worker2 # [ 0.591958] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1577worker2 # [ 0.593121] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1578lb # [ 0.594367] ACPI: \_SB_.GSIB: Enabled at IRQ 171579worker2 # [ 0.594272] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1580worker1 # [ 0.584372] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1581worker2 # [ 0.595979] ACPI: \_SB_.GSIA: Enabled at IRQ 161582lb # [ 0.596634] ACPI: \_SB_.GSIC: Enabled at IRQ 181583worker1 # [ 0.586046] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1584worker1 # [ 0.587406] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1585lb # [ 0.598758] ACPI: \_SB_.GSID: Enabled at IRQ 191586worker2 # [ 0.599212] ACPI: \_SB_.GSIB: Enabled at IRQ 171587worker1 # [ 0.588824] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1588lb # [ 0.600380] PCI: CLS 0 bytes, default 641589worker1 # [ 0.590113] TCP: Hash tables configured (established 16384 bind 16384)1590worker2 # [ 0.601204] ACPI: \_SB_.GSIC: Enabled at IRQ 181591worker1 # [ 0.591316] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1592lb # [ 0.601369] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1593client # [ 0.639865] Initialise system trusted keyrings1594lb # [ 0.603096] Trying to unpack rootfs image as initramfs...1595worker2 # [ 0.603181] ACPI: \_SB_.GSID: Enabled at IRQ 191596worker1 # [ 0.592652] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1597worker1 # [ 0.593777] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1598worker2 # [ 0.604827] PCI: CLS 0 bytes, default 641599worker1 # [ 0.595026] NET: Registered PF_UNIX/PF_LOCAL protocol family1600client # [ 0.643035] workingset: timestamp_bits=40 max_order=19 bucket_order=01601worker1 # [ 0.596004] NET: Registered PF_XDP protocol family1602worker2 # [ 0.605747] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1603worker1 # [ 0.596851] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1604worker2 # [ 0.607434] Trying to unpack rootfs image as initramfs...1605worker1 # [ 0.598433] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1606worker1 # [ 0.599988] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1607worker1 # [ 0.601159] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1608worker1 # [ 0.602321] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1609worker1 # [ 0.603480] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1610worker1 # [ 0.605486] ACPI: \_SB_.GSIA: Enabled at IRQ 161611worker1 # [ 0.608026] ACPI: \_SB_.GSIB: Enabled at IRQ 171612worker1 # [ 0.610123] ACPI: \_SB_.GSIC: Enabled at IRQ 181613worker1 # [ 0.612133] ACPI: \_SB_.GSID: Enabled at IRQ 191614worker1 # [ 0.613931] PCI: CLS 0 bytes, default 641615worker1 # [ 0.614881] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1616client # [ 0.664261] Key type asymmetric registered1617client # [ 0.664948] Asymmetric key parser 'x509' registered1618worker1 # [ 0.616713] Trying to unpack rootfs image as initramfs...1619client # [ 0.665815] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1620client # [ 0.671080] io scheduler mq-deadline registered1621client # [ 0.671870] io scheduler kyber registered1622client # [ 0.674607] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1623client # [ 0.677987] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1624client # [ 0.681589] Linux agpgart interface v0.1031625client # [ 0.682372] ACPI: bus type drm_connector registered1626lb # [ 0.647620] Initialise system trusted keyrings1627client # [ 0.685440] usbcore: registered new interface driver usbserial_generic1628client # [ 0.686555] usbserial: USB Serial support registered for generic1629lb # [ 0.650800] workingset: timestamp_bits=40 max_order=19 bucket_order=01630worker2 # [ 0.652453] Initialise system trusted keyrings1631client # [ 0.689993] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1632client # [ 0.689993] Please enable it if your BIOS has the CPPC option.1633worker2 # [ 0.655093] workingset: timestamp_bits=40 max_order=19 bucket_order=01634client # [ 0.692237] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1635client # [ 0.696128] drop_monitor: Initializing network drop monitor service1636client # [ 0.697418] NET: Registered PF_INET6 protocol family1637client # [ 0.701462] Segment Routing with IPv61638client # [ 0.704099] In-situ OAM (IOAM) with IPv61639client # [ 0.705194] IPI shorthand broadcast: enabled1640lb # [ 0.670006] Key type asymmetric registered1641lb # [ 0.672741] Asymmetric key parser 'x509' registered1642worker1 # [ 0.662913] Initialise system trusted keyrings1643lb # [ 0.673593] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1644lb # [ 0.676850] io scheduler mq-deadline registered1645client # [ 0.713471] sched_clock: Marking stable (577016236, 135952020)->(786690046, -73721790)1646worker1 # [ 0.666760] workingset: timestamp_bits=40 max_order=19 bucket_order=01647lb # [ 0.677607] io scheduler kyber registered1648worker2 # [ 0.679240] Key type asymmetric registered1649worker2 # [ 0.680000] Asymmetric key parser 'x509' registered1650client # [ 0.718059] registered taskstats version 11651worker2 # [ 0.680911] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1652client # [ 0.719080] Loading compiled-in X.509 certificates1653lb # [ 0.681897] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1654lb # [ 0.683192] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1655worker2 # [ 0.684180] io scheduler mq-deadline registered1656worker2 # [ 0.684991] io scheduler kyber registered1657lb # [ 0.688513] Linux agpgart interface v0.1031658worker2 # [ 0.688629] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1659lb # [ 0.689263] ACPI: bus type drm_connector registered1660worker2 # [ 0.689911] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1661lb # [ 0.692214] usbcore: registered new interface driver usbserial_generic1662lb # [ 0.693373] usbserial: USB Serial support registered for generic1663worker2 # [ 0.695188] Linux agpgart interface v0.1031664worker2 # [ 0.695977] ACPI: bus type drm_connector registered1665lb # [ 0.696750] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1666lb # [ 0.696750] Please enable it if your BIOS has the CPPC option.1667lb # [ 0.699084] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1668worker2 # [ 0.699316] usbcore: registered new interface driver usbserial_generic1669worker2 # [ 0.700396] usbserial: USB Serial support registered for generic1670client # [ 0.737978] Demotion targets for Node 0: null1671worker1 # [ 0.690612] Key type asymmetric registered1672client # [ 0.738876] Key type .fscrypt registered1673worker1 # [ 0.691339] Asymmetric key parser 'x509' registered1674worker1 # [ 0.692197] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1675client # [ 0.740979] Key type fscrypt-provisioning registered1676lb # [ 0.703873] drop_monitor: Initializing network drop monitor service1677worker2 # [ 0.704043] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1678client # [ 0.741990] ima: No TPM chip found, activating TPM-bypass!1679lb # [ 0.705071] NET: Registered PF_INET6 protocol family1680worker2 # [ 0.704043] Please enable it if your BIOS has the CPPC option.1681client # [ 0.742902] ima: Allocated hash algorithm: sha11682worker2 # [ 0.706243] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1683client # [ 0.744999] ima: No architecture policies found1684worker1 # [ 0.697682] io scheduler mq-deadline registered1685lb # [ 0.708337] Segment Routing with IPv61686worker1 # [ 0.698459] io scheduler kyber registered1687lb # [ 0.709036] In-situ OAM (IOAM) with IPv61688worker2 # [ 0.709199] drop_monitor: Initializing network drop monitor service1689worker2 # [ 0.710390] NET: Registered PF_INET6 protocol family1690client # [ 0.747985] PM: Magic number: 10:279:3511691worker1 # [ 0.701226] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1692lb # [ 0.712083] IPI shorthand broadcast: enabled1693client # [ 0.749701] RAS: Correctable Errors collector initialized.1694worker2 # [ 0.713537] Segment Routing with IPv61695worker1 # [ 0.704571] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1696worker2 # [ 0.716052] In-situ OAM (IOAM) with IPv61697worker2 # [ 0.717176] IPI shorthand broadcast: enabled1698worker1 # [ 0.708206] Linux agpgart interface v0.1031699worker1 # [ 0.709042] ACPI: bus type drm_connector registered1700lb # [ 0.721327] sched_clock: Marking stable (588015812, 132707687)->(794946126, -74222627)1701client # [ 0.759708] clk: Disabling unused clocks1702worker1 # [ 0.712012] usbcore: registered new interface driver usbserial_generic1703worker1 # [ 0.713116] usbserial: USB Serial support registered for generic1704lb # [ 0.724821] registered taskstats version 11705client # [ 0.761986] PM: genpd: Disabling unused power domains1706lb # [ 0.725794] Loading compiled-in X.509 certificates1707worker2 # [ 0.725445] sched_clock: Marking stable (592013867, 133009135)->(799826291, -74803289)1708worker1 # [ 0.715575] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1709worker1 # [ 0.715575] Please enable it if your BIOS has the CPPC option.1710worker1 # [ 0.717859] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1711worker2 # [ 0.730273] registered taskstats version 11712worker2 # [ 0.731276] Loading compiled-in X.509 certificates1713worker1 # [ 0.721731] drop_monitor: Initializing network drop monitor service1714worker1 # [ 0.723015] NET: Registered PF_INET6 protocol family1715worker1 # [ 0.728112] Segment Routing with IPv61716worker1 # [ 0.730607] In-situ OAM (IOAM) with IPv61717worker1 # [ 0.731760] IPI shorthand broadcast: enabled1718lb # [ 0.744733] Demotion targets for Node 0: null1719lb # [ 0.745630] Key type .fscrypt registered1720lb # [ 0.747734] Key type fscrypt-provisioning registered1721lb # [ 0.748686] ima: No TPM chip found, activating TPM-bypass!1722lb # [ 0.749614] ima: Allocated hash algorithm: sha11723worker2 # [ 0.750042] Demotion targets for Node 0: null1724worker1 # [ 0.739905] sched_clock: Marking stable (606014151, 133533223)->(821151484, -81604110)1725worker2 # [ 0.750944] Key type .fscrypt registered1726lb # [ 0.753759] ima: No architecture policies found1727worker2 # [ 0.754028] Key type fscrypt-provisioning registered1728lb # [ 0.754694] PM: Magic number: 10:279:3511729worker1 # [ 0.744787] registered taskstats version 11730worker2 # [ 0.754984] ima: No TPM chip found, activating TPM-bypass!1731worker2 # [ 0.755914] ima: Allocated hash algorithm: sha11732worker1 # [ 0.745792] Loading compiled-in X.509 certificates1733worker2 # [ 0.758055] ima: No architecture policies found1734lb # [ 0.758351] RAS: Correctable Errors collector initialized.1735worker2 # [ 0.759126] PM: Magic number: 10:279:3511736worker2 # [ 0.760789] RAS: Correctable Errors collector initialized.1737lb # [ 0.767055] clk: Disabling unused clocks1738lb # [ 0.769740] PM: genpd: Disabling unused power domains1739worker2 # [ 0.771754] clk: Disabling unused clocks1740worker2 # [ 0.774056] PM: genpd: Disabling unused power domains1741worker1 # [ 0.764564] Demotion targets for Node 0: null1742worker1 # [ 0.765560] Key type .fscrypt registered1743worker1 # [ 0.766258] Key type fscrypt-provisioning registered1744worker1 # [ 0.767266] ima: No TPM chip found, activating TPM-bypass!1745worker1 # [ 0.770571] ima: Allocated hash algorithm: sha11746worker1 # [ 0.771412] ima: No architecture policies found1747worker1 # [ 0.775569] PM: Magic number: 10:279:3511748worker1 # [ 0.777401] RAS: Correctable Errors collector initialized.1749worker1 # [ 0.787303] clk: Disabling unused clocks1750worker1 # [ 0.789569] PM: genpd: Disabling unused power domains1751client # [ 0.925401] Freeing initrd memory: 29056K1752client # [ 0.928717] Freeing unused decrypted memory: 2028K1753client # [ 0.931526] Freeing unused kernel image (initmem) memory: 3652K1754client # [ 0.932652] Write protecting the kernel read-only data: 32768k1755client # [ 0.934569] Freeing unused kernel image (text/rodata gap) memory: 1184K1756client # [ 0.936098] Freeing unused kernel image (rodata/data gap) memory: 720K1757lb # [ 0.930769] Freeing initrd memory: 29064K1758lb # [ 0.934024] Freeing unused decrypted memory: 2028K1759lb # [ 0.936795] Freeing unused kernel image (initmem) memory: 3652K1760worker2 # [ 0.937475] Freeing initrd memory: 29060K1761lb # [ 0.937910] Write protecting the kernel read-only data: 32768k1762lb # [ 0.939784] Freeing unused kernel image (text/rodata gap) memory: 1184K1763worker2 # [ 0.940731] Freeing unused decrypted memory: 2028K1764lb # [ 0.941341] Freeing unused kernel image (rodata/data gap) memory: 720K1765worker2 # [ 0.943497] Freeing unused kernel image (initmem) memory: 3652K1766worker2 # [ 0.944626] Write protecting the kernel read-only data: 32768k1767worker2 # [ 0.946583] Freeing unused kernel image (text/rodata gap) memory: 1184K1768worker2 # [ 0.948172] Freeing unused kernel image (rodata/data gap) memory: 720K1769client # [ 0.986674] x86/mm: Checked W+X mappings: passed, no W+X pages found.1770client # [ 0.987740] Run /init as init process1771client # [ 0.998602] systemd[1]: Inserted module 'autofs4'1772worker1 # [ 0.958901] Freeing initrd memory: 29056K1773worker1 # [ 0.962405] Freeing unused decrypted memory: 2028K1774worker1 # [ 0.965436] Freeing unused kernel image (initmem) memory: 3652K1775worker1 # [ 0.966611] Write protecting the kernel read-only data: 32768k1776worker1 # [ 0.968633] Freeing unused kernel image (text/rodata gap) memory: 1184K1777worker1 # [ 0.970261] Freeing unused kernel image (rodata/data gap) memory: 720K1778client # [ 1.018527] fuse: init (API version 7.45)1779client # [ 1.025941] ACPI: \_SB_.GSIG: Enabled at IRQ 221780client # [ 1.028343] ACPI: \_SB_.GSIH: Enabled at IRQ 231781lb # [ 0.992307] x86/mm: Checked W+X mappings: passed, no W+X pages found.1782lb # [ 0.993431] Run /init as init process1783client # [ 1.031339] ACPI: \_SB_.GSIE: Enabled at IRQ 201784client # [ 1.033286] ACPI: \_SB_.GSIF: Enabled at IRQ 211785worker2 # [ 0.998930] x86/mm: Checked W+X mappings: passed, no W+X pages found.1786worker2 # [ 1.000066] Run /init as init process1787client # [ 1.038325] virtiofs virtio5: discovered new tag: nix-store1788client # [ 1.039867] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1789lb # [ 1.004310] systemd[1]: Inserted module 'autofs4'1790client # [ 1.046200] virtiofs virtio6: discovered new tag: shared1791client # [ 1.047718] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1792worker2 # [ 1.010878] systemd[1]: Inserted module 'autofs4'1793client # [ 1.050941] virtiofs virtio7: discovered new tag: xchg1794client # [ 1.052422] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1795lb # [ 1.024275] fuse: init (API version 7.45)1796worker2 # [ 1.030741] fuse: init (API version 7.45)1797lb # [ 1.031785] ACPI: \_SB_.GSIG: Enabled at IRQ 221798worker1 # [ 1.021227] x86/mm: Checked W+X mappings: passed, no W+X pages found.1799worker1 # [ 1.022394] Run /init as init process1800lb # [ 1.034131] ACPI: \_SB_.GSIH: Enabled at IRQ 231801client # [ 1.073072] systemd[1]: Successfully made /usr/ read-only.1802lb # [ 1.037161] ACPI: \_SB_.GSIE: Enabled at IRQ 201803worker2 # [ 1.038179] ACPI: \_SB_.GSIG: Enabled at IRQ 221804lb # [ 1.039140] ACPI: \_SB_.GSIF: Enabled at IRQ 211805worker2 # [ 1.040614] ACPI: \_SB_.GSIH: Enabled at IRQ 231806worker2 # [ 1.043722] ACPI: \_SB_.GSIE: Enabled at IRQ 201807lb # [ 1.044621] virtiofs virtio5: discovered new tag: nix-store1808worker1 # [ 1.034337] systemd[1]: Inserted module 'autofs4'1809worker2 # [ 1.045770] ACPI: \_SB_.GSIF: Enabled at IRQ 211810lb # [ 1.046173] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1811worker2 # [ 1.051239] virtiofs virtio5: discovered new tag: nix-store1812lb # [ 1.053003] virtiofs virtio6: discovered new tag: shared1813worker2 # [ 1.052862] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1814lb # [ 1.054498] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1815lb # [ 1.057913] virtiofs virtio7: discovered new tag: xchg1816lb # [ 1.059395] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1817worker2 # [ 1.059538] virtiofs virtio6: discovered new tag: shared1818worker2 # [ 1.061239] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1819worker2 # [ 1.064708] virtiofs virtio7: discovered new tag: xchg1820worker1 # [ 1.054875] fuse: init (API version 7.45)1821worker2 # [ 1.066240] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1822worker1 # [ 1.062727] ACPI: \_SB_.GSIG: Enabled at IRQ 221823worker1 # [ 1.065571] ACPI: \_SB_.GSIH: Enabled at IRQ 231824lb # [ 1.078521] systemd[1]: Successfully made /usr/ read-only.1825worker1 # [ 1.069025] ACPI: \_SB_.GSIE: Enabled at IRQ 201826worker1 # [ 1.071339] ACPI: \_SB_.GSIF: Enabled at IRQ 211827worker1 # [ 1.077384] virtiofs virtio5: discovered new tag: nix-store1828worker2 # [ 1.089170] systemd[1]: Successfully made /usr/ read-only.1829worker1 # [ 1.079000] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1830worker1 # [ 1.086422] virtiofs virtio6: discovered new tag: shared1831worker1 # [ 1.087963] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1832worker1 # [ 1.091921] virtiofs virtio7: discovered new tag: xchg1833worker1 # [ 1.093567] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1834worker1 # [ 1.114570] systemd[1]: Successfully made /usr/ read-only.1835client # [ 1.409731] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836client # [ 1.421684] systemd[1]: Detected virtualization kvm.1837client # [ 1.423777] systemd[1]: Detected architecture x86-64.1838client # [ 1.425876] systemd[1]: Running in initrd.1839client # [ 1.428384] systemd[1]: Initializing machine ID from random generator.1840client # [ 1.431217] systemd[1]: Hostname set to <client>.1841lb # [ 1.415173] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842lb # [ 1.427199] systemd[1]: Detected virtualization kvm.1843lb # [ 1.429431] systemd[1]: Detected architecture x86-64.1844lb # [ 1.431615] systemd[1]: Running in initrd.1845lb # [ 1.434115] systemd[1]: Initializing machine ID from random generator.1846worker2 # [ 1.425832] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1847lb # [ 1.437037] systemd[1]: Hostname set to <lb>.1848worker2 # [ 1.437937] systemd[1]: Detected virtualization kvm.1849worker2 # [ 1.440163] systemd[1]: Detected architecture x86-64.1850worker2 # [ 1.442265] systemd[1]: Running in initrd.1851worker2 # [ 1.444733] systemd[1]: Initializing machine ID from random generator.1852worker2 # [ 1.447637] systemd[1]: Hostname set to <worker2>.1853worker1 # [ 1.450058] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854worker1 # [ 1.456626] systemd[1]: Detected virtualization kvm.1855worker1 # [ 1.458900] systemd[1]: Detected architecture x86-64.1856worker1 # [ 1.461041] systemd[1]: Running in initrd.1857worker1 # [ 1.463450] systemd[1]: Initializing machine ID from random generator.1858worker1 # [ 1.466340] systemd[1]: Hostname set to <worker1>.1859client # [ 1.656496] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860lb # [ 1.660940] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861client # [ 1.705597] systemd[1]: Queued start job for default target Initrd Default Target.1862worker2 # [ 1.668738] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863client # [ 1.709332] systemd[1]: Created slice Slice /system/modprobe.1864client # [ 1.710469] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1865client # [ 1.711841] systemd[1]: Expecting device /dev/disk/by-label/nixos...1866client # [ 1.712889] systemd[1]: Reached target Path Units.1867client # [ 1.713767] systemd[1]: Reached target Slice Units.1868client # [ 1.714601] systemd[1]: Reached target Swaps.1869client # [ 1.715458] systemd[1]: Reached target Timer Units.1870client # [ 1.716404] systemd[1]: Listening on D-Bus System Message Bus Socket.1871client # [ 1.717589] systemd[1]: Listening on Journal Socket (/dev/log).1872client # [ 1.718683] systemd[1]: Listening on Journal Sockets.1873client # [ 1.719631] systemd[1]: Listening on udev Control Socket.1874client # [ 1.720633] systemd[1]: Listening on udev Kernel Socket.1875client # [ 1.721536] systemd[1]: Reached target Socket Units.1876client # [ 1.723373] systemd[1]: Starting Create List of Static Device Nodes...1877client # [ 1.727040] systemd[1]: Starting Load Kernel Module configfs...1878worker1 # [ 1.695197] systemd[1]: bpf-restrict-fs: LSM BPF program attached1879client # [ 1.737560] systemd[1]: Starting Journal Service...1880lb # [ 1.714528] systemd[1]: Queued start job for default target Initrd Default Target.1881lb # [ 1.718985] systemd[1]: Created slice Slice /system/modprobe.1882lb # [ 1.720253] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1883lb # [ 1.721762] systemd[1]: Expecting device /dev/disk/by-label/nixos...1884lb # [ 1.722902] systemd[1]: Reached target Path Units.1885worker2 # [ 1.722747] systemd[1]: Queued start job for default target Initrd Default Target.1886lb # [ 1.723808] systemd[1]: Reached target Slice Units.1887lb # [ 1.724650] systemd[1]: Reached target Swaps.1888lb # [ 1.725457] systemd[1]: Reached target Timer Units.1889lb # [ 1.726444] systemd[1]: Listening on D-Bus System Message Bus Socket.1890worker2 # [ 1.727409] systemd[1]: Created slice Slice /system/modprobe.1891client # [ 1.764752] systemd[1]: Starting Load Kernel Modules...1892lb # [ 1.727660] systemd[1]: Listening on Journal Socket (/dev/log).1893lb # [ 1.728790] systemd[1]: Listening on Journal Sockets.1894worker2 # [ 1.728596] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1895lb # [ 1.729775] systemd[1]: Listening on udev Control Socket.1896worker2 # [ 1.730038] systemd[1]: Expecting device /dev/disk/by-label/nixos...1897lb # [ 1.730807] systemd[1]: Listening on udev Kernel Socket.1898worker2 # [ 1.731186] systemd[1]: Reached target Path Units.1899lb # [ 1.731746] systemd[1]: Reached target Socket Units.1900worker2 # [ 1.732129] systemd[1]: Reached target Slice Units.1901client # [ 1.769087] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1902worker2 # [ 1.732981] systemd[1]: Reached target Swaps.1903lb # [ 1.733536] systemd[1]: Starting Create List of Static Device Nodes...1904worker2 # [ 1.733816] systemd[1]: Reached target Timer Units.1905worker2 # [ 1.734847] systemd[1]: Listening on D-Bus System Message Bus Socket.1906worker2 # [ 1.736191] systemd[1]: Listening on Journal Socket (/dev/log).1907worker2 # [ 1.737403] systemd[1]: Listening on Journal Sockets.1908lb # [ 1.737792] systemd[1]: Starting Load Kernel Module configfs...1909worker2 # [ 1.738429] systemd[1]: Listening on udev Control Socket.1910worker2 # [ 1.739538] systemd[1]: Listening on udev Kernel Socket.1911worker2 # [ 1.740549] systemd[1]: Reached target Socket Units.1912client # [ 1.779238] systemd[1]: Starting Coldplug All udev Devices...1913worker2 # [ 1.742429] systemd[1]: Starting Create List of Static Device Nodes...1914worker2 # [ 1.747102] systemd[1]: Starting Load Kernel Module configfs...1915client # [ 1.786132] systemd-journald[66]: Collecting audit messages is disabled.1916client # [ 1.792623] systemd[1]: Finished Create List of Static Device Nodes.1917lb # [ 1.747314] systemd[1]: Starting Journal Service...1918worker1 # [ 1.749473] systemd[1]: Queued start job for default target Initrd Default Target.1919client # [ 1.797740] systemd[1]: modprobe@configfs.service: Deactivated successfully.1920worker1 # [ 1.753859] systemd[1]: Created slice Slice /system/modprobe.1921worker1 # [ 1.755091] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1922worker1 # [ 1.756574] systemd[1]: Expecting device /dev/disk/by-label/nixos...1923worker1 # [ 1.757732] systemd[1]: Reached target Path Units.1924worker2 # [ 1.756757] systemd[1]: Starting Journal Service...1925worker1 # [ 1.758630] systemd[1]: Reached target Slice Units.1926client # [ 1.806396] systemd[1]: Finished Load Kernel Module configfs.1927worker1 # [ 1.759511] systemd[1]: Reached target Swaps.1928worker1 # [ 1.760349] systemd[1]: Reached target Timer Units.1929worker1 # [ 1.761329] systemd[1]: Listening on D-Bus System Message Bus Socket.1930lb # [ 1.772906] systemd[1]: Starting Load Kernel Modules...1931worker1 # [ 1.762610] systemd[1]: Listening on Journal Socket (/dev/log).1932worker1 # [ 1.763801] systemd[1]: Listening on Journal Sockets.1933worker1 # [ 1.764838] systemd[1]: Listening on udev Control Socket.1934client # [ 1.812477] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1935worker1 # [ 1.765885] systemd[1]: Listening on udev Kernel Socket.1936worker1 # [ 1.766864] systemd[1]: Reached target Socket Units.1937lb # [ 1.776819] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1938worker1 # [ 1.768699] systemd[1]: Starting Create List of Static Device Nodes...1939worker2 # [ 1.782670] systemd[1]: Starting Load Kernel Modules...1940worker1 # [ 1.772625] systemd[1]: Starting Load Kernel Module configfs...1941lb # [ 1.784909] systemd[1]: Starting Coldplug All udev Devices...1942client # [ 1.822076] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1943worker2 # [ 1.787173] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1944lb # [ 1.796257] systemd-journald[66]: Collecting audit messages is disabled.1945worker2 # [ 1.797272] systemd[1]: Starting Coldplug All udev Devices...1946lb # [ 1.798190] systemd[1]: Finished Create List of Static Device Nodes.1947client # [ 1.834970] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1948worker1 # [ 1.781619] systemd[1]: Starting Journal Service...1949lb # [ 1.804193] systemd[1]: modprobe@configfs.service: Deactivated successfully.1950worker2 # [ 1.807670] systemd-journald[66]: Collecting audit messages is disabled.1951client # [ 1.845418] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1952lb # [ 1.811126] systemd[1]: Finished Load Kernel Module configfs.1953worker1 # [ 1.807072] systemd[1]: Starting Load Kernel Modules...1954worker2 # [ 1.817383] systemd[1]: Finished Create List of Static Device Nodes.1955lb # [ 1.817194] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1956worker2 # [ 1.821714] systemd[1]: modprobe@configfs.service: Deactivated successfully.1957worker1 # [ 1.811676] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1958worker2 # [ 1.828440] systemd[1]: Finished Load Kernel Module configfs.1959worker1 # [ 1.818335] systemd[1]: Starting Coldplug All udev Devices...1960lb # [ 1.830793] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1961worker2 # [ 1.834491] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1962client # [ 1.874189] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1963lb # [ 1.837411] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1964worker1 # [ 1.831971] systemd[1]: Finished Create List of Static Device Nodes.1965worker1 # [ 1.833245] systemd-journald[66]: Collecting audit messages is disabled.1966worker2 # [ 1.846111] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1967client # [ 1.883622] systemd[1]: Starting Create Static Device Nodes in /dev...1968lb # [ 1.846879] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1969worker1 # [ 1.837501] systemd[1]: modprobe@configfs.service: Deactivated successfully.1970worker2 # [ 1.850056] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1971worker1 # [ 1.844993] systemd[1]: Finished Load Kernel Module configfs.1972client # [ 1.895045] systemd[1]: Finished Load Kernel Modules.1973worker2 # [ 1.860082] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1974worker1 # [ 1.851067] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1975client # [ 1.770282] systemd-modules-load[68]: Inserted module 'dm_mod'1976client # [ 1.907805] systemd[1]: Starting Apply Kernel Variables...1977client # [ 1.772808] systemd-modules-load[68]: Inserted module 'virtio_balloon'1978lb # [ 1.871345] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1979client # [ 1.773897] systemd-modules-load[68]: Inserted module 'virtio_gpu'1980worker1 # [ 1.864614] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1981client # [ 1.914311] systemd[1]: Started Journal Service.1982lb # [ 1.879923] systemd[1]: Starting Create Static Device Nodes in /dev...1983worker1 # [ 1.873855] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1984lb # [ 1.890264] systemd[1]: Finished Load Kernel Modules.1985worker1 # [ 1.882758] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1986worker2 # [ 1.897118] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1987client # [ 1.799502] systemd[1]: Finished Create Static Device Nodes in /dev.1988lb # [ 1.898392] systemd[1]: Starting Apply Kernel Variables...1989client # [ 1.800883] systemd[1]: Reached target Preparation for Local File Systems.1990client # [ 1.803084] systemd[1]: Reached target Local File Systems.1991worker2 # [ 1.903116] systemd[1]: Finished Load Kernel Modules.1992client # [ 1.807902] systemd[1]: Starting Create System Files and Directories...1993worker2 # [ 1.910273] systemd[1]: Starting Apply Kernel Variables...1994client # [ 1.816707] systemd[1]: Starting Rule-based Manager for Device Events and Files...1995worker1 # [ 1.905645] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1996lb # [ 1.918865] systemd[1]: Finished Create Static Device Nodes in /dev.1997worker2 # [ 1.920107] systemd[1]: Starting Create Static Device Nodes in /dev...1998lb # [ 1.788306] systemd-modules-load[68]: Inserted module 'dm_mod'1999lb # [ 1.789648] systemd-modules-load[68]: Inserted module 'virtio_balloon'2000client # [ 1.823738] systemd[1]: Finished Apply Kernel Variables.
loading 2,762 hidden lines…
4763lb # [ 37.993981] niks3-server[988]: 2026/09/22 22:20:51 INFO Completed upload id=54764worker1 # [ 37.982925] nix-grpc-daemon[859]: time=2026-09-22T22:20:52.001Z level=INFO msg="Upload complete. (180ms)"4765worker1 # [ 37.985976] nix-grpc-daemon[725]: ts=2026-09-22T22:20:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_in=7524766lb # [ 37.999704] niks3-server[988]: 2026/09/22 22:20:51 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4767worker1 # [ 37.992431] nix-grpc-daemon[725]: ts=2026-09-22T22:20:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592144768worker1 # [ 37.999765] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4769worker1 # [ 38.002855] nix-daemon[846]: reaped child process 930, status = succeeded4770worker1 # [ 38.009572] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4771client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4772client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4773client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4774client # farm-top> LOG-farm-top-t14775worker1 # [ 39.247579] nix-daemon[846]: reaped child process 939, status = succeeded4776worker1 # [ 39.335383] nix-daemon[846]: accepted connection from pid 955, user nix-grpc-daemon (trusted)4777worker1 # [ 39.350200] nix-daemon[846]: reaped child process 958, status = succeeded4778lb # [ 39.369524] niks3-server[988]: 2026/09/22 22:20:52 INFO Received uploads request method=POST path=/api/pending_closures4779worker1 # [ 39.370712] nix-grpc-daemon[859]: time=2026-09-22T22:20:53.388Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4780worker1 # [ 39.372481] nix-grpc-daemon[859]: time=2026-09-22T22:20:53.390Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4781lb # [ 39.416510] niks3-server[988]: 2026/09/22 22:20:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4782lb # [ 39.424717] niks3-server[988]: 2026/09/22 22:20:52 INFO Signed narinfos id=6 count=14783worker1 # [ 39.417473] nix-grpc-daemon[859]: time=2026-09-22T22:20:53.435Z level=INFO msg="Uploading 1 narinfos"4784lb # [ 39.436550] niks3-server[988]: 2026/09/22 22:20:52 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4785lb # [ 39.441879] niks3-server[988]: 2026/09/22 22:20:52 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4786lb # [ 39.450436] niks3-server[988]: 2026/09/22 22:20:52 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4787lb # [ 39.457873] niks3-server[988]: 2026/09/22 22:20:52 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4788lb # [ 39.462652] niks3-server[988]: 2026/09/22 22:20:52 INFO Completed upload id=64789lb # [ 39.464669] niks3-server[988]: 2026/09/22 22:20:52 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4790worker1 # [ 39.451905] nix-grpc-daemon[859]: time=2026-09-22T22:20:53.470Z level=INFO msg="Upload complete. (232ms)"4791worker1 # [ 39.459857] nix-grpc-daemon[725]: ts=2026-09-22T22:20:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14792worker1 # [ 39.475597] nix-daemon[846]: reaped child process 937, status = succeeded4793client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 5.28 seconds)4794client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4795client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.12 seconds)4796client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4797client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4798client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4799client # copying 3 paths...4800client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4801client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4802client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4803client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.17 seconds)4804(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.65 seconds)4805subtest: repeat is answered Cached by the scheduler without building4806worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4807worker1 # finding garbage collector roots...4808worker1 # deleting specified paths...4809worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4810worker1 # deleting unused links...4811worker1 # note: hard linking is currently saving 0.0 KiB4812worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds)4813worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4814worker2 # finding garbage collector roots...4815worker2 # deleting specified paths...4816worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4817worker2 # deleting unused links...4818worker2 # note: hard linking is currently saving 0.0 KiB4819worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.18 seconds)4820client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24821worker2 # [ 40.352981] nix-grpc-daemon[785]: ts=2026-09-22T22:20:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471064822client # don't know how to build these paths:4823client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4824worker1 # [ 40.352528] nix-grpc-daemon[725]: ts=2026-09-22T22:20:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592144825worker1 # [ 40.363158] nix-grpc-daemon[859]: time=2026-09-22T22:20:54.381Z level=INFO msg="All 2 paths already cached"4826worker2 # [ 40.389285] nix-grpc-daemon[785]: ts=2026-09-22T22:20:53Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14827client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.20 seconds)4828client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4829client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.09 seconds)4830lb: must succeed: curl -sf localhost:9901/clusters4831lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4832lb: must succeed: curl -sf localhost:9901/clusters4833lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4834worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4835worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4836(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.80 seconds)4837subtest: inputs already in the cache are not sent to the scheduler4838lb: must succeed: curl -sf localhost:9901/clusters4839lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4840lb: must succeed: curl -sf localhost:9901/clusters4841lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4842worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4843worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4844client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24845client # don't know how to build these paths:4846client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4847worker1 # [ 40.864776] nix-grpc-daemon[725]: ts=2026-09-22T22:20:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:592144848worker1 # [ 40.871806] nix-grpc-daemon[725]: ts=2026-09-22T22:20:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592144849worker1 # [ 40.878645] nix-grpc-daemon[859]: time=2026-09-22T22:20:54.894Z level=INFO msg="All 2 paths already cached"4850worker2 # [ 40.896770] nix-grpc-daemon[785]: ts=2026-09-22T22:20:54Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04851worker2 # [ 40.901537] nix-grpc-daemon[785]: ts=2026-09-22T22:20:54Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14852worker1 # [ 40.893172] nix-grpc-daemon[725]: ts=2026-09-22T22:20:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592144853worker1 # [ 40.905387] nix-grpc-daemon[725]: ts=2026-09-22T22:20:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592144854worker1 # [ 40.919589] nix-grpc-daemon[859]: time=2026-09-22T22:20:54.937Z level=INFO msg="All 2 paths already cached"4855client # copying 1 paths to 'lb:50051'...4856client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4857worker1 # [ 40.927068] nix-grpc-daemon[725]: ts=2026-09-22T22:20:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:592144858worker1 # [ 40.935742] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4859worker1 # [ 41.036892] nix-daemon[846]: accepted connection from pid 976, user nix-grpc-daemon (trusted)4860worker1 # [ 41.052086] nix-daemon[846]: reaped child process 979, status = succeeded4861lb # [ 41.070533] niks3-server[988]: 2026/09/22 22:20:54 INFO Received uploads request method=POST path=/api/pending_closures4862worker1 # [ 41.067797] nix-grpc-daemon[859]: time=2026-09-22T22:20:55.086Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4863worker1 # [ 41.069739] nix-grpc-daemon[859]: time=2026-09-22T22:20:55.087Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4864lb # [ 41.101322] niks3-server[988]: 2026/09/22 22:20:54 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4865lb # [ 41.117142] niks3-server[988]: 2026/09/22 22:20:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4866lb # [ 41.124597] niks3-server[988]: 2026/09/22 22:20:54 INFO Signed narinfos id=7 count=14867worker1 # [ 41.116841] nix-grpc-daemon[859]: time=2026-09-22T22:20:55.135Z level=INFO msg="Uploading 1 narinfos"4868lb # [ 41.134424] niks3-server[988]: 2026/09/22 22:20:54 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4869lb # [ 41.144143] niks3-server[988]: 2026/09/22 22:20:54 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4870lb # [ 41.149184] niks3-server[988]: 2026/09/22 22:20:54 INFO Completed upload id=74871worker1 # [ 41.138368] nix-grpc-daemon[859]: time=2026-09-22T22:20:55.156Z level=INFO msg="Upload complete. (188ms)"4872lb # [ 41.151069] niks3-server[988]: 2026/09/22 22:20:54 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4873worker1 # [ 41.143079] nix-grpc-daemon[725]: ts=2026-09-22T22:20:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_in=7524874worker1 # [ 41.149304] nix-grpc-daemon[725]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592144875worker1 # [ 41.160536] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4876worker1 # [ 41.169533] nix-daemon[846]: reaped child process 974, status = succeeded4877client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4878worker1 # [ 41.178284] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)4879client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4880client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4881client # farm-top> LOG-farm-top-t1b4882worker1 # [ 42.016574] nix-daemon[846]: reaped child process 983, status = succeeded4883worker1 # [ 42.100411] nix-daemon[846]: accepted connection from pid 999, user nix-grpc-daemon (trusted)4884worker1 # [ 42.111533] nix-daemon[846]: reaped child process 1002, status = succeeded4885lb # [ 42.131284] niks3-server[988]: 2026/09/22 22:20:55 INFO Received uploads request method=POST path=/api/pending_closures4886worker1 # [ 42.129246] nix-grpc-daemon[859]: time=2026-09-22T22:20:56.147Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4887worker1 # [ 42.130657] nix-grpc-daemon[859]: time=2026-09-22T22:20:56.149Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4888lb # [ 42.172782] niks3-server[988]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4889lb # [ 42.181503] niks3-server[988]: 2026/09/22 22:20:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4890lb # [ 42.188583] niks3-server[988]: 2026/09/22 22:20:55 INFO Signed narinfos id=8 count=14891worker1 # [ 42.181735] nix-grpc-daemon[859]: time=2026-09-22T22:20:56.200Z level=INFO msg="Uploading 1 narinfos"4892lb # [ 42.195667] niks3-server[988]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4893lb # [ 42.207787] niks3-server[988]: 2026/09/22 22:20:55 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4894lb # [ 42.213129] niks3-server[988]: 2026/09/22 22:20:55 INFO Completed upload id=84895worker1 # [ 42.203575] nix-grpc-daemon[859]: time=2026-09-22T22:20:56.221Z level=INFO msg="Upload complete. (219ms)"4896lb # [ 42.216854] niks3-server[988]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4897lb # [ 42.220597] niks3-server[988]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4898worker1 # [ 42.211123] nix-grpc-daemon[725]: ts=2026-09-22T22:20:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14899worker1 # [ 42.224480] nix-daemon[846]: reaped child process 981, status = succeeded4900client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.48 seconds)4901client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4902client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.10 seconds)4903lb: must succeed: curl -sf localhost:9901/clusters4904lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4905lb: must succeed: curl -sf localhost:9901/clusters4906lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4907worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4908worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4909(finished: subtest: inputs already in the cache are not sent to the scheduler, in 1.76 seconds)4910subtest: outputs read back through any worker carry the cache signature4911client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24912worker2 # [ 42.554519] nix-grpc-daemon[785]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471064913client # don't know how to build these paths:4914client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4915worker2 # [ 42.561926] nix-grpc-daemon[785]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471064916worker2 # [ 42.576777] nix-grpc-daemon[785]: ts=2026-09-22T22:20:55Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04917worker2 # [ 42.582838] nix-grpc-daemon[785]: ts=2026-09-22T22:20:55Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.4:50051 clients=14918worker2 # [ 42.587688] nix-grpc-daemon[785]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471064919worker2 # [ 42.596471] nix-grpc-daemon[785]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471064920client # copying 1 paths to 'lb:50051'...4921client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4922worker2 # [ 42.605727] nix-grpc-daemon[785]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471064923worker2 # [ 42.616285] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4924worker2 # [ 42.632386] nix-grpc-daemon[785]: ts=2026-09-22T22:20:55Z level=info event=niks3_push_started pid=9234925worker2 # [ 42.750768] nix-grpc-daemon[923]: time=2026-09-22T22:20:55.998Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4926worker2 # [ 42.838552] nix-daemon[870]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4927worker2 # [ 42.854174] nix-daemon[870]: reaped child process 938, status = succeeded4928lb # [ 42.855238] niks3-server[988]: 2026/09/22 22:20:56 INFO Received uploads request method=POST path=/api/pending_closures4929worker2 # [ 42.863332] nix-grpc-daemon[923]: time=2026-09-22T22:20:56.111Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4930worker2 # [ 42.864838] nix-grpc-daemon[923]: time=2026-09-22T22:20:56.112Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4931lb # [ 42.893837] niks3-server[988]: 2026/09/22 22:20:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4932worker2 # [ 42.902089] nix-grpc-daemon[923]: time=2026-09-22T22:20:56.150Z level=INFO msg="Uploading 1 narinfos"4933lb # [ 42.903227] niks3-server[988]: 2026/09/22 22:20:56 INFO Signed narinfos id=9 count=14934lb # [ 42.905880] niks3-server[988]: 2026/09/22 22:20:56 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4935lb # [ 42.912946] niks3-server[988]: 2026/09/22 22:20:56 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4936lb # [ 42.927208] niks3-server[988]: 2026/09/22 22:20:56 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4937worker2 # [ 42.932114] nix-grpc-daemon[923]: time=2026-09-22T22:20:56.180Z level=INFO msg="Upload complete. (176ms)"4938lb # [ 42.932664] niks3-server[988]: 2026/09/22 22:20:56 INFO Completed upload id=94939worker2 # [ 42.934770] nix-grpc-daemon[785]: ts=2026-09-22T22:20:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_in=5044940lb # [ 42.936484] niks3-server[988]: 2026/09/22 22:20:56 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4941worker2 # [ 42.942783] nix-grpc-daemon[785]: ts=2026-09-22T22:20:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471064942worker2 # [ 42.952394] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4943worker2 # [ 42.953758] nix-daemon[870]: reaped child process 921, status = succeeded4944client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4945worker2 # [ 42.958878] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4946client # farm-a> worker2: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4947client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4948client # farm-a> LOG-farm-a-sig4949worker2 # [ 43.973427] nix-daemon[870]: reaped child process 942, status = succeeded4950worker2 # [ 44.044870] nix-daemon[870]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4951worker2 # [ 44.059084] nix-daemon[870]: reaped child process 961, status = succeeded4952lb # [ 44.064497] niks3-server[988]: 2026/09/22 22:20:57 INFO Received uploads request method=POST path=/api/pending_closures4953worker2 # [ 44.073280] nix-grpc-daemon[923]: time=2026-09-22T22:20:57.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4954worker2 # [ 44.075818] nix-grpc-daemon[923]: time=2026-09-22T22:20:57.322Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4955lb # [ 44.107231] niks3-server[988]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4956lb # [ 44.114409] niks3-server[988]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4957lb # [ 44.117412] niks3-server[988]: 2026/09/22 22:20:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4958worker2 # [ 44.122933] nix-grpc-daemon[923]: time=2026-09-22T22:20:57.370Z level=INFO msg="Uploading 1 narinfos"4959lb # [ 44.123301] niks3-server[988]: 2026/09/22 22:20:57 INFO Signed narinfos id=10 count=14960lb # [ 44.133481] niks3-server[988]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4961lb # [ 44.138583] niks3-server[988]: 2026/09/22 22:20:57 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4962lb # [ 44.144383] niks3-server[988]: 2026/09/22 22:20:57 INFO Completed upload id=104963worker2 # [ 44.144367] nix-grpc-daemon[923]: time=2026-09-22T22:20:57.392Z level=INFO msg="Upload complete. (178ms)"4964lb # [ 44.146163] niks3-server[988]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4965worker2 # [ 44.150583] nix-grpc-daemon[785]: ts=2026-09-22T22:20:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14966worker2 # [ 44.160613] nix-grpc-daemon[785]: ts=2026-09-22T22:20:57Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04967worker2 # [ 44.164595] nix-grpc-daemon[785]: ts=2026-09-22T22:20:57Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=14968worker2 # [ 44.172486] nix-daemon[870]: reaped child process 940, status = succeeded4969worker2 # [ 44.203757] nix-grpc-daemon[785]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471064970worker2 # [ 44.215224] nix-grpc-daemon[785]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471064971worker2 # [ 44.227441] nix-grpc-daemon[923]: time=2026-09-22T22:20:57.475Z level=INFO msg="All 1 paths already cached"4972client # copying 1 paths to 'lb:50051'...4973client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4974worker2 # [ 44.234688] nix-grpc-daemon[785]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471064975worker2 # [ 44.244227] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4976worker2 # [ 44.332379] nix-daemon[870]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4977worker2 # [ 44.345949] nix-daemon[870]: reaped child process 970, status = succeeded4978lb # [ 44.352773] niks3-server[988]: 2026/09/22 22:20:57 INFO Received uploads request method=POST path=/api/pending_closures4979worker2 # [ 44.359934] nix-grpc-daemon[923]: time=2026-09-22T22:20:57.607Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4980worker2 # [ 44.361415] nix-grpc-daemon[923]: time=2026-09-22T22:20:57.609Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4981lb # [ 44.382867] niks3-server[988]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4982lb # [ 44.391863] niks3-server[988]: 2026/09/22 22:20:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4983lb # [ 44.398690] niks3-server[988]: 2026/09/22 22:20:57 INFO Signed narinfos id=11 count=14984worker2 # [ 44.401609] nix-grpc-daemon[923]: time=2026-09-22T22:20:57.649Z level=INFO msg="Uploading 1 narinfos"4985lb # [ 44.405610] niks3-server[988]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4986lb # [ 44.419108] niks3-server[988]: 2026/09/22 22:20:57 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4987lb # [ 44.426262] niks3-server[988]: 2026/09/22 22:20:57 INFO Completed upload id=114988worker2 # [ 44.426043] nix-grpc-daemon[923]: time=2026-09-22T22:20:57.673Z level=INFO msg="Upload complete. (171ms)"4989lb # [ 44.429681] niks3-server[988]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4990worker2 # [ 44.430198] nix-grpc-daemon[785]: ts=2026-09-22T22:20:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_in=6244991worker2 # [ 44.435457] nix-grpc-daemon[785]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471064992worker2 # [ 44.444697] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4993worker2 # [ 44.452717] nix-daemon[870]: reaped child process 965, status = succeeded4994client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4995worker2 # [ 44.456797] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4996client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4997client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4998lb # [ 44.680918] envoy[906]: conn peer=[2001:db8:1::1]:53874 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9494 tx=4371 ms=51394999lb # [ 44.684722] envoy[906]: conn peer=[2001:db8:1::1]:53878 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=525000lb # [ 44.688609] envoy[906]: conn peer=[2001:db8:1::1]:53880 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4165 tx=1877 ms=13575001client # farm-b> LOG-farm-b-sig5002worker2 # [ 44.871315] nix-daemon[870]: reaped child process 974, status = succeeded5003worker2 # [ 44.941970] nix-daemon[870]: accepted connection from pid 990, user nix-grpc-daemon (trusted)5004worker2 # [ 44.956227] nix-daemon[870]: reaped child process 993, status = succeeded5005lb # [ 44.960786] niks3-server[988]: 2026/09/22 22:20:58 INFO Received uploads request method=POST path=/api/pending_closures5006worker2 # [ 44.968320] nix-grpc-daemon[923]: time=2026-09-22T22:20:58.216Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5007worker2 # [ 44.969866] nix-grpc-daemon[923]: time=2026-09-22T22:20:58.217Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"5008lb # [ 45.002832] niks3-server[988]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls5009lb # [ 45.005874] niks3-server[988]: 2026/09/22 22:20:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5010lb # [ 45.015584] niks3-server[988]: 2026/09/22 22:20:58 INFO Signed narinfos id=12 count=15011worker2 # [ 45.018114] nix-grpc-daemon[923]: time=2026-09-22T22:20:58.266Z level=INFO msg="Uploading 1 narinfos"5012lb # [ 45.020980] niks3-server[988]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv5013lb # [ 45.025316] niks3-server[988]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst5014lb # [ 45.040550] niks3-server[988]: 2026/09/22 22:20:58 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5015lb # [ 45.044520] niks3-server[988]: 2026/09/22 22:20:58 INFO Completed upload id=125016worker2 # [ 45.044433] nix-grpc-daemon[923]: time=2026-09-22T22:20:58.292Z level=INFO msg="Upload complete. (182ms)"5017lb # [ 45.046866] niks3-server[988]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo5018worker2 # [ 45.046868] nix-grpc-daemon[785]: ts=2026-09-22T22:20:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=15019worker2 # [ 45.054319] nix-daemon[870]: reaped child process 972, status = succeeded5020worker2 # [ 45.095458] nix-grpc-daemon[785]: ts=2026-09-22T22:20:58Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=05021worker2 # [ 45.097684] nix-grpc-daemon[785]: ts=2026-09-22T22:20:58Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=15022worker2 # [ 45.100433] nix-grpc-daemon[785]: ts=2026-09-22T22:20:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065023worker2 # [ 45.108281] nix-grpc-daemon[785]: ts=2026-09-22T22:20:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471065024worker2 # [ 45.117389] nix-grpc-daemon[923]: time=2026-09-22T22:20:58.365Z level=INFO msg="All 2 paths already cached"5025client # copying 1 paths to 'lb:50051'...5026worker2 # [ 45.123117] nix-grpc-daemon[785]: ts=2026-09-22T22:20:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471065027client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...5028worker2 # [ 45.128941] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5029worker2 # [ 45.209586] nix-daemon[870]: accepted connection from pid 999, user nix-grpc-daemon (trusted)5030worker2 # [ 45.224082] nix-daemon[870]: reaped child process 1002, status = succeeded5031lb # [ 45.230073] niks3-server[988]: 2026/09/22 22:20:58 INFO Received uploads request method=POST path=/api/pending_closures5032worker2 # [ 45.239753] nix-grpc-daemon[923]: time=2026-09-22T22:20:58.487Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5033worker2 # [ 45.241289] nix-grpc-daemon[923]: time=2026-09-22T22:20:58.489Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"5034lb # [ 45.264429] niks3-server[988]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst5035lb # [ 45.268684] niks3-server[988]: 2026/09/22 22:20:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5036lb # [ 45.273703] niks3-server[988]: 2026/09/22 22:20:58 INFO Signed narinfos id=13 count=15037worker2 # [ 45.276682] nix-grpc-daemon[923]: time=2026-09-22T22:20:58.524Z level=INFO msg="Uploading 1 narinfos"5038lb # [ 45.284204] niks3-server[988]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls5039lb # [ 45.293139] niks3-server[988]: 2026/09/22 22:20:58 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5040worker2 # [ 45.298825] nix-grpc-daemon[923]: time=2026-09-22T22:20:58.546Z level=INFO msg="Upload complete. (165ms)"5041lb # [ 45.299088] niks3-server[988]: 2026/09/22 22:20:58 INFO Completed upload id=135042worker2 # [ 45.301158] nix-grpc-daemon[785]: ts=2026-09-22T22:20:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_in=7525043worker2 # [ 45.305913] nix-grpc-daemon[785]: ts=2026-09-22T22:20:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065044lb # [ 45.306474] niks3-server[988]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo5045worker2 # [ 45.311887] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5046worker2 # [ 45.314215] nix-daemon[870]: reaped child process 997, status = succeeded5047worker2 # [ 45.319346] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5048client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...5049client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5050client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5051client # farm-top> LOG-farm-top-sig5052worker2 # [ 46.145981] nix-daemon[870]: reaped child process 1006, status = succeeded5053worker2 # [ 46.218195] nix-daemon[870]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)5054worker2 # [ 46.231681] nix-daemon[870]: reaped child process 1025, status = succeeded5055lb # [ 46.237915] niks3-server[988]: 2026/09/22 22:20:59 INFO Received uploads request method=POST path=/api/pending_closures5056worker2 # [ 46.249824] nix-grpc-daemon[923]: time=2026-09-22T22:20:59.497Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5057worker2 # [ 46.251543] nix-grpc-daemon[923]: time=2026-09-22T22:20:59.499Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5058lb # [ 46.284129] niks3-server[988]: 2026/09/22 22:20:59 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5059lb # [ 46.290864] niks3-server[988]: 2026/09/22 22:20:59 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5060lb # [ 46.295098] niks3-server[988]: 2026/09/22 22:20:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5061lb # [ 46.299770] niks3-server[988]: 2026/09/22 22:20:59 INFO Signed narinfos id=14 count=15062worker2 # [ 46.303301] nix-grpc-daemon[923]: time=2026-09-22T22:20:59.551Z level=INFO msg="Uploading 1 narinfos"5063lb # [ 46.307349] niks3-server[988]: 2026/09/22 22:20:59 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5064lb # [ 46.313074] niks3-server[988]: 2026/09/22 22:20:59 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5065lb # [ 46.316171] niks3-server[988]: 2026/09/22 22:20:59 INFO Completed upload id=145066worker2 # [ 46.317459] nix-grpc-daemon[923]: time=2026-09-22T22:20:59.565Z level=INFO msg="Upload complete. (191ms)"5067lb # [ 46.320526] niks3-server[988]: 2026/09/22 22:20:59 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5068worker2 # [ 46.324686] nix-grpc-daemon[785]: ts=2026-09-22T22:20:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15069worker2 # [ 46.336953] nix-daemon[870]: reaped child process 1004, status = succeeded5070client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 3.93 seconds)5071client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5072client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds)5073client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5074worker1 # [ 46.570566] nix-grpc-daemon[725]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:446065075client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5076client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5077client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5078client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5079worker1 # [ 46.597066] nix-grpc-daemon[725]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:446065080worker1 # [ 46.603557] nix-grpc-daemon[725]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5081worker1 # [ 46.618721] nix-grpc-daemon[725]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5082worker1 # [ 46.632264] nix-grpc-daemon[725]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5083client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.21 seconds)5084client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5085worker2 # [ 46.798934] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:378345086worker2 # [ 46.841466] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:378345087client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.22 seconds)5088client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5089client # finding garbage collector roots...5090client # removing stale temporary roots file "/nix/var/nix/temproots/856"5091client # removing stale temporary roots file "/nix/var/nix/temproots/843"5092client # removing stale temporary roots file "/nix/var/nix/temproots/877"5093client # removing stale temporary roots file "/nix/var/nix/temproots/864"5094client # removing stale temporary roots file "/nix/var/nix/temproots/806"5095client # removing stale temporary roots file "/nix/var/nix/temproots/822"5096client # removing stale temporary roots file "/nix/var/nix/temproots/900"5097client # removing stale temporary roots file "/nix/var/nix/temproots/885"5098client # deleting specified paths...5099client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5100client # deleting unused links...5101client # note: hard linking is currently saving 0.0 KiB5102worker2 # [ 47.158125] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471065103worker2 # [ 47.163851] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471065104worker2 # [ 47.168506] nix-grpc-daemon[923]: time=2026-09-22T22:21:00.415Z level=INFO msg="All 1 paths already cached"5105worker1 # [ 47.164601] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:592145106worker2 # [ 47.183101] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:471065107worker2 # [ 47.189954] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:471065108client # copying 3 paths...5109worker1 # [ 47.188881] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:592145110client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5111worker1 # [ 47.204719] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:592145112worker1 # [ 47.207769] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_out=2325113worker2 # [ 47.222957] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:471065114worker1 # [ 47.218727] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:592145115worker2 # [ 47.228634] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_out=1765116worker1 # [ 47.220697] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_out=1285117client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5118client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5119client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.40 seconds)5120(finished: subtest: outputs read back through any worker carry the cache signature, in 4.88 seconds)5121subtest: two clients wanting the same drv build it once5122client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5123client # [ 47.366664] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5124client # Running as unit: dedup1.service; invocation ID: 39e07cb1abdb42939af89b940e71f5d85125client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)5126client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5127client # [ 47.496946] nix[953]: don't know how to build these paths:5128client # [ 47.498374] nix[953]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5129worker2 # [ 47.462425] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471065130worker2 # [ 47.473735] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05131worker2 # [ 47.477959] nix-grpc-daemon[785]: ts=2026-09-22T22:21:00Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=15132worker1 # [ 47.507590] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145133worker1 # [ 47.515934] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145134client # [ 47.567158] nix[953]: copying 1 paths to 'lb:50051'...5135worker1 # [ 47.523043] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:592145136client # [ 47.569651] nix[953]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5137worker1 # [ 47.531415] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5138worker1 # [ 47.622404] nix-daemon[846]: accepted connection from pid 1009, user nix-grpc-daemon (trusted)5139worker1 # [ 47.636804] nix-daemon[846]: reaped child process 1012, status = succeeded5140lb # [ 47.656962] niks3-server[988]: 2026/09/22 22:21:01 INFO Received uploads request method=POST path=/api/pending_closures5141worker1 # [ 47.649935] nix-grpc-daemon[859]: time=2026-09-22T22:21:01.668Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5142worker1 # [ 47.651863] nix-grpc-daemon[859]: time=2026-09-22T22:21:01.669Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5143lb # [ 47.671673] niks3-server[988]: 2026/09/22 22:21:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5144lb # [ 47.674731] niks3-server[988]: 2026/09/22 22:21:01 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5145lb # [ 47.677436] niks3-server[988]: 2026/09/22 22:21:01 INFO Signed narinfos id=15 count=15146worker1 # [ 47.667661] nix-grpc-daemon[859]: time=2026-09-22T22:21:01.685Z level=INFO msg="Uploading 1 narinfos"5147lb # [ 47.679986] niks3-server[988]: 2026/09/22 22:21:01 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5148lb # [ 47.684355] niks3-server[988]: 2026/09/22 22:21:01 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5149lb # [ 47.687028] niks3-server[988]: 2026/09/22 22:21:01 INFO Completed upload id=155150worker1 # [ 47.675890] nix-grpc-daemon[859]: time=2026-09-22T22:21:01.694Z level=INFO msg="Upload complete. (138ms)"5151worker1 # [ 47.678131] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_in=4325152lb # [ 47.690130] niks3-server[988]: 2026/09/22 22:21:01 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5153worker1 # [ 47.681368] nix-grpc-daemon[725]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145154worker1 # [ 47.688134] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5155worker1 # [ 47.690359] nix-daemon[846]: reaped child process 1007, status = succeeded5156worker1 # [ 47.695454] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5157client # [ 47.739996] nix[953]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5158client # [ 48.420822] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5159client # Running as unit: dedup2.service; invocation ID: e090d59a70c74961ae6387e4d1f515e15160client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)5161worker2 # [ 48.547212] nix-grpc-daemon[785]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471065162client # [ 48.587211] nix[969]: don't know how to build these paths:5163client # [ 48.588244] nix[969]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5164worker2 # [ 48.558319] nix-grpc-daemon[785]: ts=2026-09-22T22:21:01Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05165worker1 # [ 48.552312] nix-grpc-daemon[725]: ts=2026-09-22T22:21:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145166client # [ 48.600195] nix[969]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5167client: must succeed: sleep 25168client: (finished: must succeed: sleep 2, in 2.03 seconds)5169client: waiting for success: ! systemctl is-active dedup1 dedup25170worker1 # [ 51.713692] nix-daemon[846]: reaped child process 1016, status = succeeded5171worker1 # [ 51.791457] nix-daemon[846]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)5172worker1 # [ 51.804225] nix-daemon[846]: reaped child process 1049, status = succeeded5173lb # [ 51.824489] niks3-server[988]: 2026/09/22 22:21:05 INFO Received uploads request method=POST path=/api/pending_closures5174worker1 # [ 51.821653] nix-grpc-daemon[859]: time=2026-09-22T22:21:05.839Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5175worker1 # [ 51.823502] nix-grpc-daemon[859]: time=2026-09-22T22:21:05.841Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5176lb # [ 51.868815] niks3-server[988]: 2026/09/22 22:21:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5177lb # [ 51.875259] niks3-server[988]: 2026/09/22 22:21:05 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5178lb # [ 51.881363] niks3-server[988]: 2026/09/22 22:21:05 INFO Signed narinfos id=16 count=15179worker1 # [ 51.873352] nix-grpc-daemon[859]: time=2026-09-22T22:21:05.891Z level=INFO msg="Uploading 1 narinfos"5180lb # [ 51.886757] niks3-server[988]: 2026/09/22 22:21:05 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5181lb # [ 51.896612] niks3-server[988]: 2026/09/22 22:21:05 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5182lb # [ 51.905357] niks3-server[988]: 2026/09/22 22:21:05 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5183lb # [ 51.910312] niks3-server[988]: 2026/09/22 22:21:05 INFO Completed upload id=165184lb # [ 51.912789] niks3-server[988]: 2026/09/22 22:21:05 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5185worker1 # [ 51.900638] nix-grpc-daemon[859]: time=2026-09-22T22:21:05.919Z level=INFO msg="Upload complete. (196ms)"5186worker1 # [ 51.907122] nix-grpc-daemon[725]: ts=2026-09-22T22:21:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15187worker1 # [ 51.914334] nix-grpc-daemon[725]: ts=2026-09-22T22:21:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15188worker1 # [ 51.928747] nix-daemon[846]: reaped child process 1014, status = succeeded5189client # [ 51.983177] systemd[1]: dedup2.service: Deactivated successfully.5190client # [ 51.986899] systemd[1]: dedup2.service: Consumed 134ms CPU time over 3.565s wall clock time, 35.4M memory peak, 5.9K incoming IP traffic, 5.5K outgoing IP traffic.5191client # [ 52.022789] systemd[1]: dedup1.service: Deactivated successfully.5192client # [ 52.025671] systemd[1]: dedup1.service: Consumed 114ms CPU time over 4.658s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.5193client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)5194client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5195client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5196worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5197worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)5198worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5199worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5200worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5201worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5202worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5203worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5204(finished: subtest: two clients wanting the same drv build it once, in 5.67 seconds)5205subtest: --timeout and --max-silent-time reach the builder5206worker2 # [ 53.118600] nix-grpc-daemon[785]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471065207worker2 # [ 53.130803] nix-grpc-daemon[785]: ts=2026-09-22T22:21:06Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05208worker2 # [ 53.135775] nix-grpc-daemon[785]: ts=2026-09-22T22:21:06Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=15209worker2 # [ 53.142259] nix-grpc-daemon[785]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065210worker2 # [ 53.149903] nix-grpc-daemon[785]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471065211worker2 # [ 53.160697] nix-grpc-daemon[785]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471065212worker2 # [ 53.171437] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5213worker2 # [ 53.267608] nix-daemon[870]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)5214worker2 # [ 53.281080] nix-daemon[870]: reaped child process 1059, status = succeeded5215lb # [ 53.289635] niks3-server[988]: 2026/09/22 22:21:06 INFO Received uploads request method=POST path=/api/pending_closures5216worker2 # [ 53.297655] nix-grpc-daemon[923]: time=2026-09-22T22:21:06.545Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5217worker2 # [ 53.299861] nix-grpc-daemon[923]: time=2026-09-22T22:21:06.547Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5218lb # [ 53.319382] niks3-server[988]: 2026/09/22 22:21:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5219lb # [ 53.326157] niks3-server[988]: 2026/09/22 22:21:06 INFO Signed narinfos id=17 count=15220worker2 # [ 53.329529] nix-grpc-daemon[923]: time=2026-09-22T22:21:06.577Z level=INFO msg="Uploading 1 narinfos"5221lb # [ 53.334428] niks3-server[988]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5222lb # [ 53.343512] niks3-server[988]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5223lb # [ 53.353731] niks3-server[988]: 2026/09/22 22:21:06 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5224lb # [ 53.361402] niks3-server[988]: 2026/09/22 22:21:06 INFO Completed upload id=175225worker2 # [ 53.361391] nix-grpc-daemon[923]: time=2026-09-22T22:21:06.609Z level=INFO msg="Upload complete. (179ms)"5226lb # [ 53.366703] niks3-server[988]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5227worker2 # [ 53.365636] nix-grpc-daemon[785]: ts=2026-09-22T22:21:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_in=4245228worker2 # [ 53.370322] nix-grpc-daemon[785]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065229worker2 # [ 53.379705] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5230worker2 # [ 53.390129] nix-daemon[870]: reaped child process 1054, status = succeeded5231worker2 # [ 53.393425] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5232lb # [ 54.681292] envoy[906]: conn peer=[2001:db8:1::1]:53890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9451 tx=4344 ms=37865233lb # [ 54.685001] envoy[906]: conn peer=[2001:db8:1::1]:44588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2928 ms=1395234lb # [ 54.688613] envoy[906]: conn peer=[2001:db8:1::1]:44618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=655235lb # [ 54.692829] envoy[906]: conn peer=[2001:db8:1::1]:44594 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=795236lb # [ 54.696827] envoy[906]: conn peer=[2001:db8:1::1]:44608 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=725237lb # [ 54.700291] envoy[906]: conn peer=[2001:db8:1::1]:44626 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=33865238lb # [ 54.703839] envoy[906]: conn peer=[2001:db8:1::1]:44622 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1519 ms=45095239worker2 # [ 59.204867] nix-grpc-daemon[785]: ts=2026-09-22T22:21:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=5 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05240worker2 # [ 59.218209] nix-daemon[870]: reaped child process 1061, status = succeeded5241worker2 # [ 59.219279] nix-daemon[870]: reaped child process 1063, status = succeeded5242worker2 # [ 59.406412] nix-grpc-daemon[785]: ts=2026-09-22T22:21:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:471065243worker2 # [ 59.420901] nix-grpc-daemon[785]: ts=2026-09-22T22:21:12Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05244worker2 # [ 59.425774] nix-grpc-daemon[785]: ts=2026-09-22T22:21:12Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=15245worker1 # [ 59.419850] nix-grpc-daemon[725]: ts=2026-09-22T22:21:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145246worker1 # [ 59.430809] nix-grpc-daemon[725]: ts=2026-09-22T22:21:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145247worker1 # [ 59.442337] nix-grpc-daemon[725]: ts=2026-09-22T22:21:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:592145248worker1 # [ 59.452708] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5249worker1 # [ 59.559507] nix-daemon[846]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)5250worker1 # [ 59.574284] nix-daemon[846]: reaped child process 1065, status = succeeded5251lb # [ 59.591919] niks3-server[988]: 2026/09/22 22:21:12 INFO Received uploads request method=POST path=/api/pending_closures5252worker1 # [ 59.589441] nix-grpc-daemon[859]: time=2026-09-22T22:21:13.607Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5253worker1 # [ 59.591061] nix-grpc-daemon[859]: time=2026-09-22T22:21:13.607Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5254lb # [ 59.622128] niks3-server[988]: 2026/09/22 22:21:12 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5255lb # [ 59.629613] niks3-server[988]: 2026/09/22 22:21:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5256lb # [ 59.635433] niks3-server[988]: 2026/09/22 22:21:12 INFO Signed narinfos id=18 count=15257worker1 # [ 59.627731] nix-grpc-daemon[859]: time=2026-09-22T22:21:13.646Z level=INFO msg="Uploading 1 narinfos"5258lb # [ 59.647995] niks3-server[988]: 2026/09/22 22:21:12 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5259lb # [ 59.663466] niks3-server[988]: 2026/09/22 22:21:13 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5260lb # [ 59.666444] niks3-server[988]: 2026/09/22 22:21:13 INFO Completed upload id=185261worker1 # [ 59.655365] nix-grpc-daemon[859]: time=2026-09-22T22:21:13.673Z level=INFO msg="Upload complete. (191ms)"5262lb # [ 59.668399] niks3-server[988]: 2026/09/22 22:21:13 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5263worker1 # [ 59.660506] nix-grpc-daemon[725]: ts=2026-09-22T22:21:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_in=4245264worker1 # [ 59.667159] nix-grpc-daemon[725]: ts=2026-09-22T22:21:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145265worker1 # [ 59.677806] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5266worker1 # [ 59.686684] nix-daemon[846]: reaped child process 1060, status = succeeded5267worker1 # [ 59.692963] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5268lb # [ 64.681529] envoy[906]: conn peer=[2001:db8:1::1]:47278 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=935 ms=61415269worker1 # [ 65.906760] nix-grpc-daemon[725]: ts=2026-09-22T22:21:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05270worker1 # [ 65.930850] nix-daemon[846]: reaped child process 1067, status = succeeded5271worker1 # [ 65.933723] nix-daemon[846]: reaped child process 1069, status = succeeded5272(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.06 seconds)5273subtest: the first client leaving does not fail the second5274client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5275client # [ 66.114293] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5276client # Running as unit: share1.service; invocation ID: dad388937163404999225427355116285277client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5278client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5279client # [ 66.251164] nix[1042]: don't know how to build these paths:5280client # [ 66.252062] nix[1042]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5281worker1 # [ 66.204730] nix-grpc-daemon[725]: ts=2026-09-22T22:21:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:592145282worker2 # [ 66.227502] nix-grpc-daemon[785]: ts=2026-09-22T22:21:19Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05283worker2 # [ 66.231650] nix-grpc-daemon[785]: ts=2026-09-22T22:21:19Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15284worker2 # [ 66.236513] nix-grpc-daemon[785]: ts=2026-09-22T22:21:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065285worker2 # [ 66.247317] nix-grpc-daemon[785]: ts=2026-09-22T22:21:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471065286client # [ 66.291072] nix[1042]: copying 1 paths to 'lb:50051'...5287client # [ 66.291846] nix[1042]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5288worker2 # [ 66.258112] nix-grpc-daemon[785]: ts=2026-09-22T22:21:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471065289worker2 # [ 66.266740] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5290worker2 # [ 66.367399] nix-daemon[870]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)5291worker2 # [ 66.379522] nix-daemon[870]: reaped child process 1093, status = succeeded5292lb # [ 66.388256] niks3-server[988]: 2026/09/22 22:21:19 INFO Received uploads request method=POST path=/api/pending_closures5293worker2 # [ 66.396704] nix-grpc-daemon[923]: time=2026-09-22T22:21:19.644Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5294worker2 # [ 66.398701] nix-grpc-daemon[923]: time=2026-09-22T22:21:19.646Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5295lb # [ 66.418595] niks3-server[988]: 2026/09/22 22:21:19 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5296lb # [ 66.429407] niks3-server[988]: 2026/09/22 22:21:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5297worker2 # [ 66.436679] nix-grpc-daemon[923]: time=2026-09-22T22:21:19.684Z level=INFO msg="Uploading 1 narinfos"5298lb # [ 66.437073] niks3-server[988]: 2026/09/22 22:21:19 INFO Signed narinfos id=19 count=15299lb # [ 66.448124] niks3-server[988]: 2026/09/22 22:21:19 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5300lb # [ 66.458720] niks3-server[988]: 2026/09/22 22:21:19 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5301lb # [ 66.465229] niks3-server[988]: 2026/09/22 22:21:19 INFO Completed upload id=195302worker2 # [ 66.465295] nix-grpc-daemon[923]: time=2026-09-22T22:21:19.713Z level=INFO msg="Upload complete. (185ms)"5303lb # [ 66.468420] niks3-server[988]: 2026/09/22 22:21:19 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5304worker2 # [ 66.470631] nix-grpc-daemon[785]: ts=2026-09-22T22:21:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_in=4325305worker2 # [ 66.478175] nix-grpc-daemon[785]: ts=2026-09-22T22:21:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065306worker2 # [ 66.488700] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5307worker2 # [ 66.497242] nix-daemon[870]: reaped child process 1088, status = succeeded5308worker2 # [ 66.504977] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5309client # [ 66.538876] nix[1042]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5310client # [ 67.176689] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5311client # Running as unit: share2.service; invocation ID: d11afd48c4f54db9abcf967326ab49845312client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5313worker1 # [ 67.268656] nix-grpc-daemon[725]: ts=2026-09-22T22:21:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:592145314client # [ 67.314844] nix[1058]: don't know how to build these paths:5315client # [ 67.315743] nix[1058]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5316worker2 # [ 67.287681] nix-grpc-daemon[785]: ts=2026-09-22T22:21:20Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05317client # [ 67.327488] nix[1058]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5318worker2 # [ 67.294431] nix-grpc-daemon[785]: ts=2026-09-22T22:21:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065319client: must succeed: sleep 2; systemctl kill -s INT share15320client # [ 70.434443] systemd[1]: share1.service: Sent signal SIGINT to main process 1042 (nix) on client request.5321client # [ 70.441784] nix[1042]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5322client # [ 70.448576] nix[1042]: error: interrupted by the user5323client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5324client: waiting for success: ! systemctl is-active share15325client # [ 70.491727] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5326client # [ 70.493438] systemd[1]: share1.service: Failed with result 'exit-code'.5327client # [ 70.495376] systemd[1]: share1.service: Consumed 113ms CPU time over 4.380s wall clock time, 43.4M memory peak, 6K incoming IP traffic, 8K outgoing IP traffic.5328client: (finished: waiting for success: ! systemctl is-active share1, in 1.06 seconds)5329client: waiting for success: ! systemctl is-active share25330worker2 # [ 71.730117] nix-daemon[870]: reaped child process 1097, status = succeeded5331worker2 # [ 71.796465] nix-daemon[870]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5332worker2 # [ 71.809051] nix-daemon[870]: reaped child process 1136, status = succeeded5333lb # [ 71.815836] niks3-server[988]: 2026/09/22 22:21:25 INFO Received uploads request method=POST path=/api/pending_closures5334worker2 # [ 71.820114] nix-grpc-daemon[923]: time=2026-09-22T22:21:25.067Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5335worker2 # [ 71.821773] nix-grpc-daemon[923]: time=2026-09-22T22:21:25.069Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5336lb # [ 71.841392] niks3-server[988]: 2026/09/22 22:21:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5337lb # [ 71.844219] niks3-server[988]: 2026/09/22 22:21:25 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5338lb # [ 71.846508] niks3-server[988]: 2026/09/22 22:21:25 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5339lb # [ 71.850257] niks3-server[988]: 2026/09/22 22:21:25 INFO Signed narinfos id=20 count=15340worker2 # [ 71.850362] nix-grpc-daemon[923]: time=2026-09-22T22:21:25.098Z level=INFO msg="Uploading 1 narinfos"5341lb # [ 71.853946] niks3-server[988]: 2026/09/22 22:21:25 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5342lb # [ 71.859536] niks3-server[988]: 2026/09/22 22:21:25 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5343lb # [ 71.862201] niks3-server[988]: 2026/09/22 22:21:25 INFO Completed upload id=205344worker2 # [ 71.863257] nix-grpc-daemon[923]: time=2026-09-22T22:21:25.111Z level=INFO msg="Upload complete. (146ms)"5345lb # [ 71.866710] niks3-server[988]: 2026/09/22 22:21:25 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5346worker2 # [ 71.866847] nix-grpc-daemon[785]: ts=2026-09-22T22:21:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15347worker2 # [ 71.871144] nix-grpc-daemon[785]: ts=2026-09-22T22:21:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15348worker2 # [ 71.877507] nix-daemon[870]: reaped child process 1095, status = succeeded5349client # [ 71.960633] systemd[1]: share2.service: Deactivated successfully.5350client # [ 71.962158] systemd[1]: share2.service: Consumed 109ms CPU time over 4.785s wall clock time, 35.4M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.5351client: (finished: waiting for success: ! systemctl is-active share2, in 1.06 seconds)5352client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5353client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.02 seconds)5354(finished: subtest: the first client leaving does not fail the second, in 6.70 seconds)5355subtest: build hook: nix-daemon with builders = grpc://lb5356client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5357client # [ 72.857940] systemd[1]: Started Nix Daemon.5358client # [ 72.943098] nix-daemon[1110]: accepted connection from pid 1107, user root (trusted)5359worker1 # [ 73.009274] nix-grpc-daemon[725]: ts=2026-09-22T22:21:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:592145360worker1 # [ 73.014511] nix-grpc-daemon[725]: ts=2026-09-22T22:21:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145361worker2 # [ 73.037368] nix-grpc-daemon[785]: ts=2026-09-22T22:21:26Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05362worker1 # [ 73.027804] nix-grpc-daemon[725]: ts=2026-09-22T22:21:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145363worker2 # [ 73.041533] nix-grpc-daemon[785]: ts=2026-09-22T22:21:26Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15364worker1 # [ 73.040505] nix-grpc-daemon[725]: ts=2026-09-22T22:21:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145365worker1 # [ 73.051166] nix-grpc-daemon[725]: ts=2026-09-22T22:21:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:592145366worker1 # [ 73.062398] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5367worker1 # [ 73.161574] nix-daemon[846]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)5368worker1 # [ 73.175181] nix-daemon[846]: reaped child process 1111, status = succeeded5369lb # [ 73.194728] niks3-server[988]: 2026/09/22 22:21:26 INFO Received uploads request method=POST path=/api/pending_closures5370worker1 # [ 73.189964] nix-grpc-daemon[859]: time=2026-09-22T22:21:27.208Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5371worker1 # [ 73.191944] nix-grpc-daemon[859]: time=2026-09-22T22:21:27.209Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5372lb # [ 73.228633] niks3-server[988]: 2026/09/22 22:21:26 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5373lb # [ 73.236318] niks3-server[988]: 2026/09/22 22:21:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5374worker1 # [ 73.231320] nix-grpc-daemon[859]: time=2026-09-22T22:21:27.249Z level=INFO msg="Uploading 1 narinfos"5375lb # [ 73.242076] niks3-server[988]: 2026/09/22 22:21:26 INFO Signed narinfos id=21 count=15376lb # [ 73.259569] niks3-server[988]: 2026/09/22 22:21:26 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5377lb # [ 73.266713] niks3-server[988]: 2026/09/22 22:21:26 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5378lb # [ 73.272679] niks3-server[988]: 2026/09/22 22:21:26 INFO Completed upload id=215379worker1 # [ 73.261929] nix-grpc-daemon[859]: time=2026-09-22T22:21:27.280Z level=INFO msg="Upload complete. (187ms)"5380lb # [ 73.279134] niks3-server[988]: 2026/09/22 22:21:26 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5381worker1 # [ 73.267261] nix-grpc-daemon[725]: ts=2026-09-22T22:21:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_in=5125382worker1 # [ 73.274285] nix-grpc-daemon[725]: ts=2026-09-22T22:21:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145383worker1 # [ 73.284845] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5384worker1 # [ 73.295528] nix-daemon[846]: reaped child process 1106, status = succeeded5385worker1 # [ 73.303510] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5386worker1 # [ 74.521605] nix-daemon[846]: reaped child process 1115, status = succeeded5387worker1 # [ 74.596239] nix-daemon[846]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)5388worker1 # [ 74.610078] nix-daemon[846]: reaped child process 1135, status = succeeded5389lb # [ 74.626729] niks3-server[988]: 2026/09/22 22:21:27 INFO Received uploads request method=POST path=/api/pending_closures5390worker1 # [ 74.619903] nix-grpc-daemon[859]: time=2026-09-22T22:21:28.638Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5391worker1 # [ 74.621430] nix-grpc-daemon[859]: time=2026-09-22T22:21:28.639Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5392lb # [ 74.649414] niks3-server[988]: 2026/09/22 22:21:27 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5393lb # [ 74.653200] niks3-server[988]: 2026/09/22 22:21:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5394lb # [ 74.655077] niks3-server[988]: 2026/09/22 22:21:28 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5395lb # [ 74.657760] niks3-server[988]: 2026/09/22 22:21:28 INFO Signed narinfos id=22 count=15396worker1 # [ 74.646539] nix-grpc-daemon[859]: time=2026-09-22T22:21:28.664Z level=INFO msg="Uploading 1 narinfos"5397lb # [ 74.661072] niks3-server[988]: 2026/09/22 22:21:28 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5398lb # [ 74.665835] niks3-server[988]: 2026/09/22 22:21:28 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5399lb # [ 74.669159] niks3-server[988]: 2026/09/22 22:21:28 INFO Completed upload id=225400worker1 # [ 74.658128] nix-grpc-daemon[859]: time=2026-09-22T22:21:28.676Z level=INFO msg="Upload complete. (147ms)"5401worker1 # [ 74.660755] nix-grpc-daemon[725]: ts=2026-09-22T22:21:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15402lb # [ 74.673386] niks3-server[988]: 2026/09/22 22:21:28 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5403worker1 # [ 74.667777] nix-daemon[846]: reaped child process 1113, status = succeeded5404lb # [ 74.680894] envoy[906]: conn peer=[2001:db8:1::1]:47288 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=933 ms=65215405lb # [ 74.682348] envoy[906]: conn peer=[2001:db8:1::1]:45844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=772 ms=42095406lb # [ 74.683717] envoy[906]: conn peer=[2001:db8:1::1]:45848 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1453 tx=1197 ms=46375407worker2 # [ 74.718702] nix-grpc-daemon[785]: ts=2026-09-22T22:21:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:471065408worker2 # [ 74.729607] nix-grpc-daemon[785]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5409worker2 # [ 74.756313] nix-grpc-daemon[785]: ts=2026-09-22T22:21:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:471065410worker2 # [ 74.761449] nix-grpc-daemon[785]: ts=2026-09-22T22:21:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_out=1285411worker1 # [ 74.938801] nix-grpc-daemon[725]: ts=2026-09-22T22:21:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:592145412worker2 # [ 74.952624] nix-grpc-daemon[785]: ts=2026-09-22T22:21:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471065413worker2 # [ 74.962467] nix-grpc-daemon[923]: time=2026-09-22T22:21:28.209Z level=INFO msg="All 1 paths already cached"5414worker2 # [ 74.976122] nix-grpc-daemon[785]: ts=2026-09-22T22:21:28Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05415worker1 # [ 74.967589] nix-grpc-daemon[725]: ts=2026-09-22T22:21:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145416worker2 # [ 74.980341] nix-grpc-daemon[785]: ts=2026-09-22T22:21:28Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15417worker1 # [ 74.981632] nix-grpc-daemon[725]: ts=2026-09-22T22:21:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145418worker1 # [ 74.994305] nix-grpc-daemon[859]: time=2026-09-22T22:21:29.012Z level=INFO msg="All 1 paths already cached"5419worker1 # [ 75.000941] nix-grpc-daemon[725]: ts=2026-09-22T22:21:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:592145420worker1 # [ 75.010161] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5421worker1 # [ 75.113930] nix-daemon[846]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)5422worker1 # [ 75.128710] nix-daemon[846]: reaped child process 1142, status = succeeded5423lb # [ 75.144987] niks3-server[988]: 2026/09/22 22:21:28 INFO Received uploads request method=POST path=/api/pending_closures5424worker1 # [ 75.137857] nix-grpc-daemon[859]: time=2026-09-22T22:21:29.156Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5425worker1 # [ 75.139790] nix-grpc-daemon[859]: time=2026-09-22T22:21:29.157Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5426lb # [ 75.158466] niks3-server[988]: 2026/09/22 22:21:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5427lb # [ 75.161515] niks3-server[988]: 2026/09/22 22:21:28 INFO Signed narinfos id=23 count=15428worker1 # [ 75.150661] nix-grpc-daemon[859]: time=2026-09-22T22:21:29.168Z level=INFO msg="Uploading 1 narinfos"5429lb # [ 75.165186] niks3-server[988]: 2026/09/22 22:21:28 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5430lb # [ 75.168070] niks3-server[988]: 2026/09/22 22:21:28 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5431lb # [ 75.171841] niks3-server[988]: 2026/09/22 22:21:28 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5432lb # [ 75.174442] niks3-server[988]: 2026/09/22 22:21:28 INFO Completed upload id=235433worker1 # [ 75.163392] nix-grpc-daemon[859]: time=2026-09-22T22:21:29.181Z level=INFO msg="Upload complete. (140ms)"5434lb # [ 75.177072] niks3-server[988]: 2026/09/22 22:21:28 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5435worker1 # [ 75.165697] nix-grpc-daemon[725]: ts=2026-09-22T22:21:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_in=6245436worker1 # [ 75.169805] nix-grpc-daemon[725]: ts=2026-09-22T22:21:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145437worker1 # [ 75.175639] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5438worker1 # [ 75.177823] nix-daemon[846]: reaped child process 1137, status = succeeded5439worker1 # [ 75.182848] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5440worker1 # [ 76.026247] nix-daemon[846]: reaped child process 1146, status = succeeded5441worker1 # [ 76.090482] nix-daemon[846]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5442worker1 # [ 76.103377] nix-daemon[846]: reaped child process 1165, status = succeeded5443lb # [ 76.121417] niks3-server[988]: 2026/09/22 22:21:29 INFO Received uploads request method=POST path=/api/pending_closures5444worker1 # [ 76.114645] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.132Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5445worker1 # [ 76.116132] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.134Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5446lb # [ 76.144666] niks3-server[988]: 2026/09/22 22:21:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5447lb # [ 76.146924] niks3-server[988]: 2026/09/22 22:21:29 INFO Signed narinfos id=24 count=15448worker1 # [ 76.138320] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.156Z level=INFO msg="Uploading 1 narinfos"5449lb # [ 76.150254] niks3-server[988]: 2026/09/22 22:21:29 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5450lb # [ 76.153229] niks3-server[988]: 2026/09/22 22:21:29 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5451lb # [ 76.155325] niks3-server[988]: 2026/09/22 22:21:29 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5452lb # [ 76.160597] niks3-server[988]: 2026/09/22 22:21:29 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5453lb # [ 76.163956] niks3-server[988]: 2026/09/22 22:21:29 INFO Completed upload id=245454worker1 # [ 76.153056] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.171Z level=INFO msg="Upload complete. (163ms)"5455lb # [ 76.166951] niks3-server[988]: 2026/09/22 22:21:29 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5456worker1 # [ 76.155937] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15457worker1 # [ 76.163170] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:592145458worker1 # [ 76.165157] nix-daemon[846]: reaped child process 1144, status = succeeded5459worker1 # [ 76.177336] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:592145460worker1 # [ 76.179558] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_out=1765461worker1 # [ 76.360326] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:592145462worker1 # [ 76.366556] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145463worker1 # [ 76.372366] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.389Z level=INFO msg="All 2 paths already cached"5464worker2 # [ 76.392842] nix-grpc-daemon[785]: ts=2026-09-22T22:21:29Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05465worker2 # [ 76.397429] nix-grpc-daemon[785]: ts=2026-09-22T22:21:29Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15466worker1 # [ 76.388408] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145467worker1 # [ 76.400084] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145468worker1 # [ 76.412515] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.430Z level=INFO msg="All 2 paths already cached"5469worker1 # [ 76.419225] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:592145470worker1 # [ 76.429159] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5471worker1 # [ 76.524645] nix-daemon[846]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5472worker1 # [ 76.538651] nix-daemon[846]: reaped child process 1172, status = succeeded5473lb # [ 76.556530] niks3-server[988]: 2026/09/22 22:21:29 INFO Received uploads request method=POST path=/api/pending_closures5474worker1 # [ 76.554393] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.572Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5475worker1 # [ 76.555965] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.574Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5476lb # [ 76.586889] niks3-server[988]: 2026/09/22 22:21:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5477worker1 # [ 76.580377] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.598Z level=INFO msg="Uploading 1 narinfos"5478lb # [ 76.592132] niks3-server[988]: 2026/09/22 22:21:29 INFO Signed narinfos id=25 count=15479lb # [ 76.598628] niks3-server[988]: 2026/09/22 22:21:29 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5480lb # [ 76.606051] niks3-server[988]: 2026/09/22 22:21:29 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5481lb # [ 76.618191] niks3-server[988]: 2026/09/22 22:21:29 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5482lb # [ 76.624643] niks3-server[988]: 2026/09/22 22:21:29 INFO Completed upload id=255483worker1 # [ 76.613531] nix-grpc-daemon[859]: time=2026-09-22T22:21:30.631Z level=INFO msg="Upload complete. (172ms)"5484lb # [ 76.628280] niks3-server[988]: 2026/09/22 22:21:29 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5485worker1 # [ 76.617990] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_in=7525486worker1 # [ 76.624055] nix-grpc-daemon[725]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145487worker1 # [ 76.635231] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5488worker1 # [ 76.637551] nix-daemon[846]: reaped child process 1167, status = succeeded5489worker1 # [ 76.642904] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5490worker1 # [ 77.475248] nix-daemon[846]: reaped child process 1176, status = succeeded5491worker1 # [ 77.544276] nix-daemon[846]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5492worker1 # [ 77.557888] nix-daemon[846]: reaped child process 1196, status = succeeded5493lb # [ 77.575778] niks3-server[988]: 2026/09/22 22:21:30 INFO Received uploads request method=POST path=/api/pending_closures5494worker1 # [ 77.569557] nix-grpc-daemon[859]: time=2026-09-22T22:21:31.587Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5495worker1 # [ 77.571099] nix-grpc-daemon[859]: time=2026-09-22T22:21:31.589Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5496lb # [ 77.598297] niks3-server[988]: 2026/09/22 22:21:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5497lb # [ 77.600598] niks3-server[988]: 2026/09/22 22:21:30 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5498lb # [ 77.602761] niks3-server[988]: 2026/09/22 22:21:30 INFO Signed narinfos id=26 count=15499worker1 # [ 77.591805] nix-grpc-daemon[859]: time=2026-09-22T22:21:31.610Z level=INFO msg="Uploading 1 narinfos"5500lb # [ 77.606442] niks3-server[988]: 2026/09/22 22:21:30 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5501lb # [ 77.610602] niks3-server[988]: 2026/09/22 22:21:30 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5502lb # [ 77.615577] niks3-server[988]: 2026/09/22 22:21:30 INFO Completed upload id=265503worker1 # [ 77.604503] nix-grpc-daemon[859]: time=2026-09-22T22:21:31.622Z level=INFO msg="Upload complete. (154ms)"5504lb # [ 77.617067] niks3-server[988]: 2026/09/22 22:21:30 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5505worker1 # [ 77.607149] nix-grpc-daemon[725]: ts=2026-09-22T22:21:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15506lb # [ 77.619960] niks3-server[988]: 2026/09/22 22:21:30 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5507worker1 # [ 77.614338] nix-daemon[846]: reaped child process 1174, status = succeeded5508worker2 # [ 77.626760] nix-grpc-daemon[785]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:471065509worker2 # [ 77.633678] nix-grpc-daemon[785]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5510worker2 # [ 77.638509] nix-grpc-daemon[785]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5511worker2 # [ 77.655270] nix-grpc-daemon[785]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:471065512worker2 # [ 77.657373] nix-grpc-daemon[785]: ts=2026-09-22T22:21:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_out=2325513client # [ 77.800622] nix-daemon[1110]: reaped child process 1112, status = succeeded5514client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.05 seconds)5515client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5516client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5517client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5518client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5519client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5520client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5521(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.11 seconds)5522subtest: an input only one worker has still reaches the builder5523client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5524client # this derivation will be built:5525client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5526client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5527client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds)5528client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5529client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5530worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5531worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.12 seconds)5532worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5533worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5534client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25535client # [ 78.463398] nix-daemon[1110]: accepted connection from pid 1192, user root (trusted)5536client # this derivation will be built:5537client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5538worker2 # [ 78.540964] nix-grpc-daemon[785]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:471065539client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5540worker1 # [ 78.533412] nix-grpc-daemon[725]: ts=2026-09-22T22:21:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145541worker1 # [ 78.622449] nix-daemon[846]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5542lb # [ 78.647733] niks3-server[988]: 2026/09/22 22:21:31 INFO Received uploads request method=POST path=/api/pending_closures5543worker1 # [ 78.637639] nix-daemon[846]: reaped child process 1206, status = succeeded5544worker1 # [ 78.641160] nix-grpc-daemon[859]: time=2026-09-22T22:21:32.659Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5545worker1 # [ 78.643073] nix-grpc-daemon[859]: time=2026-09-22T22:21:32.660Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5546lb # [ 78.663437] niks3-server[988]: 2026/09/22 22:21:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5547lb # [ 78.668102] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5548lb # [ 78.674540] niks3-server[988]: 2026/09/22 22:21:32 INFO Signed narinfos id=27 count=15549worker1 # [ 78.666265] nix-grpc-daemon[859]: time=2026-09-22T22:21:32.684Z level=INFO msg="Uploading 1 narinfos"5550lb # [ 78.679745] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5551lb # [ 78.684673] niks3-server[988]: 2026/09/22 22:21:32 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5552lb # [ 78.688767] niks3-server[988]: 2026/09/22 22:21:32 INFO Completed upload id=275553worker1 # [ 78.677492] nix-grpc-daemon[859]: time=2026-09-22T22:21:32.695Z level=INFO msg="Upload complete. (142ms)"5554lb # [ 78.693577] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5555worker2 # [ 78.704903] nix-grpc-daemon[785]: ts=2026-09-22T22:21:31Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05556worker2 # [ 78.710806] nix-grpc-daemon[785]: ts=2026-09-22T22:21:31Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15557worker2 # [ 78.717392] nix-grpc-daemon[785]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065558worker2 # [ 78.726601] nix-grpc-daemon[785]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471065559client # copying 2 paths to 'lb:50051'...5560client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5561worker2 # [ 78.738710] nix-grpc-daemon[785]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471065562client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5563worker2 # [ 78.752688] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5564worker2 # [ 78.860444] nix-daemon[870]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5565worker2 # [ 78.875639] nix-daemon[870]: reaped child process 1158, status = succeeded5566lb # [ 78.880428] niks3-server[988]: 2026/09/22 22:21:32 INFO Received uploads request method=POST path=/api/pending_closures5567lb # [ 78.892684] niks3-server[988]: 2026/09/22 22:21:32 INFO Received uploads request method=POST path=/api/pending_closures5568worker2 # [ 78.904325] nix-grpc-daemon[923]: time=2026-09-22T22:21:32.151Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5569worker2 # [ 78.905793] nix-grpc-daemon[923]: time=2026-09-22T22:21:32.151Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5570worker2 # [ 78.908664] nix-grpc-daemon[923]: time=2026-09-22T22:21:32.151Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5571lb # [ 78.927280] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5572lb # [ 78.932045] niks3-server[988]: 2026/09/22 22:21:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5573lb # [ 78.934265] niks3-server[988]: 2026/09/22 22:21:32 INFO Signed narinfos id=28 count=15574lb # [ 78.935798] niks3-server[988]: 2026/09/22 22:21:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5575worker2 # [ 78.937656] nix-grpc-daemon[923]: time=2026-09-22T22:21:32.185Z level=INFO msg="Uploading 3 narinfos"5576lb # [ 78.938279] niks3-server[988]: 2026/09/22 22:21:32 INFO Signed narinfos id=29 count=25577lb # [ 78.940806] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5578lb # [ 78.943697] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5579lb # [ 78.947878] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5580lb # [ 78.957535] niks3-server[988]: 2026/09/22 22:21:32 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5581lb # [ 78.960322] niks3-server[988]: 2026/09/22 22:21:32 INFO Completed upload id=285582lb # [ 78.963236] niks3-server[988]: 2026/09/22 22:21:32 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5583lb # [ 78.966882] niks3-server[988]: 2026/09/22 22:21:32 INFO Completed upload id=295584worker2 # [ 78.966827] nix-grpc-daemon[923]: time=2026-09-22T22:21:32.214Z level=INFO msg="Upload complete. (200ms)"5585lb # [ 78.969625] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5586lb # [ 78.973579] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5587worker2 # [ 78.971257] nix-grpc-daemon[785]: ts=2026-09-22T22:21:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=2 nar_bytes_in=9765588lb # [ 78.975932] niks3-server[988]: 2026/09/22 22:21:32 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5589worker2 # [ 78.976543] nix-grpc-daemon[785]: ts=2026-09-22T22:21:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065590worker2 # [ 78.988120] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5591worker2 # [ 78.992924] nix-daemon[870]: reaped child process 1153, status = succeeded5592worker2 # [ 78.998738] nix-grpc-daemon[785]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5593worker2 # [ 79.004836] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5594client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5595client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5596client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5597worker2 # [ 80.220402] nix-daemon[870]: reaped child process 1163, status = succeeded5598worker2 # [ 80.297265] nix-daemon[870]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)5599worker2 # [ 80.309543] nix-daemon[870]: reaped child process 1182, status = succeeded5600lb # [ 80.317909] niks3-server[988]: 2026/09/22 22:21:33 INFO Received uploads request method=POST path=/api/pending_closures5601worker2 # [ 80.328231] nix-grpc-daemon[923]: time=2026-09-22T22:21:33.575Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5602lb # [ 80.351843] niks3-server[988]: 2026/09/22 22:21:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5603worker2 # [ 80.358188] nix-grpc-daemon[923]: time=2026-09-22T22:21:33.606Z level=INFO msg="Uploading 1 narinfos"5604lb # [ 80.358324] niks3-server[988]: 2026/09/22 22:21:33 INFO Signed narinfos id=30 count=15605lb # [ 80.365865] niks3-server[988]: 2026/09/22 22:21:33 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5606lb # [ 80.372218] niks3-server[988]: 2026/09/22 22:21:33 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5607lb # [ 80.384701] niks3-server[988]: 2026/09/22 22:21:33 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5608lb # [ 80.390606] niks3-server[988]: 2026/09/22 22:21:33 INFO Completed upload id=305609worker2 # [ 80.390777] nix-grpc-daemon[923]: time=2026-09-22T22:21:33.638Z level=INFO msg="Upload complete. (178ms)"5610lb # [ 80.396696] niks3-server[988]: 2026/09/22 22:21:33 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5611worker2 # [ 80.396810] nix-grpc-daemon[785]: ts=2026-09-22T22:21:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15612client # copying 1 paths...5613worker1 # [ 80.396272] nix-grpc-daemon[725]: ts=2026-09-22T22:21:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:592145614worker2 # [ 80.410151] nix-daemon[870]: reaped child process 1160, status = succeeded5615worker1 # [ 80.401623] nix-grpc-daemon[725]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5616client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5617worker2 # [ 80.432252] nix-grpc-daemon[785]: ts=2026-09-22T22:21:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:471065618worker2 # [ 80.437176] nix-grpc-daemon[785]: ts=2026-09-22T22:21:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_out=1205619client # [ 80.581578] nix-daemon[1110]: reaped child process 1195, status = succeeded5620client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.24 seconds)5621client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25622client # [ 80.686163] nix-daemon[1110]: accepted connection from pid 1214, user root (trusted)5623client # this derivation will be built:5624client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5625worker2 # [ 80.765672] nix-grpc-daemon[785]: ts=2026-09-22T22:21:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:471065626client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5627worker1 # [ 80.757840] nix-grpc-daemon[725]: ts=2026-09-22T22:21:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145628worker1 # [ 80.768104] nix-grpc-daemon[859]: time=2026-09-22T22:21:34.785Z level=INFO msg="All 1 paths already cached"5629worker2 # [ 80.793749] nix-grpc-daemon[785]: ts=2026-09-22T22:21:34Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05630worker1 # [ 80.785653] nix-grpc-daemon[725]: ts=2026-09-22T22:21:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145631worker2 # [ 80.798509] nix-grpc-daemon[785]: ts=2026-09-22T22:21:34Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15632worker1 # [ 80.797935] nix-grpc-daemon[725]: ts=2026-09-22T22:21:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:592145633client # copying 1 paths to 'lb:50051'...5634client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5635worker1 # [ 80.814716] nix-grpc-daemon[725]: ts=2026-09-22T22:21:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:592145636worker1 # [ 80.825868] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5637worker1 # [ 80.841263] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 1/5)5638worker1 # [ 80.879479] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 187 ms (attempt 2/5)5639worker1 # [ 81.074475] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 3/5)5640worker1 # [ 81.214533] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 389 ms (attempt 4/5)5641worker1 # [ 81.605382] nix-grpc-daemon[725]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5642worker1 # [ 81.607239] nix-grpc-daemon[725]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5643worker1 # [ 81.609221] nix-grpc-daemon[725]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5644worker1 # [ 81.701796] nix-daemon[846]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)5645worker1 # [ 81.718408] nix-daemon[846]: reaped child process 1219, status = succeeded5646lb # [ 81.732743] niks3-server[988]: 2026/09/22 22:21:35 INFO Received uploads request method=POST path=/api/pending_closures5647worker1 # [ 81.725685] nix-grpc-daemon[859]: time=2026-09-22T22:21:35.744Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5648worker1 # [ 81.727181] nix-grpc-daemon[859]: time=2026-09-22T22:21:35.745Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5649lb # [ 81.748759] niks3-server[988]: 2026/09/22 22:21:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5650lb # [ 81.751509] niks3-server[988]: 2026/09/22 22:21:35 INFO Signed narinfos id=31 count=15651worker1 # [ 81.740670] nix-grpc-daemon[859]: time=2026-09-22T22:21:35.758Z level=INFO msg="Uploading 1 narinfos"5652lb # [ 81.755115] niks3-server[988]: 2026/09/22 22:21:35 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5653lb # [ 81.757644] niks3-server[988]: 2026/09/22 22:21:35 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5654lb # [ 81.762058] niks3-server[988]: 2026/09/22 22:21:35 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5655lb # [ 81.766970] niks3-server[988]: 2026/09/22 22:21:35 INFO Completed upload id=315656worker1 # [ 81.757234] nix-grpc-daemon[859]: time=2026-09-22T22:21:35.775Z level=INFO msg="Upload complete. (135ms)"5657lb # [ 81.768864] niks3-server[988]: 2026/09/22 22:21:35 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5658worker1 # [ 81.759510] nix-grpc-daemon[725]: ts=2026-09-22T22:21:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_in=5605659worker1 # [ 81.762527] nix-grpc-daemon[725]: ts=2026-09-22T22:21:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:592145660worker1 # [ 81.767567] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5661worker1 # [ 81.774200] nix-daemon[846]: reaped child process 1209, status = succeeded5662worker1 # [ 81.778365] nix-daemon[846]: accepted connection from pid 725, user nix-grpc-daemon (trusted)5663client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5664client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5665client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5666worker1 # [ 82.797335] nix-daemon[846]: reaped child process 1223, status = succeeded5667worker1 # [ 82.878616] nix-daemon[846]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5668worker1 # [ 82.893466] nix-daemon[846]: reaped child process 1242, status = succeeded5669lb # [ 82.911617] niks3-server[988]: 2026/09/22 22:21:36 INFO Received uploads request method=POST path=/api/pending_closures5670worker1 # [ 82.909032] nix-grpc-daemon[859]: time=2026-09-22T22:21:36.927Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5671lb # [ 82.945153] niks3-server[988]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5672lb # [ 82.950544] niks3-server[988]: 2026/09/22 22:21:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5673worker1 # [ 82.948556] nix-grpc-daemon[859]: time=2026-09-22T22:21:36.966Z level=INFO msg="Uploading 1 narinfos"5674lb # [ 82.959996] niks3-server[988]: 2026/09/22 22:21:36 INFO Signed narinfos id=32 count=15675lb # [ 82.965155] niks3-server[988]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5676lb # [ 82.979368] niks3-server[988]: 2026/09/22 22:21:36 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5677lb # [ 82.985925] niks3-server[988]: 2026/09/22 22:21:36 INFO Completed upload id=325678worker1 # [ 82.974944] nix-grpc-daemon[859]: time=2026-09-22T22:21:36.993Z level=INFO msg="Upload complete. (188ms)"5679lb # [ 82.987854] niks3-server[988]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5680client # copying 1 paths...5681worker1 # [ 82.980430] nix-grpc-daemon[725]: ts=2026-09-22T22:21:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15682worker1 # [ 82.987731] nix-grpc-daemon[725]: ts=2026-09-22T22:21:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:592145683client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5684worker1 # [ 82.998571] nix-daemon[846]: reaped child process 1221, status = succeeded5685worker2 # [ 83.016817] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:471065686worker2 # [ 83.026465] nix-grpc-daemon[785]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5687worker2 # [ 83.034244] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_out=1205688client # [ 83.168348] nix-daemon[1110]: reaped child process 1217, status = succeeded5689client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.58 seconds)5690client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25691client # [ 83.261726] nix-daemon[1110]: accepted connection from pid 1236, user root (trusted)5692client # this derivation will be built:5693client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5694worker2 # [ 83.340689] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:471065695worker2 # [ 83.345568] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471065696client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5697worker2 # [ 83.351532] nix-grpc-daemon[923]: time=2026-09-22T22:21:36.599Z level=INFO msg="All 1 paths already cached"5698worker2 # [ 83.361160] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05699worker2 # [ 83.363357] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15700worker2 # [ 83.405094] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065701worker2 # [ 83.416829] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:471065702worker2 # [ 83.437776] nix-grpc-daemon[923]: time=2026-09-22T22:21:36.685Z level=INFO msg="All 1 paths already cached"5703client # copying 1 paths to 'lb:50051'...5704client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5705worker2 # [ 83.446803] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:471065706worker2 # [ 83.458581] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5707worker2 # [ 83.555597] nix-daemon[870]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)5708worker2 # [ 83.568323] nix-daemon[870]: reaped child process 1194, status = succeeded5709lb # [ 83.575035] niks3-server[988]: 2026/09/22 22:21:36 INFO Received uploads request method=POST path=/api/pending_closures5710worker2 # [ 83.583121] nix-grpc-daemon[923]: time=2026-09-22T22:21:36.831Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5711worker2 # [ 83.584986] nix-grpc-daemon[923]: time=2026-09-22T22:21:36.832Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5712lb # [ 83.594060] niks3-server[988]: 2026/09/22 22:21:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5713lb # [ 83.596739] niks3-server[988]: 2026/09/22 22:21:36 INFO Signed narinfos id=33 count=15714worker2 # [ 83.597905] nix-grpc-daemon[923]: time=2026-09-22T22:21:36.845Z level=INFO msg="Uploading 1 narinfos"5715lb # [ 83.599354] niks3-server[988]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5716lb # [ 83.604365] niks3-server[988]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5717lb # [ 83.607680] niks3-server[988]: 2026/09/22 22:21:36 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5718lb # [ 83.612051] niks3-server[988]: 2026/09/22 22:21:36 INFO Completed upload id=335719worker2 # [ 83.611630] nix-grpc-daemon[923]: time=2026-09-22T22:21:36.859Z level=INFO msg="Upload complete. (141ms)"5720lb # [ 83.613877] niks3-server[988]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5721worker2 # [ 83.613711] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=0 paths=1 nar_bytes_in=5605722worker2 # [ 83.617583] nix-grpc-daemon[785]: ts=2026-09-22T22:21:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:471065723worker2 # [ 83.623624] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5724worker2 # [ 83.625854] nix-daemon[870]: reaped child process 1189, status = succeeded5725worker2 # [ 83.630736] nix-daemon[870]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5726client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5727client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5728client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5729lb # [ 84.682155] envoy[906]: conn peer=[2001:db8:1::1]:54330 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=585730lb # [ 84.684138] envoy[906]: conn peer=[2001:db8:1::1]:54322 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3411 tx=1922 ms=17945731lb # [ 84.686118] envoy[906]: conn peer=[2001:db8:1::1]:54336 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3850 tx=2137 ms=12925732lb # [ 84.688082] envoy[906]: conn peer=[2001:db8:1::1]:54346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=525733lb # [ 84.689873] envoy[906]: conn peer=[2001:db8:1::1]:54358 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4324 tx=2367 ms=13375734lb # [ 84.691681] envoy[906]: conn peer=[2001:db8:1::1]:54360 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=525735lb # [ 84.693474] envoy[906]: conn peer=[2001:db8:1::1]:54370 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3879 tx=2065 ms=19475736lb # [ 84.695298] envoy[906]: conn peer=[2001:db8:1::1]:54378 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575737lb # [ 84.697053] envoy[906]: conn peer=[2001:db8:1::1]:43608 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=685738lb # [ 84.698895] envoy[906]: conn peer=[2001:db8:1::1]:54388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3744 tx=2121 ms=23215739worker2 # [ 84.850136] nix-daemon[870]: reaped child process 1198, status = succeeded5740worker2 # [ 84.916075] nix-daemon[870]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)5741worker2 # [ 84.929106] nix-daemon[870]: reaped child process 1217, status = succeeded5742lb # [ 84.935302] niks3-server[988]: 2026/09/22 22:21:38 INFO Received uploads request method=POST path=/api/pending_closures5743worker2 # [ 84.942361] nix-grpc-daemon[923]: time=2026-09-22T22:21:38.188Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5744lb # [ 84.952508] niks3-server[988]: 2026/09/22 22:21:38 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5745lb # [ 84.954575] niks3-server[988]: 2026/09/22 22:21:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5746lb # [ 84.956780] niks3-server[988]: 2026/09/22 22:21:38 INFO Signed narinfos id=34 count=15747worker2 # [ 84.956777] nix-grpc-daemon[923]: time=2026-09-22T22:21:38.204Z level=INFO msg="Uploading 1 narinfos"5748lb # [ 84.962049] niks3-server[988]: 2026/09/22 22:21:38 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5749lb # [ 84.965734] niks3-server[988]: 2026/09/22 22:21:38 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5750lb # [ 84.969468] niks3-server[988]: 2026/09/22 22:21:38 INFO Completed upload id=345751worker2 # [ 84.968978] nix-grpc-daemon[923]: time=2026-09-22T22:21:38.216Z level=INFO msg="Upload complete. (131ms)"5752lb # [ 84.972710] niks3-server[988]: 2026/09/22 22:21:38 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5753worker2 # [ 84.972729] nix-grpc-daemon[785]: ts=2026-09-22T22:21:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47106 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15754worker2 # [ 84.980405] nix-daemon[870]: reaped child process 1196, status = succeeded5755client # copying 1 paths...5756worker2 # [ 85.017343] nix-grpc-daemon[785]: ts=2026-09-22T22:21:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:471065757client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5758worker1 # [ 85.025743] nix-grpc-daemon[725]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:592145759worker1 # [ 85.036510] nix-grpc-daemon[725]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5760worker1 # [ 85.050466] nix-grpc-daemon[725]: ts=2026-09-22T22:21:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:59214 duration_s=0 paths=1 nar_bytes_out=1205761client # [ 85.203720] nix-daemon[1110]: reaped child process 1239, status = succeeded5762client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.04 seconds)5763(finished: subtest: an input only one worker has still reaches the builder, in 7.34 seconds)5764subtest: upload whose reference the worker lacks is completed from the cache5765client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5766client # this derivation will be built:5767client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5768client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5769client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.25 seconds)5770client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5771client # this derivation will be built:5772client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5773client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5774client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.24 seconds)5775client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5776client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5777worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5778worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.14 seconds)5779worker2 # [ 85.944740] systemd[1]: Stopping Nix worker-protocol over gRPC...5780worker2 # [ 86.322668] nix-grpc-daemon[785]: ts=2026-09-22T22:21:39Z level=info event=signal_exit5781lb # [ 86.908472] niks3-server[988]: 2026/09/22 22:21:40 INFO lead: released remote=192.168.1.4:432305782worker1 # [ 87.901645] nix-grpc-daemon[725]: ts=2026-09-22T22:21:41Z level=info event=scheduler_restarting addr=lb:500515783worker2 # [ 87.913646] nix-grpc-daemon[785]: ts=2026-09-22T22:21:41Z level=info event=scheduler_restarting5784worker2 # [ 87.917910] nix-grpc-daemon[785]: ts=2026-09-22T22:21:41Z level=info event=scheduler_restarting addr=lb:500515785worker2 # [ 87.921267] nix-grpc-daemon[785]: ts=2026-09-22T22:21:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453925786worker2 # [ 87.924731] nix-grpc-daemon[785]: ts=2026-09-22T22:21:41Z level=info event=worker_gone addr=192.168.1.4:500515787worker2 # [ 87.927835] nix-grpc-daemon[785]: ts=2026-09-22T22:21:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453925788worker2 # [ 87.931558] nix-grpc-daemon[785]: ts=2026-09-22T22:21:41Z level=info event=worker_gone addr=192.168.1.3:500515789worker2 # [ 87.947673] nix-daemon[870]: reaped child process 872, status = succeeded5790worker1 # [ 88.102085] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_connected addr=lb:500515791worker1 # [ 88.105971] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5792worker2 # [ 88.124397] nix-grpc-daemon[785]: ts=2026-09-22T22:21:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5793worker1 # [ 88.306431] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_connected addr=lb:500515794worker1 # [ 88.308377] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5795worker2 # [ 88.435836] nix-grpc-daemon[785]: E0000 00:00:1790115701.683552 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5796worker2 # [ 88.441127] nix-grpc-daemon[785]: E0000 00:00:1790115701.684020 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5797worker1 # [ 88.509049] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_connected addr=lb:500515798worker1 # [ 88.510922] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5799worker1 # [ 88.711714] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_connected addr=lb:500515800worker1 # [ 88.713626] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5801worker1 # [ 88.915471] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_connected addr=lb:500515802worker1 # [ 88.918730] nix-grpc-daemon[725]: ts=2026-09-22T22:21:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5803worker2 # [ 89.037829] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5804worker2 # [ 89.040720] systemd[1]: Stopped Nix worker-protocol over gRPC.5805worker2 # [ 89.044599] systemd[1]: nix-grpc-daemon.service: Consumed 1.820s CPU time over 1min 13.878s wall clock time, 63.8M memory peak, 93.1K incoming IP traffic, 251.6K outgoing IP traffic.5806worker2 # [ 89.056250] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5807worker2 # [ 89.060645] systemd[1]: Closed Nix worker-protocol over gRPC.5808worker2 # [ 89.063978] systemd[1]: nix-grpc-daemon.socket: Consumed 333us CPU time over 1min 21.537s wall clock time, 256K memory peak, 108K incoming IP traffic, 91.6K outgoing IP traffic.5809lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5810lb: must succeed: curl -sf localhost:9901/clusters5811lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5812lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.02 seconds)5813client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25814worker1 # [ 89.116275] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_connected addr=lb:500515815worker1 # [ 89.118273] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5816worker1 # [ 89.193945] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:592145817worker1 # [ 89.198824] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:592145818client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5819client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.13 seconds)5820worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5821worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5822worker2 # [ 89.292315] systemd[1]: Listening on Nix worker-protocol over gRPC.5823worker1 # [ 89.305863] systemd[1]: Stopping Nix worker-protocol over gRPC...5824worker1 # [ 89.318818] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_connected addr=lb:500515825worker1 # [ 89.322130] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5826worker1 # [ 89.522806] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_connected addr=lb:500515827worker1 # [ 89.526811] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5828worker1 # [ 89.727370] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_connected addr=lb:500515829worker1 # [ 89.729286] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5830worker1 # [ 89.873850] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=signal_exit5831worker1 # [ 89.931135] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_connected addr=lb:500515832worker1 # [ 89.934868] nix-grpc-daemon[725]: ts=2026-09-22T22:21:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5833worker2 # [ 90.029348] systemd[1]: Starting scheduler-token.service...5834worker2 # [ 90.064690] systemd[1]: scheduler-token.service: Deactivated successfully.5835worker2 # [ 90.066183] systemd[1]: Finished scheduler-token.service.5836worker2 # [ 90.067427] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 36ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5837worker2 # [ 90.071459] systemd[1]: Starting Nix worker-protocol over gRPC...5838worker2 # [ 90.148126] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5839worker1 # [ 90.135964] nix-grpc-daemon[725]: ts=2026-09-22T22:21:44Z level=info event=scheduler_connected addr=lb:500515840worker2 # [ 90.150283] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5841worker1 # [ 90.140325] nix-grpc-daemon[725]: ts=2026-09-22T22:21:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5842worker2 # [ 90.153630] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15843worker2 # [ 90.157954] nix-grpc-daemon[1241]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5844worker2 # [ 90.159304] nix-grpc-daemon[1241]: E0000 00:00:1790115703.405844 1241 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5845worker2 # [ 90.165412] nix-grpc-daemon[1241]: E0000 00:00:1790115703.411389 1241 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5846worker2 # [ 90.169122] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5847worker2 # [ 90.170801] systemd[1]: Started Nix worker-protocol over gRPC.5848worker2 # [ 90.176059] nix-grpc-daemon[1241]: E0000 00:00:1790115703.423970 1282 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5849worker2 # [ 90.179334] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=scheduler_take_over5850lb # [ 90.179835] niks3-server[988]: 2026/09/22 22:21:43 INFO lead: acquired remote=192.168.1.4:464185851worker2 # [ 90.192057] nix-grpc-daemon[1241]: E0000 00:00:1790115703.439988 1282 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5852worker2 # [ 90.196132] nix-grpc-daemon[1241]: E0000 00:00:1790115703.442307 1282 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5853worker2 # [ 90.206368] nix-grpc-daemon[1241]: E0000 00:00:1790115703.454307 1282 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5854worker2 # [ 90.215304] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=scheduler_connected addr=lb:500515855worker2 # [ 90.221965] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=oidc_jwks provider=mock keys=15856worker2 # [ 90.223457] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:351925857worker2 # [ 90.225072] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05858worker1 # [ 90.340856] nix-grpc-daemon[725]: ts=2026-09-22T22:21:44Z level=info event=scheduler_connected addr=lb:500515859worker2 # [ 90.353451] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:351925860worker2 # [ 90.355274] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05861worker1 # [ 90.901966] nix-grpc-daemon[725]: ts=2026-09-22T22:21:44Z level=info event=scheduler_restarting5862worker2 # [ 90.920824] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:351925863worker2 # [ 90.924304] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:44Z level=info event=worker_gone addr=192.168.1.3:500515864worker1 # [ 90.924587] nix-grpc-daemon[725]: ts=2026-09-22T22:21:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5865worker1 # [ 90.944423] nix-daemon[846]: reaped child process 848, status = succeeded5866worker1 # [ 90.989523] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5867worker1 # [ 90.993335] systemd[1]: Stopped Nix worker-protocol over gRPC.5868worker1 # [ 90.998095] systemd[1]: nix-grpc-daemon.service: Consumed 2.480s CPU time over 1min 16.320s wall clock time, 66.2M memory peak, 109.8K incoming IP traffic, 205K outgoing IP traffic.5869worker1 # [ 91.007838] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5870worker1 # [ 91.011224] systemd[1]: Closed Nix worker-protocol over gRPC.5871worker1 # [ 91.013771] systemd[1]: nix-grpc-daemon.socket: Consumed 339us CPU time over 1min 23.298s wall clock time, 256K memory peak, 99.5K incoming IP traffic, 86.8K outgoing IP traffic.5872lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5873lb: must succeed: curl -sf localhost:9901/clusters5874lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5875lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5876client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25877worker2 # [ 91.220875] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352005878worker2 # [ 91.232967] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)5879worker2 # [ 91.248881] nix-grpc-daemon[1241]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)5880worker2 # [ 91.290159] nix-grpc-daemon[1241]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 2/5)5881worker2 # [ 91.318925] nix-grpc-daemon[1241]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 3/5)5882worker2 # [ 91.326615] nix-grpc-daemon[1241]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 468 ms (attempt 4/5)5883worker2 # [ 91.799517] nix-grpc-daemon[1241]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5884worker2 # [ 91.802966] nix-grpc-daemon[1241]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5885client # copying 2 paths...5886client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5887worker2 # [ 91.821665] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352005888client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5889worker2 # [ 91.831392] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)5890worker2 # [ 91.854366] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:45Z level=info event=niks3_push_started pid=13015891worker2 # [ 91.956831] nix-grpc-daemon[1301]: time=2026-09-22T22:21:45.203Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5892worker2 # [ 92.042761] nix-daemon[870]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)5893worker2 # [ 92.054754] nix-daemon[870]: reaped child process 1317, status = succeeded5894lb # [ 92.062822] niks3-server[988]: 2026/09/22 22:21:45 INFO Received uploads request method=POST path=/api/pending_closures5895lb # [ 92.073387] niks3-server[988]: 2026/09/22 22:21:45 INFO Received uploads request method=POST path=/api/pending_closures5896worker2 # [ 92.080671] nix-grpc-daemon[1301]: time=2026-09-22T22:21:45.328Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5897worker2 # [ 92.082183] nix-grpc-daemon[1301]: time=2026-09-22T22:21:45.330Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5898worker2 # [ 92.084146] nix-grpc-daemon[1301]: time=2026-09-22T22:21:45.332Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5899lb # [ 92.127638] niks3-server[988]: 2026/09/22 22:21:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5900lb # [ 92.132107] niks3-server[988]: 2026/09/22 22:21:45 INFO Signed narinfos id=36 count=25901lb # [ 92.136167] niks3-server[988]: 2026/09/22 22:21:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5902lb # [ 92.141679] niks3-server[988]: 2026/09/22 22:21:45 INFO Signed narinfos id=35 count=15903worker2 # [ 92.142089] nix-grpc-daemon[1301]: time=2026-09-22T22:21:45.389Z level=INFO msg="Uploading 3 narinfos"5904lb # [ 92.144551] niks3-server[988]: 2026/09/22 22:21:45 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5905lb # [ 92.148167] niks3-server[988]: 2026/09/22 22:21:45 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5906lb # [ 92.153253] niks3-server[988]: 2026/09/22 22:21:45 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5907lb # [ 92.163878] niks3-server[988]: 2026/09/22 22:21:45 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5908lb # [ 92.174473] niks3-server[988]: 2026/09/22 22:21:45 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5909lb # [ 92.175949] niks3-server[988]: 2026/09/22 22:21:45 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5910lb # [ 92.179508] niks3-server[988]: 2026/09/22 22:21:45 INFO Completed upload id=355911lb # [ 92.181345] niks3-server[988]: 2026/09/22 22:21:45 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5912lb # [ 92.183208] niks3-server[988]: 2026/09/22 22:21:45 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5913lb # [ 92.188242] niks3-server[988]: 2026/09/22 22:21:45 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5914lb # [ 92.191562] niks3-server[988]: 2026/09/22 22:21:45 INFO Completed upload id=365915worker2 # [ 92.191695] nix-grpc-daemon[1301]: time=2026-09-22T22:21:45.439Z level=INFO msg="Upload complete. (232ms)"5916worker2 # [ 92.199351] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35200 duration_s=0 paths=2 nar_bytes_in=3125917worker2 # [ 92.212905] nix-daemon[870]: reaped child process 1299, status = succeeded5918client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.15 seconds)5919worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5920worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5921worker1 # [ 92.284887] systemd[1]: Listening on Nix worker-protocol over gRPC.5922worker1 # [ 92.292312] systemd[1]: Starting Nix worker-protocol over gRPC...5923worker1 # [ 92.385994] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5924worker1 # [ 92.392146] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5925worker1 # [ 92.395232] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15926worker1 # [ 92.397264] nix-grpc-daemon[1266]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5927worker1 # [ 92.398564] nix-grpc-daemon[1266]: E0000 00:00:1790115706.410459 1266 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5928worker1 # [ 92.407526] nix-grpc-daemon[1266]: E0000 00:00:1790115706.419876 1266 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5929worker1 # [ 92.413084] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5930worker1 # [ 92.414697] systemd[1]: Started Nix worker-protocol over gRPC.5931worker1 # [ 92.417221] nix-grpc-daemon[1266]: E0000 00:00:1790115706.429604 1307 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5932worker1 # [ 92.426875] nix-grpc-daemon[1266]: E0000 00:00:1790115706.445278 1307 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5933worker1 # [ 92.430129] nix-grpc-daemon[1266]: E0000 00:00:1790115706.447612 1307 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5934lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5935lb: must succeed: curl -sf localhost:9901/clusters5936worker1 # [ 92.442140] nix-grpc-daemon[1266]: E0000 00:00:1790115706.460491 1307 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5937worker1 # [ 92.454746] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:46Z level=info event=scheduler_connected addr=lb:500515938worker2 # [ 92.469292] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:351925939worker2 # [ 92.470811] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05940lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5941lb: must succeed: curl -sf localhost:9901/clusters5942lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5943lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.07 seconds)5944(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 8.34 seconds)5945subtest: a niks3 restart does not cost the leader its role5946lb: must succeed: curl -sf localhost:9901/clusters5947lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5948lb: must succeed: curl -sf localhost:9901/clusters5949lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5950lb # [ 93.576921] systemd[1]: Stopping niks3 server...5951lb # [ 93.578582] niks3-server[988]: 2026/09/22 22:21:46 INFO Shutdown signal received, draining in-flight requests timeout=10s5952lb # [ 93.582794] niks3-server[988]: 2026/09/22 22:21:46 INFO lead: released remote=192.168.1.4:464185953worker1 # [ 93.572435] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:47Z level=info event=scheduler_lock_lost status=2005954worker2 # [ 93.586235] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:46Z level=info event=scheduler_lock_lost status=2005955lb # [ 93.598770] systemd[1]: niks3.service: Deactivated successfully.5956lb # [ 93.600403] systemd[1]: Stopped niks3 server.5957lb # [ 93.601178] systemd[1]: niks3.service: Consumed 658ms CPU time over 1min 12.948s wall clock time, 28.7M memory peak, 96.4K incoming IP traffic, 139K outgoing IP traffic.5958lb # [ 93.608747] systemd[1]: Starting niks3 server...5959lb # [ 93.658226] niks3-server[1129]: 2026/09/22 22:21:47 goose: no migrations to run. current version: 202609200000005960lb # [ 93.663930] niks3-server[1129]: 2026/09/22 22:21:47 OK 1_commit_pending_closure.sql (3.9ms)5961lb # [ 93.667561] niks3-server[1129]: 2026/09/22 22:21:47 OK 2_object_stats_trigger.sql (2.01ms)5962lb # [ 93.668704] niks3-server[1129]: 2026/09/22 22:21:47 goose: up to current file version: 25963lb # [ 93.669962] niks3-server[1129]: 2026/09/22 22:21:47 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5964lb # [ 93.680990] niks3-server[1129]: 2026/09/22 22:21:47 INFO Read proxy enabled — serving cache objects from S35965lb # [ 93.682575] niks3-server[1129]: 2026/09/22 22:21:47 INFO Using socket-activated listener address=0.0.0.0:57515966lb # [ 93.684283] niks3-server[1129]: 2026/09/22 22:21:47 INFO systemd watchdog enabled interval=15s5967lb # [ 93.685472] systemd[1]: Started niks3 server.5968lb # [ 93.689150] niks3-server[1129]: 2026/09/22 22:21:47 INFO Starting HTTP server address=0.0.0.0:57515969lb: waiting for unit niks3.service5970lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5971lb: must succeed: curl -sf localhost:9901/clusters5972lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5973worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5974worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5975lb: must succeed: curl -sf localhost:9901/clusters5976lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5977lb: must succeed: curl -sf localhost:9901/clusters5978lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5979worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5980worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)5981client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25982worker1 # [ 93.995290] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:360965983worker1 # [ 94.007529] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)5984client # don't know how to build these paths:5985client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5986worker1 # [ 94.024487] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360965987worker1 # [ 94.036063] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5)5988worker1 # [ 94.050936] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 2/5)5989worker1 # [ 94.205179] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 3/5)5990worker1 # [ 94.254951] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 4/5)5991worker1 # [ 94.299086] nix-grpc-daemon[1266]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5992worker1 # [ 94.302383] nix-grpc-daemon[1266]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5993worker2 # [ 94.341492] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:47Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05994worker2 # [ 94.344232] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:47Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15995worker2 # [ 94.388770] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352005996worker2 # [ 94.399730] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352005997client # copying 1 paths to 'lb:50051'...5998client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5999worker2 # [ 94.409319] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352006000worker2 # [ 94.416889] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)6001worker2 # [ 94.507704] nix-daemon[870]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6002worker2 # [ 94.522578] nix-daemon[870]: reaped child process 1337, status = succeeded6003lb # [ 94.527170] niks3-server[1129]: 2026/09/22 22:21:47 INFO Received uploads request method=POST path=/api/pending_closures6004worker2 # [ 94.538347] nix-grpc-daemon[1301]: time=2026-09-22T22:21:47.786Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6005worker2 # [ 94.540557] nix-grpc-daemon[1301]: time=2026-09-22T22:21:47.787Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"6006lb # [ 94.571432] niks3-server[1129]: 2026/09/22 22:21:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6007lb # [ 94.576538] niks3-server[1129]: 2026/09/22 22:21:47 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls6008lb # [ 94.593221] niks3-server[1129]: 2026/09/22 22:21:47 INFO lead: acquired remote=192.168.1.4:584566009lb # [ 94.597737] niks3-server[1129]: 2026/09/22 22:21:47 INFO Signed narinfos id=37 count=16010worker2 # [ 94.599485] nix-grpc-daemon[1301]: time=2026-09-22T22:21:47.847Z level=INFO msg="Uploading 1 narinfos"6011lb # [ 94.603948] niks3-server[1129]: 2026/09/22 22:21:47 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst6012lb # [ 94.610331] niks3-server[1129]: 2026/09/22 22:21:47 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6013lb # [ 94.616952] niks3-server[1129]: 2026/09/22 22:21:47 INFO Completed upload id=376014worker2 # [ 94.616739] nix-grpc-daemon[1301]: time=2026-09-22T22:21:47.864Z level=INFO msg="Upload complete. (193ms)"6015lb # [ 94.621389] niks3-server[1129]: 2026/09/22 22:21:47 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo6016worker2 # [ 94.621996] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35200 duration_s=0 paths=1 nar_bytes_in=5366017worker2 # [ 94.628503] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352006018worker2 # [ 94.639130] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)6019worker2 # [ 94.647957] nix-daemon[870]: reaped child process 1332, status = succeeded6020worker2 # [ 94.653819] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)6021client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...6022client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6023lb # [ 94.683643] envoy[906]: conn peer=[2001:db8:1::1]:43624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3729 tx=2084 ms=17726024lb # [ 94.686967] envoy[906]: conn peer=[2001:db8:1::1]:43634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=796025lb # [ 94.690476] envoy[906]: conn peer=[2001:db8:1::4]:52396 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=67842 tx=3543 ms=727616026lb # [ 94.694307] envoy[906]: conn peer=[2001:db8:1::1]:43640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=206027lb # [ 94.697852] envoy[906]: conn peer=[2001:db8:1::3]:47902 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=21092 tx=4214 ms=760156028lb # [ 94.701648] envoy[906]: conn peer=[2001:db8:1::1]:43642 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=281 ms=9926029client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6030client # farm-a> LOG-farm-a-after-niks3-restart6031worker2 # [ 95.688547] nix-daemon[870]: reaped child process 1343, status = succeeded6032worker2 # [ 95.765175] nix-daemon[870]: accepted connection from pid 1359, user nix-grpc-daemon (trusted)6033worker2 # [ 95.777439] nix-daemon[870]: reaped child process 1362, status = succeeded6034lb # [ 95.784578] niks3-server[1129]: 2026/09/22 22:21:49 INFO Received uploads request method=POST path=/api/pending_closures6035worker2 # [ 95.790494] nix-grpc-daemon[1301]: time=2026-09-22T22:21:49.038Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6036worker2 # [ 95.791976] nix-grpc-daemon[1301]: time=2026-09-22T22:21:49.039Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"6037lb # [ 95.826156] niks3-server[1129]: 2026/09/22 22:21:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6038lb # [ 95.833131] niks3-server[1129]: 2026/09/22 22:21:49 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6039lb # [ 95.838500] niks3-server[1129]: 2026/09/22 22:21:49 INFO Signed narinfos id=38 count=16040worker2 # [ 95.841191] nix-grpc-daemon[1301]: time=2026-09-22T22:21:49.088Z level=INFO msg="Uploading 1 narinfos"6041lb # [ 95.846721] niks3-server[1129]: 2026/09/22 22:21:49 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls6042lb # [ 95.859185] niks3-server[1129]: 2026/09/22 22:21:49 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6043lb # [ 95.861915] niks3-server[1129]: 2026/09/22 22:21:49 INFO Completed upload id=386044worker2 # [ 95.861909] nix-grpc-daemon[1301]: time=2026-09-22T22:21:49.109Z level=INFO msg="Upload complete. (199ms)"6045lb # [ 95.865396] niks3-server[1129]: 2026/09/22 22:21:49 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6046lb # [ 95.868479] niks3-server[1129]: 2026/09/22 22:21:49 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst6047worker2 # [ 95.870473] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35200 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16048worker2 # [ 95.880333] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:49Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06049worker2 # [ 95.884361] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:49Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=16050worker2 # [ 95.892278] nix-daemon[870]: reaped child process 1341, status = succeeded6051worker2 # [ 95.923898] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352006052worker2 # [ 95.932153] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352006053worker2 # [ 95.942438] nix-grpc-daemon[1301]: time=2026-09-22T22:21:49.190Z level=INFO msg="All 1 paths already cached"6054client # copying 1 paths to 'lb:50051'...6055client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6056worker2 # [ 95.947442] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352006057worker2 # [ 95.952323] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)6058worker2 # [ 96.032760] nix-daemon[870]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)6059worker2 # [ 96.045768] nix-daemon[870]: reaped child process 1371, status = succeeded6060lb # [ 96.051902] niks3-server[1129]: 2026/09/22 22:21:49 INFO Received uploads request method=POST path=/api/pending_closures6061worker2 # [ 96.061998] nix-grpc-daemon[1301]: time=2026-09-22T22:21:49.309Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6062worker2 # [ 96.063463] nix-grpc-daemon[1301]: time=2026-09-22T22:21:49.311Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6063lb # [ 96.086201] niks3-server[1129]: 2026/09/22 22:21:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6064lb # [ 96.091415] niks3-server[1129]: 2026/09/22 22:21:49 INFO Signed narinfos id=39 count=16065worker2 # [ 96.097676] nix-grpc-daemon[1301]: time=2026-09-22T22:21:49.345Z level=INFO msg="Uploading 1 narinfos"6066lb # [ 96.103132] niks3-server[1129]: 2026/09/22 22:21:49 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6067lb # [ 96.107917] niks3-server[1129]: 2026/09/22 22:21:49 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6068lb # [ 96.117315] niks3-server[1129]: 2026/09/22 22:21:49 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6069lb # [ 96.125470] niks3-server[1129]: 2026/09/22 22:21:49 INFO Completed upload id=396070worker2 # [ 96.125391] nix-grpc-daemon[1301]: time=2026-09-22T22:21:49.373Z level=INFO msg="Upload complete. (168ms)"6071lb # [ 96.127023] niks3-server[1129]: 2026/09/22 22:21:49 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6072worker2 # [ 96.129805] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35200 duration_s=0 paths=1 nar_bytes_in=6566073worker2 # [ 96.135609] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352006074worker2 # [ 96.147524] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)6075worker2 # [ 96.153874] nix-daemon[870]: reaped child process 1366, status = succeeded6076worker2 # [ 96.160942] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)6077client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6078client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6079client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6080client # farm-b> LOG-farm-b-after-niks3-restart6081worker2 # [ 96.802528] nix-daemon[870]: reaped child process 1375, status = succeeded6082worker2 # [ 96.885435] nix-daemon[870]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)6083worker2 # [ 96.898648] nix-daemon[870]: reaped child process 1394, status = succeeded6084lb # [ 96.907741] niks3-server[1129]: 2026/09/22 22:21:50 INFO Received uploads request method=POST path=/api/pending_closures6085worker2 # [ 96.916484] nix-grpc-daemon[1301]: time=2026-09-22T22:21:50.164Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6086worker2 # [ 96.917956] nix-grpc-daemon[1301]: time=2026-09-22T22:21:50.165Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6087lb # [ 96.946593] niks3-server[1129]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6088lb # [ 96.957295] niks3-server[1129]: 2026/09/22 22:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6089lb # [ 96.964347] niks3-server[1129]: 2026/09/22 22:21:50 INFO Signed narinfos id=40 count=16090worker2 # [ 96.966658] nix-grpc-daemon[1301]: time=2026-09-22T22:21:50.214Z level=INFO msg="Uploading 1 narinfos"6091lb # [ 96.971748] niks3-server[1129]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6092lb # [ 96.980521] niks3-server[1129]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6093lb # [ 96.984895] niks3-server[1129]: 2026/09/22 22:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6094lb # [ 96.990279] niks3-server[1129]: 2026/09/22 22:21:50 INFO Completed upload id=406095worker2 # [ 96.990335] nix-grpc-daemon[1301]: time=2026-09-22T22:21:50.238Z level=INFO msg="Upload complete. (222ms)"6096lb # [ 96.992109] niks3-server[1129]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6097worker2 # [ 96.996757] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35200 duration_s=0 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16098worker2 # [ 97.008622] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:50Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06099worker2 # [ 97.013217] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:50Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=16100worker2 # [ 97.022560] nix-daemon[870]: reaped child process 1373, status = succeeded6101worker2 # [ 97.052572] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352006102worker2 # [ 97.064635] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352006103worker2 # [ 97.078485] nix-grpc-daemon[1301]: time=2026-09-22T22:21:50.326Z level=INFO msg="All 2 paths already cached"6104client # copying 1 paths to 'lb:50051'...6105client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6106worker2 # [ 97.087103] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352006107worker2 # [ 97.097149] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)6108worker2 # [ 97.193521] nix-daemon[870]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)6109worker2 # [ 97.208840] nix-daemon[870]: reaped child process 1403, status = succeeded6110lb # [ 97.214669] niks3-server[1129]: 2026/09/22 22:21:50 INFO Received uploads request method=POST path=/api/pending_closures6111worker2 # [ 97.224600] nix-grpc-daemon[1301]: time=2026-09-22T22:21:50.472Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6112worker2 # [ 97.226100] nix-grpc-daemon[1301]: time=2026-09-22T22:21:50.474Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6113lb # [ 97.249986] niks3-server[1129]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6114lb # [ 97.257489] niks3-server[1129]: 2026/09/22 22:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6115worker2 # [ 97.264128] nix-grpc-daemon[1301]: time=2026-09-22T22:21:50.512Z level=INFO msg="Uploading 1 narinfos"6116lb # [ 97.264299] niks3-server[1129]: 2026/09/22 22:21:50 INFO Signed narinfos id=41 count=16117lb # [ 97.277340] niks3-server[1129]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6118lb # [ 97.283399] niks3-server[1129]: 2026/09/22 22:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6119lb # [ 97.290601] niks3-server[1129]: 2026/09/22 22:21:50 INFO Completed upload id=416120worker2 # [ 97.290700] nix-grpc-daemon[1301]: time=2026-09-22T22:21:50.538Z level=INFO msg="Upload complete. (182ms)"6121lb # [ 97.292938] niks3-server[1129]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6122worker2 # [ 97.295942] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35200 duration_s=0 paths=1 nar_bytes_in=7846123worker2 # [ 97.302690] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352006124worker2 # [ 97.314146] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)6125worker2 # [ 97.326394] nix-daemon[870]: reaped child process 1398, status = succeeded6126worker2 # [ 97.329336] nix-daemon[870]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)6127client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6128client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6129client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6130client # farm-top> LOG-farm-top-after-niks3-restart6131worker2 # [ 98.543739] nix-daemon[870]: reaped child process 1407, status = succeeded6132worker2 # [ 98.619922] nix-daemon[870]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6133worker2 # [ 98.633094] nix-daemon[870]: reaped child process 1427, status = succeeded6134lb # [ 98.641711] niks3-server[1129]: 2026/09/22 22:21:51 INFO Received uploads request method=POST path=/api/pending_closures6135worker2 # [ 98.651050] nix-grpc-daemon[1301]: time=2026-09-22T22:21:51.898Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6136worker2 # [ 98.652535] nix-grpc-daemon[1301]: time=2026-09-22T22:21:51.900Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6137lb # [ 98.682831] niks3-server[1129]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6138lb # [ 98.688712] niks3-server[1129]: 2026/09/22 22:21:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6139lb # [ 98.695499] niks3-server[1129]: 2026/09/22 22:21:52 INFO Signed narinfos id=42 count=16140worker2 # [ 98.698325] nix-grpc-daemon[1301]: time=2026-09-22T22:21:51.946Z level=INFO msg="Uploading 1 narinfos"6141lb # [ 98.700246] niks3-server[1129]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6142lb # [ 98.706835] niks3-server[1129]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6143lb # [ 98.713056] niks3-server[1129]: 2026/09/22 22:21:52 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6144lb # [ 98.717097] niks3-server[1129]: 2026/09/22 22:21:52 INFO Completed upload id=426145worker2 # [ 98.716990] nix-grpc-daemon[1301]: time=2026-09-22T22:21:51.964Z level=INFO msg="Upload complete. (179ms)"6146lb # [ 98.718800] niks3-server[1129]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6147worker2 # [ 98.723740] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35200 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16148worker2 # [ 98.735489] nix-daemon[870]: reaped child process 1405, status = succeeded6149client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 4.89 seconds)6150client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6151client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)6152(finished: subtest: a niks3 restart does not cost the leader its role, in 5.35 seconds)6153subtest: scheduler down: the other node takes the lock and keeps it6154lb: must succeed: curl -sf localhost:9901/clusters6155lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6156lb: must succeed: curl -sf localhost:9901/clusters6157lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6158lb: must succeed: curl -sf localhost:9901/clusters6159lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6160lb: must succeed: curl -sf localhost:9901/clusters6161lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6162worker2 # [ 99.013731] systemd[1]: Stopping Nix worker-protocol over gRPC...6163worker2 # [ 99.168487] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:52Z level=info event=signal_exit6164lb # [ 99.592078] niks3-server[1129]: 2026/09/22 22:21:52 INFO lead: released remote=192.168.1.4:584566165worker1 # [ 100.583967] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:54Z level=info event=scheduler_restarting addr=lb:500516166worker2 # [ 100.598186] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:53Z level=info event=scheduler_restarting6167worker2 # [ 100.601805] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:53Z level=info event=scheduler_restarting addr=lb:500516168worker2 # [ 100.605227] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:351926169worker2 # [ 100.608790] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:53Z level=info event=worker_gone addr=192.168.1.3:500516170worker2 # [ 100.612271] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:351926171worker2 # [ 100.615849] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:53Z level=info event=worker_gone addr=192.168.1.4:500516172worker2 # [ 100.633921] nix-daemon[870]: reaped child process 1293, status = succeeded6173worker1 # [ 100.783697] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:54Z level=info event=scheduler_connected addr=lb:500516174worker1 # [ 100.787989] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6175worker2 # [ 100.807209] nix-grpc-daemon[1241]: ts=2026-09-22T22:21:54Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6176worker1 # [ 100.989070] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_connected addr=lb:500516177worker1 # [ 100.993366] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6178worker1 # [ 101.194148] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_connected addr=lb:500516179worker1 # [ 101.198158] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6180worker2 # [ 101.213879] nix-grpc-daemon[1241]: E0000 00:00:1790115714.461685 1286 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6181worker2 # [ 101.215922] nix-grpc-daemon[1241]: E0000 00:00:1790115714.461774 1286 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6182worker1 # [ 101.398634] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_connected addr=lb:500516183worker1 # [ 101.400700] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6184worker1 # [ 101.602593] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_connected addr=lb:500516185worker1 # [ 101.606732] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6186worker2 # [ 101.804335] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6187worker2 # [ 101.808079] systemd[1]: Stopped Nix worker-protocol over gRPC.6188worker2 # [ 101.812629] systemd[1]: nix-grpc-daemon.service: Consumed 842ms CPU time over 11.735s wall clock time, 37.6M memory peak, 45.6K incoming IP traffic, 79.7K outgoing IP traffic.6189worker1 # [ 101.807244] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_connected addr=lb:500516190worker2 # [ 101.823345] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6191worker1 # [ 101.810981] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6192worker2 # [ 101.827509] systemd[1]: Closed Nix worker-protocol over gRPC.6193worker2 # [ 101.830845] systemd[1]: nix-grpc-daemon.socket: Consumed 302us CPU time over 12.538s wall clock time, 256K memory peak, 35.9K incoming IP traffic, 25.9K outgoing IP traffic.6194worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6195worker1 # [ 102.011939] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_connected addr=lb:500516196worker1 # [ 102.015472] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6197worker1 # [ 102.216196] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_connected addr=lb:500516198worker1 # [ 102.218254] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6199worker1 # [ 102.418751] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_connected addr=lb:500516200worker1 # [ 102.420489] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6201worker1 # [ 102.622546] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_connected addr=lb:500516202worker1 # [ 102.626426] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6203worker1 # [ 102.827052] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_connected addr=lb:500516204worker1 # [ 102.828917] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6205worker1 # [ 103.029902] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_connected addr=lb:500516206worker1 # [ 103.034285] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6207worker1 # [ 103.234713] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_connected addr=lb:500516208worker1 # [ 103.236539] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6209worker1 # [ 103.437795] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_connected addr=lb:500516210worker1 # [ 103.441700] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6211worker1 # [ 103.642827] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_connected addr=lb:500516212worker1 # [ 103.644767] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6213worker1 # [ 103.845526] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_connected addr=lb:500516214worker1 # [ 103.848044] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6215worker1 # [ 104.048390] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_connected addr=lb:500516216worker1 # [ 104.050263] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6217worker1 # [ 104.251136] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_connected addr=lb:500516218worker1 # [ 104.255316] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6219worker1 # [ 104.456171] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_connected addr=lb:500516220worker1 # [ 104.460168] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6221worker1 # [ 104.579071] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_take_over6222lb # [ 104.590367] niks3-server[1129]: 2026/09/22 22:21:57 INFO lead: acquired remote=192.168.1.3:586006223worker1 # [ 104.661936] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_connected addr=lb:500516224worker1 # [ 104.666448] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6225lb # [ 104.684906] envoy[906]: conn peer=[2001:db8:1::1]:51280 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9705 tx=4377 ms=47346226lb # [ 104.688762] envoy[906]: conn peer=[2001:db8:1::4]:50412 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1755 tx=425 ms=105866227worker1 # [ 104.867426] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_connected addr=lb:500516228worker1 # [ 104.871507] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6229worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 3.17 seconds)6230worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6231worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6232worker1 # [ 105.072105] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:59Z level=info event=scheduler_connected addr=lb:500516233worker1 # [ 105.078143] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:455166234worker1 # [ 105.081111] nix-grpc-daemon[1266]: ts=2026-09-22T22:21:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06235worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6236worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6237client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26238worker1 # [ 106.214885] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:360966239client # don't know how to build these paths:6240client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6241worker1 # [ 106.223990] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360966242worker1 # [ 106.246270] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06243worker1 # [ 106.250284] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=16244worker1 # [ 106.300409] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360966245worker1 # [ 106.314332] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360966246client # copying 1 paths to 'lb:50051'...6247client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6248worker1 # [ 106.328289] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:360966249worker1 # [ 106.340099] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6250worker1 # [ 106.361475] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=info event=niks3_push_started pid=13606251worker1 # [ 106.468798] nix-grpc-daemon[1360]: time=2026-09-22T22:22:00.486Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6252worker1 # [ 106.561094] nix-daemon[846]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)6253worker1 # [ 106.574200] nix-daemon[846]: reaped child process 1376, status = succeeded6254lb # [ 106.593638] niks3-server[1129]: 2026/09/22 22:21:59 INFO Received uploads request method=POST path=/api/pending_closures6255worker1 # [ 106.591458] nix-grpc-daemon[1360]: time=2026-09-22T22:22:00.609Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6256worker1 # [ 106.593036] nix-grpc-daemon[1360]: time=2026-09-22T22:22:00.611Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6257lb # [ 106.635221] niks3-server[1129]: 2026/09/22 22:21:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6258lb # [ 106.642880] niks3-server[1129]: 2026/09/22 22:21:59 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6259lb # [ 106.656199] niks3-server[1129]: 2026/09/22 22:22:00 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6260lb # [ 106.665482] niks3-server[1129]: 2026/09/22 22:22:00 INFO Signed narinfos id=43 count=16261worker1 # [ 106.657085] nix-grpc-daemon[1360]: time=2026-09-22T22:22:00.675Z level=INFO msg="Uploading 1 narinfos"6262lb # [ 106.680647] niks3-server[1129]: 2026/09/22 22:22:00 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6263lb # [ 106.687353] niks3-server[1129]: 2026/09/22 22:22:00 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6264lb # [ 106.691603] niks3-server[1129]: 2026/09/22 22:22:00 INFO Completed upload id=436265worker1 # [ 106.680848] nix-grpc-daemon[1360]: time=2026-09-22T22:22:00.699Z level=INFO msg="Upload complete. (208ms)"6266worker1 # [ 106.685715] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36096 duration_s=0 paths=1 nar_bytes_in=5206267worker1 # [ 106.692497] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360966268worker1 # [ 106.705111] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6269worker1 # [ 106.713899] nix-daemon[846]: reaped child process 1358, status = succeeded6270worker1 # [ 106.719574] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6271client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6272client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6273client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6274client # farm-a> LOG-farm-a-on-standby6275worker1 # [ 107.535558] nix-daemon[846]: reaped child process 1380, status = succeeded6276worker1 # [ 107.614580] nix-daemon[846]: accepted connection from pid 1396, user nix-grpc-daemon (trusted)6277worker1 # [ 107.629543] nix-daemon[846]: reaped child process 1399, status = succeeded6278lb # [ 107.646885] niks3-server[1129]: 2026/09/22 22:22:00 INFO Received uploads request method=POST path=/api/pending_closures6279worker1 # [ 107.648485] nix-grpc-daemon[1360]: time=2026-09-22T22:22:01.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6280worker1 # [ 107.650045] nix-grpc-daemon[1360]: time=2026-09-22T22:22:01.668Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6281lb # [ 107.694967] niks3-server[1129]: 2026/09/22 22:22:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6282worker1 # [ 107.690356] nix-grpc-daemon[1360]: time=2026-09-22T22:22:01.708Z level=INFO msg="Uploading 1 narinfos"6283lb # [ 107.701524] niks3-server[1129]: 2026/09/22 22:22:01 INFO Signed narinfos id=44 count=16284lb # [ 107.711935] niks3-server[1129]: 2026/09/22 22:22:01 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6285lb # [ 107.717695] niks3-server[1129]: 2026/09/22 22:22:01 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6286lb # [ 107.725848] niks3-server[1129]: 2026/09/22 22:22:01 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6287lb # [ 107.728944] niks3-server[1129]: 2026/09/22 22:22:01 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6288lb # [ 107.733258] niks3-server[1129]: 2026/09/22 22:22:01 INFO Completed upload id=446289worker1 # [ 107.722402] nix-grpc-daemon[1360]: time=2026-09-22T22:22:01.740Z level=INFO msg="Upload complete. (195ms)"6290lb # [ 107.735557] niks3-server[1129]: 2026/09/22 22:22:01 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6291worker1 # [ 107.728666] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36096 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16292worker1 # [ 107.743459] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:01Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06293worker1 # [ 107.748109] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:01Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=16294worker1 # [ 107.757138] nix-daemon[846]: reaped child process 1378, status = succeeded6295worker1 # [ 107.786856] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360966296worker1 # [ 107.800902] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360966297worker1 # [ 107.814382] nix-grpc-daemon[1360]: time=2026-09-22T22:22:01.832Z level=INFO msg="All 1 paths already cached"6298client # copying 1 paths to 'lb:50051'...6299client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6300worker1 # [ 107.823295] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:360966301worker1 # [ 107.833154] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6302worker1 # [ 107.934024] nix-daemon[846]: accepted connection from pid 1405, user nix-grpc-daemon (trusted)6303worker1 # [ 107.949133] nix-daemon[846]: reaped child process 1408, status = succeeded6304lb # [ 107.965992] niks3-server[1129]: 2026/09/22 22:22:01 INFO Received uploads request method=POST path=/api/pending_closures6305worker1 # [ 107.964234] nix-grpc-daemon[1360]: time=2026-09-22T22:22:01.982Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6306worker1 # [ 107.965748] nix-grpc-daemon[1360]: time=2026-09-22T22:22:01.984Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6307lb # [ 107.997604] niks3-server[1129]: 2026/09/22 22:22:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6308worker1 # [ 108.000036] nix-grpc-daemon[1360]: time=2026-09-22T22:22:02.018Z level=INFO msg="Uploading 1 narinfos"6309lb # [ 108.010660] niks3-server[1129]: 2026/09/22 22:22:01 INFO Signed narinfos id=45 count=16310lb # [ 108.016658] niks3-server[1129]: 2026/09/22 22:22:01 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6311lb # [ 108.021187] niks3-server[1129]: 2026/09/22 22:22:01 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6312lb # [ 108.029793] niks3-server[1129]: 2026/09/22 22:22:01 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6313lb # [ 108.036973] niks3-server[1129]: 2026/09/22 22:22:01 INFO Completed upload id=456314worker1 # [ 108.026132] nix-grpc-daemon[1360]: time=2026-09-22T22:22:02.044Z level=INFO msg="Upload complete. (181ms)"6315lb # [ 108.040367] niks3-server[1129]: 2026/09/22 22:22:01 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6316worker1 # [ 108.030569] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36096 duration_s=0 paths=1 nar_bytes_in=6406317worker1 # [ 108.035473] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360966318worker1 # [ 108.047911] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6319worker1 # [ 108.054252] nix-daemon[846]: reaped child process 1403, status = succeeded6320worker1 # [ 108.057274] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6321client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6322client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6323client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6324client # farm-b> LOG-farm-b-on-standby6325worker1 # [ 109.274894] nix-daemon[846]: reaped child process 1412, status = succeeded6326worker1 # [ 109.352834] nix-daemon[846]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)6327worker1 # [ 109.367340] nix-daemon[846]: reaped child process 1431, status = succeeded6328lb # [ 109.385437] niks3-server[1129]: 2026/09/22 22:22:02 INFO Received uploads request method=POST path=/api/pending_closures6329worker1 # [ 109.385570] nix-grpc-daemon[1360]: time=2026-09-22T22:22:03.403Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6330worker1 # [ 109.387180] nix-grpc-daemon[1360]: time=2026-09-22T22:22:03.405Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6331lb # [ 109.424061] niks3-server[1129]: 2026/09/22 22:22:02 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6332lb # [ 109.433107] niks3-server[1129]: 2026/09/22 22:22:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6333lb # [ 109.438967] niks3-server[1129]: 2026/09/22 22:22:02 INFO Signed narinfos id=46 count=16334worker1 # [ 109.430504] nix-grpc-daemon[1360]: time=2026-09-22T22:22:03.448Z level=INFO msg="Uploading 1 narinfos"6335lb # [ 109.446404] niks3-server[1129]: 2026/09/22 22:22:02 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6336lb # [ 109.449716] niks3-server[1129]: 2026/09/22 22:22:02 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6337lb # [ 109.455251] niks3-server[1129]: 2026/09/22 22:22:02 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6338lb # [ 109.459291] niks3-server[1129]: 2026/09/22 22:22:02 INFO Completed upload id=466339worker1 # [ 109.448423] nix-grpc-daemon[1360]: time=2026-09-22T22:22:03.466Z level=INFO msg="Upload complete. (183ms)"6340lb # [ 109.462059] niks3-server[1129]: 2026/09/22 22:22:02 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6341worker1 # [ 109.454863] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36096 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16342worker1 # [ 109.464141] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:03Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06343worker1 # [ 109.469211] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:03Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16344worker1 # [ 109.476763] nix-daemon[846]: reaped child process 1410, status = succeeded6345worker1 # [ 109.508523] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360966346worker1 # [ 109.520682] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360966347worker1 # [ 109.534321] nix-grpc-daemon[1360]: time=2026-09-22T22:22:03.552Z level=INFO msg="All 2 paths already cached"6348client # copying 1 paths to 'lb:50051'...6349client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6350worker1 # [ 109.543741] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:360966351worker1 # [ 109.554103] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6352worker1 # [ 109.651870] nix-daemon[846]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)6353worker1 # [ 109.666842] nix-daemon[846]: reaped child process 1440, status = succeeded6354lb # [ 109.683837] niks3-server[1129]: 2026/09/22 22:22:03 INFO Received uploads request method=POST path=/api/pending_closures6355worker1 # [ 109.683768] nix-grpc-daemon[1360]: time=2026-09-22T22:22:03.702Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6356worker1 # [ 109.685326] nix-grpc-daemon[1360]: time=2026-09-22T22:22:03.703Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6357lb # [ 109.717647] niks3-server[1129]: 2026/09/22 22:22:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6358worker1 # [ 109.711618] nix-grpc-daemon[1360]: time=2026-09-22T22:22:03.730Z level=INFO msg="Uploading 1 narinfos"6359lb # [ 109.722613] niks3-server[1129]: 2026/09/22 22:22:03 INFO Signed narinfos id=47 count=16360lb # [ 109.729622] niks3-server[1129]: 2026/09/22 22:22:03 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6361lb # [ 109.736993] niks3-server[1129]: 2026/09/22 22:22:03 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6362lb # [ 109.741942] niks3-server[1129]: 2026/09/22 22:22:03 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6363lb # [ 109.745090] niks3-server[1129]: 2026/09/22 22:22:03 INFO Completed upload id=476364worker1 # [ 109.733899] nix-grpc-daemon[1360]: time=2026-09-22T22:22:03.752Z level=INFO msg="Upload complete. (170ms)"6365worker1 # [ 109.737154] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36096 duration_s=0 paths=1 nar_bytes_in=7686366lb # [ 109.750918] niks3-server[1129]: 2026/09/22 22:22:03 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6367worker1 # [ 109.742226] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360966368worker1 # [ 109.750586] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6369worker1 # [ 109.756412] nix-daemon[846]: reaped child process 1435, status = succeeded6370worker1 # [ 109.760834] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6371client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6372client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6373client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6374client # farm-top> LOG-farm-top-on-standby6375worker1 # [ 111.402227] nix-daemon[846]: reaped child process 1444, status = succeeded6376worker1 # [ 111.490274] nix-daemon[846]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)6377worker1 # [ 111.505178] nix-daemon[846]: reaped child process 1463, status = succeeded6378lb # [ 111.525314] niks3-server[1129]: 2026/09/22 22:22:04 INFO Received uploads request method=POST path=/api/pending_closures6379worker1 # [ 111.522899] nix-grpc-daemon[1360]: time=2026-09-22T22:22:05.541Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6380worker1 # [ 111.524651] nix-grpc-daemon[1360]: time=2026-09-22T22:22:05.543Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6381lb # [ 111.568717] niks3-server[1129]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6382lb # [ 111.574738] niks3-server[1129]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6383lb # [ 111.584253] niks3-server[1129]: 2026/09/22 22:22:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6384lb # [ 111.590261] niks3-server[1129]: 2026/09/22 22:22:04 INFO Signed narinfos id=48 count=16385worker1 # [ 111.583070] nix-grpc-daemon[1360]: time=2026-09-22T22:22:05.600Z level=INFO msg="Uploading 1 narinfos"6386lb # [ 111.594350] niks3-server[1129]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6387lb # [ 111.605033] niks3-server[1129]: 2026/09/22 22:22:04 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6388lb # [ 111.608098] niks3-server[1129]: 2026/09/22 22:22:04 INFO Completed upload id=486389lb # [ 111.610525] niks3-server[1129]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6390worker1 # [ 111.597643] nix-grpc-daemon[1360]: time=2026-09-22T22:22:05.615Z level=INFO msg="Upload complete. (227ms)"6391worker1 # [ 111.605955] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36096 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16392worker1 # [ 111.622056] nix-daemon[846]: reaped child process 1442, status = succeeded6393client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.58 seconds)6394client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6395client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6396worker2 # [ 111.821698] systemd[1]: Listening on Nix worker-protocol over gRPC.6397worker2 # [ 111.827648] systemd[1]: Starting scheduler-token.service...6398worker2 # [ 111.870470] systemd[1]: scheduler-token.service: Deactivated successfully.6399worker2 # [ 111.871987] systemd[1]: Finished scheduler-token.service.6400worker2 # [ 111.873190] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 45ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6401worker2 # [ 111.879101] systemd[1]: Starting Nix worker-protocol over gRPC...6402worker2 # [ 111.955796] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6403worker2 # [ 111.957850] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6404worker2 # [ 111.961042] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16405worker2 # [ 111.965454] nix-grpc-daemon[1448]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6406worker2 # [ 111.966817] nix-grpc-daemon[1448]: E0000 00:00:1790115725.213258 1448 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6407worker2 # [ 111.972934] nix-grpc-daemon[1448]: E0000 00:00:1790115725.218609 1448 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6408worker2 # [ 111.977912] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6409worker2 # [ 111.981092] systemd[1]: Started Nix worker-protocol over gRPC.6410worker2 # [ 111.981933] nix-grpc-daemon[1448]: E0000 00:00:1790115725.228995 1489 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6411worker2 # [ 111.997053] nix-grpc-daemon[1448]: E0000 00:00:1790115725.243532 1489 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6412worker2 # [ 111.999202] nix-grpc-daemon[1448]: E0000 00:00:1790115725.243702 1489 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6413lb: must succeed: curl -sf localhost:9901/clusters6414worker2 # [ 112.010342] nix-grpc-daemon[1448]: E0000 00:00:1790115725.258328 1489 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6415worker2 # [ 112.022616] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500516416worker1 # [ 112.020184] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=info event=oidc_jwks provider=mock keys=16417worker1 # [ 112.025257] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:455166418worker1 # [ 112.029885] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06419lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6420worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6421worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6422lb: must succeed: curl -sf localhost:9901/clusters6423lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6424lb: must succeed: curl -sf localhost:9901/clusters6425lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6426worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6427worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6428(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.34 seconds)6429subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6430client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6431client # [ 112.287886] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6432client # Running as unit: rs.service; invocation ID: 6323af5ff60b40c9bd4bdc938ca560366433client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6434worker1 # [ 112.376765] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:360966435client # [ 112.429411] nix[1363]: don't know how to build these paths:6436client # [ 112.430352] nix[1363]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6437worker1 # [ 112.391135] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06438worker1 # [ 112.397939] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16439worker1 # [ 112.435724] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360966440worker1 # [ 112.449357] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:360966441client # [ 112.505170] nix[1363]: copying 1 paths to 'lb:50051'...6442client # [ 112.507902] nix[1363]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6443worker1 # [ 112.463981] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:360966444worker1 # [ 112.476484] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6445worker1 # [ 112.579611] nix-daemon[846]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)6446worker1 # [ 112.595681] nix-daemon[846]: reaped child process 1479, status = succeeded6447lb # [ 112.612713] niks3-server[1129]: 2026/09/22 22:22:05 INFO Received uploads request method=POST path=/api/pending_closures6448worker1 # [ 112.610191] nix-grpc-daemon[1360]: time=2026-09-22T22:22:06.628Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6449worker1 # [ 112.612481] nix-grpc-daemon[1360]: time=2026-09-22T22:22:06.630Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6450lb # [ 112.644944] niks3-server[1129]: 2026/09/22 22:22:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6451lb # [ 112.650966] niks3-server[1129]: 2026/09/22 22:22:05 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6452lb # [ 112.656248] niks3-server[1129]: 2026/09/22 22:22:06 INFO Signed narinfos id=49 count=16453worker1 # [ 112.648374] nix-grpc-daemon[1360]: time=2026-09-22T22:22:06.666Z level=INFO msg="Uploading 1 narinfos"6454lb # [ 112.669360] niks3-server[1129]: 2026/09/22 22:22:06 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6455lb # [ 112.676594] niks3-server[1129]: 2026/09/22 22:22:06 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6456lb # [ 112.682728] niks3-server[1129]: 2026/09/22 22:22:06 INFO Completed upload id=496457worker1 # [ 112.671549] nix-grpc-daemon[1360]: time=2026-09-22T22:22:06.689Z level=INFO msg="Upload complete. (183ms)"6458lb # [ 112.687611] niks3-server[1129]: 2026/09/22 22:22:06 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6459worker1 # [ 112.676711] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36096 duration_s=0 paths=1 nar_bytes_in=4166460worker1 # [ 112.683296] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:360966461worker1 # [ 112.691721] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6462worker1 # [ 112.698618] nix-daemon[846]: reaped child process 1474, status = succeeded6463worker1 # [ 112.702929] nix-daemon[846]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)6464client # [ 112.747887] nix[1363]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6465lb: must succeed: curl -sf localhost:9901/clusters6466lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6467lb: must succeed: curl -sf localhost:9901/clusters6468lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6469lb: must succeed: curl -sf localhost:9901/clusters6470lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6471lb: must succeed: curl -sf localhost:9901/clusters6472lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6473worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6474worker2 # [ 113.818198] systemd-journald[283]: Received client request to rotate journal, rotating.6475worker2 # [ 113.838675] systemd-journald[283]: Vacuuming done, freed 0B of archived journals from /var/log/journal/09bb6c2dceb44973b0e4d0ee675dff46.6476worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6477worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6478worker1 # [ 113.736620] systemd[1]: Stopping Nix worker-protocol over gRPC...6479worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6480client: must succeed: sleep 5.06481worker1 # [ 114.410812] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:08Z level=info event=draining6482lb # [ 114.686497] envoy[906]: conn peer=[2001:db8:1::1]:44874 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9603 tx=4390 ms=54096483client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6484worker1 # [ 118.923291] nix-daemon[846]: reaped child process 1483, status = succeeded6485worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6486worker1 # [ 119.002392] nix-daemon[846]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)6487worker1 # [ 119.016614] nix-daemon[846]: reaped child process 1523, status = succeeded6488lb # [ 119.035898] niks3-server[1129]: 2026/09/22 22:22:12 INFO Received uploads request method=POST path=/api/pending_closures6489worker1 # [ 119.033648] nix-grpc-daemon[1360]: time=2026-09-22T22:22:13.051Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6490worker1 # [ 119.035424] nix-grpc-daemon[1360]: time=2026-09-22T22:22:13.053Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6491lb # [ 119.086799] niks3-server[1129]: 2026/09/22 22:22:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6492lb # [ 119.093476] niks3-server[1129]: 2026/09/22 22:22:12 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6493worker1 # [ 119.087640] nix-grpc-daemon[1360]: time=2026-09-22T22:22:13.105Z level=INFO msg="Uploading 1 narinfos"6494lb # [ 119.099908] niks3-server[1129]: 2026/09/22 22:22:12 INFO Signed narinfos id=50 count=16495lb # [ 119.101854] niks3-server[1129]: 2026/09/22 22:22:12 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6496lb # [ 119.112266] niks3-server[1129]: 2026/09/22 22:22:12 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6497lb # [ 119.119134] niks3-server[1129]: 2026/09/22 22:22:12 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6498lb # [ 119.121653] niks3-server[1129]: 2026/09/22 22:22:12 INFO Completed upload id=506499worker1 # [ 119.110720] nix-grpc-daemon[1360]: time=2026-09-22T22:22:13.129Z level=INFO msg="Upload complete. (196ms)"6500lb # [ 119.124673] niks3-server[1129]: 2026/09/22 22:22:12 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6501worker1 # [ 119.119772] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36096 duration_s=6 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16502worker1 # [ 119.133156] nix-daemon[846]: reaped child process 1481, status = succeeded6503client # [ 119.194259] systemd[1]: rs.service: Deactivated successfully.6504client # [ 119.197393] systemd[1]: rs.service: Consumed 123ms CPU time over 6.905s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6505worker1 # [ 119.414377] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:13Z level=info event=signal_exit6506lb # [ 119.590168] niks3-server[1129]: 2026/09/22 22:22:12 INFO lead: released remote=192.168.1.3:586006507worker2 # [ 120.593538] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:13Z level=info event=scheduler_restarting addr=lb:500516508worker1 # [ 120.579866] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:14Z level=info event=scheduler_restarting6509worker1 # [ 120.590521] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:455166510worker1 # [ 120.595349] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:14Z level=info event=worker_gone addr=192.168.1.4:500516511worker1 # [ 120.599156] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:14Z level=info event=scheduler_restarting addr=lb:500516512worker1 # [ 120.602872] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:455166513worker1 # [ 120.606817] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:14Z level=info event=worker_gone addr=192.168.1.3:500516514worker1 # [ 120.624955] nix-daemon[846]: reaped child process 1318, status = succeeded6515worker2 # [ 120.793923] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_connected addr=lb:500516516worker2 # [ 120.798305] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6517worker1 # [ 120.800225] nix-grpc-daemon[1266]: ts=2026-09-22T22:22:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6518worker2 # [ 120.999104] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_connected addr=lb:500516519worker2 # [ 121.003160] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6520worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.11 seconds)6521client: waiting for success: ! systemctl is-active rs6522worker1 # [ 121.070402] nix-grpc-daemon[1266]: E0000 00:00:1790115735.088404 1311 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6523worker1 # [ 121.076395] nix-grpc-daemon[1266]: E0000 00:00:1790115735.089130 1311 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6524client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6525client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6526client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6527client: must fail: journalctl -u rs | grep -q 'reconnecting'6528client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6529lb: must succeed: curl -sf localhost:9901/clusters6530lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6531worker2 # [ 121.204386] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_connected addr=lb:500516532worker2 # [ 121.208372] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6533worker2 # [ 121.409647] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_connected addr=lb:500516534worker2 # [ 121.413542] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6535worker2 # [ 121.615327] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_connected addr=lb:500516536worker2 # [ 121.619200] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6537worker2 # [ 121.820283] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_connected addr=lb:500516538worker2 # [ 121.824298] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6539lb # [ 121.983754] niks3-server[1129]: 2026/09/22 22:22:15 INFO lead: acquired remote=192.168.1.4:564726540worker2 # [ 121.986718] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_take_over6541worker2 # [ 122.025155] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_connected addr=lb:500516542worker2 # [ 122.029174] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6543worker1 # [ 122.117992] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6544worker1 # [ 122.121508] systemd[1]: Stopped Nix worker-protocol over gRPC.6545worker1 # [ 122.124677] systemd[1]: nix-grpc-daemon.service: Consumed 1.113s CPU time over 29.825s wall clock time, 37.4M memory peak, 50.1K incoming IP traffic, 104.9K outgoing IP traffic.6546worker1 # [ 122.140793] systemd[1]: Starting Nix worker-protocol over gRPC...6547lb: must succeed: curl -sf localhost:9901/clusters6548worker2 # [ 122.230281] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_connected addr=lb:500516549worker2 # [ 122.234196] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6550worker1 # [ 122.234128] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6551worker1 # [ 122.236437] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6552worker1 # [ 122.240060] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16553lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6554worker1 # [ 122.244559] nix-grpc-daemon[1530]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6555worker1 # [ 122.246117] nix-grpc-daemon[1530]: E0000 00:00:1790115736.262951 1530 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6556worker1 # [ 122.253426] nix-grpc-daemon[1530]: E0000 00:00:1790115736.268954 1530 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6557worker1 # [ 122.256924] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6558worker1 # [ 122.258995] systemd[1]: Started Nix worker-protocol over gRPC.6559worker1 # [ 122.262065] nix-grpc-daemon[1530]: E0000 00:00:1790115736.279918 1571 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6560worker1 # [ 122.273992] nix-grpc-daemon[1530]: E0000 00:00:1790115736.292359 1571 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6561worker1 # [ 122.279056] nix-grpc-daemon[1530]: E0000 00:00:1790115736.295051 1571 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6562worker1 # [ 122.288981] nix-grpc-daemon[1530]: E0000 00:00:1790115736.307361 1571 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6563worker1 # [ 122.298828] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:500516564worker1 # [ 122.300589] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6565worker2 # [ 122.435215] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_connected addr=lb:500516566worker2 # [ 122.438999] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6567worker1 # [ 122.501681] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:500516568worker1 # [ 122.506279] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6569worker2 # [ 122.640903] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_connected addr=lb:500516570worker2 # [ 122.644812] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6571worker1 # [ 122.707538] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:500516572worker1 # [ 122.712197] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6573worker2 # [ 122.847116] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:500516574worker2 # [ 122.860158] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=info event=oidc_jwks provider=mock keys=16575worker2 # [ 122.864206] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:536486576worker2 # [ 122.868165] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06577worker1 # [ 122.913458] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:500516578worker2 # [ 122.926278] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:536486579worker2 # [ 122.929988] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06580lb: must succeed: curl -sf localhost:9901/clusters6581lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6582worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6583worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6584(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.14 seconds)6585subtest: scheduler restart mid-build: build finishes, no double build6586client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6587client # [ 123.423290] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6588client # Running as unit: mid.service; invocation ID: 9a2e0d689f3149c5914395117b650d3d6589client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6590worker2 # [ 123.532632] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536586591worker2 # [ 123.543662] nix-daemon[870]: accepted connection from pid 1448, user nix-grpc-daemon (trusted)6592client # [ 123.590207] nix[1399]: don't know how to build these paths:6593client # [ 123.591183] nix[1399]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6594worker2 # [ 123.562224] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06595worker2 # [ 123.565979] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16596worker2 # [ 123.610820] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536586597worker2 # [ 123.621885] nix-grpc-daemon[1448]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 1/5)6598worker2 # [ 123.654833] nix-grpc-daemon[1448]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)6599worker2 # [ 123.678830] nix-grpc-daemon[1448]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 126 ms (attempt 3/5)6600worker2 # [ 123.807134] nix-grpc-daemon[1448]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 4/5)6601worker2 # [ 123.976251] nix-grpc-daemon[1448]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6602worker2 # [ 123.979831] nix-grpc-daemon[1448]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6603worker2 # [ 123.997235] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536586604client # [ 124.040923] nix[1399]: copying 1 paths to 'lb:50051'...6605client # [ 124.045152] nix[1399]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6606worker2 # [ 124.010903] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536586607worker2 # [ 124.019409] nix-daemon[870]: accepted connection from pid 1448, user nix-grpc-daemon (trusted)6608worker2 # [ 124.042416] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:17Z level=info event=niks3_push_started pid=15636609worker2 # [ 124.145464] nix-grpc-daemon[1563]: time=2026-09-22T22:22:17.393Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6610worker2 # [ 124.231975] nix-daemon[870]: accepted connection from pid 1576, user nix-grpc-daemon (trusted)6611worker2 # [ 124.243426] nix-daemon[870]: reaped child process 1579, status = succeeded6612lb # [ 124.252886] niks3-server[1129]: 2026/09/22 22:22:17 INFO Received uploads request method=POST path=/api/pending_closures6613worker2 # [ 124.263497] nix-grpc-daemon[1563]: time=2026-09-22T22:22:17.511Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6614worker2 # [ 124.265032] nix-grpc-daemon[1563]: time=2026-09-22T22:22:17.513Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6615lb # [ 124.291308] niks3-server[1129]: 2026/09/22 22:22:17 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6616lb # [ 124.299151] niks3-server[1129]: 2026/09/22 22:22:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6617lb # [ 124.306899] niks3-server[1129]: 2026/09/22 22:22:17 INFO Signed narinfos id=51 count=16618worker2 # [ 124.310119] nix-grpc-daemon[1563]: time=2026-09-22T22:22:17.558Z level=INFO msg="Uploading 1 narinfos"6619lb # [ 124.318133] niks3-server[1129]: 2026/09/22 22:22:17 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6620lb # [ 124.327919] niks3-server[1129]: 2026/09/22 22:22:17 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6621lb # [ 124.332800] niks3-server[1129]: 2026/09/22 22:22:17 INFO Completed upload id=516622worker2 # [ 124.332644] nix-grpc-daemon[1563]: time=2026-09-22T22:22:17.580Z level=INFO msg="Upload complete. (184ms)"6623lb # [ 124.334894] niks3-server[1129]: 2026/09/22 22:22:17 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6624worker2 # [ 124.336653] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53658 duration_s=0 paths=1 nar_bytes_in=4246625worker2 # [ 124.341488] nix-grpc-daemon[1448]: ts=2026-09-22T22:22:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536586626worker2 # [ 124.353217] nix-daemon[870]: accepted connection from pid 1448, user nix-grpc-daemon (trusted)6627worker2 # [ 124.355251] nix-daemon[870]: reaped child process 1561, status = succeeded6628worker2 # [ 124.360176] nix-daemon[870]: accepted connection from pid 1448, user nix-grpc-daemon (trusted)6629client # [ 124.394122] nix[1399]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6630lb # [ 124.686600] envoy[906]: conn peer=[2001:db8:1::1]:44882 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1498 ms=67526631lb # [ 124.690942] envoy[906]: conn peer=[2001:db8:1::3]:49712 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5809 tx=1354 ms=283506632lb: must succeed: curl -sf localhost:9901/clusters6633lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6634lb: must succeed: curl -sf localhost:9901/clusters6635lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6636worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6637worker2 # [ 125.895442] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1448 (nix-grpc-daemon) on client request.6638worker2 # [ 125.899344] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6639lb # [ 125.906400] niks3-server[1129]: 2026/09/22 22:22:19 INFO lead: released remote=192.168.1.4:564726640worker2 # [ 125.917499] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6641client # [ 125.949500] nix[1399]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6642worker2 # [ 125.918884] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6643worker1 # [ 125.904417] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:19Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6644worker2 # [ 125.920091] systemd[1]: nix-grpc-daemon.service: Consumed 369ms CPU time over 14.035s wall clock time, 35.6M memory peak, 16.4K incoming IP traffic, 33.7K outgoing IP traffic.6645client # [ 125.954288] nix[1399]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6646worker2 # [ 125.925477] systemd[1]: Starting scheduler-token.service...6647worker2 # [ 125.948180] nix-daemon[870]: reaped child process 1553, status = succeeded6648worker2 # [ 125.950095] nix-daemon[870]: reaped child process 1581, status = succeeded6649worker2 # [ 125.959340] nix-daemon[1583]: unexpected Nix daemon error: error: interrupted by the user6650worker2 # [ 125.965198] nix-daemon[870]: reaped child process 1583, status = failed with exit code 16651worker2 # [ 125.981352] systemd[1]: scheduler-token.service: Deactivated successfully.6652worker2 # [ 125.982875] systemd[1]: Finished scheduler-token.service.6653worker2 # [ 125.984300] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 57ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6654worker2 # [ 125.987509] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6655worker2 # [ 125.990414] systemd[1]: Starting scheduler-token.service...6656worker2 # [ 126.024276] systemd[1]: scheduler-token.service: Deactivated successfully.6657worker2 # [ 126.025794] systemd[1]: Finished scheduler-token.service.6658worker2 # [ 126.027329] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 35ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6659worker2 # [ 126.031507] systemd[1]: Starting Nix worker-protocol over gRPC...6660worker2 # [ 126.109635] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6661worker2 # [ 126.114055] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6662worker1 # [ 126.104835] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:20Z level=info event=scheduler_connected addr=lb:500516663worker2 # [ 126.119170] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16664worker2 # [ 126.123155] nix-grpc-daemon[1626]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6665worker2 # [ 126.128467] nix-grpc-daemon[1626]: E0000 00:00:1790115739.364620 1626 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6666worker2 # [ 126.131584] nix-grpc-daemon[1626]: E0000 00:00:1790115739.366873 1626 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6667worker2 # [ 126.135434] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6668worker2 # [ 126.138584] systemd[1]: Started Nix worker-protocol over gRPC.6669lb # [ 126.137561] niks3-server[1129]: 2026/09/22 22:22:19 INFO lead: acquired remote=192.168.1.4:494346670worker1 # [ 126.126450] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:20Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6671worker2 # [ 126.140153] nix-grpc-daemon[1626]: E0000 00:00:1790115739.387462 1668 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6672worker2 # [ 126.145102] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=scheduler_take_over6673worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.28 seconds)6674client: must succeed: sleep 5.06675worker2 # [ 126.155086] nix-grpc-daemon[1626]: E0000 00:00:1790115739.401803 1668 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6676worker2 # [ 126.157178] nix-grpc-daemon[1626]: E0000 00:00:1790115739.401996 1668 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6677worker2 # [ 126.167969] nix-grpc-daemon[1626]: E0000 00:00:1790115739.415908 1668 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6678worker2 # [ 126.176872] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=scheduler_connected addr=lb:500516679worker2 # [ 126.182060] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=oidc_jwks provider=mock keys=16680worker2 # [ 126.183581] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:536726681worker2 # [ 126.185218] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06682worker1 # [ 126.327746] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:20Z level=info event=scheduler_connected addr=lb:500516683worker2 # [ 126.341115] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:536726684worker2 # [ 126.344838] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06685worker2 # [ 126.429339] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06686worker2 # [ 126.434726] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16687worker2 # [ 126.482167] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536866688worker2 # [ 126.492176] nix-daemon[870]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6689worker2 # [ 126.506643] nix-daemon[870]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6690client # [ 126.553822] nix[1399]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6691worker2 # [ 126.521426] nix-daemon[870]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6692client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6693client: waiting for success: ! systemctl is-active mid6694worker2 # [ 131.545365] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:24Z level=info event=niks3_push_started pid=17096695worker2 # [ 131.584684] nix-daemon[870]: reaped child process 1684, status = succeeded6696worker2 # [ 131.666868] nix-grpc-daemon[1709]: time=2026-09-22T22:22:24.914Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6697worker2 # [ 131.754099] nix-daemon[870]: accepted connection from pid 1721, user nix-grpc-daemon (trusted)6698worker2 # [ 131.766100] nix-daemon[870]: reaped child process 1724, status = succeeded6699lb # [ 131.774959] niks3-server[1129]: 2026/09/22 22:22:25 INFO Received uploads request method=POST path=/api/pending_closures6700worker2 # [ 131.783694] nix-grpc-daemon[1709]: time=2026-09-22T22:22:25.031Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6701worker2 # [ 131.785280] nix-grpc-daemon[1709]: time=2026-09-22T22:22:25.033Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6702lb # [ 131.829241] niks3-server[1129]: 2026/09/22 22:22:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6703worker2 # [ 131.853892] nix-grpc-daemon[1709]: time=2026-09-22T22:22:25.101Z level=INFO msg="Uploading 1 narinfos"6704lb # [ 131.854364] niks3-server[1129]: 2026/09/22 22:22:25 INFO Signed narinfos id=52 count=16705lb # [ 131.865701] niks3-server[1129]: 2026/09/22 22:22:25 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6706lb # [ 131.873324] niks3-server[1129]: 2026/09/22 22:22:25 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6707lb # [ 131.878601] niks3-server[1129]: 2026/09/22 22:22:25 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6708lb # [ 131.883910] niks3-server[1129]: 2026/09/22 22:22:25 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6709lb # [ 131.887591] niks3-server[1129]: 2026/09/22 22:22:25 INFO Completed upload id=526710worker2 # [ 131.887458] nix-grpc-daemon[1709]: time=2026-09-22T22:22:25.135Z level=INFO msg="Upload complete. (217ms)"6711lb # [ 131.890251] niks3-server[1129]: 2026/09/22 22:22:25 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6712worker2 # [ 131.893971] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53686 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16713worker2 # [ 131.911239] nix-daemon[870]: reaped child process 1682, status = succeeded6714client # [ 131.967373] systemd[1]: mid.service: Deactivated successfully.6715client # [ 131.970336] systemd[1]: mid.service: Consumed 140ms CPU time over 8.543s wall clock time, 41.4M memory peak, 8.8K incoming IP traffic, 10.4K outgoing IP traffic.6716client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6717client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6718client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6719(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.13 seconds)6720subtest: interrupting the client stops the build on the worker6721client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6722client # [ 132.527447] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6723client # Running as unit: intr.service; invocation ID: 81bde05649d0419faaf2f1e45c0fa6e46724client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6725worker2 # [ 132.611564] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536866726client # [ 132.651939] nix[1435]: don't know how to build these paths:6727client # [ 132.653578] nix[1435]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6728worker2 # [ 132.629175] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:25Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06729worker2 # [ 132.633329] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:25Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=16730worker1 # [ 132.672843] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584506731worker1 # [ 132.685630] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6732worker1 # [ 132.703211] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 1/5)6733worker1 # [ 132.781905] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)6734worker1 # [ 132.826584] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 189 ms (attempt 3/5)6735worker1 # [ 133.017924] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 4/5)6736worker1 # [ 133.190157] nix-grpc-daemon[1530]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6737worker1 # [ 133.193876] nix-grpc-daemon[1530]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6738worker1 # [ 133.212230] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584506739client # [ 133.268264] nix[1435]: copying 1 paths to 'lb:50051'...6740client # [ 133.272137] nix[1435]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6741worker1 # [ 133.227509] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584506742worker1 # [ 133.238158] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6743worker1 # [ 133.259360] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:27Z level=info event=niks3_push_started pid=16186744worker1 # [ 133.366270] nix-grpc-daemon[1618]: time=2026-09-22T22:22:27.382Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6745worker1 # [ 133.455616] nix-daemon[846]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6746worker1 # [ 133.469557] nix-daemon[846]: reaped child process 1634, status = succeeded6747lb # [ 133.487411] niks3-server[1129]: 2026/09/22 22:22:26 INFO Received uploads request method=POST path=/api/pending_closures6748worker1 # [ 133.488063] nix-grpc-daemon[1618]: time=2026-09-22T22:22:27.506Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6749worker1 # [ 133.489824] nix-grpc-daemon[1618]: time=2026-09-22T22:22:27.508Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6750lb # [ 133.528987] niks3-server[1129]: 2026/09/22 22:22:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6751worker1 # [ 133.525600] nix-grpc-daemon[1618]: time=2026-09-22T22:22:27.544Z level=INFO msg="Uploading 1 narinfos"6752lb # [ 133.536900] niks3-server[1129]: 2026/09/22 22:22:26 INFO Signed narinfos id=53 count=16753lb # [ 133.545465] niks3-server[1129]: 2026/09/22 22:22:26 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6754lb # [ 133.552768] niks3-server[1129]: 2026/09/22 22:22:26 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6755lb # [ 133.563278] niks3-server[1129]: 2026/09/22 22:22:26 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6756lb # [ 133.566610] niks3-server[1129]: 2026/09/22 22:22:26 INFO Completed upload id=536757worker1 # [ 133.556850] nix-grpc-daemon[1618]: time=2026-09-22T22:22:27.575Z level=INFO msg="Upload complete. (192ms)"6758lb # [ 133.569759] niks3-server[1129]: 2026/09/22 22:22:26 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6759worker1 # [ 133.561676] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=4246760worker1 # [ 133.567457] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584506761worker1 # [ 133.579292] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6762worker1 # [ 133.584074] nix-daemon[846]: reaped child process 1616, status = succeeded6763worker1 # [ 133.587522] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6764client # [ 133.632367] nix[1435]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6765lb # [ 134.687752] envoy[906]: conn peer=[2001:db8:1::4]:34430 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=10313 tx=691 ms=138986766lb # [ 134.692244] envoy[906]: conn peer=[2001:db8:1::1]:33420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3921 tx=1942 ms=83866767client: must succeed: systemctl kill -s INT intr6768client # [ 134.835593] systemd[1]: intr.service: Sent signal SIGINT to main process 1435 (nix) on client request.6769client # [ 134.842567] nix[1435]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6770client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6771client # [ 134.849334] nix[1435]: error: interrupted by the user6772worker1 # [ 134.812778] nix-daemon[846]: reaped child process 1636, status = succeeded6773worker1 # [ 134.816731] nix-daemon[1638]: unexpected Nix daemon error: error: interrupted by the user6774worker1 # [ 134.823160] nix-daemon[846]: reaped child process 1638, status = failed with exit code 16775client # [ 134.875394] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6776client # [ 134.877151] systemd[1]: intr.service: Failed with result 'exit-code'.6777client # [ 134.879097] systemd[1]: intr.service: Consumed 115ms CPU time over 2.350s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.6778(finished: subtest: interrupting the client stops the build on the worker, in 2.42 seconds)6779subtest: a deploy mid-build drains: the build finishes, the new generation takes over6780client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6781client # [ 134.964128] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6782client # Running as unit: sw.service; invocation ID: 6c6463b5b9fe4491aa32228e58d939626783client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.04 seconds)6784worker1 # [ 135.039497] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584506785client # [ 135.090832] nix[1456]: don't know how to build these paths:6786client # [ 135.091896] nix[1456]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6787worker2 # [ 135.066734] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:28Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06788worker2 # [ 135.070927] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:28Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=16789worker2 # [ 135.109439] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536866790worker2 # [ 135.121265] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 1/5)6791worker2 # [ 135.154273] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 2/5)6792worker2 # [ 135.177814] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 145 ms (attempt 3/5)6793worker2 # [ 135.325218] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 712 ms (attempt 4/5)6794worker2 # [ 136.039618] nix-grpc-daemon[1626]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6795worker2 # [ 136.043537] nix-grpc-daemon[1626]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6796worker2 # [ 136.058113] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536866797client # [ 136.103634] nix[1456]: copying 1 paths to 'lb:50051'...6798client # [ 136.108148] nix[1456]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6799worker2 # [ 136.073620] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536866800worker2 # [ 136.089293] nix-daemon[870]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6801worker2 # [ 136.191678] nix-daemon[870]: accepted connection from pid 1760, user nix-grpc-daemon (trusted)6802worker2 # [ 136.204585] nix-daemon[870]: reaped child process 1763, status = succeeded6803lb # [ 136.212137] niks3-server[1129]: 2026/09/22 22:22:29 INFO Received uploads request method=POST path=/api/pending_closures6804worker2 # [ 136.220532] nix-grpc-daemon[1709]: time=2026-09-22T22:22:29.468Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6805worker2 # [ 136.222126] nix-grpc-daemon[1709]: time=2026-09-22T22:22:29.469Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6806lb # [ 136.234824] niks3-server[1129]: 2026/09/22 22:22:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6807lb # [ 136.237283] niks3-server[1129]: 2026/09/22 22:22:29 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6808lb # [ 136.240334] niks3-server[1129]: 2026/09/22 22:22:29 INFO Signed narinfos id=54 count=16809worker2 # [ 136.240645] nix-grpc-daemon[1709]: time=2026-09-22T22:22:29.488Z level=INFO msg="Uploading 1 narinfos"6810lb # [ 136.247261] niks3-server[1129]: 2026/09/22 22:22:29 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6811lb # [ 136.250779] niks3-server[1129]: 2026/09/22 22:22:29 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6812lb # [ 136.255806] niks3-server[1129]: 2026/09/22 22:22:29 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6813lb # [ 136.259472] niks3-server[1129]: 2026/09/22 22:22:29 INFO Completed upload id=546814worker2 # [ 136.259283] nix-grpc-daemon[1709]: time=2026-09-22T22:22:29.507Z level=INFO msg="Upload complete. (155ms)"6815worker2 # [ 136.261485] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53686 duration_s=0 paths=1 nar_bytes_in=4166816worker2 # [ 136.264319] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536866817worker2 # [ 136.271312] nix-daemon[870]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6818worker2 # [ 136.273613] nix-daemon[870]: reaped child process 1757, status = succeeded6819worker2 # [ 136.278791] nix-daemon[870]: accepted connection from pid 1626, user nix-grpc-daemon (trusted)6820client # [ 136.312747] nix[1456]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6821worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26822worker2 # Checking switch inhibitors... done6823worker2 # [ 137.692911] nixos[1795]: switching to system configuration /nix/store/jws3bxmrcfqnv642m2fh00aa1j2lax95-nixos-system-worker2-test6824worker2 # activating the configuration...6825worker2 # [ 137.694991] systemd[1]: Stopped target Local File Systems.6826worker2 # [ 137.697580] systemd[1]: Stopped target Remote File Systems.6827worker2 # setting up /etc...6828worker2 # [ 138.114839] systemd[1]: Reload requested from client PID 1795 ('.switch-to-conf') (unit backdoor.service)...6829worker2 # [ 138.116564] systemd[1]: Reloading...6830worker2 # [ 139.471317] systemd[1]: Reloading finished in 1351 ms.6831worker2 # restarting sysinit-reactivation.target6832worker2 # [ 139.533296] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6833worker2 # [ 139.640423] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6834worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6835worker2 # [ 139.645789] systemd[1]: Reached target Reactivate sysinit units.6836worker2 # [ 139.647277] systemd[1]: Reloading D-Bus System Message Bus...6837worker2 # [ 139.648497] dbus-broker-launch[518]: Caught SIGHUP, trigger reload.6838worker2 # [ 139.654075] systemd[1]: Reloading Nix worker-protocol over gRPC...6839worker2 # [ 139.664932] dbus-broker-launch[518]: Looking up NSS user entry for 'systemd-timesync'...6840worker2 # [ 139.669375] dbus-broker-launch[518]: NSS returned no entry for 'systemd-timesync'6841worker2 # [ 139.670686] dbus-broker-launch[518]: Invalid user-name in /nix/store/9f31zf0wixjfmyvwg6m53sif40vrkysg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6842worker2 # [ 139.688882] systemd[1]: Reloaded Nix worker-protocol over gRPC.6843worker2 # [ 139.696025] systemd[1]: Reloaded D-Bus System Message Bus.6844worker2 # [ 139.699921] systemd[1]: Reached target Local File Systems.6845worker2 # [ 139.706511] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6846worker2 # [ 139.711931] systemd[1]: Starting Create SUID/SGID Wrappers...6847worker2 # [ 139.712778] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6848worker2 # [ 139.713934] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676849worker2 # [ 139.715939] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6850worker2 # [ 139.717408] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6851worker2 # [ 139.718616] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6852worker2 # [ 139.724624] systemd[1]: Reached target Remote File Systems.6853worker2 # [ 139.728693] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6854worker2 # [ 139.729987] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6855worker2 # [ 139.980654] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6856worker2 # [ 139.985255] systemd[1]: Finished Create SUID/SGID Wrappers.6857worker2 # [ 139.986834] systemd[1]: Starting linger-users.service...6858worker2 # [ 140.013356] systemd[1]: linger-users.service: Deactivated successfully.6859worker2 # [ 140.016966] systemd[1]: Finished linger-users.service.6860worker2 # [ 140.128520] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:33Z level=info event=draining6861worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6862worker2 # [ 140.271856] nixos[1795]: finished switching to system configuration /nix/store/jws3bxmrcfqnv642m2fh00aa1j2lax95-nixos-system-worker2-test6863worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.94 seconds)6864worker2: must succeed: pgrep -f 'read -t [9]0 x'6865worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6866client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26867worker1 # [ 140.449723] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584506868client # don't know how to build these paths:6869client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6870worker1 # [ 140.463517] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584506871worker2 # [ 140.496205] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:33Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06872worker2 # [ 140.498138] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:33Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16873worker1 # [ 140.530087] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584506874worker1 # [ 140.542711] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584506875client # copying 1 paths to 'lb:50051'...6876worker1 # [ 140.555572] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584506877client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6878worker1 # [ 140.582644] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6879worker1 # [ 140.691315] nix-daemon[846]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6880worker1 # [ 140.704973] nix-daemon[846]: reaped child process 1688, status = succeeded6881lb # [ 140.725468] niks3-server[1129]: 2026/09/22 22:22:34 INFO Received uploads request method=POST path=/api/pending_closures6882worker1 # [ 140.725335] nix-grpc-daemon[1618]: time=2026-09-22T22:22:34.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6883worker1 # [ 140.726794] nix-grpc-daemon[1618]: time=2026-09-22T22:22:34.743Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6884lb # [ 140.760159] niks3-server[1129]: 2026/09/22 22:22:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6885worker1 # [ 140.757938] nix-grpc-daemon[1618]: time=2026-09-22T22:22:34.776Z level=INFO msg="Uploading 1 narinfos"6886lb # [ 140.768559] niks3-server[1129]: 2026/09/22 22:22:34 INFO Signed narinfos id=55 count=16887lb # [ 140.778560] niks3-server[1129]: 2026/09/22 22:22:34 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6888lb # [ 140.784371] niks3-server[1129]: 2026/09/22 22:22:34 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6889lb # [ 140.796205] niks3-server[1129]: 2026/09/22 22:22:34 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6890lb # [ 140.801917] niks3-server[1129]: 2026/09/22 22:22:34 INFO Completed upload id=556891lb # [ 140.803890] niks3-server[1129]: 2026/09/22 22:22:34 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6892worker1 # [ 140.792327] nix-grpc-daemon[1618]: time=2026-09-22T22:22:34.810Z level=INFO msg="Upload complete. (198ms)"6893worker1 # [ 140.796981] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=5286894worker1 # [ 140.802857] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584506895worker1 # [ 140.813893] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6896worker1 # [ 140.825679] nix-daemon[846]: reaped child process 1683, status = succeeded6897worker1 # [ 140.829791] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6898client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6899client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6900client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6901client # farm-a> LOG-farm-a-during-drain6902worker1 # [ 141.846253] nix-daemon[846]: reaped child process 1692, status = succeeded6903worker1 # [ 141.918621] nix-daemon[846]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6904worker1 # [ 141.933144] nix-daemon[846]: reaped child process 1711, status = succeeded6905lb # [ 141.952525] niks3-server[1129]: 2026/09/22 22:22:35 INFO Received uploads request method=POST path=/api/pending_closures6906worker1 # [ 141.949485] nix-grpc-daemon[1618]: time=2026-09-22T22:22:35.967Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6907worker1 # [ 141.951276] nix-grpc-daemon[1618]: time=2026-09-22T22:22:35.969Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6908lb # [ 141.991358] niks3-server[1129]: 2026/09/22 22:22:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6909lb # [ 141.994062] niks3-server[1129]: 2026/09/22 22:22:35 INFO Signed narinfos id=56 count=16910worker1 # [ 141.983123] nix-grpc-daemon[1618]: time=2026-09-22T22:22:36.001Z level=INFO msg="Uploading 1 narinfos"6911lb # [ 142.000140] niks3-server[1129]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6912lb # [ 142.003481] niks3-server[1129]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6913lb # [ 142.008540] niks3-server[1129]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6914lb # [ 142.014450] niks3-server[1129]: 2026/09/22 22:22:35 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6915lb # [ 142.018394] niks3-server[1129]: 2026/09/22 22:22:35 INFO Completed upload id=566916worker1 # [ 142.008878] nix-grpc-daemon[1618]: time=2026-09-22T22:22:36.027Z level=INFO msg="Upload complete. (172ms)"6917lb # [ 142.020741] niks3-server[1129]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6918worker1 # [ 142.012546] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16919worker2 # [ 142.028848] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:35Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06920worker1 # [ 142.020627] nix-daemon[846]: reaped child process 1690, status = succeeded6921worker2 # [ 142.031216] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:35Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16922worker1 # [ 142.061205] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584506923worker1 # [ 142.072583] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584506924worker1 # [ 142.084689] nix-grpc-daemon[1618]: time=2026-09-22T22:22:36.103Z level=INFO msg="All 1 paths already cached"6925client # copying 1 paths to 'lb:50051'...6926worker1 # [ 142.093310] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584506927client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6928worker1 # [ 142.107430] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6929worker1 # [ 142.206445] nix-daemon[846]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6930worker1 # [ 142.221901] nix-daemon[846]: reaped child process 1718, status = succeeded6931lb # [ 142.238144] niks3-server[1129]: 2026/09/22 22:22:35 INFO Received uploads request method=POST path=/api/pending_closures6932worker1 # [ 142.235383] nix-grpc-daemon[1618]: time=2026-09-22T22:22:36.253Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6933worker1 # [ 142.237456] nix-grpc-daemon[1618]: time=2026-09-22T22:22:36.255Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6934lb # [ 142.271683] niks3-server[1129]: 2026/09/22 22:22:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6935lb # [ 142.276765] niks3-server[1129]: 2026/09/22 22:22:35 INFO Signed narinfos id=57 count=16936worker1 # [ 142.268698] nix-grpc-daemon[1618]: time=2026-09-22T22:22:36.286Z level=INFO msg="Uploading 1 narinfos"6937lb # [ 142.284802] niks3-server[1129]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6938lb # [ 142.295147] niks3-server[1129]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6939lb # [ 142.305156] niks3-server[1129]: 2026/09/22 22:22:35 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6940lb # [ 142.311594] niks3-server[1129]: 2026/09/22 22:22:35 INFO Completed upload id=576941worker1 # [ 142.300760] nix-grpc-daemon[1618]: time=2026-09-22T22:22:36.319Z level=INFO msg="Upload complete. (181ms)"6942lb # [ 142.313483] niks3-server[1129]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6943worker1 # [ 142.304822] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=6406944worker1 # [ 142.309857] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584506945worker1 # [ 142.319143] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6946worker1 # [ 142.327948] nix-daemon[846]: reaped child process 1713, status = succeeded6947worker1 # [ 142.332470] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6948client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6949client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6950client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6951client # farm-b> LOG-farm-b-during-drain6952worker1 # [ 143.749834] nix-daemon[846]: reaped child process 1722, status = succeeded6953worker1 # [ 143.831559] nix-daemon[846]: accepted connection from pid 1738, user nix-grpc-daemon (trusted)6954worker1 # [ 143.844687] nix-daemon[846]: reaped child process 1741, status = succeeded6955lb # [ 143.863603] niks3-server[1129]: 2026/09/22 22:22:37 INFO Received uploads request method=POST path=/api/pending_closures6956worker1 # [ 143.866115] nix-grpc-daemon[1618]: time=2026-09-22T22:22:37.884Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6957worker1 # [ 143.867628] nix-grpc-daemon[1618]: time=2026-09-22T22:22:37.886Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6958lb # [ 143.904339] niks3-server[1129]: 2026/09/22 22:22:37 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6959lb # [ 143.921977] niks3-server[1129]: 2026/09/22 22:22:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6960lb # [ 143.927246] niks3-server[1129]: 2026/09/22 22:22:37 INFO Signed narinfos id=58 count=16961worker1 # [ 143.918681] nix-grpc-daemon[1618]: time=2026-09-22T22:22:37.936Z level=INFO msg="Uploading 1 narinfos"6962lb # [ 143.934914] niks3-server[1129]: 2026/09/22 22:22:37 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6963lb # [ 143.945829] niks3-server[1129]: 2026/09/22 22:22:37 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6964lb # [ 143.947523] niks3-server[1129]: 2026/09/22 22:22:37 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6965lb # [ 143.950975] niks3-server[1129]: 2026/09/22 22:22:37 INFO Completed upload id=586966worker1 # [ 143.940106] nix-grpc-daemon[1618]: time=2026-09-22T22:22:37.958Z level=INFO msg="Upload complete. (199ms)"6967lb # [ 143.953408] niks3-server[1129]: 2026/09/22 22:22:37 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6968worker1 # [ 143.946280] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16969worker2 # [ 143.961576] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:37Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06970worker2 # [ 143.963337] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:37Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16971worker1 # [ 143.961631] nix-daemon[846]: reaped child process 1720, status = succeeded6972worker1 # [ 143.994844] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584506973worker1 # [ 144.013168] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584506974worker1 # [ 144.026586] nix-grpc-daemon[1618]: time=2026-09-22T22:22:38.044Z level=INFO msg="All 2 paths already cached"6975client # copying 1 paths to 'lb:50051'...6976worker1 # [ 144.035485] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584506977client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6978worker1 # [ 144.055165] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6979worker1 # [ 144.160191] nix-daemon[846]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6980worker1 # [ 144.176077] nix-daemon[846]: reaped child process 1748, status = succeeded6981lb # [ 144.193253] niks3-server[1129]: 2026/09/22 22:22:37 INFO Received uploads request method=POST path=/api/pending_closures6982worker1 # [ 144.190740] nix-grpc-daemon[1618]: time=2026-09-22T22:22:38.209Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6983worker1 # [ 144.192197] nix-grpc-daemon[1618]: time=2026-09-22T22:22:38.210Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6984lb # [ 144.222375] niks3-server[1129]: 2026/09/22 22:22:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6985worker1 # [ 144.217087] nix-grpc-daemon[1618]: time=2026-09-22T22:22:38.235Z level=INFO msg="Uploading 1 narinfos"6986lb # [ 144.228115] niks3-server[1129]: 2026/09/22 22:22:37 INFO Signed narinfos id=59 count=16987lb # [ 144.236298] niks3-server[1129]: 2026/09/22 22:22:37 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6988lb # [ 144.248653] niks3-server[1129]: 2026/09/22 22:22:37 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6989lb # [ 144.255122] niks3-server[1129]: 2026/09/22 22:22:37 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6990lb # [ 144.262487] niks3-server[1129]: 2026/09/22 22:22:37 INFO Completed upload id=596991worker1 # [ 144.251308] nix-grpc-daemon[1618]: time=2026-09-22T22:22:38.269Z level=INFO msg="Upload complete. (183ms)"6992lb # [ 144.268322] niks3-server[1129]: 2026/09/22 22:22:37 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6993worker1 # [ 144.255065] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=7686994worker1 # [ 144.259966] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584506995worker1 # [ 144.271957] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6996worker1 # [ 144.276307] nix-daemon[846]: reaped child process 1743, status = succeeded6997worker1 # [ 144.281084] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6998client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6999client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv7000lb # [ 144.687623] envoy[906]: conn peer=[2001:db8:1::1]:33428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2967 tx=827 ms=22197001client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7002client # farm-top> LOG-farm-top-during-drain7003worker1 # [ 145.497569] nix-daemon[846]: reaped child process 1752, status = succeeded7004worker1 # [ 145.577451] nix-daemon[846]: accepted connection from pid 1768, user nix-grpc-daemon (trusted)7005worker1 # [ 145.591498] nix-daemon[846]: reaped child process 1771, status = succeeded7006lb # [ 145.609585] niks3-server[1129]: 2026/09/22 22:22:38 INFO Received uploads request method=POST path=/api/pending_closures7007worker1 # [ 145.608396] nix-grpc-daemon[1618]: time=2026-09-22T22:22:39.626Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7008worker1 # [ 145.611606] nix-grpc-daemon[1618]: time=2026-09-22T22:22:39.628Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"7009lb # [ 145.650465] niks3-server[1129]: 2026/09/22 22:22:38 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv7010lb # [ 145.658826] niks3-server[1129]: 2026/09/22 22:22:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign7011lb # [ 145.668462] niks3-server[1129]: 2026/09/22 22:22:39 INFO Signed narinfos id=60 count=17012worker1 # [ 145.660706] nix-grpc-daemon[1618]: time=2026-09-22T22:22:39.678Z level=INFO msg="Uploading 1 narinfos"7013lb # [ 145.674673] niks3-server[1129]: 2026/09/22 22:22:39 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls7014lb # [ 145.677857] niks3-server[1129]: 2026/09/22 22:22:39 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst7015lb # [ 145.688767] niks3-server[1129]: 2026/09/22 22:22:39 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete7016lb # [ 145.692609] niks3-server[1129]: 2026/09/22 22:22:39 INFO Completed upload id=607017worker1 # [ 145.681713] nix-grpc-daemon[1618]: time=2026-09-22T22:22:39.700Z level=INFO msg="Upload complete. (193ms)"7018lb # [ 145.695463] niks3-server[1129]: 2026/09/22 22:22:39 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo7019worker1 # [ 145.687896] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=17020worker1 # [ 145.702531] nix-daemon[846]: reaped child process 1750, status = succeeded7021client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.42 seconds)7022client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath7023client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)7024worker2: must succeed: pkill -f 'read -t [9]0 x'7025worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)7026client: waiting for success: ! systemctl is-active sw7027worker2 # [ 145.904739] nix-daemon[870]: reaped child process 1767, status = succeeded7028worker2 # [ 145.982096] nix-daemon[870]: accepted connection from pid 1941, user nix-grpc-daemon (trusted)7029worker2 # [ 145.994646] nix-daemon[870]: reaped child process 1944, status = succeeded7030lb # [ 146.001176] niks3-server[1129]: 2026/09/22 22:22:39 INFO Received uploads request method=POST path=/api/pending_closures7031worker2 # [ 146.009561] nix-grpc-daemon[1709]: time=2026-09-22T22:22:39.257Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7032worker2 # [ 146.011555] nix-grpc-daemon[1709]: time=2026-09-22T22:22:39.259Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"7033lb # [ 146.044520] niks3-server[1129]: 2026/09/22 22:22:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign7034lb # [ 146.049739] niks3-server[1129]: 2026/09/22 22:22:39 INFO Signed narinfos id=61 count=17035worker2 # [ 146.052672] nix-grpc-daemon[1709]: time=2026-09-22T22:22:39.300Z level=INFO msg="Uploading 1 narinfos"7036lb # [ 146.058164] niks3-server[1129]: 2026/09/22 22:22:39 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv7037lb # [ 146.063482] niks3-server[1129]: 2026/09/22 22:22:39 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls7038lb # [ 146.067880] niks3-server[1129]: 2026/09/22 22:22:39 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst7039lb # [ 146.078073] niks3-server[1129]: 2026/09/22 22:22:39 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete7040lb # [ 146.082811] niks3-server[1129]: 2026/09/22 22:22:39 INFO Completed upload id=617041worker2 # [ 146.082756] nix-grpc-daemon[1709]: time=2026-09-22T22:22:39.330Z level=INFO msg="Upload complete. (186ms)"7042lb # [ 146.084648] niks3-server[1129]: 2026/09/22 22:22:39 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo7043worker2 # [ 146.089700] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53686 duration_s=9 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=17044worker2 # [ 146.104468] nix-daemon[870]: reaped child process 1765, status = succeeded7045client # [ 146.160070] systemd[1]: sw.service: Deactivated successfully.7046worker2 # [ 146.130167] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:39Z level=info event=signal_exit7047client # [ 146.164536] systemd[1]: sw.service: Consumed 124ms CPU time over 11.196s wall clock time, 41.4M memory peak, 7.6K incoming IP traffic, 8.5K outgoing IP traffic.7048lb # [ 146.139237] niks3-server[1129]: 2026/09/22 22:22:39 INFO lead: released remote=192.168.1.4:494347049client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)7050client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }7051client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)7052worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service7053worker2 # [ 147.037699] systemd[1]: Starting scheduler-token.service...7054worker2 # [ 147.075708] systemd[1]: scheduler-token.service: Deactivated successfully.7055worker2 # [ 147.077379] systemd[1]: Finished scheduler-token.service.7056worker2 # [ 147.078807] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 39ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.7057worker2 # [ 147.138780] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:40Z level=info event=scheduler_restarting7058worker1 # [ 147.130481] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_restarting addr=lb:500517059worker2 # [ 147.148738] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:536727060worker2 # [ 147.152856] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:40Z level=info event=worker_gone addr=192.168.1.3:500517061worker2 # [ 147.155914] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:40Z level=info event=scheduler_restarting addr=lb:500517062worker2 # [ 147.159189] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:536727063worker2 # [ 147.162867] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:40Z level=info event=worker_gone addr=192.168.1.4:500517064worker1 # [ 147.263603] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_take_over7065lb # [ 147.274964] niks3-server[1129]: 2026/09/22 22:22:40 INFO lead: acquired remote=192.168.1.3:519427066worker1 # [ 147.330433] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_connected addr=lb:500517067worker2 # [ 147.344325] nix-daemon[870]: reaped child process 1680, status = succeeded7068worker1 # [ 147.334899] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7069worker2 # [ 147.358764] nix-grpc-daemon[1626]: ts=2026-09-22T22:22:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"7070worker1 # [ 147.535709] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_connected addr=lb:500517071worker1 # [ 147.539791] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7072worker1 # [ 147.740726] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_connected addr=lb:500517073worker1 # [ 147.744759] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7074worker2 # [ 147.897727] nix-grpc-daemon[1626]: E0000 00:00:1790115761.145460 1671 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true7075worker2 # [ 147.902652] nix-grpc-daemon[1626]: E0000 00:00:1790115761.145608 1671 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true7076worker1 # [ 147.945736] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_connected addr=lb:500517077worker1 # [ 147.949830] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7078worker1 # [ 148.152400] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:42Z level=info event=scheduler_connected addr=lb:500517079worker2 # [ 148.164963] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.7080worker1 # [ 148.155749] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7081worker2 # [ 148.168444] systemd[1]: nix-grpc-daemon.service: Consumed 569ms CPU time over 22.249s wall clock time, 36.4M memory peak, 25.4K incoming IP traffic, 33.7K outgoing IP traffic.7082worker2 # [ 148.186188] systemd[1]: Starting Nix worker-protocol over gRPC...7083worker2 # [ 148.277259] nix-grpc-daemon[1961]: ts=2026-09-22T22:22:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"7084worker2 # [ 148.281679] nix-grpc-daemon[1961]: ts=2026-09-22T22:22:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc7085worker2 # [ 148.285309] nix-grpc-daemon[1961]: ts=2026-09-22T22:22:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=17086worker2 # [ 148.287336] nix-grpc-daemon[1961]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR7087worker2 # [ 148.288709] nix-grpc-daemon[1961]: E0000 00:00:1790115761.532723 1961 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7088worker2 # [ 148.297829] nix-grpc-daemon[1961]: E0000 00:00:1790115761.540093 1961 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7089worker2 # [ 148.300558] nix-grpc-daemon[1961]: ts=2026-09-22T22:22:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket7090worker2 # [ 148.305173] systemd[1]: Started Nix worker-protocol over gRPC.7091worker2 # [ 148.309125] nix-grpc-daemon[1961]: E0000 00:00:1790115761.550654 2001 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7092worker2 # [ 148.319078] nix-grpc-daemon[1961]: E0000 00:00:1790115761.565031 2001 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7093worker2 # [ 148.322130] nix-grpc-daemon[1961]: E0000 00:00:1790115761.565202 2001 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7094worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.35 seconds)7095lb: waiting until exactly no endpoints of x86_64-linux are unhealthy7096lb: must succeed: curl -sf localhost:9901/clusters7097worker2 # [ 148.335133] nix-grpc-daemon[1961]: E0000 00:00:1790115761.582392 2001 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7098worker2 # [ 148.349253] nix-grpc-daemon[1961]: ts=2026-09-22T22:22:41Z level=info event=scheduler_connected addr=lb:500517099worker1 # [ 148.353643] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:42Z level=info event=scheduler_connected addr=lb:500517100worker1 # [ 148.358810] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:462847101worker1 # [ 148.365532] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07102worker1 # [ 148.371400] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:42Z level=info event=oidc_jwks provider=mock keys=17103lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7104worker1 # [ 148.374363] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:462847105worker1 # [ 148.378358] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07106lb: must succeed: curl -sf localhost:9901/clusters7107lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7108lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)7109lb: must succeed: curl -sf localhost:9901/clusters7110lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7111worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7112worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7113(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.62 seconds)7114subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7115client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17116worker2 # [ 149.663505] nix-grpc-daemon[1961]: ts=2026-09-22T22:22:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:567267117worker2 # [ 149.675634] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7118worker2 # [ 149.690104] nix-grpc-daemon[1961]: ts=2026-09-22T22:22:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567267119worker2 # [ 149.700832] nix-grpc-daemon[1961]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)7120worker2 # [ 149.722486] nix-grpc-daemon[1961]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 2/5)7121worker2 # [ 149.894106] nix-grpc-daemon[1961]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 3/5)7122worker2 # [ 150.063402] nix-grpc-daemon[1961]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 443 ms (attempt 4/5)7123worker2 # [ 150.509272] nix-grpc-daemon[1961]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7124worker2 # [ 150.512786] nix-grpc-daemon[1961]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7125worker1 # [ 150.548934] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:44Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07126worker1 # [ 150.556405] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:44Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17127worker1 # [ 150.561802] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:44Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07128worker1 # [ 150.565986] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:44Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07129worker1 # [ 150.599516] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507130worker1 # [ 150.613192] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507131worker1 # [ 150.625904] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584507132worker1 # [ 150.639101] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7133worker1 # [ 150.747949] nix-daemon[846]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)7134worker1 # [ 150.763343] nix-daemon[846]: reaped child process 1783, status = succeeded7135lb # [ 150.782255] niks3-server[1129]: 2026/09/22 22:22:44 INFO Received uploads request method=POST path=/api/pending_closures7136worker1 # [ 150.905823] nix-grpc-daemon[1618]: time=2026-09-22T22:22:44.923Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7137worker1 # [ 150.910308] nix-grpc-daemon[1618]: time=2026-09-22T22:22:44.923Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7138lb # [ 150.946732] niks3-server[1129]: 2026/09/22 22:22:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7139lb # [ 150.953917] niks3-server[1129]: 2026/09/22 22:22:44 INFO Signed narinfos id=62 count=17140worker1 # [ 150.949632] nix-grpc-daemon[1618]: time=2026-09-22T22:22:44.967Z level=INFO msg="Uploading 1 narinfos"7141lb # [ 150.965154] niks3-server[1129]: 2026/09/22 22:22:44 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7142lb # [ 150.975101] niks3-server[1129]: 2026/09/22 22:22:44 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7143lb # [ 150.984315] niks3-server[1129]: 2026/09/22 22:22:44 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7144lb # [ 150.991237] niks3-server[1129]: 2026/09/22 22:22:44 INFO Completed upload id=627145worker1 # [ 150.980301] nix-grpc-daemon[1618]: time=2026-09-22T22:22:44.998Z level=INFO msg="Upload complete. (329ms)"7146lb # [ 150.994471] niks3-server[1129]: 2026/09/22 22:22:44 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7147worker1 # [ 150.985972] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=5047148worker1 # [ 150.991926] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507149worker1 # [ 151.006140] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7150worker1 # [ 151.017562] nix-daemon[846]: reaped child process 1778, status = succeeded7151worker1 # [ 151.027540] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7152worker1 # [ 152.263767] nix-daemon[846]: reaped child process 1787, status = succeeded7153worker1 # [ 152.349982] nix-daemon[846]: accepted connection from pid 1803, user nix-grpc-daemon (trusted)7154worker1 # [ 152.363133] nix-daemon[846]: reaped child process 1806, status = succeeded7155lb # [ 152.382382] niks3-server[1129]: 2026/09/22 22:22:45 INFO Received uploads request method=POST path=/api/pending_closures7156worker1 # [ 152.380376] nix-grpc-daemon[1618]: time=2026-09-22T22:22:46.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7157worker1 # [ 152.384896] nix-grpc-daemon[1618]: time=2026-09-22T22:22:46.400Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7158lb # [ 152.427167] niks3-server[1129]: 2026/09/22 22:22:45 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7159lb # [ 152.430549] niks3-server[1129]: 2026/09/22 22:22:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7160lb # [ 152.432830] niks3-server[1129]: 2026/09/22 22:22:45 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7161lb # [ 152.435221] niks3-server[1129]: 2026/09/22 22:22:45 INFO Signed narinfos id=63 count=17162worker1 # [ 152.424508] nix-grpc-daemon[1618]: time=2026-09-22T22:22:46.442Z level=INFO msg="Uploading 1 narinfos"7163lb # [ 152.438980] niks3-server[1129]: 2026/09/22 22:22:45 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7164lb # [ 152.444721] niks3-server[1129]: 2026/09/22 22:22:45 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7165lb # [ 152.448276] niks3-server[1129]: 2026/09/22 22:22:45 INFO Completed upload id=637166worker1 # [ 152.437264] nix-grpc-daemon[1618]: time=2026-09-22T22:22:46.455Z level=INFO msg="Upload complete. (196ms)"7167lb # [ 152.450132] niks3-server[1129]: 2026/09/22 22:22:45 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7168worker1 # [ 152.444329] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=17169worker1 # [ 152.449167] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:46Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17170worker1 # [ 152.453098] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:46Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07171worker1 # [ 152.458231] nix-daemon[846]: reaped child process 1785, status = succeeded7172worker1 # [ 152.487974] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507173worker1 # [ 152.498546] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507174worker1 # [ 152.508708] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584507175worker1 # [ 152.516297] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7176worker1 # [ 152.613448] nix-daemon[846]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)7177worker1 # [ 152.628749] nix-daemon[846]: reaped child process 1815, status = succeeded7178lb # [ 152.646923] niks3-server[1129]: 2026/09/22 22:22:45 INFO Received uploads request method=POST path=/api/pending_closures7179worker1 # [ 152.643464] nix-grpc-daemon[1618]: time=2026-09-22T22:22:46.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7180worker1 # [ 152.645075] nix-grpc-daemon[1618]: time=2026-09-22T22:22:46.663Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7181lb # [ 152.677487] niks3-server[1129]: 2026/09/22 22:22:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7182worker1 # [ 152.676341] nix-grpc-daemon[1618]: time=2026-09-22T22:22:46.694Z level=INFO msg="Uploading 1 narinfos"7183lb # [ 152.687492] niks3-server[1129]: 2026/09/22 22:22:46 INFO Signed narinfos id=64 count=17184lb # [ 152.692345] niks3-server[1129]: 2026/09/22 22:22:46 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7185lb # [ 152.698098] niks3-server[1129]: 2026/09/22 22:22:46 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7186lb # [ 152.710288] niks3-server[1129]: 2026/09/22 22:22:46 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7187lb # [ 152.717485] niks3-server[1129]: 2026/09/22 22:22:46 INFO Completed upload id=647188worker1 # [ 152.708610] nix-grpc-daemon[1618]: time=2026-09-22T22:22:46.727Z level=INFO msg="Upload complete. (183ms)"7189lb # [ 152.721101] niks3-server[1129]: 2026/09/22 22:22:46 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7190worker1 # [ 152.712352] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=5047191worker1 # [ 152.717387] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507192worker1 # [ 152.727142] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7193worker1 # [ 152.735372] nix-daemon[846]: reaped child process 1810, status = succeeded7194worker1 # [ 152.740657] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7195worker1 # [ 153.785647] nix-daemon[846]: reaped child process 1819, status = succeeded7196worker1 # [ 153.870827] nix-daemon[846]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)7197worker1 # [ 153.883915] nix-daemon[846]: reaped child process 1838, status = succeeded7198lb # [ 153.903206] niks3-server[1129]: 2026/09/22 22:22:47 INFO Received uploads request method=POST path=/api/pending_closures7199worker1 # [ 153.900654] nix-grpc-daemon[1618]: time=2026-09-22T22:22:47.918Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7200worker1 # [ 153.902532] nix-grpc-daemon[1618]: time=2026-09-22T22:22:47.920Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7201lb # [ 153.943122] niks3-server[1129]: 2026/09/22 22:22:47 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7202lb # [ 153.954612] niks3-server[1129]: 2026/09/22 22:22:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7203lb # [ 153.965267] niks3-server[1129]: 2026/09/22 22:22:47 INFO Signed narinfos id=65 count=17204worker1 # [ 153.957378] nix-grpc-daemon[1618]: time=2026-09-22T22:22:47.975Z level=INFO msg="Uploading 1 narinfos"7205lb # [ 153.976740] niks3-server[1129]: 2026/09/22 22:22:47 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7206lb # [ 153.986717] niks3-server[1129]: 2026/09/22 22:22:47 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7207lb # [ 153.991051] niks3-server[1129]: 2026/09/22 22:22:47 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7208lb # [ 153.994063] niks3-server[1129]: 2026/09/22 22:22:47 INFO Completed upload id=657209worker1 # [ 153.983170] nix-grpc-daemon[1618]: time=2026-09-22T22:22:48.001Z level=INFO msg="Upload complete. (234ms)"7210lb # [ 153.999296] niks3-server[1129]: 2026/09/22 22:22:47 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7211worker1 # [ 153.993164] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=17212worker1 # [ 154.000403] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:48Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17213worker1 # [ 154.013949] nix-daemon[846]: reaped child process 1817, status = succeeded7214worker1 # [ 154.035709] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507215worker1 # [ 154.043889] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:48Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07216worker1 # [ 154.053112] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507217worker1 # [ 154.065380] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584507218worker1 # [ 154.073838] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7219worker1 # [ 154.175126] nix-daemon[846]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)7220worker1 # [ 154.189250] nix-daemon[846]: reaped child process 1847, status = succeeded7221lb # [ 154.209543] niks3-server[1129]: 2026/09/22 22:22:47 INFO Received uploads request method=POST path=/api/pending_closures7222worker1 # [ 154.206052] nix-grpc-daemon[1618]: time=2026-09-22T22:22:48.224Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7223worker1 # [ 154.207592] nix-grpc-daemon[1618]: time=2026-09-22T22:22:48.226Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7224lb # [ 154.241656] niks3-server[1129]: 2026/09/22 22:22:47 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7225lb # [ 154.248705] niks3-server[1129]: 2026/09/22 22:22:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7226lb # [ 154.255394] niks3-server[1129]: 2026/09/22 22:22:47 INFO Signed narinfos id=66 count=17227worker1 # [ 154.247074] nix-grpc-daemon[1618]: time=2026-09-22T22:22:48.265Z level=INFO msg="Uploading 1 narinfos"7228lb # [ 154.263366] niks3-server[1129]: 2026/09/22 22:22:47 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7229lb # [ 154.272977] niks3-server[1129]: 2026/09/22 22:22:47 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7230lb # [ 154.280188] niks3-server[1129]: 2026/09/22 22:22:47 INFO Completed upload id=667231worker1 # [ 154.271500] nix-grpc-daemon[1618]: time=2026-09-22T22:22:48.289Z level=INFO msg="Upload complete. (186ms)"7232lb # [ 154.285221] niks3-server[1129]: 2026/09/22 22:22:47 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7233worker1 # [ 154.276373] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=5047234worker1 # [ 154.282718] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507235worker1 # [ 154.292339] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7236worker1 # [ 154.302189] nix-daemon[846]: reaped child process 1842, status = succeeded7237worker1 # [ 154.305861] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7238lb # [ 154.689181] envoy[906]: conn peer=[2001:db8:1::1]:55472 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9542 tx=4380 ms=52537239lb # [ 154.692838] envoy[906]: conn peer=[2001:db8:1::1]:55462 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1561 ms=110587240lb # [ 154.696477] envoy[906]: conn peer=[2001:db8:1::4]:34252 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1601 tx=361 ms=211767241worker1 # [ 155.335567] nix-daemon[846]: reaped child process 1851, status = succeeded7242worker1 # [ 155.412650] nix-daemon[846]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)7243worker1 # [ 155.428082] nix-daemon[846]: reaped child process 1870, status = succeeded7244lb # [ 155.445717] niks3-server[1129]: 2026/09/22 22:22:48 INFO Received uploads request method=POST path=/api/pending_closures7245worker1 # [ 155.442883] nix-grpc-daemon[1618]: time=2026-09-22T22:22:49.461Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7246worker1 # [ 155.444856] nix-grpc-daemon[1618]: time=2026-09-22T22:22:49.462Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7247lb # [ 155.478081] niks3-server[1129]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7248lb # [ 155.492366] niks3-server[1129]: 2026/09/22 22:22:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7249lb # [ 155.500377] niks3-server[1129]: 2026/09/22 22:22:48 INFO Signed narinfos id=67 count=17250worker1 # [ 155.493605] nix-grpc-daemon[1618]: time=2026-09-22T22:22:49.511Z level=INFO msg="Uploading 1 narinfos"7251lb # [ 155.508603] niks3-server[1129]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7252lb # [ 155.512199] niks3-server[1129]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7253lb # [ 155.524828] niks3-server[1129]: 2026/09/22 22:22:48 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7254lb # [ 155.529782] niks3-server[1129]: 2026/09/22 22:22:48 INFO Completed upload id=677255worker1 # [ 155.518988] nix-grpc-daemon[1618]: time=2026-09-22T22:22:49.537Z level=INFO msg="Upload complete. (204ms)"7256lb # [ 155.534131] niks3-server[1129]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7257worker1 # [ 155.531777] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=17258worker1 # [ 155.537798] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:49Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17259worker1 # [ 155.542244] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507260worker1 # [ 155.545418] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:49Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07261worker1 # [ 155.553879] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507262worker1 # [ 155.558825] nix-daemon[846]: reaped child process 1849, status = succeeded7263worker1 # [ 155.563936] nix-grpc-daemon[1618]: time=2026-09-22T22:22:49.582Z level=INFO msg="All 1 paths already cached"7264worker1 # [ 155.568418] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584507265worker1 # [ 155.573288] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7266worker1 # [ 155.664933] nix-daemon[846]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)7267worker1 # [ 155.679098] nix-daemon[846]: reaped child process 1879, status = succeeded7268lb # [ 155.698062] niks3-server[1129]: 2026/09/22 22:22:49 INFO Received uploads request method=POST path=/api/pending_closures7269worker1 # [ 155.696029] nix-grpc-daemon[1618]: time=2026-09-22T22:22:49.714Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7270worker1 # [ 155.698091] nix-grpc-daemon[1618]: time=2026-09-22T22:22:49.716Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7271lb # [ 155.730963] niks3-server[1129]: 2026/09/22 22:22:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7272lb # [ 155.735937] niks3-server[1129]: 2026/09/22 22:22:49 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7273worker1 # [ 155.732605] nix-grpc-daemon[1618]: time=2026-09-22T22:22:49.750Z level=INFO msg="Uploading 1 narinfos"7274lb # [ 155.743803] niks3-server[1129]: 2026/09/22 22:22:49 INFO Signed narinfos id=68 count=17275lb # [ 155.750775] niks3-server[1129]: 2026/09/22 22:22:49 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7276lb # [ 155.763945] niks3-server[1129]: 2026/09/22 22:22:49 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7277worker1 # [ 155.759957] nix-grpc-daemon[1618]: time=2026-09-22T22:22:49.778Z level=INFO msg="Upload complete. (180ms)"7278lb # [ 155.771082] niks3-server[1129]: 2026/09/22 22:22:49 INFO Completed upload id=687279lb # [ 155.775087] niks3-server[1129]: 2026/09/22 22:22:49 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7280worker1 # [ 155.762827] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=6247281worker1 # [ 155.768268] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507282worker1 # [ 155.778679] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7283worker1 # [ 155.784282] nix-daemon[846]: reaped child process 1874, status = succeeded7284worker1 # [ 155.789223] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7285worker1 # [ 156.607063] nix-daemon[846]: reaped child process 1883, status = succeeded7286worker1 # [ 156.691103] nix-daemon[846]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)7287worker1 # [ 156.706342] nix-daemon[846]: reaped child process 1902, status = succeeded7288lb # [ 156.724588] niks3-server[1129]: 2026/09/22 22:22:50 INFO Received uploads request method=POST path=/api/pending_closures7289worker1 # [ 156.722051] nix-grpc-daemon[1618]: time=2026-09-22T22:22:50.740Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7290worker1 # [ 156.723603] nix-grpc-daemon[1618]: time=2026-09-22T22:22:50.742Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7291lb # [ 156.772322] niks3-server[1129]: 2026/09/22 22:22:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7292lb # [ 156.777079] niks3-server[1129]: 2026/09/22 22:22:50 INFO Signed narinfos id=69 count=17293worker1 # [ 156.769101] nix-grpc-daemon[1618]: time=2026-09-22T22:22:50.787Z level=INFO msg="Uploading 1 narinfos"7294lb # [ 156.788539] niks3-server[1129]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7295lb # [ 156.794406] niks3-server[1129]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7296lb # [ 156.804644] niks3-server[1129]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7297lb # [ 156.807660] niks3-server[1129]: 2026/09/22 22:22:50 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7298lb # [ 156.814935] niks3-server[1129]: 2026/09/22 22:22:50 INFO Completed upload id=697299worker1 # [ 156.804234] nix-grpc-daemon[1618]: time=2026-09-22T22:22:50.822Z level=INFO msg="Upload complete. (207ms)"7300lb # [ 156.816926] niks3-server[1129]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7301worker1 # [ 156.811654] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=17302worker1 # [ 156.822572] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:50Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17303worker1 # [ 156.827391] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:50Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07304worker1 # [ 156.838133] nix-daemon[846]: reaped child process 1881, status = succeeded7305worker1 # [ 156.862907] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507306worker1 # [ 156.875518] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507307worker1 # [ 156.889340] nix-grpc-daemon[1618]: time=2026-09-22T22:22:50.907Z level=INFO msg="All 1 paths already cached"7308worker1 # [ 156.897212] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584507309worker1 # [ 156.907165] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7310worker1 # [ 157.005307] nix-daemon[846]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)7311worker1 # [ 157.019592] nix-daemon[846]: reaped child process 1911, status = succeeded7312lb # [ 157.037677] niks3-server[1129]: 2026/09/22 22:22:50 INFO Received uploads request method=POST path=/api/pending_closures7313worker1 # [ 157.034252] nix-grpc-daemon[1618]: time=2026-09-22T22:22:51.052Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7314worker1 # [ 157.035730] nix-grpc-daemon[1618]: time=2026-09-22T22:22:51.054Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7315lb # [ 157.067830] niks3-server[1129]: 2026/09/22 22:22:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7316lb # [ 157.076843] niks3-server[1129]: 2026/09/22 22:22:50 INFO Signed narinfos id=70 count=17317worker1 # [ 157.069608] nix-grpc-daemon[1618]: time=2026-09-22T22:22:51.087Z level=INFO msg="Uploading 1 narinfos"7318lb # [ 157.083526] niks3-server[1129]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7319lb # [ 157.089415] niks3-server[1129]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7320lb # [ 157.102059] niks3-server[1129]: 2026/09/22 22:22:50 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7321lb # [ 157.107492] niks3-server[1129]: 2026/09/22 22:22:50 INFO Completed upload id=707322worker1 # [ 157.096730] nix-grpc-daemon[1618]: time=2026-09-22T22:22:51.115Z level=INFO msg="Upload complete. (178ms)"7323lb # [ 157.113952] niks3-server[1129]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7324worker1 # [ 157.100924] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=6247325worker1 # [ 157.105954] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507326worker1 # [ 157.117106] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7327worker1 # [ 157.121866] nix-daemon[846]: reaped child process 1906, status = succeeded7328worker1 # [ 157.126978] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7329worker1 # [ 158.562295] nix-daemon[846]: reaped child process 1915, status = succeeded7330worker1 # [ 158.650286] nix-daemon[846]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)7331worker1 # [ 158.664120] nix-daemon[846]: reaped child process 1934, status = succeeded7332lb # [ 158.683325] niks3-server[1129]: 2026/09/22 22:22:52 INFO Received uploads request method=POST path=/api/pending_closures7333worker1 # [ 158.684951] nix-grpc-daemon[1618]: time=2026-09-22T22:22:52.703Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7334worker1 # [ 158.686452] nix-grpc-daemon[1618]: time=2026-09-22T22:22:52.704Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7335lb # [ 158.726481] niks3-server[1129]: 2026/09/22 22:22:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7336worker1 # [ 158.723260] nix-grpc-daemon[1618]: time=2026-09-22T22:22:52.741Z level=INFO msg="Uploading 1 narinfos"7337lb # [ 158.734146] niks3-server[1129]: 2026/09/22 22:22:52 INFO Signed narinfos id=71 count=17338lb # [ 158.740623] niks3-server[1129]: 2026/09/22 22:22:52 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7339lb # [ 158.743378] niks3-server[1129]: 2026/09/22 22:22:52 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7340lb # [ 158.745851] niks3-server[1129]: 2026/09/22 22:22:52 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7341lb # [ 158.751797] niks3-server[1129]: 2026/09/22 22:22:52 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7342lb # [ 158.754804] niks3-server[1129]: 2026/09/22 22:22:52 INFO Completed upload id=717343worker1 # [ 158.743926] nix-grpc-daemon[1618]: time=2026-09-22T22:22:52.762Z level=INFO msg="Upload complete. (206ms)"7344lb # [ 158.758188] niks3-server[1129]: 2026/09/22 22:22:52 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7345worker1 # [ 158.750441] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=17346worker1 # [ 158.761166] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:52Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17347worker1 # [ 158.764686] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:52Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07348worker1 # [ 158.773104] nix-daemon[846]: reaped child process 1913, status = succeeded7349worker1 # [ 158.800246] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507350worker1 # [ 158.810385] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507351worker1 # [ 158.820601] nix-grpc-daemon[1618]: time=2026-09-22T22:22:52.838Z level=INFO msg="All 1 paths already cached"7352worker1 # [ 158.827126] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584507353worker1 # [ 158.836133] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7354worker1 # [ 158.927571] nix-daemon[846]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)7355worker1 # [ 158.941811] nix-daemon[846]: reaped child process 1943, status = succeeded7356lb # [ 158.959335] niks3-server[1129]: 2026/09/22 22:22:52 INFO Received uploads request method=POST path=/api/pending_closures7357worker1 # [ 158.956689] nix-grpc-daemon[1618]: time=2026-09-22T22:22:52.975Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7358worker1 # [ 158.958320] nix-grpc-daemon[1618]: time=2026-09-22T22:22:52.976Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7359lb # [ 158.994565] niks3-server[1129]: 2026/09/22 22:22:52 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7360lb # [ 158.998929] niks3-server[1129]: 2026/09/22 22:22:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7361worker1 # [ 158.994119] nix-grpc-daemon[1618]: time=2026-09-22T22:22:53.012Z level=INFO msg="Uploading 1 narinfos"7362lb # [ 159.005826] niks3-server[1129]: 2026/09/22 22:22:52 INFO Signed narinfos id=72 count=17363lb # [ 159.011594] niks3-server[1129]: 2026/09/22 22:22:52 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7364lb # [ 159.024744] niks3-server[1129]: 2026/09/22 22:22:52 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7365lb # [ 159.031636] niks3-server[1129]: 2026/09/22 22:22:52 INFO Completed upload id=727366worker1 # [ 159.020576] nix-grpc-daemon[1618]: time=2026-09-22T22:22:53.038Z level=INFO msg="Upload complete. (175ms)"7367lb # [ 159.032942] niks3-server[1129]: 2026/09/22 22:22:52 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7368worker1 # [ 159.025619] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=6247369worker1 # [ 159.031694] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507370worker1 # [ 159.041769] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7371worker1 # [ 159.050323] nix-daemon[846]: reaped child process 1938, status = succeeded7372worker1 # [ 159.055805] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7373worker1 # [ 159.885674] nix-daemon[846]: reaped child process 1947, status = succeeded7374worker1 # [ 159.967193] nix-daemon[846]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)7375worker1 # [ 159.981391] nix-daemon[846]: reaped child process 1966, status = succeeded7376lb # [ 160.001910] niks3-server[1129]: 2026/09/22 22:22:53 INFO Received uploads request method=POST path=/api/pending_closures7377worker1 # [ 159.998088] nix-grpc-daemon[1618]: time=2026-09-22T22:22:54.016Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7378worker1 # [ 159.999597] nix-grpc-daemon[1618]: time=2026-09-22T22:22:54.018Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7379lb # [ 160.043893] niks3-server[1129]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7380lb # [ 160.052204] niks3-server[1129]: 2026/09/22 22:22:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7381worker1 # [ 160.048068] nix-grpc-daemon[1618]: time=2026-09-22T22:22:54.066Z level=INFO msg="Uploading 1 narinfos"7382lb # [ 160.059128] niks3-server[1129]: 2026/09/22 22:22:53 INFO Signed narinfos id=73 count=17383lb # [ 160.063750] niks3-server[1129]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7384lb # [ 160.073871] niks3-server[1129]: 2026/09/22 22:22:53 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7385lb # [ 160.079114] niks3-server[1129]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7386lb # [ 160.085697] niks3-server[1129]: 2026/09/22 22:22:53 INFO Completed upload id=737387lb # [ 160.088353] niks3-server[1129]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7388worker1 # [ 160.076393] nix-grpc-daemon[1618]: time=2026-09-22T22:22:54.094Z level=INFO msg="Upload complete. (212ms)"7389worker1 # [ 160.083687] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=17390worker1 # [ 160.094377] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:54Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17391worker1 # [ 160.100278] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:54Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07392worker1 # [ 160.110482] nix-daemon[846]: reaped child process 1945, status = succeeded7393worker1 # [ 160.134919] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507394worker1 # [ 160.147944] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507395worker1 # [ 160.160873] nix-grpc-daemon[1618]: time=2026-09-22T22:22:54.179Z level=INFO msg="All 2 paths already cached"7396worker1 # [ 160.169185] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584507397worker1 # [ 160.179132] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7398worker1 # [ 160.278283] nix-daemon[846]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)7399worker1 # [ 160.293413] nix-daemon[846]: reaped child process 1975, status = succeeded7400lb # [ 160.311574] niks3-server[1129]: 2026/09/22 22:22:53 INFO Received uploads request method=POST path=/api/pending_closures7401worker1 # [ 160.308462] nix-grpc-daemon[1618]: time=2026-09-22T22:22:54.326Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7402worker1 # [ 160.310025] nix-grpc-daemon[1618]: time=2026-09-22T22:22:54.328Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7403lb # [ 160.344091] niks3-server[1129]: 2026/09/22 22:22:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7404lb # [ 160.350144] niks3-server[1129]: 2026/09/22 22:22:53 INFO Signed narinfos id=74 count=17405worker1 # [ 160.342966] nix-grpc-daemon[1618]: time=2026-09-22T22:22:54.361Z level=INFO msg="Uploading 1 narinfos"7406lb # [ 160.356630] niks3-server[1129]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7407lb # [ 160.367928] niks3-server[1129]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7408lb # [ 160.376351] niks3-server[1129]: 2026/09/22 22:22:53 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7409lb # [ 160.382124] niks3-server[1129]: 2026/09/22 22:22:53 INFO Completed upload id=747410worker1 # [ 160.371084] nix-grpc-daemon[1618]: time=2026-09-22T22:22:54.389Z level=INFO msg="Upload complete. (181ms)"7411lb # [ 160.385329] niks3-server[1129]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7412worker1 # [ 160.375535] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=7527413worker1 # [ 160.380265] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507414worker1 # [ 160.392845] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7415worker1 # [ 160.398620] nix-daemon[846]: reaped child process 1970, status = succeeded7416worker1 # [ 160.401873] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7417worker1 # [ 161.219710] nix-daemon[846]: reaped child process 1979, status = succeeded7418worker1 # [ 161.298271] nix-daemon[846]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)7419worker1 # [ 161.314044] nix-daemon[846]: reaped child process 1998, status = succeeded7420lb # [ 161.330847] niks3-server[1129]: 2026/09/22 22:22:54 INFO Received uploads request method=POST path=/api/pending_closures7421worker1 # [ 161.328722] nix-grpc-daemon[1618]: time=2026-09-22T22:22:55.346Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7422worker1 # [ 161.330272] nix-grpc-daemon[1618]: time=2026-09-22T22:22:55.348Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7423lb # [ 161.370512] niks3-server[1129]: 2026/09/22 22:22:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7424worker1 # [ 161.369887] nix-grpc-daemon[1618]: time=2026-09-22T22:22:55.388Z level=INFO msg="Uploading 1 narinfos"7425lb # [ 161.382418] niks3-server[1129]: 2026/09/22 22:22:54 INFO Signed narinfos id=75 count=17426lb # [ 161.388261] niks3-server[1129]: 2026/09/22 22:22:54 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7427lb # [ 161.393533] niks3-server[1129]: 2026/09/22 22:22:54 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7428lb # [ 161.399391] niks3-server[1129]: 2026/09/22 22:22:54 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7429lb # [ 161.410141] niks3-server[1129]: 2026/09/22 22:22:54 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7430lb # [ 161.413136] niks3-server[1129]: 2026/09/22 22:22:54 INFO Completed upload id=757431worker1 # [ 161.403414] nix-grpc-daemon[1618]: time=2026-09-22T22:22:55.421Z level=INFO msg="Upload complete. (194ms)"7432lb # [ 161.416725] niks3-server[1129]: 2026/09/22 22:22:54 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7433worker1 # [ 161.414129] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=17434worker1 # [ 161.419917] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:55Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17435worker1 # [ 161.431933] nix-daemon[846]: reaped child process 1977, status = succeeded7436worker1 # [ 161.455950] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507437worker1 # [ 161.468731] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507438worker1 # [ 161.482454] nix-grpc-daemon[1618]: time=2026-09-22T22:22:55.500Z level=INFO msg="All 2 paths already cached"7439worker1 # [ 161.490921] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584507440worker1 # [ 161.501167] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7441worker1 # [ 161.598581] nix-daemon[846]: accepted connection from pid 2002, user nix-grpc-daemon (trusted)7442worker1 # [ 161.613511] nix-daemon[846]: reaped child process 2005, status = succeeded7443lb # [ 161.630618] niks3-server[1129]: 2026/09/22 22:22:54 INFO Received uploads request method=POST path=/api/pending_closures7444worker1 # [ 161.627655] nix-grpc-daemon[1618]: time=2026-09-22T22:22:55.646Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7445worker1 # [ 161.629209] nix-grpc-daemon[1618]: time=2026-09-22T22:22:55.647Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7446lb # [ 161.666520] niks3-server[1129]: 2026/09/22 22:22:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7447worker1 # [ 161.661068] nix-grpc-daemon[1618]: time=2026-09-22T22:22:55.679Z level=INFO msg="Uploading 1 narinfos"7448lb # [ 161.672977] niks3-server[1129]: 2026/09/22 22:22:55 INFO Signed narinfos id=76 count=17449lb # [ 161.677983] niks3-server[1129]: 2026/09/22 22:22:55 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7450lb # [ 161.683931] niks3-server[1129]: 2026/09/22 22:22:55 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7451lb # [ 161.695968] niks3-server[1129]: 2026/09/22 22:22:55 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7452lb # [ 161.701359] niks3-server[1129]: 2026/09/22 22:22:55 INFO Completed upload id=767453worker1 # [ 161.690672] nix-grpc-daemon[1618]: time=2026-09-22T22:22:55.709Z level=INFO msg="Upload complete. (178ms)"7454lb # [ 161.708102] niks3-server[1129]: 2026/09/22 22:22:55 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7455worker1 # [ 161.694811] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=7527456worker1 # [ 161.699995] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507457worker1 # [ 161.711115] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7458worker1 # [ 161.716529] nix-daemon[846]: reaped child process 2000, status = succeeded7459worker1 # [ 161.721770] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7460worker1 # [ 162.564428] nix-daemon[846]: reaped child process 2009, status = succeeded7461worker1 # [ 162.646098] nix-daemon[846]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)7462worker1 # [ 162.659241] nix-daemon[846]: reaped child process 2028, status = succeeded7463lb # [ 162.677707] niks3-server[1129]: 2026/09/22 22:22:56 INFO Received uploads request method=POST path=/api/pending_closures7464worker1 # [ 162.675889] nix-grpc-daemon[1618]: time=2026-09-22T22:22:56.694Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7465worker1 # [ 162.677430] nix-grpc-daemon[1618]: time=2026-09-22T22:22:56.695Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7466lb # [ 162.720394] niks3-server[1129]: 2026/09/22 22:22:56 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7467lb # [ 162.728730] niks3-server[1129]: 2026/09/22 22:22:56 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7468lb # [ 162.740072] niks3-server[1129]: 2026/09/22 22:22:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7469lb # [ 162.745107] niks3-server[1129]: 2026/09/22 22:22:56 INFO Signed narinfos id=77 count=17470worker1 # [ 162.734661] nix-grpc-daemon[1618]: time=2026-09-22T22:22:56.752Z level=INFO msg="Uploading 1 narinfos"7471lb # [ 162.755140] niks3-server[1129]: 2026/09/22 22:22:56 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7472lb # [ 162.760972] niks3-server[1129]: 2026/09/22 22:22:56 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7473lb # [ 162.764309] niks3-server[1129]: 2026/09/22 22:22:56 INFO Completed upload id=777474lb # [ 162.766083] niks3-server[1129]: 2026/09/22 22:22:56 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7475worker1 # [ 162.753337] nix-grpc-daemon[1618]: time=2026-09-22T22:22:56.771Z level=INFO msg="Upload complete. (222ms)"7476worker1 # [ 162.762321] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=17477worker1 # [ 162.769293] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:56Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17478worker1 # [ 162.781265] nix-daemon[846]: reaped child process 2007, status = succeeded7479worker1 # [ 162.805168] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507480worker1 # [ 162.818322] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507481worker1 # [ 162.831926] nix-grpc-daemon[1618]: time=2026-09-22T22:22:56.850Z level=INFO msg="All 2 paths already cached"7482worker1 # [ 162.840365] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:584507483worker1 # [ 162.848749] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7484worker1 # [ 162.949847] nix-daemon[846]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)7485worker1 # [ 162.964907] nix-daemon[846]: reaped child process 2035, status = succeeded7486lb # [ 162.981924] niks3-server[1129]: 2026/09/22 22:22:56 INFO Received uploads request method=POST path=/api/pending_closures7487worker1 # [ 162.978814] nix-grpc-daemon[1618]: time=2026-09-22T22:22:56.997Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7488worker1 # [ 162.980344] nix-grpc-daemon[1618]: time=2026-09-22T22:22:56.998Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7489lb # [ 163.016216] niks3-server[1129]: 2026/09/22 22:22:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7490lb # [ 163.020921] niks3-server[1129]: 2026/09/22 22:22:56 INFO Signed narinfos id=78 count=17491worker1 # [ 163.012987] nix-grpc-daemon[1618]: time=2026-09-22T22:22:57.031Z level=INFO msg="Uploading 1 narinfos"7492lb # [ 163.031408] niks3-server[1129]: 2026/09/22 22:22:56 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7493lb # [ 163.035209] niks3-server[1129]: 2026/09/22 22:22:56 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7494lb # [ 163.050411] niks3-server[1129]: 2026/09/22 22:22:56 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7495lb # [ 163.057134] niks3-server[1129]: 2026/09/22 22:22:56 INFO Completed upload id=787496worker1 # [ 163.046296] nix-grpc-daemon[1618]: time=2026-09-22T22:22:57.064Z level=INFO msg="Upload complete. (184ms)"7497lb # [ 163.059307] niks3-server[1129]: 2026/09/22 22:22:56 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7498worker1 # [ 163.051199] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=0 paths=1 nar_bytes_in=7527499worker1 # [ 163.057411] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:584507500worker1 # [ 163.070187] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7501worker1 # [ 163.077383] nix-daemon[846]: reaped child process 2030, status = succeeded7502worker1 # [ 163.082588] nix-daemon[846]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)7503worker1 # [ 164.123625] nix-daemon[846]: reaped child process 2039, status = succeeded7504worker1 # [ 164.213098] nix-daemon[846]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)7505worker1 # [ 164.226788] nix-daemon[846]: reaped child process 2058, status = succeeded7506lb # [ 164.246767] niks3-server[1129]: 2026/09/22 22:22:57 INFO Received uploads request method=POST path=/api/pending_closures7507worker1 # [ 164.243161] nix-grpc-daemon[1618]: time=2026-09-22T22:22:58.261Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7508worker1 # [ 164.244741] nix-grpc-daemon[1618]: time=2026-09-22T22:22:58.263Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7509lb # [ 164.289345] niks3-server[1129]: 2026/09/22 22:22:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7510worker1 # [ 164.283344] nix-grpc-daemon[1618]: time=2026-09-22T22:22:58.301Z level=INFO msg="Uploading 1 narinfos"7511lb # [ 164.295103] niks3-server[1129]: 2026/09/22 22:22:57 INFO Signed narinfos id=79 count=17512lb # [ 164.301341] niks3-server[1129]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7513lb # [ 164.305850] niks3-server[1129]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7514lb # [ 164.313649] niks3-server[1129]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7515lb # [ 164.321154] niks3-server[1129]: 2026/09/22 22:22:57 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7516lb # [ 164.325710] niks3-server[1129]: 2026/09/22 22:22:57 INFO Completed upload id=797517worker1 # [ 164.314767] nix-grpc-daemon[1618]: time=2026-09-22T22:22:58.333Z level=INFO msg="Upload complete. (223ms)"7518lb # [ 164.327439] niks3-server[1129]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7519worker1 # [ 164.322161] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58450 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=17520worker1 # [ 164.337309] nix-daemon[846]: reaped child process 2037, status = succeeded7521client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.87 seconds)7522worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7523worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7524client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7525client # [ 164.508856] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7526client # Running as unit: gpu.service; invocation ID: 7132cf2263b741e3ad3b872958fdfbc17527client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7528lb: must succeed: curl -sf localhost:9901/clusters7529lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)7530lb: must succeed: curl -sf localhost:9901/clusters7531lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7532worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7533worker1 # [ 164.600131] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:584507534client # [ 164.651100] nix[1536]: don't know how to build these paths:7535client # [ 164.652144] nix[1536]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7536worker1 # [ 164.609829] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:584507537worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7538worker1 # [ 164.632177] nix-grpc-daemon[1530]: ts=2026-09-22T22:22:58Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07539client # [ 164.681966] nix[1536]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7540lb # [ 164.690134] envoy[906]: conn peer=[2001:db8:1::1]:53344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27081 tx=12305 ms=146927541lb: must succeed: curl -sf localhost:9901/clusters7542lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7543lb: must succeed: curl -sf localhost:9901/clusters7544lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7545worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7546worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7547client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7548client # [ 194.683377] nix[1536]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7549client # [ 194.693631] nix[1536]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7550client # [ 194.725609] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7551lb # [ 194.691571] envoy[906]: conn peer=[2001:db8:1::1]:41674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300617552client # [ 194.728992] systemd[1]: gpu.service: Failed with result 'exit-code'.7553client # [ 194.733974] systemd[1]: gpu.service: Consumed 155ms CPU time over 30.224s wall clock time, 37.6M memory peak, 5.5K incoming IP traffic, 5.2K outgoing IP traffic.7554client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.41 seconds)7555client: must succeed: journalctl -u gpu -o cat7556client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7557lb: must succeed: curl -sf localhost:9901/clusters7558lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7559lb: must succeed: curl -sf localhost:9901/clusters7560lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7561worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7562worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7563(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.80 seconds)7564subtest: low disk drains a worker and builds go to the other7565worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7566worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.96 seconds)7567worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7568worker1 # [ 196.285915] nix-grpc-daemon[1530]: ts=2026-09-22T22:23:30Z level=info event=unhealthy reason=min_free7569worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.04 seconds)7570client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27571client # don't know how to build these paths:7572client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7573worker2 # [ 196.465778] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:567267574worker2 # [ 196.471850] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567267575worker1 # [ 196.483465] nix-grpc-daemon[1530]: ts=2026-09-22T22:23:30Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07576worker1 # [ 196.488853] nix-grpc-daemon[1530]: ts=2026-09-22T22:23:30Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17577worker2 # [ 196.544454] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567267578worker2 # [ 196.557924] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567267579client # copying 1 paths to 'lb:50051'...7580client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7581worker2 # [ 196.570903] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567267582worker2 # [ 196.582330] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7583worker2 # [ 196.601661] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:29Z level=info event=niks3_push_started pid=20217584worker2 # [ 196.717636] nix-grpc-daemon[2021]: time=2026-09-22T22:23:29.965Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""7585worker2 # [ 196.805820] nix-daemon[870]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)7586worker2 # [ 196.817255] nix-daemon[870]: reaped child process 2037, status = succeeded7587lb # [ 196.827364] niks3-server[1129]: 2026/09/22 22:23:30 INFO Received uploads request method=POST path=/api/pending_closures7588worker2 # [ 196.836136] nix-grpc-daemon[2021]: time=2026-09-22T22:23:30.084Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7589worker2 # [ 196.837704] nix-grpc-daemon[2021]: time=2026-09-22T22:23:30.085Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7590lb # [ 196.867421] niks3-server[1129]: 2026/09/22 22:23:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7591worker2 # [ 196.878530] nix-grpc-daemon[2021]: time=2026-09-22T22:23:30.126Z level=INFO msg="Uploading 1 narinfos"7592lb # [ 196.879266] niks3-server[1129]: 2026/09/22 22:23:30 INFO Signed narinfos id=80 count=17593lb # [ 196.883149] niks3-server[1129]: 2026/09/22 22:23:30 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7594lb # [ 196.892843] niks3-server[1129]: 2026/09/22 22:23:30 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7595lb # [ 196.906356] niks3-server[1129]: 2026/09/22 22:23:30 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7596lb # [ 196.909623] niks3-server[1129]: 2026/09/22 22:23:30 INFO Completed upload id=807597worker2 # [ 196.909367] nix-grpc-daemon[2021]: time=2026-09-22T22:23:30.157Z level=INFO msg="Upload complete. (188ms)"7598worker2 # [ 196.913309] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56726 duration_s=0 paths=1 nar_bytes_in=5127599lb # [ 196.917102] niks3-server[1129]: 2026/09/22 22:23:30 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7600worker2 # [ 196.918833] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567267601worker2 # [ 196.929059] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7602worker2 # [ 196.933510] nix-daemon[870]: reaped child process 2019, status = succeeded7603worker2 # [ 196.938112] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7604client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7605client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7606client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7607client # farm-a> LOG-farm-a-drain7608worker2 # [ 198.174171] nix-daemon[870]: reaped child process 2041, status = succeeded7609worker2 # [ 198.260513] nix-daemon[870]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)7610worker2 # [ 198.272232] nix-daemon[870]: reaped child process 2060, status = succeeded7611lb # [ 198.280633] niks3-server[1129]: 2026/09/22 22:23:31 INFO Received uploads request method=POST path=/api/pending_closures7612worker2 # [ 198.289679] nix-grpc-daemon[2021]: time=2026-09-22T22:23:31.537Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7613worker2 # [ 198.291182] nix-grpc-daemon[2021]: time=2026-09-22T22:23:31.539Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7614lb # [ 198.325453] niks3-server[1129]: 2026/09/22 22:23:31 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7615lb # [ 198.330158] niks3-server[1129]: 2026/09/22 22:23:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7616lb # [ 198.335149] niks3-server[1129]: 2026/09/22 22:23:31 INFO Signed narinfos id=81 count=17617worker2 # [ 198.337147] nix-grpc-daemon[2021]: time=2026-09-22T22:23:31.585Z level=INFO msg="Uploading 1 narinfos"7618lb # [ 198.338681] niks3-server[1129]: 2026/09/22 22:23:31 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7619lb # [ 198.347903] niks3-server[1129]: 2026/09/22 22:23:31 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7620lb # [ 198.350116] niks3-server[1129]: 2026/09/22 22:23:31 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7621lb # [ 198.353051] niks3-server[1129]: 2026/09/22 22:23:31 INFO Completed upload id=817622lb # [ 198.355301] niks3-server[1129]: 2026/09/22 22:23:31 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7623worker2 # [ 198.354271] nix-grpc-daemon[2021]: time=2026-09-22T22:23:31.602Z level=INFO msg="Upload complete. (208ms)"7624worker2 # [ 198.359484] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56726 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17625worker1 # [ 198.356533] nix-grpc-daemon[1530]: ts=2026-09-22T22:23:32Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07626worker2 # [ 198.372157] nix-daemon[870]: reaped child process 2039, status = succeeded7627worker1 # [ 198.360867] nix-grpc-daemon[1530]: ts=2026-09-22T22:23:32Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17628worker2 # [ 198.410453] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567267629worker2 # [ 198.422744] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567267630worker2 # [ 198.436873] nix-grpc-daemon[2021]: time=2026-09-22T22:23:31.684Z level=INFO msg="All 1 paths already cached"7631client # copying 1 paths to 'lb:50051'...7632client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7633worker2 # [ 198.446449] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567267634worker2 # [ 198.454914] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7635worker2 # [ 198.553595] nix-daemon[870]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)7636worker2 # [ 198.566468] nix-daemon[870]: reaped child process 2067, status = succeeded7637lb # [ 198.574712] niks3-server[1129]: 2026/09/22 22:23:31 INFO Received uploads request method=POST path=/api/pending_closures7638worker2 # [ 198.585637] nix-grpc-daemon[2021]: time=2026-09-22T22:23:31.833Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7639worker2 # [ 198.587171] nix-grpc-daemon[2021]: time=2026-09-22T22:23:31.835Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7640lb # [ 198.611129] niks3-server[1129]: 2026/09/22 22:23:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7641lb # [ 198.617123] niks3-server[1129]: 2026/09/22 22:23:31 INFO Signed narinfos id=82 count=17642worker2 # [ 198.621613] nix-grpc-daemon[2021]: time=2026-09-22T22:23:31.869Z level=INFO msg="Uploading 1 narinfos"7643lb # [ 198.625724] niks3-server[1129]: 2026/09/22 22:23:31 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7644lb # [ 198.633136] niks3-server[1129]: 2026/09/22 22:23:31 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7645lb # [ 198.644569] niks3-server[1129]: 2026/09/22 22:23:31 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7646lb # [ 198.650210] niks3-server[1129]: 2026/09/22 22:23:31 INFO Completed upload id=827647worker2 # [ 198.650056] nix-grpc-daemon[2021]: time=2026-09-22T22:23:31.897Z level=INFO msg="Upload complete. (181ms)"7648lb # [ 198.652822] niks3-server[1129]: 2026/09/22 22:23:32 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7649worker2 # [ 198.654876] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56726 duration_s=0 paths=1 nar_bytes_in=6247650worker2 # [ 198.660886] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567267651worker2 # [ 198.673592] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7652worker2 # [ 198.682087] nix-daemon[870]: reaped child process 2062, status = succeeded7653worker2 # [ 198.685233] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7654client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7655client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7656client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7657client # farm-b> LOG-farm-b-drain7658worker2 # [ 199.699990] nix-daemon[870]: reaped child process 2071, status = succeeded7659worker2 # [ 199.777081] nix-daemon[870]: accepted connection from pid 2087, user nix-grpc-daemon (trusted)7660worker2 # [ 199.789476] nix-daemon[870]: reaped child process 2090, status = succeeded7661lb # [ 199.796896] niks3-server[1129]: 2026/09/22 22:23:33 INFO Received uploads request method=POST path=/api/pending_closures7662worker2 # [ 199.806326] nix-grpc-daemon[2021]: time=2026-09-22T22:23:33.054Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7663worker2 # [ 199.808141] nix-grpc-daemon[2021]: time=2026-09-22T22:23:33.055Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7664lb # [ 199.843229] niks3-server[1129]: 2026/09/22 22:23:33 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7665lb # [ 199.850160] niks3-server[1129]: 2026/09/22 22:23:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7666worker2 # [ 199.857655] nix-grpc-daemon[2021]: time=2026-09-22T22:23:33.105Z level=INFO msg="Uploading 1 narinfos"7667lb # [ 199.856989] niks3-server[1129]: 2026/09/22 22:23:33 INFO Signed narinfos id=83 count=17668lb # [ 199.864189] niks3-server[1129]: 2026/09/22 22:23:33 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7669lb # [ 199.869077] niks3-server[1129]: 2026/09/22 22:23:33 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7670lb # [ 199.878061] niks3-server[1129]: 2026/09/22 22:23:33 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7671lb # [ 199.882819] niks3-server[1129]: 2026/09/22 22:23:33 INFO Completed upload id=837672worker2 # [ 199.882702] nix-grpc-daemon[2021]: time=2026-09-22T22:23:33.130Z level=INFO msg="Upload complete. (190ms)"7673lb # [ 199.886172] niks3-server[1129]: 2026/09/22 22:23:33 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7674worker2 # [ 199.888880] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56726 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17675worker1 # [ 199.884840] nix-grpc-daemon[1530]: ts=2026-09-22T22:23:33Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07676worker1 # [ 199.889496] nix-grpc-daemon[1530]: ts=2026-09-22T22:23:33Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17677worker2 # [ 199.905219] nix-daemon[870]: reaped child process 2069, status = succeeded7678worker2 # [ 199.945281] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567267679worker2 # [ 199.958780] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:567267680worker2 # [ 199.974535] nix-grpc-daemon[2021]: time=2026-09-22T22:23:33.222Z level=INFO msg="All 2 paths already cached"7681client # copying 1 paths to 'lb:50051'...7682client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7683worker2 # [ 199.984701] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:567267684worker2 # [ 199.992901] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7685worker2 # [ 200.094901] nix-daemon[870]: accepted connection from pid 2094, user nix-grpc-daemon (trusted)7686worker2 # [ 200.107775] nix-daemon[870]: reaped child process 2097, status = succeeded7687lb # [ 200.117111] niks3-server[1129]: 2026/09/22 22:23:33 INFO Received uploads request method=POST path=/api/pending_closures7688worker2 # [ 200.124717] nix-grpc-daemon[2021]: time=2026-09-22T22:23:33.372Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7689worker2 # [ 200.126192] nix-grpc-daemon[2021]: time=2026-09-22T22:23:33.374Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7690lb # [ 200.147089] niks3-server[1129]: 2026/09/22 22:23:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7691lb # [ 200.153489] niks3-server[1129]: 2026/09/22 22:23:33 INFO Signed narinfos id=84 count=17692worker2 # [ 200.156705] nix-grpc-daemon[2021]: time=2026-09-22T22:23:33.404Z level=INFO msg="Uploading 1 narinfos"7693lb # [ 200.162715] niks3-server[1129]: 2026/09/22 22:23:33 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7694lb # [ 200.175239] niks3-server[1129]: 2026/09/22 22:23:33 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7695lb # [ 200.181099] niks3-server[1129]: 2026/09/22 22:23:33 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7696lb # [ 200.185812] niks3-server[1129]: 2026/09/22 22:23:33 INFO Completed upload id=847697worker2 # [ 200.185277] nix-grpc-daemon[2021]: time=2026-09-22T22:23:33.433Z level=INFO msg="Upload complete. (178ms)"7698lb # [ 200.190026] niks3-server[1129]: 2026/09/22 22:23:33 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7699worker2 # [ 200.189533] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56726 duration_s=0 paths=1 nar_bytes_in=7527700worker2 # [ 200.194211] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:567267701worker2 # [ 200.203145] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7702worker2 # [ 200.210863] nix-daemon[870]: reaped child process 2092, status = succeeded7703client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7704worker2 # [ 200.216787] nix-daemon[870]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7705client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7706client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7707client # farm-top> LOG-farm-top-drain7708worker2 # [ 200.853333] nix-daemon[870]: reaped child process 2101, status = succeeded7709worker2 # [ 200.932099] nix-daemon[870]: accepted connection from pid 2117, user nix-grpc-daemon (trusted)7710worker2 # [ 200.944962] nix-daemon[870]: reaped child process 2120, status = succeeded7711lb # [ 200.954681] niks3-server[1129]: 2026/09/22 22:23:34 INFO Received uploads request method=POST path=/api/pending_closures7712worker2 # [ 200.961688] nix-grpc-daemon[2021]: time=2026-09-22T22:23:34.209Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7713worker2 # [ 200.963793] nix-grpc-daemon[2021]: time=2026-09-22T22:23:34.211Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7714lb # [ 200.997418] niks3-server[1129]: 2026/09/22 22:23:34 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7715lb # [ 201.003438] niks3-server[1129]: 2026/09/22 22:23:34 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7716lb # [ 201.006907] niks3-server[1129]: 2026/09/22 22:23:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7717lb # [ 201.013734] niks3-server[1129]: 2026/09/22 22:23:34 INFO Signed narinfos id=85 count=17718worker2 # [ 201.015922] nix-grpc-daemon[2021]: time=2026-09-22T22:23:34.263Z level=INFO msg="Uploading 1 narinfos"7719lb # [ 201.020225] niks3-server[1129]: 2026/09/22 22:23:34 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7720lb # [ 201.028826] niks3-server[1129]: 2026/09/22 22:23:34 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7721lb # [ 201.034800] niks3-server[1129]: 2026/09/22 22:23:34 INFO Completed upload id=857722worker2 # [ 201.034508] nix-grpc-daemon[2021]: time=2026-09-22T22:23:34.282Z level=INFO msg="Upload complete. (211ms)"7723lb # [ 201.037728] niks3-server[1129]: 2026/09/22 22:23:34 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7724worker2 # [ 201.040972] nix-grpc-daemon[1961]: ts=2026-09-22T22:23:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56726 duration_s=0 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17725worker2 # [ 201.055499] nix-daemon[870]: reaped child process 2099, status = succeeded7726client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.76 seconds)7727client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7728client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7729worker1: must succeed: rm /nix/.rw-store/fill7730worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7731worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7732worker1 # [ 202.288967] nix-grpc-daemon[1530]: ts=2026-09-22T22:23:36Z level=info event=healthy reason=min_free7733worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7734(finished: subtest: low disk drains a worker and builds go to the other, in 7.13 seconds)7735(finished: run the VM test script, in 203.38 seconds)7736test script finished in 203.57s7737cleanup7738kill QemuMachine (pid 49)7739client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7740client # [2026-09-22T22:23:37Z INFO virtiofsd] Client disconnected, shutting down7741client # [2026-09-22T22:23:37Z INFO virtiofsd] Client disconnected, shutting down7742client # [2026-09-22T22:23:37Z INFO virtiofsd] Client disconnected, shutting down7743kill QemuMachine (pid 51)7744lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7745lb # [2026-09-22T22:23:37Z INFO virtiofsd] Client disconnected, shutting down7746lb # [2026-09-22T22:23:37Z INFO virtiofsd] Client disconnected, shutting down7747lb # [2026-09-22T22:23:37Z INFO virtiofsd] Client disconnected, shutting down7748kill QemuMachine (pid 50)7749worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7750worker1 # [2026-09-22T22:23:37Z INFO virtiofsd] Client disconnected, shutting down7751worker1 # [2026-09-22T22:23:37Z INFO virtiofsd] Client disconnected, shutting down7752worker1 # [2026-09-22T22:23:37Z INFO virtiofsd] Client disconnected, shutting down7753kill QemuMachine (pid 52)7754worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7755worker2 # [2026-09-22T22:23:38Z INFO virtiofsd] Client disconnected, shutting down7756worker2 # [2026-09-22T22:23:38Z INFO virtiofsd] Client disconnected, shutting down7757worker2 # [2026-09-22T22:23:38Z INFO virtiofsd] Client disconnected, shutting down7758(finished: cleanup, in 1.01 seconds)7759additionally exposed symbols:7760 client, lb, worker1, worker2,7761 vlan1,7762 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh