vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #501
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.bSsJETmhH5', fmt=raw size=107374182414client: QEMU running (pid 49)15worker1 # Disk image does not exist, creating the virtualisation disk image...16worker1: QEMU running (pid 51)17client # mke2fs 1.47.4 (6-Mar-2025)18lb: QEMU running (pid 50)19worker2 # Disk image does not exist, creating the virtualisation disk image...20worker1 # Formatting '/build/vm-state-worker1/tmp.KxsjmRpxYH', fmt=raw size=107374182421lb # Disk image does not exist, creating the virtualisation disk image...22worker2 # Formatting '/build/vm-state-worker2/tmp.gFq2ay4We5', fmt=raw size=107374182423worker1 # mke2fs 1.47.4 (6-Mar-2025)24worker2: QEMU running (pid 52)25worker2 # mke2fs 1.47.4 (6-Mar-2025)26worker1 # Discarding device blocks: 0/262144 done27lb # Formatting '/build/vm-state-lb/tmp.Ig69nvJVHX', fmt=raw size=107374182428client # Discarding device blocks: 0/262144 done29worker2 # Discarding device blocks: 0/262144 done30worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # Filesystem UUID: a6ecff3c-92fe-436f-87f9-7ba7358fceec32worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes33worker2 # Filesystem UUID: 617606b7-6b13-41d5-879e-78f08e71f5dc34worker1 # Superblock backups stored on blocks:35lb # mke2fs 1.47.4 (6-Mar-2025)36worker2 # Superblock backups stored on blocks:37worker1 # 32768, 98304, 163840, 22937638lb # Discarding device blocks: 0/262144 done39worker2 # 32768, 98304, 163840, 22937640worker1 # 41lb # Creating filesystem with 262144 4k blocks and 65536 inodes42worker2 # 43worker1 # Allocating group tables: 0/8 done44worker1 # Writing inode tables: 0/8 done45lb # Filesystem UUID: 11545468-e34f-4655-b646-834c34ea92d846(finished: start all VMs, in 0.27 seconds)47client # Creating filesystem with 262144 4k blocks and 65536 inodes48lb # Superblock backups stored on blocks:49client: waiting for unit network-addresses-eth1.service50client # Filesystem UUID: 88b9297a-0752-4a49-bb7a-2b420a5d11a951lb # 32768, 98304, 163840, 22937652client: waiting for the VM to finish booting53client # Superblock backups stored on blocks:54lb # 55worker2 # Allocating group tables: 0/8 done56lb # Allocating group tables: 0/8 done57worker2 # Writing inode tables: 0/8 done58client # 32768, 98304, 163840, 22937659worker1 # Creating journal (8192 blocks): done60lb # Writing inode tables: 0/8 done61worker2 # Creating journal (8192 blocks): done62client # 63worker1 # Writing superblocks and filesystem accounting information: 0/8 done64lb # Creating journal (8192 blocks): done65worker2 # Writing superblocks and filesystem accounting information: 0/8 done66client # Allocating group tables: 0/8 done67worker1 # 68lb # Writing superblocks and filesystem accounting information: 0/8 done69worker2 # 70client # Writing inode tables: 0/8 done71worker1 # Virtualisation disk image created.72lb # 73worker2 # Virtualisation disk image created.74client # Creating journal (8192 blocks): done75worker1 # Starting virtiofs daemons...76lb # Virtualisation disk image created.77worker2 # Starting virtiofs daemons...78client # Writing superblocks and filesystem accounting information: 0/8 done79worker1 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80lb # Starting virtiofs daemons...81worker2 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)82client # 83worker1 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84lb # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker2 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether86client # Virtualisation disk image created.87worker1 # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...88lb # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker2 # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...90client # Starting virtiofs daemons...91worker1 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)92lb # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker2 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94client # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)95worker1 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether96lb # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)97worker2 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether98client # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether99worker1 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100lb # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether101worker2 # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...102client # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...103worker1 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104lb # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...105worker2 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)106client # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)107worker1 # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...108lb # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)109worker2 # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether110client # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether111worker1 # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...112lb # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether113worker2 # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...114client # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...115worker1 # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests116lb # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...117worker2 # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests118client # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)119worker1 # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests120lb # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests121worker2 # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests122client # [2026-09-22T14:50:30Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether123worker1 # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests124lb # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests125worker2 # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests126lb # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests127client # [2026-09-22T14:50:30Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-22T14:50:30Z INFO virtiofsd] Client connected, servicing requests131worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker1 # 136worker1 # 137worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139worker1 # 140worker1 # 141worker1 # 142worker1 # 143worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00144client # 145client # 146lb # 147lb # 148client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00149lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00150worker2 # 151worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...152worker2 # 153worker1 # 154worker1 # 155client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...156client # 157client # 158client # 159client # 160worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00161lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...162lb # 163lb # 164client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00165lb # 166lb # 167lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00168worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...169worker2 # 170worker2 # 171worker2 # 172worker2 # 173worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00174client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...175client # 176client # 177lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178lb # 179lb # 180worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker2 # 182worker2 # 183worker1 # Booting from ROM...184client # Booting from ROM...185lb # Booting from ROM...186worker2 # Booting from ROM...187worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b5wjb92n6pk0dvawkzyxrvwqjkpdl5dh-nixos-system-worker1-test/init regInfo=/nix/store/rimqqvv97hgfqmwpzjdxhyp5f1xkg23w-closure-info/registration console=ttyS0,115200n8 console=tty0189worker1 # [ 0.000000] BIOS-provided physical RAM map:190worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026199worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved200worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved201worker1 # [ 0.000000] NX (Execute Disable) protection: active202worker1 # [ 0.000000] APIC: Static calls initialized203worker1 # [ 0.000000] SMBIOS 2.8 present.204worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014205client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gf7wpg39jwq284cdjajvxqgxnm2p4dqn-nixos-system-client-test/init regInfo=/nix/store/8yv2mzydx25rdvyb7iz1gsd7iqxlkjg4-closure-info/registration console=ttyS0,115200n8 console=tty0206worker1 # [ 0.000000] DMI: Memory slots populated: 1/1207client # [ 0.000000] BIOS-provided physical RAM map:208worker1 # [ 0.000000] Hypervisor detected: KVM209worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000210client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable211worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00212client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved213worker1 # [ 0.000000] kvm-clock: using sched offset of 449023645 cycles214client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved215worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns216client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable217worker1 # [ 0.000006] tsc: Detected 2400.012 MHz processor218client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved219worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026220worker1 # [ 0.000806] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000221client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved222worker1 # [ 0.000834] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs223lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026224client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved225worker1 # [ 0.000837] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT226client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved227worker1 # [ 0.002725] found SMP MP-table at [mem 0x000f5450-0x000f545f]228worker1 # [ 0.002738] Using GB pages for direct mapping229client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved230worker1 # [ 0.002868] RAMDISK: [mem 0x5e370000-0x5ffcffff]231worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3x9dnnqnj54467v5gvfcy6bbd182ar8b-nixos-system-worker2-test/init regInfo=/nix/store/000yyak5fh8kvhsg7sjzgbzg7ffh39qz-closure-info/registration console=ttyS0,115200n8 console=tty0232client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved233worker2 # [ 0.000000] BIOS-provided physical RAM map:234worker1 # [ 0.002877] ACPI: Early table checksum verification disabled235client # [ 0.000000] NX (Execute Disable) protection: active236client # [ 0.000000] APIC: Static calls initialized237worker1 # [ 0.002880] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )238worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable239client # [ 0.000000] SMBIOS 2.8 present.240lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qfr7fghmwm537hb3f4f5is3yb8phm6jx-nixos-system-lb-test/init regInfo=/nix/store/s1480xaqc00gh7hz31b4z2fwjpnxi3sa-closure-info/registration console=ttyS0,115200n8 console=tty0241worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved242worker1 # [ 0.002884] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)243lb # [ 0.000000] BIOS-provided physical RAM map:244client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014245worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved246worker1 # [ 0.002889] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)247client # [ 0.000000] DMI: Memory slots populated: 1/1248lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable249worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable250client # [ 0.000000] Hypervisor detected: KVM251worker1 # [ 0.002896] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)252client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000253lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved254worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved255worker1 # [ 0.002898] ACPI: FACS 0x000000005FFE0000 000040256client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00257worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved258lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved259worker1 # [ 0.002900] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)260client # [ 0.000000] kvm-clock: using sched offset of 464914187 cycles261worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved262lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable263worker1 # [ 0.002901] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)264worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved265client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns266lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved267client # [ 0.000004] tsc: Detected 2400.012 MHz processor268worker1 # [ 0.002903] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)269worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved270client # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000271lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved272worker1 # [ 0.002904] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)273worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved274client # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs275worker2 # [ 0.000000] NX (Execute Disable) protection: active276lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved277worker1 # [ 0.002906] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]278worker2 # [ 0.000000] APIC: Static calls initialized279client # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT280worker2 # [ 0.000000] SMBIOS 2.8 present.281lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved282worker1 # [ 0.002907] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]283client # [ 0.002734] found SMP MP-table at [mem 0x000f5450-0x000f545f]284client # [ 0.002746] Using GB pages for direct mapping285worker1 # [ 0.002907] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]286lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved287worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014288client # [ 0.002909] RAMDISK: [mem 0x5e370000-0x5ffcffff]289worker2 # [ 0.000000] DMI: Memory slots populated: 1/1290worker1 # [ 0.002908] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]291worker2 # [ 0.000000] Hypervisor detected: KVM292lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved293client # [ 0.002917] ACPI: Early table checksum verification disabled294worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000295worker1 # [ 0.002908] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]296client # [ 0.002920] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )297lb # [ 0.000000] NX (Execute Disable) protection: active298lb # [ 0.000000] APIC: Static calls initialized299worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00300worker1 # [ 0.002909] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]301lb # [ 0.000000] SMBIOS 2.8 present.302client # [ 0.002924] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)303worker2 # [ 0.000001] kvm-clock: using sched offset of 448238059 cycles304worker1 # [ 0.002909] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]305worker1 # [ 0.003137] No NUMA configuration found306client # [ 0.002928] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)307lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014308worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns309worker1 # [ 0.003138] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]310lb # [ 0.000000] DMI: Memory slots populated: 1/1311worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor312client # [ 0.002936] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)313lb # [ 0.000000] Hypervisor detected: KVM314worker1 # [ 0.003140] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]315client # [ 0.002938] ACPI: FACS 0x000000005FFE0000 000040316worker1 # [ 0.005367] Zone ranges:317worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000318lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000319worker1 # [ 0.005369] DMA [mem 0x0000000000001000-0x0000000000ffffff]320client # [ 0.002939] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)321worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs322lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00323worker1 # [ 0.005370] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]324worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT325worker1 # [ 0.005371] Normal empty326client # [ 0.002941] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)327lb # [ 0.000000] kvm-clock: using sched offset of 466496707 cycles328worker1 # [ 0.005372] Device empty329worker2 # [ 0.002737] found SMP MP-table at [mem 0x000f5450-0x000f545f]330worker1 # [ 0.005373] Movable zone start for each node331client # [ 0.002942] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)332worker2 # [ 0.002748] Using GB pages for direct mapping333worker1 # [ 0.005374] Early memory node ranges334worker2 # [ 0.002845] RAMDISK: [mem 0x5e36f000-0x5ffcffff]335lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns336worker1 # [ 0.005374] node 0: [mem 0x0000000000001000-0x000000000009efff]337client # [ 0.002944] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)338worker2 # [ 0.002853] ACPI: Early table checksum verification disabled339lb # [ 0.000005] tsc: Detected 2400.012 MHz processor340worker1 # [ 0.005375] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]341client # [ 0.002945] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]342worker2 # [ 0.002856] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )343lb # [ 0.000818] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000344worker1 # [ 0.005376] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]345client # [ 0.002946] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]346worker2 # [ 0.002860] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)347worker1 # [ 0.005396] On node 0, zone DMA: 1 pages in unavailable ranges348lb # [ 0.000846] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs349client # [ 0.002947] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]350worker1 # [ 0.005654] On node 0, zone DMA: 97 pages in unavailable ranges351worker2 # [ 0.002865] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)352lb # [ 0.000848] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT353client # [ 0.002947] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]354worker1 # [ 0.031353] On node 0, zone DMA32: 40 pages in unavailable ranges355lb # [ 0.002738] found SMP MP-table at [mem 0x000f5450-0x000f545f]356worker2 # [ 0.002871] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)357client # [ 0.002948] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]358worker1 # [ 0.031812] ACPI: PM-Timer IO Port: 0x608359worker2 # [ 0.002873] ACPI: FACS 0x000000005FFE0000 000040360lb # [ 0.002749] Using GB pages for direct mapping361worker1 # [ 0.031825] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])362client # [ 0.002948] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]363lb # [ 0.002911] RAMDISK: [mem 0x5e36e000-0x5ffcffff]364worker1 # [ 0.031853] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23365worker2 # [ 0.002875] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)366client # [ 0.002949] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]367lb # [ 0.002920] ACPI: Early table checksum verification disabled368worker1 # [ 0.031855] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)369client # [ 0.003175] No NUMA configuration found370worker2 # [ 0.002876] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)371lb # [ 0.002924] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )372worker1 # [ 0.031857] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)373client # [ 0.003176] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]374worker2 # [ 0.002878] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)375client # [ 0.003178] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]376worker1 # [ 0.031858] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)377lb # [ 0.002928] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)378client # [ 0.005337] Zone ranges:379worker2 # [ 0.002879] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)380worker1 # [ 0.031860] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)381client # [ 0.005338] DMA [mem 0x0000000000001000-0x0000000000ffffff]382lb # [ 0.002933] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)383worker1 # [ 0.031860] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)384client # [ 0.005339] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]385worker2 # [ 0.002881] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]386client # [ 0.005340] Normal empty387client # [ 0.005341] Device empty388worker2 # [ 0.002882] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]389worker1 # [ 0.031863] ACPI: Using ACPI (MADT) for SMP configuration information390lb # [ 0.002940] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)391client # [ 0.005342] Movable zone start for each node392worker1 # [ 0.031864] ACPI: HPET id: 0x8086a201 base: 0xfed00000393lb # [ 0.002942] ACPI: FACS 0x000000005FFE0000 000040394worker2 # [ 0.002882] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]395client # [ 0.005342] Early memory node ranges396worker1 # [ 0.031868] TSC deadline timer available397client # [ 0.005343] node 0: [mem 0x0000000000001000-0x000000000009efff]398worker1 # [ 0.031872] CPU topo: Max. logical packages: 1399worker2 # [ 0.002883] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]400lb # [ 0.002943] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)401worker1 # [ 0.031873] CPU topo: Max. logical dies: 1402client # [ 0.005344] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]403worker2 # [ 0.002883] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]404worker1 # [ 0.031873] CPU topo: Max. dies per package: 1405lb # [ 0.002945] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)406client # [ 0.005345] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]407worker1 # [ 0.031877] CPU topo: Max. threads per core: 1408worker2 # [ 0.002884] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]409worker1 # [ 0.031877] CPU topo: Num. cores per package: 1410client # [ 0.005364] On node 0, zone DMA: 1 pages in unavailable ranges411worker2 # [ 0.002884] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]412lb # [ 0.002946] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)413worker1 # [ 0.031878] CPU topo: Num. threads per package: 1414client # [ 0.005616] On node 0, zone DMA: 97 pages in unavailable ranges415worker2 # [ 0.003110] No NUMA configuration found416worker1 # [ 0.031878] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs417client # [ 0.031393] On node 0, zone DMA32: 40 pages in unavailable ranges418lb # [ 0.002948] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)419worker2 # [ 0.003111] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]420client # [ 0.031888] ACPI: PM-Timer IO Port: 0x608421worker1 # [ 0.031903] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()422worker2 # [ 0.003113] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]423worker2 # [ 0.005386] Zone ranges:424client # [ 0.031901] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])425lb # [ 0.002949] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]426worker1 # [ 0.031942] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]427worker2 # [ 0.005387] DMA [mem 0x0000000000001000-0x0000000000ffffff]428client # [ 0.031927] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23429lb # [ 0.002950] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]430worker1 # [ 0.031944] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]431worker2 # [ 0.005388] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]432client # [ 0.031930] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)433worker2 # [ 0.005390] Normal empty434worker1 # [ 0.031945] [mem 0x60000000-0xafffffff] available for PCI devices435lb # [ 0.002951] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]436worker2 # [ 0.005391] Device empty437client # [ 0.031932] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)438worker1 # [ 0.031947] Booting paravirtualized kernel on KVM439worker2 # [ 0.005391] Movable zone start for each node440lb # [ 0.002951] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]441worker2 # [ 0.005392] Early memory node ranges442client # [ 0.031933] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)443worker1 # [ 0.031950] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns444worker2 # [ 0.005392] node 0: [mem 0x0000000000001000-0x000000000009efff]445lb # [ 0.002951] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]446client # [ 0.031934] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)447worker2 # [ 0.005393] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]448worker1 # [ 0.036435] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1449client # [ 0.031935] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)450lb # [ 0.002952] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]451worker2 # [ 0.005394] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]452worker1 # [ 0.038530] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152453client # [ 0.031937] ACPI: Using ACPI (MADT) for SMP configuration information454lb # [ 0.002952] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]455worker1 # [ 0.038573] kvm-guest: PV spinlocks disabled, single CPU456worker2 # [ 0.005414] On node 0, zone DMA: 1 pages in unavailable ranges457client # [ 0.031938] ACPI: HPET id: 0x8086a201 base: 0xfed00000458lb # [ 0.003186] No NUMA configuration found459client # [ 0.031942] TSC deadline timer available460worker2 # [ 0.005677] On node 0, zone DMA: 97 pages in unavailable ranges461client # [ 0.031946] CPU topo: Max. logical packages: 1462lb # [ 0.003186] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]463worker2 # [ 0.032165] On node 0, zone DMA32: 40 pages in unavailable ranges464client # [ 0.031947] CPU topo: Max. logical dies: 1465worker2 # [ 0.032618] ACPI: PM-Timer IO Port: 0x608466lb # [ 0.003189] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]467client # [ 0.031948] CPU topo: Max. dies per package: 1468lb # [ 0.005439] Zone ranges:469worker2 # [ 0.032630] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])470client # [ 0.031951] CPU topo: Max. threads per core: 1471client # [ 0.031951] CPU topo: Num. cores per package: 1472worker2 # [ 0.032660] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23473lb # [ 0.005440] DMA [mem 0x0000000000001000-0x0000000000ffffff]474worker1 # [ 0.038574] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b5wjb92n6pk0dvawkzyxrvwqjkpdl5dh-nixos-system-worker1-test/init regInfo=/nix/store/rimqqvv97hgfqmwpzjdxhyp5f1xkg23w-closure-info/registration console=ttyS0,115200n8 console=tty0475client # [ 0.031952] CPU topo: Num. threads per package: 1476worker2 # [ 0.032663] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)477lb # [ 0.005442] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]478client # [ 0.031952] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs479lb # [ 0.005444] Normal empty480worker2 # [ 0.032665] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)481lb # [ 0.005444] Device empty482worker1 # [ 0.038668] Unknown kernel command line parameters "regInfo=/nix/store/rimqqvv97hgfqmwpzjdxhyp5f1xkg23w-closure-info/registration", will be passed to user space.483client # [ 0.031973] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()484worker2 # [ 0.032666] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)485lb # [ 0.005445] Movable zone start for each node486worker1 # [ 0.038681] random: crng init done487lb # [ 0.005446] Early memory node ranges488client # [ 0.032004] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]489worker2 # [ 0.032667] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)490worker1 # [ 0.038682] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes491lb # [ 0.005446] node 0: [mem 0x0000000000001000-0x000000000009efff]492client # [ 0.032006] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]493worker2 # [ 0.032667] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)494worker1 # [ 0.040759] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)495client # [ 0.032007] [mem 0x60000000-0xafffffff] available for PCI devices496lb # [ 0.005447] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]497worker2 # [ 0.032670] ACPI: Using ACPI (MADT) for SMP configuration information498client # [ 0.032009] Booting paravirtualized kernel on KVM499worker1 # [ 0.041987] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)500worker2 # [ 0.032671] ACPI: HPET id: 0x8086a201 base: 0xfed00000501lb # [ 0.005448] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]502worker1 # [ 0.042021] Fallback order for Node 0: 0503worker2 # [ 0.032674] TSC deadline timer available504client # [ 0.032012] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns505lb # [ 0.005469] On node 0, zone DMA: 1 pages in unavailable ranges506worker1 # [ 0.042024] Built 1 zonelists, mobility grouping on. Total pages: 393078507worker2 # [ 0.032678] CPU topo: Max. logical packages: 1508worker1 # [ 0.042025] Policy zone: DMA32509worker2 # [ 0.032679] CPU topo: Max. logical dies: 1510lb # [ 0.005733] On node 0, zone DMA: 97 pages in unavailable ranges511client # [ 0.036502] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1512worker2 # [ 0.032679] CPU topo: Max. dies per package: 1513worker1 # [ 0.044449] mem auto-init: stack:all(zero), heap alloc:on, heap free:off514worker2 # [ 0.032682] CPU topo: Max. threads per core: 1515client # [ 0.038826] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152516lb # [ 0.032209] On node 0, zone DMA32: 40 pages in unavailable ranges517worker1 # [ 0.047605] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1518worker2 # [ 0.032683] CPU topo: Num. cores per package: 1519lb # [ 0.032661] ACPI: PM-Timer IO Port: 0x608520client # [ 0.038869] kvm-guest: PV spinlocks disabled, single CPU521worker1 # [ 0.050828] allocated 3145728 bytes of page_ext522worker2 # [ 0.032683] CPU topo: Num. threads per package: 1523lb # [ 0.032674] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])524worker1 # [ 0.060661] ftrace: allocating 48787 entries in 192 pages525worker2 # [ 0.032684] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs526worker1 # [ 0.060663] ftrace: allocated 192 pages with 2 groups527lb # [ 0.032701] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23528worker1 # [ 0.061531] Dynamic Preempt: lazy529worker2 # [ 0.032703] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()530worker1 # [ 0.061662] rcu: Preemptible hierarchical RCU implementation.531lb # [ 0.032704] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)532worker2 # [ 0.032734] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]533worker1 # [ 0.061662] rcu: RCU event tracing is enabled.534lb # [ 0.032705] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)535client # [ 0.038870] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gf7wpg39jwq284cdjajvxqgxnm2p4dqn-nixos-system-client-test/init regInfo=/nix/store/8yv2mzydx25rdvyb7iz1gsd7iqxlkjg4-closure-info/registration console=ttyS0,115200n8 console=tty0536worker1 # [ 0.061663] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.537worker2 # [ 0.032736] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]538worker1 # [ 0.061664] Trampoline variant of Tasks RCU enabled.539lb # [ 0.032707] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)540worker2 # [ 0.032737] [mem 0x60000000-0xafffffff] available for PCI devices541worker1 # [ 0.061665] Rude variant of Tasks RCU enabled.542worker2 # [ 0.032739] Booting paravirtualized kernel on KVM543client # [ 0.038964] Unknown kernel command line parameters "regInfo=/nix/store/8yv2mzydx25rdvyb7iz1gsd7iqxlkjg4-closure-info/registration", will be passed to user space.544lb # [ 0.032708] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)545worker1 # [ 0.061665] Tracing variant of Tasks RCU enabled.546client # [ 0.038978] random: crng init done547lb # [ 0.032708] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)548worker2 # [ 0.032742] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns549worker1 # [ 0.061666] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.550client # [ 0.038979] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes551lb # [ 0.032711] ACPI: Using ACPI (MADT) for SMP configuration information552worker1 # [ 0.061666] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1553worker2 # [ 0.037227] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1554client # [ 0.041067] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)555lb # [ 0.032712] ACPI: HPET id: 0x8086a201 base: 0xfed00000556worker2 # [ 0.039394] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152557worker1 # [ 0.061715] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.558client # [ 0.042104] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)559lb # [ 0.032716] TSC deadline timer available560worker2 # [ 0.039437] kvm-guest: PV spinlocks disabled, single CPU561client # [ 0.042138] Fallback order for Node 0: 0562lb # [ 0.032720] CPU topo: Max. logical packages: 1563worker1 # [ 0.061717] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.564client # [ 0.042141] Built 1 zonelists, mobility grouping on. Total pages: 393078565lb # [ 0.032721] CPU topo: Max. logical dies: 1566client # [ 0.042142] Policy zone: DMA32567lb # [ 0.032721] CPU topo: Max. dies per package: 1568worker1 # [ 0.061718] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.569client # [ 0.044584] mem auto-init: stack:all(zero), heap alloc:on, heap free:off570worker1 # [ 0.066072] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16571lb # [ 0.032724] CPU topo: Max. threads per core: 1572client # [ 0.047809] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1573lb # [ 0.032725] CPU topo: Num. cores per package: 1574worker1 # [ 0.066578] rcu: srcu_init: Setting srcu_struct sizes based on contention.575client # [ 0.051247] allocated 3145728 bytes of page_ext576worker2 # [ 0.039438] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3x9dnnqnj54467v5gvfcy6bbd182ar8b-nixos-system-worker2-test/init regInfo=/nix/store/000yyak5fh8kvhsg7sjzgbzg7ffh39qz-closure-info/registration console=ttyS0,115200n8 console=tty0577lb # [ 0.032725] CPU topo: Num. threads per package: 1578client # [ 0.060959] ftrace: allocating 48787 entries in 192 pages579worker1 # [ 0.066585] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns580lb # [ 0.032725] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs581client # [ 0.060961] ftrace: allocated 192 pages with 2 groups582client # [ 0.061764] Dynamic Preempt: lazy583worker2 # [ 0.039533] Unknown kernel command line parameters "regInfo=/nix/store/000yyak5fh8kvhsg7sjzgbzg7ffh39qz-closure-info/registration", will be passed to user space.584worker1 # [ 0.066688] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)585lb # [ 0.032745] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()586client # [ 0.061927] rcu: Preemptible hierarchical RCU implementation.587worker2 # [ 0.039546] random: crng init done588worker1 # [ 0.070224] Console: colour VGA+ 80x25589client # [ 0.061928] rcu: RCU event tracing is enabled.590worker1 # [ 0.070227] printk: legacy console [tty0] enabled591lb # [ 0.032777] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]592worker2 # [ 0.039546] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes593worker1 # [ 0.099686] printk: legacy console [ttyS0] enabled594client # [ 0.061928] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.595worker2 # [ 0.041671] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)596client # [ 0.061930] Trampoline variant of Tasks RCU enabled.597lb # [ 0.032779] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]598worker1 # [ 0.196275] ACPI: Core revision 20250807599client # [ 0.061930] Rude variant of Tasks RCU enabled.600lb # [ 0.032780] [mem 0x60000000-0xafffffff] available for PCI devices601worker2 # [ 0.042731] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)602client # [ 0.061930] Tracing variant of Tasks RCU enabled.603worker2 # [ 0.042761] Fallback order for Node 0: 0604lb # [ 0.032782] Booting paravirtualized kernel on KVM605worker1 # [ 0.197139] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns606client # [ 0.061931] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.607worker2 # [ 0.042763] Built 1 zonelists, mobility grouping on. Total pages: 393078608worker2 # [ 0.042765] Policy zone: DMA32609worker1 # [ 0.198608] APIC: Switch to symmetric I/O mode setup610client # [ 0.061932] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1611lb # [ 0.032786] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns612worker1 # [ 0.199534] x2apic enabled613worker2 # [ 0.045635] mem auto-init: stack:all(zero), heap alloc:on, heap free:off614client # [ 0.061954] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.615lb # [ 0.037295] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1616worker2 # [ 0.048822] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1617worker1 # [ 0.200264] APIC: Switched APIC routing to: physical x2apic618worker2 # [ 0.052131] allocated 3145728 bytes of page_ext619client # [ 0.061956] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.620lb # [ 0.039477] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152621worker2 # [ 0.062021] ftrace: allocating 48787 entries in 192 pages622lb # [ 0.039519] kvm-guest: PV spinlocks disabled, single CPU623worker2 # [ 0.062024] ftrace: allocated 192 pages with 2 groups624client # [ 0.061957] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.625worker2 # [ 0.062872] Dynamic Preempt: lazy626worker1 # [ 0.202055] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1627client # [ 0.066324] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16628worker2 # [ 0.063050] rcu: Preemptible hierarchical RCU implementation.629worker2 # [ 0.063051] rcu: RCU event tracing is enabled.630client # [ 0.066600] rcu: srcu_init: Setting srcu_struct sizes based on contention.631worker1 # [ 0.203011] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns632worker2 # [ 0.063052] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.633client # [ 0.066605] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns634worker2 # [ 0.063053] Trampoline variant of Tasks RCU enabled.635worker1 # [ 0.204638] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)636worker2 # [ 0.063054] Rude variant of Tasks RCU enabled.637client # [ 0.066707] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)638worker2 # [ 0.063054] Tracing variant of Tasks RCU enabled.639lb # [ 0.039520] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qfr7fghmwm537hb3f4f5is3yb8phm6jx-nixos-system-lb-test/init regInfo=/nix/store/s1480xaqc00gh7hz31b4z2fwjpnxi3sa-closure-info/registration console=ttyS0,115200n8 console=tty0640client # [ 0.070246] Console: colour VGA+ 80x25641worker1 # [ 0.205721] x86/cpu: User Mode Instruction Prevention (UMIP) activated642client # [ 0.070249] printk: legacy console [tty0] enabled643worker2 # [ 0.063054] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.644worker1 # [ 0.206754] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127645client # [ 0.099813] printk: legacy console [ttyS0] enabled646worker2 # [ 0.063055] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1647lb # [ 0.039617] Unknown kernel command line parameters "regInfo=/nix/store/s1480xaqc00gh7hz31b4z2fwjpnxi3sa-closure-info/registration", will be passed to user space.648client # [ 0.196119] ACPI: Core revision 20250807649worker1 # [ 0.207636] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0650lb # [ 0.039630] random: crng init done651worker2 # [ 0.063070] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.652lb # [ 0.039631] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes653worker2 # [ 0.063072] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.654client # [ 0.197018] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns655worker1 # [ 0.208639] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto656lb # [ 0.041752] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)657client # [ 0.198510] APIC: Switch to symmetric I/O mode setup658worker2 # [ 0.063072] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.659worker1 # [ 0.209636] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl660client # [ 0.199429] x2apic enabled661worker2 # [ 0.067436] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16662lb # [ 0.043000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)663lb # [ 0.043037] Fallback order for Node 0: 0664worker1 # [ 0.210636] Transient Scheduler Attacks: Vulnerable: No microcode665worker2 # [ 0.067729] rcu: srcu_init: Setting srcu_struct sizes based on contention.666client # [ 0.200153] APIC: Switched APIC routing to: physical x2apic667lb # [ 0.043040] Built 1 zonelists, mobility grouping on. Total pages: 393078668worker1 # [ 0.211636] Spectre V2 : Mitigation: Enhanced / Automatic IBRS669worker2 # [ 0.067736] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns670lb # [ 0.043042] Policy zone: DMA32671client # [ 0.201905] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1672worker1 # [ 0.212636] Speculative Return Stack Overflow: Mitigation: Safe RET673lb # [ 0.045521] mem auto-init: stack:all(zero), heap alloc:on, heap free:off674worker2 # [ 0.067841] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)675worker2 # [ 0.071360] Console: colour VGA+ 80x25676lb # [ 0.048814] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1677worker2 # [ 0.071363] printk: legacy console [tty0] enabled678client # [ 0.202843] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns679lb # [ 0.052145] allocated 3145728 bytes of page_ext680worker2 # [ 0.101144] printk: legacy console [ttyS0] enabled681lb # [ 0.062163] ftrace: allocating 48787 entries in 192 pages682client # [ 0.204425] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)683worker2 # [ 0.197559] ACPI: Core revision 20250807684lb # [ 0.062165] ftrace: allocated 192 pages with 2 groups685lb # [ 0.063012] Dynamic Preempt: lazy686worker1 # [ 0.213635] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization687client # [ 0.206514] x86/cpu: User Mode Instruction Prevention (UMIP) activated688lb # [ 0.063196] rcu: Preemptible hierarchical RCU implementation.689worker2 # [ 0.198469] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns690worker1 # [ 0.215642] Spectre V2 : Enabling IBPB for BPF691client # [ 0.207590] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127692lb # [ 0.063197] rcu: RCU event tracing is enabled.693worker2 # [ 0.200009] APIC: Switch to symmetric I/O mode setup694worker1 # [ 0.216637] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier695client # [ 0.208423] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0696lb # [ 0.063198] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.697worker2 # [ 0.200993] x2apic enabled698lb # [ 0.063199] Trampoline variant of Tasks RCU enabled.699worker1 # [ 0.217636] active return thunk: srso_alias_return_thunk700lb # [ 0.063200] Rude variant of Tasks RCU enabled.701worker2 # [ 0.201781] APIC: Switched APIC routing to: physical x2apic702lb # [ 0.063200] Tracing variant of Tasks RCU enabled.703client # [ 0.209426] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto704worker1 # [ 0.218656] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'705lb # [ 0.063201] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.706client # [ 0.210422] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl707worker1 # [ 0.219635] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'708worker2 # [ 0.203581] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1709lb # [ 0.063201] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1710client # [ 0.211423] Transient Scheduler Attacks: Vulnerable: No microcode711worker1 # [ 0.220635] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'712lb # [ 0.063218] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.713client # [ 0.212422] Spectre V2 : Mitigation: Enhanced / Automatic IBRS714worker2 # [ 0.204557] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns715worker1 # [ 0.221635] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'716lb # [ 0.063220] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.717client # [ 0.213422] Speculative Return Stack Overflow: Mitigation: Safe RET718worker1 # [ 0.222635] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'719worker2 # [ 0.206213] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)720lb # [ 0.063221] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.721lb # [ 0.067898] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16722worker1 # [ 0.223635] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'723worker2 # [ 0.207298] x86/cpu: User Mode Instruction Prevention (UMIP) activated724lb # [ 0.068183] rcu: srcu_init: Setting srcu_struct sizes based on contention.725worker1 # [ 0.224635] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'726worker2 # [ 0.208337] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127727lb # [ 0.068189] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns728worker1 # [ 0.225636] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256729worker2 # [ 0.209899] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0730client # [ 0.214422] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization731lb # [ 0.068298] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)732client # [ 0.216429] Spectre V2 : Enabling IBPB for BPF733worker1 # [ 0.227341] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64734lb # [ 0.071803] Console: colour VGA+ 80x25735worker2 # [ 0.210215] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto736lb # [ 0.071805] printk: legacy console [tty0] enabled737worker1 # [ 0.227636] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512738lb # [ 0.101254] printk: legacy console [ttyS0] enabled739client # [ 0.217126] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier740worker1 # [ 0.228635] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024741worker2 # [ 0.212210] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl742lb # [ 0.212102] ACPI: Core revision 20250807743client # [ 0.218424] active return thunk: srso_alias_return_thunk744worker1 # [ 0.229635] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8745worker2 # [ 0.213211] Transient Scheduler Attacks: Vulnerable: No microcode746lb # [ 0.212978] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns747client # [ 0.219451] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'748worker2 # [ 0.214210] Spectre V2 : Mitigation: Enhanced / Automatic IBRS749worker1 # [ 0.230635] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.750lb # [ 0.214459] APIC: Switch to symmetric I/O mode setup751client # [ 0.220424] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'752worker2 # [ 0.215210] Speculative Return Stack Overflow: Mitigation: Safe RET753lb # [ 0.215390] x2apic enabled754client # [ 0.221424] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'755lb # [ 0.216140] APIC: Switched APIC routing to: physical x2apic756client # [ 0.222423] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'757client # [ 0.224289] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'758lb # [ 0.217907] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1759worker2 # [ 0.216210] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization760client # [ 0.225218] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'761worker2 # [ 0.218218] Spectre V2 : Enabling IBPB for BPF762lb # [ 0.218884] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns763client # [ 0.226213] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'764lb # [ 0.220489] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)765worker2 # [ 0.219211] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier766client # [ 0.227423] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256767worker2 # [ 0.220211] active return thunk: srso_alias_return_thunk768lb # [ 0.221573] x86/cpu: User Mode Instruction Prevention (UMIP) activated769client # [ 0.228422] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64770lb # [ 0.222665] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127771client # [ 0.229422] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512772worker2 # [ 0.221871] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'773client # [ 0.230422] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024774lb # [ 0.223486] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0775worker2 # [ 0.223103] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'776client # [ 0.231422] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8777worker2 # [ 0.223947] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'778lb # [ 0.224489] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto779worker2 # [ 0.224950] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'780client # [ 0.232422] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.781lb # [ 0.225486] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl782worker2 # [ 0.225976] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'783lb # [ 0.226486] Transient Scheduler Attacks: Vulnerable: No microcode784worker2 # [ 0.226965] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'785lb # [ 0.227486] Spectre V2 : Mitigation: Enhanced / Automatic IBRS786worker2 # [ 0.228017] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'787lb # [ 0.228486] Speculative Return Stack Overflow: Mitigation: Safe RET788worker2 # [ 0.229211] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256789worker2 # [ 0.230211] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64790worker2 # [ 0.231210] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512791worker2 # [ 0.232210] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024792lb # [ 0.229485] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization793worker2 # [ 0.233210] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8794lb # [ 0.231493] Spectre V2 : Enabling IBPB for BPF795lb # [ 0.232207] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier796worker2 # [ 0.234210] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.797lb # [ 0.233486] active return thunk: srso_alias_return_thunk798lb # [ 0.234328] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'799lb # [ 0.235485] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'800lb # [ 0.236485] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'801lb # [ 0.237486] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'802lb # [ 0.238485] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'803lb # [ 0.239428] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'804lb # [ 0.240260] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'805lb # [ 0.241486] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256806lb # [ 0.242485] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807lb # [ 0.243486] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808worker1 # [ 0.263872] Freeing SMP alternatives memory: 44K809lb # [ 0.244485] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024810worker1 # [ 0.264639] pid_max: default: 32768 minimum: 301811lb # [ 0.245406] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8812worker1 # [ 0.265453] LSM: initializing lsm=capability,landlock,yama,bpf,ima813worker1 # [ 0.266509] landlock: Up and running.814lb # [ 0.246197] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.815worker1 # [ 0.267084] Yama: becoming mindful.816worker1 # [ 0.267847] LSM support for eBPF active817worker1 # [ 0.268735] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818worker1 # [ 0.269678] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819worker1 # [ 0.272835] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820worker1 # [ 0.274166] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821worker1 # [ 0.274645] ... version: 2822worker1 # [ 0.275385] ... bit width: 48823client # [ 0.266637] Freeing SMP alternatives memory: 44K824worker1 # [ 0.275638] ... generic counters: 6825client # [ 0.267425] pid_max: default: 32768 minimum: 301826worker1 # [ 0.276364] ... generic bitmap: 000000000000003f827client # [ 0.268274] LSM: initializing lsm=capability,landlock,yama,bpf,ima828worker1 # [ 0.276638] ... fixed-purpose counters: 0829client # [ 0.269442] landlock: Up and running.830worker1 # [ 0.277404] ... fixed-purpose bitmap: 0000000000000000831client # [ 0.270084] Yama: becoming mindful.832client # [ 0.270642] LSM support for eBPF active833worker1 # [ 0.277638] ... value mask: 0000ffffffffffff834worker1 # [ 0.278577] ... max period: 00007fffffffffff835client # [ 0.271524] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)836worker1 # [ 0.279383] ... global_ctrl mask: 000000000000003f837client # [ 0.272460] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)838worker1 # [ 0.279743] signal: max sigframe size: 3376839worker1 # [ 0.280557] rcu: Hierarchical SRCU implementation.840worker1 # [ 0.281262] rcu: Max phase no-delay instances is 400.841client # [ 0.275210] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)842worker2 # [ 0.268923] Freeing SMP alternatives memory: 44K843worker2 # [ 0.269214] pid_max: default: 32768 minimum: 301844client # [ 0.275942] Performance Events: Fam17h+ core perfctr, AMD PMU driver.845client # [ 0.276427] ... version: 2846worker2 # [ 0.270313] LSM: initializing lsm=capability,landlock,yama,bpf,ima847client # [ 0.277150] ... bit width: 48848worker2 # [ 0.271315] landlock: Up and running.849worker2 # [ 0.272211] Yama: becoming mindful.850client # [ 0.277425] ... generic counters: 6851worker1 # [ 0.286072] smp: Bringing up secondary CPUs ...852worker2 # [ 0.273300] LSM support for eBPF active853client # [ 0.278159] ... generic bitmap: 000000000000003f854worker1 # [ 0.286651] smp: Brought up 1 node, 1 CPU855client # [ 0.278434] ... fixed-purpose counters: 0856worker2 # [ 0.274065] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)857worker1 # [ 0.287317] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)858client # [ 0.279155] ... fixed-purpose bitmap: 0000000000000000859client # [ 0.279425] ... value mask: 0000ffffffffffff860worker2 # [ 0.275251] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)861client # [ 0.280358] ... max period: 00007fffffffffff862worker1 # [ 0.287856] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)863client # [ 0.281158] ... global_ctrl mask: 000000000000003f864worker1 # [ 0.288857] devtmpfs: initialized865worker2 # [ 0.277650] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)866client # [ 0.281542] signal: max sigframe size: 3376867worker1 # [ 0.289658] x86/mm: Memory block size: 128MB868client # [ 0.282336] rcu: Hierarchical SRCU implementation.869worker2 # [ 0.278798] Performance Events: Fam17h+ core perfctr, AMD PMU driver.870client # [ 0.283042] rcu: Max phase no-delay instances is 400.871worker2 # [ 0.279216] ... version: 2872worker1 # [ 0.291391] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)873worker2 # [ 0.279961] ... bit width: 48874worker2 # [ 0.280213] ... generic counters: 6875lb # [ 0.280350] Freeing SMP alternatives memory: 44K876worker1 # [ 0.292603] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).877worker2 # [ 0.280906] ... generic bitmap: 000000000000003f878lb # [ 0.281026] pid_max: default: 32768 minimum: 301879worker1 # [ 0.293736] pinctrl core: initialized pinctrl subsystem880worker2 # [ 0.281218] ... fixed-purpose counters: 0881lb # [ 0.281586] LSM: initializing lsm=capability,landlock,yama,bpf,ima882worker1 # [ 0.294824] PM: RTC time: 14:50:31, date: 2026-09-22883worker2 # [ 0.281947] ... fixed-purpose bitmap: 0000000000000000884lb # [ 0.282589] landlock: Up and running.885lb # [ 0.283486] Yama: becoming mindful.886client # [ 0.287936] smp: Bringing up secondary CPUs ...887worker2 # [ 0.282213] ... value mask: 0000ffffffffffff888client # [ 0.288439] smp: Brought up 1 node, 1 CPU889lb # [ 0.284267] LSM support for eBPF active890worker2 # [ 0.283111] ... max period: 00007fffffffffff891client # [ 0.289125] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)892worker2 # [ 0.283907] ... global_ctrl mask: 000000000000003f893lb # [ 0.285058] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)894worker2 # [ 0.284340] signal: max sigframe size: 3376895worker1 # [ 0.298338] NET: Registered PF_NETLINK/PF_ROUTE protocol family896lb # [ 0.285530] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)897worker2 # [ 0.285098] rcu: Hierarchical SRCU implementation.898client # [ 0.289574] Memory: 1454580K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)899worker2 # [ 0.285786] rcu: Max phase no-delay instances is 400.900worker1 # [ 0.299137] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations901client # [ 0.290647] devtmpfs: initialized902lb # [ 0.288565] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)903worker1 # [ 0.299664] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations904client # [ 0.291521] x86/mm: Memory block size: 128MB905worker1 # [ 0.300915] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations906lb # [ 0.290090] Performance Events: Fam17h+ core perfctr, AMD PMU driver.907lb # [ 0.290494] ... version: 2908client # [ 0.293258] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)909worker1 # [ 0.301647] audit: initializing netlink subsys (disabled)910lb # [ 0.291161] ... bit width: 48911lb # [ 0.291488] ... generic counters: 6912worker1 # [ 0.302809] thermal_sys: Registered thermal governor 'fair_share'913client # [ 0.294366] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).914worker2 # [ 0.290779] smp: Bringing up secondary CPUs ...915lb # [ 0.292175] ... generic bitmap: 000000000000003f916worker1 # [ 0.302811] thermal_sys: Registered thermal governor 'bang_bang'917client # [ 0.295517] pinctrl core: initialized pinctrl subsystem918worker2 # [ 0.291227] smp: Brought up 1 node, 1 CPU919lb # [ 0.292488] ... fixed-purpose counters: 0920worker1 # [ 0.303639] thermal_sys: Registered thermal governor 'step_wise'921client # [ 0.296603] PM: RTC time: 14:50:30, date: 2026-09-22922worker2 # [ 0.291872] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)923lb # [ 0.293161] ... fixed-purpose bitmap: 0000000000000000924worker1 # [ 0.304525] thermal_sys: Registered thermal governor 'user_space'925lb # [ 0.293488] ... value mask: 0000ffffffffffff926lb # [ 0.294348] ... max period: 00007fffffffffff927worker1 # [ 0.305377] audit: type=2000 audit(1790088631.634:1): state=initialized audit_enabled=0 res=1928worker2 # [ 0.292384] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)929lb # [ 0.295134] ... global_ctrl mask: 000000000000003f930worker2 # [ 0.293510] devtmpfs: initialized931worker1 # [ 0.306641] thermal_sys: Registered thermal governor 'power_allocator'932lb # [ 0.295584] signal: max sigframe size: 3376933worker1 # [ 0.306654] cpuidle: using governor menu934client # [ 0.300398] NET: Registered PF_NETLINK/PF_ROUTE protocol family935worker2 # [ 0.294329] x86/mm: Memory block size: 128MB936lb # [ 0.296338] rcu: Hierarchical SRCU implementation.937lb # [ 0.297049] rcu: Max phase no-delay instances is 400.938worker1 # [ 0.308839] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5939client # [ 0.301630] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations940worker2 # [ 0.296100] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)941client # [ 0.302444] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations942worker1 # [ 0.309847] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]943worker2 # [ 0.297145] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).944worker1 # [ 0.310641] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry945client # [ 0.303701] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations946worker2 # [ 0.298300] pinctrl core: initialized pinctrl subsystem947worker1 # [ 0.311650] PCI: Using configuration type 1 for base access948client # [ 0.304435] audit: initializing netlink subsys (disabled)949worker2 # [ 0.299477] PM: RTC time: 14:50:31, date: 2026-09-22950client # [ 0.305591] thermal_sys: Registered thermal governor 'fair_share'951lb # [ 0.302041] smp: Bringing up secondary CPUs ...952worker1 # [ 0.312779] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.953lb # [ 0.302501] smp: Brought up 1 node, 1 CPU954client # [ 0.305593] thermal_sys: Registered thermal governor 'bang_bang'955lb # [ 0.303140] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)956client # [ 0.306426] thermal_sys: Registered thermal governor 'step_wise'957client # [ 0.307379] thermal_sys: Registered thermal governor 'user_space'958worker2 # [ 0.302980] NET: Registered PF_NETLINK/PF_ROUTE protocol family959lb # [ 0.303659] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)960client # [ 0.308164] audit: type=2000 audit(1790088631.649:1): state=initialized audit_enabled=0 res=1961lb # [ 0.304748] devtmpfs: initialized962worker2 # [ 0.303726] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations963client # [ 0.309428] thermal_sys: Registered thermal governor 'power_allocator'964worker1 # [ 0.317859] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages965lb # [ 0.305615] x86/mm: Memory block size: 128MB966client # [ 0.309441] cpuidle: using governor menu967worker2 # [ 0.304238] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations968worker1 # [ 0.318638] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page969client # [ 0.311606] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5970worker2 # [ 0.305495] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations971lb # [ 0.307426] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)972worker2 # [ 0.306223] audit: initializing netlink subsys (disabled)973client # [ 0.312659] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]974lb # [ 0.308370] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).975worker2 # [ 0.307379] thermal_sys: Registered thermal governor 'fair_share'976client # [ 0.313427] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry977lb # [ 0.309559] pinctrl core: initialized pinctrl subsystem978worker2 # [ 0.307381] thermal_sys: Registered thermal governor 'bang_bang'979client # [ 0.314436] PCI: Using configuration type 1 for base access980lb # [ 0.310680] PM: RTC time: 14:50:30, date: 2026-09-22981worker1 # [ 0.323640] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages982worker2 # [ 0.308214] thermal_sys: Registered thermal governor 'step_wise'983client # [ 0.315527] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.984worker1 # [ 0.324638] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page985worker2 # [ 0.309185] thermal_sys: Registered thermal governor 'user_space'986worker2 # [ 0.309954] audit: type=2000 audit(1790088631.658:1): state=initialized audit_enabled=0 res=1987lb # [ 0.314323] NET: Registered PF_NETLINK/PF_ROUTE protocol family988worker2 # [ 0.311216] thermal_sys: Registered thermal governor 'power_allocator'989worker2 # [ 0.311242] cpuidle: using governor menu990lb # [ 0.315658] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations991worker2 # [ 0.313472] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5992lb # [ 0.316510] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations993worker2 # [ 0.314454] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]994lb # [ 0.317764] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations995client # [ 0.322707] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages996worker2 # [ 0.315216] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry997lb # [ 0.318498] audit: initializing netlink subsys (disabled)998client # [ 0.323426] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page999worker2 # [ 0.316225] PCI: Using configuration type 1 for base access1000lb # [ 0.319587] thermal_sys: Registered thermal governor 'fair_share'1001lb # [ 0.319590] thermal_sys: Registered thermal governor 'bang_bang'1002worker1 # [ 0.334980] ACPI: Added _OSI(Module Device)1003worker2 # [ 0.317353] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1004lb # [ 0.320489] thermal_sys: Registered thermal governor 'step_wise'1005worker1 # [ 0.335640] ACPI: Added _OSI(Processor Device)1006lb # [ 0.321403] thermal_sys: Registered thermal governor 'user_space'1007worker1 # [ 0.336355] ACPI: Added _OSI(Processor Aggregator Device)1008lb # [ 0.322195] audit: type=2000 audit(1790088631.657:1): state=initialized audit_enabled=0 res=11009client # [ 0.328426] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1010lb # [ 0.323492] thermal_sys: Registered thermal governor 'power_allocator'1011client # [ 0.329426] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1012lb # [ 0.323506] cpuidle: using governor menu1013lb # [ 0.325682] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51014worker2 # [ 0.324403] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1015lb # [ 0.326744] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1016worker2 # [ 0.325214] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1017lb # [ 0.327491] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1018lb # [ 0.328500] PCI: Using configuration type 1 for base access1019lb # [ 0.329565] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1020worker1 # [ 0.345204] ACPI: 1 ACPI AML tables successfully acquired and loaded1021client # [ 0.338040] ACPI: Added _OSI(Module Device)1022worker1 # [ 0.346598] ACPI: Interpreter enabled1023client # [ 0.338426] ACPI: Added _OSI(Processor Device)1024worker1 # [ 0.347157] ACPI: PM: (supports S0 S3 S4 S5)1025worker2 # [ 0.330214] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1026client # [ 0.339152] ACPI: Added _OSI(Processor Aggregator Device)1027worker2 # [ 0.331214] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1028worker1 # [ 0.349639] ACPI: Using IOAPIC for interrupt routing1029lb # [ 0.334737] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1030lb # [ 0.335489] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1031worker1 # [ 0.350502] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1032worker1 # [ 0.353638] PCI: Using E820 reservations for host bridge windows1033worker1 # [ 0.354783] ACPI: Enabled 2 GPEs in block 00 to 3F1034client # [ 0.346039] ACPI: 1 ACPI AML tables successfully acquired and loaded1035lb # [ 0.340489] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1036lb # [ 0.341489] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1037worker2 # [ 0.341610] ACPI: Added _OSI(Module Device)1038worker2 # [ 0.342214] ACPI: Added _OSI(Processor Device)1039client # [ 0.349307] ACPI: Interpreter enabled1040client # [ 0.349878] ACPI: PM: (supports S0 S3 S4 S5)1041worker2 # [ 0.342971] ACPI: Added _OSI(Processor Aggregator Device)1042client # [ 0.350425] ACPI: Using IOAPIC for interrupt routing1043client # [ 0.351338] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1044worker1 # [ 0.363431] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1045worker1 # [ 0.365133] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1046client # [ 0.356425] PCI: Using E820 reservations for host bridge windows1047lb # [ 0.351887] ACPI: Added _OSI(Module Device)1048worker1 # [ 0.365717] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1049client # [ 0.357559] ACPI: Enabled 2 GPEs in block 00 to 3F1050lb # [ 0.352490] ACPI: Added _OSI(Processor Device)1051lb # [ 0.353218] ACPI: Added _OSI(Processor Aggregator Device)1052worker1 # [ 0.366762] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1053worker2 # [ 0.351562] ACPI: 1 ACPI AML tables successfully acquired and loaded1054worker1 # [ 0.367978] PCI host bridge to bus 0000:001055worker2 # [ 0.353174] ACPI: Interpreter enabled1056worker1 # [ 0.368642] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1057worker2 # [ 0.353689] ACPI: PM: (supports S0 S3 S4 S5)1058worker1 # [ 0.369639] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1059worker1 # [ 0.370638] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1060worker2 # [ 0.356214] ACPI: Using IOAPIC for interrupt routing1061worker1 # [ 0.371638] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1062worker2 # [ 0.357111] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1063worker1 # [ 0.372638] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1064client # [ 0.366196] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1065worker1 # [ 0.373639] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1066worker1 # [ 0.374639] pci_bus 0000:00: root bus resource [bus 00-ff]1067lb # [ 0.362086] ACPI: 1 ACPI AML tables successfully acquired and loaded1068client # [ 0.367729] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1069worker2 # [ 0.360213] PCI: Using E820 reservations for host bridge windows1070worker1 # [ 0.375592] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1071worker2 # [ 0.361319] ACPI: Enabled 2 GPEs in block 00 to 3F1072lb # [ 0.363420] ACPI: Interpreter enabled1073client # [ 0.368502] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1074lb # [ 0.363931] ACPI: PM: (supports S0 S3 S4 S5)1075client # [ 0.369546] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1076worker1 # [ 0.377094] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1077client # [ 0.370787] PCI host bridge to bus 0000:001078client # [ 0.371430] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1079lb # [ 0.366490] ACPI: Using IOAPIC for interrupt routing1080client # [ 0.372425] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1081lb # [ 0.367375] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1082client # [ 0.373425] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1083client # [ 0.374425] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1084worker2 # [ 0.369117] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1085lb # [ 0.370489] PCI: Using E820 reservations for host bridge windows1086client # [ 0.375425] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1087lb # [ 0.371585] ACPI: Enabled 2 GPEs in block 00 to 3F1088worker1 # [ 0.379719] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1089worker2 # [ 0.370002] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1090client # [ 0.376425] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1091worker1 # [ 0.380650] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1092client # [ 0.377425] pci_bus 0000:00: root bus resource [bus 00-ff]1093worker2 # [ 0.371291] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1094worker1 # [ 0.381643] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1095client # [ 0.378384] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1096worker2 # [ 0.372336] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1097worker2 # [ 0.373600] PCI host bridge to bus 0000:001098worker1 # [ 0.383051] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1099client # [ 0.379880] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1100worker2 # [ 0.374218] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1101worker2 # [ 0.375213] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1102worker1 # [ 0.384381] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1103worker2 # [ 0.376216] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1104worker2 # [ 0.377213] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1105lb # [ 0.380172] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1106worker1 # [ 0.386673] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1107worker2 # [ 0.378213] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1108worker1 # [ 0.387553] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1109lb # [ 0.381184] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1110worker2 # [ 0.379213] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1111worker1 # [ 0.388408] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1112lb # [ 0.382567] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1113client # [ 0.382490] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1114worker2 # [ 0.380214] pci_bus 0000:00: root bus resource [bus 00-ff]1115worker1 # [ 0.388644] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1116client # [ 0.383438] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1117lb # [ 0.383614] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1118worker2 # [ 0.381166] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1119client # [ 0.384384] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1120lb # [ 0.384873] PCI host bridge to bus 0000:001121worker1 # [ 0.390204] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1122worker2 # [ 0.382649] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1123lb # [ 0.385493] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1124client # [ 0.385470] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1125lb # [ 0.386489] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1126worker1 # [ 0.391683] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1127lb # [ 0.387488] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1128worker1 # [ 0.392578] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1129client # [ 0.387323] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1130lb # [ 0.388488] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1131worker1 # [ 0.393390] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1132lb # [ 0.389488] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1133worker1 # [ 0.394167] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1134client # [ 0.390452] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1135lb # [ 0.390489] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1136client # [ 0.391290] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1137lb # [ 0.391488] pci_bus 0000:00: root bus resource [bus 00-ff]1138client # [ 0.392207] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1139worker2 # [ 0.385373] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1140lb # [ 0.392536] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1141client # [ 0.393265] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1142worker2 # [ 0.386296] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1143worker1 # [ 0.396704] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1144worker2 # [ 0.387238] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1145lb # [ 0.393994] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1146worker1 # [ 0.397645] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1147client # [ 0.394757] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1148worker2 # [ 0.388570] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1149worker1 # [ 0.398660] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1150client # [ 0.396439] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1151worker1 # [ 0.400222] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1152worker2 # [ 0.389939] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1153client # [ 0.397291] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1154client # [ 0.398198] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1155worker1 # [ 0.401646] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1156worker2 # [ 0.392253] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1157worker1 # [ 0.402644] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158client # [ 0.399883] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1159worker2 # [ 0.393136] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1160lb # [ 0.396552] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1161worker1 # [ 0.403600] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1162lb # [ 0.397500] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1163worker2 # [ 0.393970] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1164worker1 # [ 0.404433] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1165lb # [ 0.398476] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1166worker2 # [ 0.394219] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1167worker1 # [ 0.406169] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1168lb # [ 0.399497] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1169client # [ 0.402440] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1170worker2 # [ 0.395786] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1171client # [ 0.403432] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1172worker1 # [ 0.407489] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1173lb # [ 0.401186] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1174client # [ 0.404416] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1175worker1 # [ 0.408407] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1176worker2 # [ 0.398252] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1177client # [ 0.405832] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1178worker2 # [ 0.399206] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1179worker1 # [ 0.410111] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1180lb # [ 0.403504] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1181worker2 # [ 0.399991] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1182lb # [ 0.404338] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1183client # [ 0.407433] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1184worker1 # [ 0.411677] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1185lb # [ 0.405266] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1186worker2 # [ 0.401680] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1187client # [ 0.408265] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1188lb # [ 0.406297] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1189worker1 # [ 0.412638] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1190client # [ 0.409128] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1191client # [ 0.409443] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1192lb # [ 0.407761] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1193worker1 # [ 0.414189] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1194worker2 # [ 0.404411] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1195client # [ 0.410970] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1196worker1 # [ 0.415456] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1197worker2 # [ 0.405219] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1198lb # [ 0.409504] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1199worker1 # [ 0.416418] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1200worker2 # [ 0.406193] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1201lb # [ 0.410331] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1202client # [ 0.412438] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1203lb # [ 0.411232] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1204worker1 # [ 0.418052] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1205client # [ 0.413414] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1206worker2 # [ 0.407652] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1207lb # [ 0.412039] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1208client # [ 0.414800] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1209worker1 # [ 0.419651] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1210worker2 # [ 0.409249] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1211worker1 # [ 0.420606] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1212worker2 # [ 0.410131] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1213client # [ 0.416437] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1214worker2 # [ 0.410990] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1215worker1 # [ 0.422513] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1216client # [ 0.417435] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1217lb # [ 0.414497] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1218worker2 # [ 0.412094] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1219lb # [ 0.415494] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1220worker1 # [ 0.424678] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1221client # [ 0.419002] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1222lb # [ 0.416458] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1223worker2 # [ 0.413826] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1224worker1 # [ 0.425535] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1225worker1 # [ 0.426415] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1226client # [ 0.420447] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1227lb # [ 0.417839] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1228worker2 # [ 0.415046] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1229client # [ 0.421380] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1230worker2 # [ 0.415973] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1231worker1 # [ 0.428032] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1232lb # [ 0.419505] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1233client # [ 0.422833] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1234worker2 # [ 0.416843] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1235lb # [ 0.420361] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1236lb # [ 0.421235] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1237worker1 # [ 0.429672] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1238client # [ 0.424439] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1239lb # [ 0.421494] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1240worker2 # [ 0.418227] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1241worker1 # [ 0.430644] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1242client # [ 0.425402] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1243worker1 # [ 0.431582] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1244worker2 # [ 0.419189] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1245lb # [ 0.423510] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1246client # [ 0.426968] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1247worker1 # [ 0.433052] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1248worker2 # [ 0.420742] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1249lb # [ 0.425421] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1250worker1 # [ 0.434197] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1251client # [ 0.428440] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1252worker2 # [ 0.422003] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1253lb # [ 0.426271] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1254client # [ 0.429250] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1255worker1 # [ 0.434837] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1256worker2 # [ 0.422923] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1257client # [ 0.430171] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1258lb # [ 0.427855] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1259worker1 # [ 0.436190] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1260worker2 # [ 0.423751] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1261client # [ 0.430981] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1262lb # [ 0.429503] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1263worker1 # [ 0.436838] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1264lb # [ 0.430473] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1265worker2 # [ 0.425259] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1266worker1 # [ 0.438392] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1267worker2 # [ 0.426235] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1268client # [ 0.432656] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1269lb # [ 0.432379] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1270client # [ 0.433432] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1271worker2 # [ 0.427796] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1272client # [ 0.434447] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1273lb # [ 0.434241] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1274lb # [ 0.434511] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1275worker2 # [ 0.429255] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1276client # [ 0.435979] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1277worker2 # [ 0.430093] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1278lb # [ 0.436063] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1279client # [ 0.437127] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1280worker2 # [ 0.430954] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1281client # [ 0.437626] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1282lb # [ 0.437501] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1283worker2 # [ 0.432614] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1284lb # [ 0.438416] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1285client # [ 0.439051] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1286client # [ 0.439623] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1287lb # [ 0.439877] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1288worker2 # [ 0.434255] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1289worker1 # [ 0.438938] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1290worker2 # [ 0.435220] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1291client # [ 0.441121] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1292worker1 # [ 0.441167] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1293lb # [ 0.442496] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1294worker2 # [ 0.436202] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1295lb # [ 0.443341] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1296worker1 # [ 0.441890] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1297lb # [ 0.444275] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1298worker2 # [ 0.437678] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1299worker1 # [ 0.442922] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1300worker2 # [ 0.438737] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1301lb # [ 0.445912] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1302worker1 # [ 0.443907] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1303worker2 # [ 0.439413] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1304worker1 # [ 0.445577] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1305worker2 # [ 0.440773] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1306lb # [ 0.447516] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1307worker1 # [ 0.446292] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1308lb # [ 0.448497] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1309worker2 # [ 0.441428] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1310worker1 # [ 0.446951] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1311lb # [ 0.449472] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1312worker2 # [ 0.442759] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1313worker1 # [ 0.448222] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1314lb # [ 0.450955] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1315client # [ 0.441653] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1316lb # [ 0.452071] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1317client # [ 0.444073] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1318worker1 # [ 0.451622] ACPI: PCI: Interrupt link LNKA configured for IRQ 101319lb # [ 0.452692] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1320client # [ 0.444713] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1321worker1 # [ 0.452468] ACPI: PCI: Interrupt link LNKB configured for IRQ 101322lb # [ 0.454063] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1323worker1 # [ 0.453645] ACPI: PCI: Interrupt link LNKC configured for IRQ 111324client # [ 0.445710] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1325lb # [ 0.454680] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1326worker1 # [ 0.454680] ACPI: PCI: Interrupt link LNKD configured for IRQ 111327worker1 # [ 0.455715] ACPI: PCI: Interrupt link LNKE configured for IRQ 101328client # [ 0.446698] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1329lb # [ 0.456450] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1330worker1 # [ 0.456730] ACPI: PCI: Interrupt link LNKF configured for IRQ 101331client # [ 0.448453] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1332worker1 # [ 0.457737] ACPI: PCI: Interrupt link LNKG configured for IRQ 111333client # [ 0.449375] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1334worker1 # [ 0.458736] ACPI: PCI: Interrupt link LNKH configured for IRQ 111335worker1 # [ 0.459674] ACPI: PCI: Interrupt link GSIA configured for IRQ 161336worker2 # [ 0.443502] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1337client # [ 0.450547] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1338worker1 # [ 0.460606] ACPI: PCI: Interrupt link GSIB configured for IRQ 171339worker2 # [ 0.445760] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1340worker1 # [ 0.461382] ACPI: PCI: Interrupt link GSIC configured for IRQ 181341client # [ 0.452319] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1342worker1 # [ 0.462388] ACPI: PCI: Interrupt link GSID configured for IRQ 191343worker2 # [ 0.446457] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1344worker1 # [ 0.462652] ACPI: PCI: Interrupt link GSIE configured for IRQ 201345worker2 # [ 0.447529] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1346worker1 # [ 0.463651] ACPI: PCI: Interrupt link GSIF configured for IRQ 211347worker1 # [ 0.464625] ACPI: PCI: Interrupt link GSIG configured for IRQ 221348worker2 # [ 0.448460] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1349client # [ 0.456719] ACPI: PCI: Interrupt link LNKA configured for IRQ 101350worker1 # [ 0.465404] ACPI: PCI: Interrupt link GSIH configured for IRQ 231351client # [ 0.457529] ACPI: PCI: Interrupt link LNKB configured for IRQ 101352worker2 # [ 0.450262] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1353worker1 # [ 0.466665] iommu: Default domain type: Translated1354lb # [ 0.457377] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1355client # [ 0.458521] ACPI: PCI: Interrupt link LNKC configured for IRQ 111356worker2 # [ 0.451105] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1357worker1 # [ 0.467451] iommu: DMA domain TLB invalidation policy: lazy mode1358client # [ 0.459552] ACPI: PCI: Interrupt link LNKD configured for IRQ 111359worker1 # [ 0.468749] ACPI: bus type USB registered1360lb # [ 0.459315] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1361worker2 # [ 0.452254] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1362client # [ 0.460525] ACPI: PCI: Interrupt link LNKE configured for IRQ 101363worker1 # [ 0.469472] usbcore: registered new interface driver usbfs1364lb # [ 0.460511] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1365worker2 # [ 0.453793] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1366worker1 # [ 0.470334] usbcore: registered new interface driver hub1367client # [ 0.461535] ACPI: PCI: Interrupt link LNKF configured for IRQ 101368worker1 # [ 0.470648] usbcore: registered new device driver usb1369client # [ 0.462525] ACPI: PCI: Interrupt link LNKG configured for IRQ 111370lb # [ 0.461785] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1371client # [ 0.463524] ACPI: PCI: Interrupt link LNKH configured for IRQ 111372worker1 # [ 0.472374] NetLabel: Initializing1373worker1 # [ 0.472644] NetLabel: domain hash size = 1281374lb # [ 0.462783] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1375client # [ 0.464463] ACPI: PCI: Interrupt link GSIA configured for IRQ 161376worker1 # [ 0.473373] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1377worker2 # [ 0.457224] ACPI: PCI: Interrupt link LNKA configured for IRQ 101378client # [ 0.465441] ACPI: PCI: Interrupt link GSIB configured for IRQ 171379lb # [ 0.464383] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1380worker1 # [ 0.473695] NetLabel: unlabeled traffic allowed by default1381worker2 # [ 0.458318] ACPI: PCI: Interrupt link LNKB configured for IRQ 101382client # [ 0.466438] ACPI: PCI: Interrupt link GSIC configured for IRQ 181383worker1 # [ 0.474588] PCI: Using ACPI for IRQ routing1384lb # [ 0.465087] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1385worker2 # [ 0.459314] ACPI: PCI: Interrupt link LNKC configured for IRQ 111386client # [ 0.467437] ACPI: PCI: Interrupt link GSID configured for IRQ 191387worker2 # [ 0.460314] ACPI: PCI: Interrupt link LNKD configured for IRQ 111388lb # [ 0.465808] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1389client # [ 0.468443] ACPI: PCI: Interrupt link GSIE configured for IRQ 201390worker2 # [ 0.461314] ACPI: PCI: Interrupt link LNKE configured for IRQ 101391client # [ 0.469441] ACPI: PCI: Interrupt link GSIF configured for IRQ 211392lb # [ 0.467286] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1393worker2 # [ 0.462309] ACPI: PCI: Interrupt link LNKF configured for IRQ 101394client # [ 0.470440] ACPI: PCI: Interrupt link GSIG configured for IRQ 221395worker2 # [ 0.463312] ACPI: PCI: Interrupt link LNKG configured for IRQ 111396client # [ 0.471440] ACPI: PCI: Interrupt link GSIH configured for IRQ 231397worker2 # [ 0.464312] ACPI: PCI: Interrupt link LNKH configured for IRQ 111398client # [ 0.473497] iommu: Default domain type: Translated1399worker2 # [ 0.465256] ACPI: PCI: Interrupt link GSIA configured for IRQ 161400worker2 # [ 0.466196] ACPI: PCI: Interrupt link GSIB configured for IRQ 171401client # [ 0.474307] iommu: DMA domain TLB invalidation policy: lazy mode1402lb # [ 0.471219] ACPI: PCI: Interrupt link LNKA configured for IRQ 101403client # [ 0.475440] ACPI: bus type USB registered1404worker2 # [ 0.466936] ACPI: PCI: Interrupt link GSIC configured for IRQ 181405lb # [ 0.471597] ACPI: PCI: Interrupt link LNKB configured for IRQ 101406client # [ 0.476169] usbcore: registered new interface driver usbfs1407worker2 # [ 0.467226] ACPI: PCI: Interrupt link GSID configured for IRQ 191408lb # [ 0.472585] ACPI: PCI: Interrupt link LNKC configured for IRQ 111409client # [ 0.476438] usbcore: registered new interface driver hub1410worker2 # [ 0.468197] ACPI: PCI: Interrupt link GSIE configured for IRQ 201411lb # [ 0.473585] ACPI: PCI: Interrupt link LNKD configured for IRQ 111412client # [ 0.477296] usbcore: registered new device driver usb1413worker2 # [ 0.468930] ACPI: PCI: Interrupt link GSIF configured for IRQ 211414lb # [ 0.474597] ACPI: PCI: Interrupt link LNKE configured for IRQ 101415worker2 # [ 0.469226] ACPI: PCI: Interrupt link GSIG configured for IRQ 221416client # [ 0.478896] NetLabel: Initializing1417lb # [ 0.475584] ACPI: PCI: Interrupt link LNKF configured for IRQ 101418client # [ 0.479426] NetLabel: domain hash size = 1281419worker2 # [ 0.470162] ACPI: PCI: Interrupt link GSIH configured for IRQ 231420lb # [ 0.476590] ACPI: PCI: Interrupt link LNKG configured for IRQ 111421client # [ 0.480121] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1422lb # [ 0.477586] ACPI: PCI: Interrupt link LNKH configured for IRQ 111423client # [ 0.480461] NetLabel: unlabeled traffic allowed by default1424worker2 # [ 0.471953] iommu: Default domain type: Translated1425client # [ 0.481349] PCI: Using ACPI for IRQ routing1426lb # [ 0.478522] ACPI: PCI: Interrupt link GSIA configured for IRQ 161427worker2 # [ 0.472221] iommu: DMA domain TLB invalidation policy: lazy mode1428lb # [ 0.479443] ACPI: PCI: Interrupt link GSIB configured for IRQ 171429worker2 # [ 0.473410] ACPI: bus type USB registered1430worker2 # [ 0.474147] usbcore: registered new interface driver usbfs1431lb # [ 0.480285] ACPI: PCI: Interrupt link GSIC configured for IRQ 181432worker2 # [ 0.474883] usbcore: registered new interface driver hub1433lb # [ 0.481398] ACPI: PCI: Interrupt link GSID configured for IRQ 191434worker2 # [ 0.475221] usbcore: registered new device driver usb1435lb # [ 0.482271] ACPI: PCI: Interrupt link GSIE configured for IRQ 201436worker2 # [ 0.476896] NetLabel: Initializing1437lb # [ 0.483258] ACPI: PCI: Interrupt link GSIF configured for IRQ 211438worker2 # [ 0.477214] NetLabel: domain hash size = 1281439lb # [ 0.483504] ACPI: PCI: Interrupt link GSIG configured for IRQ 221440worker2 # [ 0.477933] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1441lb # [ 0.484478] ACPI: PCI: Interrupt link GSIH configured for IRQ 231442worker2 # [ 0.478297] NetLabel: unlabeled traffic allowed by default1443worker2 # [ 0.479161] PCI: Using ACPI for IRQ routing1444lb # [ 0.486435] iommu: Default domain type: Translated1445lb # [ 0.487134] iommu: DMA domain TLB invalidation policy: lazy mode1446lb # [ 0.487758] ACPI: bus type USB registered1447lb # [ 0.488519] usbcore: registered new interface driver usbfs1448lb # [ 0.489458] usbcore: registered new interface driver hub1449lb # [ 0.490183] usbcore: registered new device driver usb1450lb # [ 0.491321] NetLabel: Initializing1451lb # [ 0.491957] NetLabel: domain hash size = 1281452lb # [ 0.492488] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453lb # [ 0.493446] NetLabel: unlabeled traffic allowed by default1454lb # [ 0.494201] PCI: Using ACPI for IRQ routing1455worker1 # [ 0.517853] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456worker1 # [ 0.518634] pci 0000:00:01.0: vgaarb: bridge control possible1457worker1 # [ 0.518634] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458worker1 # [ 0.518645] vgaarb: loaded1459worker1 # [ 0.519346] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460worker1 # [ 0.519640] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461worker1 # [ 0.523719] clocksource: Switched to clocksource kvm-clock1462worker1 # [ 0.527061] VFS: Disk quotas dquot_6.6.01463worker1 # [ 0.527748] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464worker1 # [ 0.529124] pnp: PnP ACPI init1465worker1 # [ 0.529865] ACPI: IRQ 4 override to edge(!), high(!)1466client # [ 0.525387] pci 0000:00:01.0: vgaarb: setting as boot VGA device1467worker1 # [ 0.530758] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1468client # [ 0.525421] pci 0000:00:01.0: vgaarb: bridge control possible1469worker1 # [ 0.532292] pnp: PnP ACPI: found 5 devices1470client # [ 0.525421] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1471client # [ 0.525429] vgaarb: loaded1472client # [ 0.526171] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01473client # [ 0.526425] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1474worker2 # [ 0.522945] pci 0000:00:01.0: vgaarb: setting as boot VGA device1475client # [ 0.529510] clocksource: Switched to clocksource kvm-clock1476worker2 # [ 0.523209] pci 0000:00:01.0: vgaarb: bridge control possible1477worker1 # [ 0.539838] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1478worker1 # [ 0.541285] clocksource: Switched to clocksource acpi_pm1479worker2 # [ 0.523209] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1480worker2 # [ 0.523217] vgaarb: loaded1481worker1 # [ 0.542235] NET: Registered PF_INET protocol family1482client # [ 0.533047] VFS: Disk quotas dquot_6.6.01483worker2 # [ 0.523941] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01484worker1 # [ 0.543373] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1485worker2 # [ 0.524215] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1486client # [ 0.533752] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1487client # [ 0.535075] pnp: PnP ACPI init1488client # [ 0.535874] ACPI: IRQ 4 override to edge(!), high(!)1489client # [ 0.536867] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1490client # [ 0.538426] pnp: PnP ACPI: found 5 devices1491lb # [ 0.537374] pci 0000:00:01.0: vgaarb: setting as boot VGA device1492worker2 # [ 0.529318] clocksource: Switched to clocksource kvm-clock1493lb # [ 0.537484] pci 0000:00:01.0: vgaarb: bridge control possible1494lb # [ 0.537484] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1495lb # [ 0.537493] vgaarb: loaded1496lb # [ 0.538236] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01497worker2 # [ 0.532836] VFS: Disk quotas dquot_6.6.01498lb # [ 0.538488] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1499worker2 # [ 0.533587] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1500worker2 # [ 0.534925] pnp: PnP ACPI init1501worker2 # [ 0.535712] ACPI: IRQ 4 override to edge(!), high(!)1502client # [ 0.546036] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1503client # [ 0.547440] clocksource: Switched to clocksource acpi_pm1504worker2 # [ 0.536677] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1505lb # [ 0.541583] clocksource: Switched to clocksource kvm-clock1506client # [ 0.548407] NET: Registered PF_INET protocol family1507worker2 # [ 0.538204] pnp: PnP ACPI: found 5 devices1508lb # [ 0.543269] VFS: Disk quotas dquot_6.6.01509worker1 # [ 0.559496] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1510client # [ 0.549569] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1511worker1 # [ 0.560969] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1512lb # [ 0.545842] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1513worker1 # [ 0.562263] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1514lb # [ 0.547154] pnp: PnP ACPI init1515worker1 # [ 0.563573] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1516lb # [ 0.547912] ACPI: IRQ 4 override to edge(!), high(!)1517worker1 # [ 0.564850] TCP: Hash tables configured (established 16384 bind 16384)1518lb # [ 0.548823] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1519worker1 # [ 0.566014] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1520lb # [ 0.550322] pnp: PnP ACPI: found 5 devices1521worker2 # [ 0.545791] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1522worker1 # [ 0.567303] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1523worker2 # [ 0.547217] clocksource: Switched to clocksource acpi_pm1524worker1 # [ 0.568396] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1525worker2 # [ 0.548211] NET: Registered PF_INET protocol family1526worker1 # [ 0.569598] NET: Registered PF_UNIX/PF_LOCAL protocol family1527worker1 # [ 0.570566] NET: Registered PF_XDP protocol family1528worker2 # [ 0.549364] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1529worker1 # [ 0.571407] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1530worker1 # [ 0.572438] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1531worker1 # [ 0.573488] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1532lb # [ 0.557917] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1533worker1 # [ 0.574629] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1534lb # [ 0.559318] clocksource: Switched to clocksource acpi_pm1535worker1 # [ 0.575738] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1536client # [ 0.565688] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1537lb # [ 0.560251] NET: Registered PF_INET protocol family1538worker1 # [ 0.576819] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1539client # [ 0.567202] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1540lb # [ 0.561420] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1541worker1 # [ 0.578573] ACPI: \_SB_.GSIA: Enabled at IRQ 161542client # [ 0.568537] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1543client # [ 0.569945] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1544worker1 # [ 0.580718] ACPI: \_SB_.GSIB: Enabled at IRQ 171545client # [ 0.571273] TCP: Hash tables configured (established 16384 bind 16384)1546worker1 # [ 0.582633] ACPI: \_SB_.GSIC: Enabled at IRQ 181547client # [ 0.572481] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1548client # [ 0.573806] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1549worker1 # [ 0.584554] ACPI: \_SB_.GSID: Enabled at IRQ 191550client # [ 0.574966] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1551worker1 # [ 0.586250] PCI: CLS 0 bytes, default 641552client # [ 0.576233] NET: Registered PF_UNIX/PF_LOCAL protocol family1553worker1 # [ 0.587177] Trying to unpack rootfs image as initramfs...1554client # [ 0.577260] NET: Registered PF_XDP protocol family1555worker2 # [ 0.566142] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1556client # [ 0.578110] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1557worker2 # [ 0.567706] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1558client # [ 0.579153] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1559worker2 # [ 0.569077] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1560client # [ 0.580196] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1561client # [ 0.581312] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1562worker2 # [ 0.570503] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1563worker1 # [ 0.591213] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1564client # [ 0.582442] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1565worker2 # [ 0.571762] TCP: Hash tables configured (established 16384 bind 16384)1566client # [ 0.583557] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1567worker2 # [ 0.572936] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1568lb # [ 0.577893] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1569client # [ 0.585287] ACPI: \_SB_.GSIA: Enabled at IRQ 161570worker2 # [ 0.574212] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1571lb # [ 0.579400] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1572worker2 # [ 0.575329] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1573lb # [ 0.580670] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1574worker2 # [ 0.576607] NET: Registered PF_UNIX/PF_LOCAL protocol family1575client # [ 0.587787] ACPI: \_SB_.GSIB: Enabled at IRQ 171576worker2 # [ 0.577575] NET: Registered PF_XDP protocol family1577lb # [ 0.581953] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1578worker2 # [ 0.578413] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1579client # [ 0.589706] ACPI: \_SB_.GSIC: Enabled at IRQ 181580lb # [ 0.583161] TCP: Hash tables configured (established 16384 bind 16384)1581worker2 # [ 0.579440] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1582lb # [ 0.584379] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1583worker2 # [ 0.580511] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1584client # [ 0.591595] ACPI: \_SB_.GSID: Enabled at IRQ 191585lb # [ 0.585746] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1586worker2 # [ 0.581602] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1587client # [ 0.593314] PCI: CLS 0 bytes, default 641588lb # [ 0.586933] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1589worker2 # [ 0.582722] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1590lb # [ 0.588180] NET: Registered PF_UNIX/PF_LOCAL protocol family1591worker2 # [ 0.583810] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1592client # [ 0.594206] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1593lb # [ 0.589139] NET: Registered PF_XDP protocol family1594client # [ 0.595947] Trying to unpack rootfs image as initramfs...1595worker2 # [ 0.585534] ACPI: \_SB_.GSIA: Enabled at IRQ 161596lb # [ 0.590036] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1597lb # [ 0.591061] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1598worker2 # [ 0.587745] ACPI: \_SB_.GSIB: Enabled at IRQ 171599lb # [ 0.592124] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1600lb # [ 0.593274] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1601worker2 # [ 0.589681] ACPI: \_SB_.GSIC: Enabled at IRQ 181602lb # [ 0.594465] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1603lb # [ 0.595621] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1604worker2 # [ 0.591647] ACPI: \_SB_.GSID: Enabled at IRQ 191605lb # [ 0.597397] ACPI: \_SB_.GSIA: Enabled at IRQ 161606worker2 # [ 0.593319] PCI: CLS 0 bytes, default 641607worker2 # [ 0.594244] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1608lb # [ 0.599678] ACPI: \_SB_.GSIB: Enabled at IRQ 171609worker2 # [ 0.596037] Trying to unpack rootfs image as initramfs...1610lb # [ 0.601722] ACPI: \_SB_.GSIC: Enabled at IRQ 181611lb # [ 0.603719] ACPI: \_SB_.GSID: Enabled at IRQ 191612lb # [ 0.605429] PCI: CLS 0 bytes, default 641613lb # [ 0.606423] Trying to unpack rootfs image as initramfs...1614lb # [ 0.610810] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1615worker1 # [ 0.632840] Initialise system trusted keyrings1616worker1 # [ 0.636120] workingset: timestamp_bits=40 max_order=19 bucket_order=01617client # [ 0.641272] Initialise system trusted keyrings1618client # [ 0.644897] workingset: timestamp_bits=40 max_order=19 bucket_order=01619worker1 # [ 0.655341] Key type asymmetric registered1620worker1 # [ 0.660083] Asymmetric key parser 'x509' registered1621worker2 # [ 0.640581] Initialise system trusted keyrings1622worker1 # [ 0.660947] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1623worker1 # [ 0.664176] io scheduler mq-deadline registered1624worker2 # [ 0.643556] workingset: timestamp_bits=40 max_order=19 bucket_order=01625worker1 # [ 0.664972] io scheduler kyber registered1626worker1 # [ 0.667600] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1627lb # [ 0.651739] Initialise system trusted keyrings1628lb # [ 0.652720] workingset: timestamp_bits=40 max_order=19 bucket_order=01629worker1 # [ 0.668873] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630worker1 # [ 0.674827] Linux agpgart interface v0.1031631worker1 # [ 0.675624] ACPI: bus type drm_connector registered1632client # [ 0.666278] Key type asymmetric registered1633worker1 # [ 0.678204] usbcore: registered new interface driver usbserial_generic1634client # [ 0.668855] Asymmetric key parser 'x509' registered1635worker1 # [ 0.679330] usbserial: USB Serial support registered for generic1636client # [ 0.669715] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1637worker1 # [ 0.682092] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1638client # [ 0.672992] io scheduler mq-deadline registered1639worker1 # [ 0.682092] Please enable it if your BIOS has the CPPC option.1640client # [ 0.673795] io scheduler kyber registered1641worker1 # [ 0.684403] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1642worker2 # [ 0.664772] Key type asymmetric registered1643worker2 # [ 0.665718] Asymmetric key parser 'x509' registered1644client # [ 0.676461] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1645client # [ 0.677729] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1646worker1 # [ 0.688221] drop_monitor: Initializing network drop monitor service1647worker1 # [ 0.689453] NET: Registered PF_INET6 protocol family1648worker2 # [ 0.668545] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1649worker2 # [ 0.671543] io scheduler mq-deadline registered1650lb # [ 0.676555] Key type asymmetric registered1651worker2 # [ 0.672367] io scheduler kyber registered1652client # [ 0.683519] Linux agpgart interface v0.1031653lb # [ 0.677276] Asymmetric key parser 'x509' registered1654worker1 # [ 0.694182] Segment Routing with IPv61655client # [ 0.684282] ACPI: bus type drm_connector registered1656worker1 # [ 0.694880] In-situ OAM (IOAM) with IPv61657lb # [ 0.678134] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1658worker2 # [ 0.675106] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1659worker1 # [ 0.697349] IPI shorthand broadcast: enabled1660client # [ 0.687314] usbcore: registered new interface driver usbserial_generic1661worker2 # [ 0.676441] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1662lb # [ 0.681469] io scheduler mq-deadline registered1663lb # [ 0.682218] io scheduler kyber registered1664client # [ 0.688384] usbserial: USB Serial support registered for generic1665lb # [ 0.684946] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1666client # [ 0.690854] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1667client # [ 0.690854] Please enable it if your BIOS has the CPPC option.1668worker2 # [ 0.681920] Linux agpgart interface v0.1031669lb # [ 0.686193] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1670worker2 # [ 0.682688] ACPI: bus type drm_connector registered1671client # [ 0.693128] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1672worker1 # [ 0.704548] sched_clock: Marking stable (570014752, 134050340)->(776993345, -72928253)1673worker2 # [ 0.685955] usbcore: registered new interface driver usbserial_generic1674worker2 # [ 0.687074] usbserial: USB Serial support registered for generic1675lb # [ 0.691870] Linux agpgart interface v0.1031676client # [ 0.698026] drop_monitor: Initializing network drop monitor service1677lb # [ 0.692636] ACPI: bus type drm_connector registered1678client # [ 0.699254] NET: Registered PF_INET6 protocol family1679worker1 # [ 0.710152] registered taskstats version 11680worker2 # [ 0.689506] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1681worker1 # [ 0.711153] Loading compiled-in X.509 certificates1682worker2 # [ 0.689506] Please enable it if your BIOS has the CPPC option.1683client # [ 0.702303] Segment Routing with IPv61684lb # [ 0.695785] usbcore: registered new interface driver usbserial_generic1685worker2 # [ 0.691786] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1686lb # [ 0.696813] usbserial: USB Serial support registered for generic1687client # [ 0.704862] In-situ OAM (IOAM) with IPv61688worker2 # [ 0.694647] drop_monitor: Initializing network drop monitor service1689client # [ 0.705935] IPI shorthand broadcast: enabled1690lb # [ 0.699337] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1691worker2 # [ 0.695832] NET: Registered PF_INET6 protocol family1692lb # [ 0.699337] Please enable it if your BIOS has the CPPC option.1693lb # [ 0.701588] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1694lb # [ 0.704443] drop_monitor: Initializing network drop monitor service1695worker2 # [ 0.700970] Segment Routing with IPv61696lb # [ 0.705650] NET: Registered PF_INET6 protocol family1697worker2 # [ 0.701635] In-situ OAM (IOAM) with IPv61698client # [ 0.714111] sched_clock: Marking stable (580013925, 133816340)->(784696106, -70865841)1699worker2 # [ 0.704785] IPI shorthand broadcast: enabled1700lb # [ 0.710807] Segment Routing with IPv61701worker1 # [ 0.727511] Demotion targets for Node 0: null1702client # [ 0.718101] registered taskstats version 11703client # [ 0.719203] Loading compiled-in X.509 certificates1704lb # [ 0.713344] In-situ OAM (IOAM) with IPv61705lb # [ 0.714416] IPI shorthand broadcast: enabled1706worker1 # [ 0.731111] Key type .fscrypt registered1707worker1 # [ 0.731802] Key type fscrypt-provisioning registered1708worker1 # [ 0.732755] ima: No TPM chip found, activating TPM-bypass!1709worker2 # [ 0.713152] sched_clock: Marking stable (578013983, 134474038)->(786426667, -73938646)1710worker1 # [ 0.735076] ima: Allocated hash algorithm: sha11711worker1 # [ 0.735860] ima: No architecture policies found1712worker2 # [ 0.716696] registered taskstats version 11713worker1 # [ 0.738075] PM: Magic number: 10:943:8391714worker2 # [ 0.717740] Loading compiled-in X.509 certificates1715lb # [ 0.722688] sched_clock: Marking stable (574013951, 148295899)->(790798886, -68489036)1716worker1 # [ 0.739680] RAS: Correctable Errors collector initialized.1717lb # [ 0.727508] registered taskstats version 11718lb # [ 0.728595] Loading compiled-in X.509 certificates1719client # [ 0.737423] Demotion targets for Node 0: null1720worker1 # [ 0.748726] clk: Disabling unused clocks1721client # [ 0.740885] Key type .fscrypt registered1722client # [ 0.741616] Key type fscrypt-provisioning registered1723client # [ 0.742590] ima: No TPM chip found, activating TPM-bypass!1724worker1 # [ 0.753100] PM: genpd: Disabling unused power domains1725client # [ 0.744843] ima: Allocated hash algorithm: sha11726client # [ 0.745678] ima: No architecture policies found1727worker2 # [ 0.736508] Demotion targets for Node 0: null1728client # [ 0.747843] PM: Magic number: 10:943:8391729worker2 # [ 0.737400] Key type .fscrypt registered1730client # [ 0.749546] RAS: Correctable Errors collector initialized.1731worker2 # [ 0.740493] Key type fscrypt-provisioning registered1732worker2 # [ 0.741468] ima: No TPM chip found, activating TPM-bypass!1733worker2 # [ 0.742432] ima: Allocated hash algorithm: sha11734lb # [ 0.747318] Demotion targets for Node 0: null1735lb # [ 0.748189] Key type .fscrypt registered1736worker2 # [ 0.744523] ima: No architecture policies found1737lb # [ 0.750321] Key type fscrypt-provisioning registered1738lb # [ 0.751328] ima: No TPM chip found, activating TPM-bypass!1739worker2 # [ 0.747504] PM: Magic number: 10:943:8391740lb # [ 0.752271] ima: Allocated hash algorithm: sha11741worker2 # [ 0.749143] RAS: Correctable Errors collector initialized.1742client # [ 0.760642] clk: Disabling unused clocks1743lb # [ 0.756352] ima: No architecture policies found1744client # [ 0.762870] PM: genpd: Disabling unused power domains1745lb # [ 0.757385] PM: Magic number: 10:943:8391746lb # [ 0.759182] RAS: Correctable Errors collector initialized.1747worker2 # [ 0.758262] clk: Disabling unused clocks1748worker2 # [ 0.761523] PM: genpd: Disabling unused power domains1749lb # [ 0.767994] clk: Disabling unused clocks1750lb # [ 0.772343] PM: genpd: Disabling unused power domains1751worker1 # [ 0.914189] Freeing initrd memory: 29056K1752worker1 # [ 0.917386] Freeing unused decrypted memory: 2028K1753worker1 # [ 0.919977] Freeing unused kernel image (initmem) memory: 3652K1754worker1 # [ 0.921116] Write protecting the kernel read-only data: 32768k1755worker1 # [ 0.922959] Freeing unused kernel image (text/rodata gap) memory: 1184K1756worker1 # [ 0.924510] Freeing unused kernel image (rodata/data gap) memory: 720K1757client # [ 0.922721] Freeing initrd memory: 29056K1758client # [ 0.925875] Freeing unused decrypted memory: 2028K1759client # [ 0.928476] Freeing unused kernel image (initmem) memory: 3652K1760client # [ 0.929574] Write protecting the kernel read-only data: 32768k1761client # [ 0.931415] Freeing unused kernel image (text/rodata gap) memory: 1184K1762client # [ 0.932961] Freeing unused kernel image (rodata/data gap) memory: 720K1763worker2 # [ 0.924518] Freeing initrd memory: 29060K1764worker2 # [ 0.927872] Freeing unused decrypted memory: 2028K1765worker2 # [ 0.930660] Freeing unused kernel image (initmem) memory: 3652K1766lb # [ 0.935410] Freeing initrd memory: 29064K1767worker2 # [ 0.931761] Write protecting the kernel read-only data: 32768k1768worker2 # [ 0.933681] Freeing unused kernel image (text/rodata gap) memory: 1184K1769lb # [ 0.938721] Freeing unused decrypted memory: 2028K1770worker2 # [ 0.935220] Freeing unused kernel image (rodata/data gap) memory: 720K1771lb # [ 0.941492] Freeing unused kernel image (initmem) memory: 3652K1772lb # [ 0.942606] Write protecting the kernel read-only data: 32768k1773lb # [ 0.944491] Freeing unused kernel image (text/rodata gap) memory: 1184K1774lb # [ 0.946044] Freeing unused kernel image (rodata/data gap) memory: 720K1775worker1 # [ 0.975474] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776worker1 # [ 0.976573] Run /init as init process1777worker1 # [ 0.987016] systemd[1]: Inserted module 'autofs4'1778client # [ 0.983841] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779client # [ 0.984952] Run /init as init process1780client # [ 0.995796] systemd[1]: Inserted module 'autofs4'1781worker1 # [ 1.006654] fuse: init (API version 7.45)1782worker2 # [ 0.986338] x86/mm: Checked W+X mappings: passed, no W+X pages found.1783worker2 # [ 0.987509] Run /init as init process1784lb # [ 0.996799] x86/mm: Checked W+X mappings: passed, no W+X pages found.1785lb # [ 0.997816] Run /init as init process1786worker1 # [ 1.014261] ACPI: \_SB_.GSIG: Enabled at IRQ 221787worker1 # [ 1.016496] ACPI: \_SB_.GSIH: Enabled at IRQ 231788worker2 # [ 0.998351] systemd[1]: Inserted module 'autofs4'1789worker1 # [ 1.019454] ACPI: \_SB_.GSIE: Enabled at IRQ 201790worker1 # [ 1.021347] ACPI: \_SB_.GSIF: Enabled at IRQ 211791lb # [ 1.008580] systemd[1]: Inserted module 'autofs4'1792client # [ 1.015480] fuse: init (API version 7.45)1793worker1 # [ 1.026571] virtiofs virtio5: discovered new tag: nix-store1794worker1 # [ 1.028162] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1795client # [ 1.022868] ACPI: \_SB_.GSIG: Enabled at IRQ 221796worker1 # [ 1.035255] virtiofs virtio6: discovered new tag: shared1797client # [ 1.025372] ACPI: \_SB_.GSIH: Enabled at IRQ 231798worker1 # [ 1.036898] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1799client # [ 1.028689] ACPI: \_SB_.GSIE: Enabled at IRQ 201800worker2 # [ 1.018284] fuse: init (API version 7.45)1801worker1 # [ 1.040940] virtiofs virtio7: discovered new tag: xchg1802client # [ 1.030964] ACPI: \_SB_.GSIF: Enabled at IRQ 211803worker1 # [ 1.042480] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1804lb # [ 1.028611] fuse: init (API version 7.45)1805client # [ 1.036213] virtiofs virtio5: discovered new tag: nix-store1806worker2 # [ 1.025848] ACPI: \_SB_.GSIG: Enabled at IRQ 221807client # [ 1.037792] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1808worker2 # [ 1.028221] ACPI: \_SB_.GSIH: Enabled at IRQ 231809worker2 # [ 1.031082] ACPI: \_SB_.GSIE: Enabled at IRQ 201810lb # [ 1.035938] ACPI: \_SB_.GSIG: Enabled at IRQ 221811worker2 # [ 1.033115] ACPI: \_SB_.GSIF: Enabled at IRQ 211812lb # [ 1.038477] ACPI: \_SB_.GSIH: Enabled at IRQ 231813client # [ 1.046306] virtiofs virtio6: discovered new tag: shared1814lb # [ 1.041543] ACPI: \_SB_.GSIE: Enabled at IRQ 201815client # [ 1.047976] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1816worker2 # [ 1.038245] virtiofs virtio5: discovered new tag: nix-store1817lb # [ 1.043498] ACPI: \_SB_.GSIF: Enabled at IRQ 211818worker2 # [ 1.039862] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1819client # [ 1.051355] virtiofs virtio7: discovered new tag: xchg1820client # [ 1.052895] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1821worker1 # [ 1.063830] systemd[1]: Successfully made /usr/ read-only.1822lb # [ 1.048954] virtiofs virtio5: discovered new tag: nix-store1823lb # [ 1.050589] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1824worker2 # [ 1.047922] virtiofs virtio6: discovered new tag: shared1825worker2 # [ 1.049739] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1826lb # [ 1.057748] virtiofs virtio6: discovered new tag: shared1827worker2 # [ 1.053555] virtiofs virtio7: discovered new tag: xchg1828worker2 # [ 1.055113] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1829lb # [ 1.059504] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1830lb # [ 1.062925] virtiofs virtio7: discovered new tag: xchg1831lb # [ 1.064385] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1832client # [ 1.078596] systemd[1]: Successfully made /usr/ read-only.1833worker2 # [ 1.076599] systemd[1]: Successfully made /usr/ read-only.1834lb # [ 1.085420] systemd[1]: Successfully made /usr/ read-only.1835worker1 # [ 1.399373] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836worker1 # [ 1.407366] systemd[1]: Detected virtualization kvm.1837worker1 # [ 1.409499] systemd[1]: Detected architecture x86-64.1838worker1 # [ 1.411643] systemd[1]: Running in initrd.1839worker1 # [ 1.414226] systemd[1]: Initializing machine ID from random generator.1840worker1 # [ 1.417309] systemd[1]: Hostname set to <worker1>.1841client # [ 1.414137] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842client # [ 1.419193] systemd[1]: Detected virtualization kvm.1843client # [ 1.420070] systemd[1]: Detected architecture x86-64.1844client # [ 1.421004] systemd[1]: Running in initrd.1845client # [ 1.422115] systemd[1]: Initializing machine ID from random generator.1846client # [ 1.423352] systemd[1]: Hostname set to <client>.1847worker2 # [ 1.413809] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848worker2 # [ 1.427137] systemd[1]: Detected virtualization kvm.1849lb # [ 1.422007] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1850worker2 # [ 1.429401] systemd[1]: Detected architecture x86-64.1851worker2 # [ 1.430570] systemd[1]: Running in initrd.1852lb # [ 1.434006] systemd[1]: Detected virtualization kvm.1853worker2 # [ 1.432016] systemd[1]: Initializing machine ID from random generator.1854lb # [ 1.436205] systemd[1]: Detected architecture x86-64.1855worker2 # [ 1.433639] systemd[1]: Hostname set to <worker2>.1856lb # [ 1.438367] systemd[1]: Running in initrd.1857lb # [ 1.440885] systemd[1]: Initializing machine ID from random generator.1858lb # [ 1.443774] systemd[1]: Hostname set to <lb>.1859client # [ 1.620789] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860worker1 # [ 1.652688] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861lb # [ 1.652011] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862worker2 # [ 1.651366] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863client # [ 1.664358] systemd[1]: Queued start job for default target Initrd Default Target.1864client # [ 1.668153] systemd[1]: Created slice Slice /system/modprobe.1865client # [ 1.669328] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866client # [ 1.670687] systemd[1]: Expecting device /dev/disk/by-label/nixos...1867client # [ 1.671767] systemd[1]: Reached target Path Units.1868client # [ 1.672600] systemd[1]: Reached target Slice Units.1869client # [ 1.673504] systemd[1]: Reached target Swaps.1870client # [ 1.674281] systemd[1]: Reached target Timer Units.1871client # [ 1.675247] systemd[1]: Listening on D-Bus System Message Bus Socket.1872client # [ 1.676440] systemd[1]: Listening on Journal Socket (/dev/log).1873client # [ 1.677568] systemd[1]: Listening on Journal Sockets.1874client # [ 1.678518] systemd[1]: Listening on udev Control Socket.1875client # [ 1.679479] systemd[1]: Listening on udev Kernel Socket.1876client # [ 1.680394] systemd[1]: Reached target Socket Units.1877client # [ 1.682162] systemd[1]: Starting Create List of Static Device Nodes...1878client # [ 1.685902] systemd[1]: Starting Load Kernel Module configfs...1879lb # [ 1.691818] systemd[1]: Queued start job for default target Initrd Default Target.1880lb # [ 1.695738] systemd[1]: Created slice Slice /system/modprobe.1881lb # [ 1.696985] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1882worker1 # [ 1.713915] systemd[1]: Queued start job for default target Initrd Default Target.1883lb # [ 1.698416] systemd[1]: Expecting device /dev/disk/by-label/nixos...1884lb # [ 1.699525] systemd[1]: Reached target Path Units.1885lb # [ 1.700456] systemd[1]: Reached target Slice Units.1886client # [ 1.695061] systemd[1]: Starting Journal Service...1887lb # [ 1.701364] systemd[1]: Reached target Swaps.1888worker1 # [ 1.718342] systemd[1]: Created slice Slice /system/modprobe.1889lb # [ 1.702184] systemd[1]: Reached target Timer Units.1890lb # [ 1.703188] systemd[1]: Listening on D-Bus System Message Bus Socket.1891worker1 # [ 1.719505] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1892lb # [ 1.704425] systemd[1]: Listening on Journal Socket (/dev/log).1893worker1 # [ 1.720857] systemd[1]: Expecting device /dev/disk/by-label/nixos...1894worker1 # [ 1.721925] systemd[1]: Reached target Path Units.1895lb # [ 1.705603] systemd[1]: Listening on Journal Sockets.1896worker2 # [ 1.701049] systemd[1]: Queued start job for default target Initrd Default Target.1897worker1 # [ 1.722790] systemd[1]: Reached target Slice Units.1898lb # [ 1.706649] systemd[1]: Listening on udev Control Socket.1899worker1 # [ 1.723702] systemd[1]: Reached target Swaps.1900lb # [ 1.707695] systemd[1]: Listening on udev Kernel Socket.1901worker1 # [ 1.724512] systemd[1]: Reached target Timer Units.1902lb # [ 1.708676] systemd[1]: Reached target Socket Units.1903worker1 # [ 1.725474] systemd[1]: Listening on D-Bus System Message Bus Socket.1904worker2 # [ 1.704878] systemd[1]: Created slice Slice /system/modprobe.1905worker1 # [ 1.726684] systemd[1]: Listening on Journal Socket (/dev/log).1906lb # [ 1.710522] systemd[1]: Starting Create List of Static Device Nodes...1907worker2 # [ 1.706011] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1908worker1 # [ 1.727830] systemd[1]: Listening on Journal Sockets.1909worker2 # [ 1.707391] systemd[1]: Expecting device /dev/disk/by-label/nixos...1910worker1 # [ 1.728866] systemd[1]: Listening on udev Control Socket.1911worker2 # [ 1.708446] systemd[1]: Reached target Path Units.1912worker1 # [ 1.729963] systemd[1]: Listening on udev Kernel Socket.1913worker2 # [ 1.709381] systemd[1]: Reached target Slice Units.1914lb # [ 1.714156] systemd[1]: Starting Load Kernel Module configfs...1915worker2 # [ 1.710223] systemd[1]: Reached target Swaps.1916worker1 # [ 1.730966] systemd[1]: Reached target Socket Units.1917client # [ 1.721029] systemd[1]: Starting Load Kernel Modules...1918worker2 # [ 1.710970] systemd[1]: Reached target Timer Units.1919worker2 # [ 1.711898] systemd[1]: Listening on D-Bus System Message Bus Socket.1920worker1 # [ 1.732742] systemd[1]: Starting Create List of Static Device Nodes...1921worker2 # [ 1.713052] systemd[1]: Listening on Journal Socket (/dev/log).1922worker2 # [ 1.714146] systemd[1]: Listening on Journal Sockets.1923client # [ 1.724944] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1924worker2 # [ 1.715101] systemd[1]: Listening on udev Control Socket.1925worker2 # [ 1.716069] systemd[1]: Listening on udev Kernel Socket.1926worker1 # [ 1.737130] systemd[1]: Starting Load Kernel Module configfs...1927worker2 # [ 1.716976] systemd[1]: Reached target Socket Units.1928worker2 # [ 1.718782] systemd[1]: Starting Create List of Static Device Nodes...1929client # [ 1.732905] systemd[1]: Starting Coldplug All udev Devices...1930worker2 # [ 1.722399] systemd[1]: Starting Load Kernel Module configfs...1931lb # [ 1.725032] systemd[1]: Starting Journal Service...1932client # [ 1.744393] systemd-journald[66]: Collecting audit messages is disabled.1933client # [ 1.745800] systemd[1]: Finished Create List of Static Device Nodes.1934worker1 # [ 1.748139] systemd[1]: Starting Journal Service...1935client # [ 1.753280] systemd[1]: modprobe@configfs.service: Deactivated successfully.1936worker2 # [ 1.731560] systemd[1]: Starting Journal Service...1937lb # [ 1.748927] systemd[1]: Starting Load Kernel Modules...1938client # [ 1.758948] systemd[1]: Finished Load Kernel Module configfs.1939lb # [ 1.753440] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1940worker1 # [ 1.773258] systemd[1]: Starting Load Kernel Modules...1941client # [ 1.764254] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1942worker1 # [ 1.777166] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1943worker2 # [ 1.758120] systemd[1]: Starting Load Kernel Modules...1944lb # [ 1.762565] systemd[1]: Starting Coldplug All udev Devices...1945worker2 # [ 1.762627] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1946worker1 # [ 1.787259] systemd[1]: Starting Coldplug All udev Devices...1947client # [ 1.777898] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1948lb # [ 1.773873] systemd-journald[66]: Collecting audit messages is disabled.1949worker2 # [ 1.772317] systemd[1]: Starting Coldplug All udev Devices...1950lb # [ 1.778378] systemd[1]: Finished Create List of Static Device Nodes.1951client # [ 1.784255] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1952worker1 # [ 1.795871] systemd-journald[66]: Collecting audit messages is disabled.1953lb # [ 1.780995] systemd[1]: modprobe@configfs.service: Deactivated successfully.1954worker1 # [ 1.803138] systemd[1]: Finished Create List of Static Device Nodes.1955worker2 # [ 1.782597] systemd-journald[66]: Collecting audit messages is disabled.1956client # [ 1.795021] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1957lb # [ 1.789750] systemd[1]: Finished Load Kernel Module configfs.1958worker2 # [ 1.786569] systemd[1]: Finished Create List of Static Device Nodes.1959worker1 # [ 1.809763] systemd[1]: modprobe@configfs.service: Deactivated successfully.1960worker2 # [ 1.791756] systemd[1]: modprobe@configfs.service: Deactivated successfully.1961lb # [ 1.796752] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1962worker1 # [ 1.816513] systemd[1]: Finished Load Kernel Module configfs.1963worker2 # [ 1.797893] systemd[1]: Finished Load Kernel Module configfs.1964worker1 # [ 1.822494] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1965lb # [ 1.808385] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1966worker2 # [ 1.803887] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1967client # [ 1.818405] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1968lb # [ 1.816167] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1969worker1 # [ 1.835175] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1970worker2 # [ 1.815685] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1971client # [ 1.827045] systemd[1]: Starting Create Static Device Nodes in /dev...1972lb # [ 1.827502] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1973worker1 # [ 1.843469] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1974client # [ 1.838355] systemd[1]: Finished Load Kernel Modules.1975worker2 # [ 1.827844] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1976worker1 # [ 1.851378] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1977client # [ 1.846228] systemd[1]: Starting Apply Kernel Variables...1978worker2 # [ 1.836538] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1979lb # [ 1.849977] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1980client # [ 1.864651] systemd[1]: Finished Create Static Device Nodes in /dev.1981lb # [ 1.858509] systemd[1]: Starting Create Static Device Nodes in /dev...1982client # [ 1.732437] systemd-modules-load[68]: Inserted module 'dm_mod'1983worker2 # [ 1.858095] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1984client # [ 1.736383] systemd-modules-load[68]: Inserted module 'virtio_balloon'1985client # [ 1.737461] systemd-modules-load[68]: Inserted module 'virtio_gpu'1986worker1 # [ 1.882152] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1987client # [ 1.873642] systemd[1]: Started Journal Service.1988worker1 # [ 1.888598] systemd[1]: Starting Create Static Device Nodes in /dev...1989lb # [ 1.872460] systemd[1]: Finished Load Kernel Modules.1990client # [ 1.747182] systemd[1]: Finished Apply Kernel Variables.1991worker2 # [ 1.871571] systemd[1]: Starting Create Static Device Nodes in /dev...1992client # [ 1.750903] systemd[1]: Reached target Preparation for Local File Systems.1993lb # [ 1.880509] systemd[1]: Starting Apply Kernel Variables...1994client # [ 1.753905] systemd[1]: Reached target Local File Systems.1995client # [ 1.755153] systemd[1]: Starting Create System Files and Directories...1996worker1 # [ 1.899135] systemd[1]: Finished Load Kernel Modules.1997worker2 # [ 1.879029] systemd[1]: Finished Load Kernel Modules.1998client # [ 1.762504] systemd[1]: Starting Rule-based Manager for Device Events and Files...1999worker1 # [ 1.907658] systemd[1]: Starting Apply Kernel Variables...2000lb # [ 1.895448] systemd[1]: Started Journal Service.
loading 2,898 hidden lines…
4899lb # [ 49.252899] niks3-server[1059]: 2026/09/22 14:51:19 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4900lb # [ 49.259244] niks3-server[1059]: 2026/09/22 14:51:19 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4901lb # [ 49.261608] niks3-server[1059]: 2026/09/22 14:51:19 INFO Completed upload id=64902worker2 # [ 49.272319] nix-grpc-daemon[884]: time=2026-09-22T14:51:21.298Z level=INFO msg="Upload complete. (184ms)"4903lb # [ 49.264463] niks3-server[1059]: 2026/09/22 14:51:19 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4904worker2 # [ 49.275659] nix-grpc-daemon[782]: ts=2026-09-22T14:51:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14905worker2 # [ 49.284780] nix-daemon[869]: reaped child process 981, status = succeeded4906client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 5.90 seconds)4907client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4908client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds)4909client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4910client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4911client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4912client # copying 3 paths...4913client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4914client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4915client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4916client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.17 seconds)4917(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.26 seconds)4918subtest: repeat is answered Cached by the scheduler without building4919worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4920worker1 # finding garbage collector roots...4921worker1 # deleting specified paths...4922worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4923worker1 # deleting unused links...4924worker1 # note: hard linking is currently saving 0.0 KiB4925worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.18 seconds)4926worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4927worker2 # finding garbage collector roots...4928worker2 # deleting specified paths...4929worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4930worker2 # deleting unused links...4931worker2 # note: hard linking is currently saving 0.0 KiB4932worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds)4933client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24934worker1 # [ 50.136297] nix-grpc-daemon[781]: ts=2026-09-22T14:51:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575504935client # don't know how to build these paths:4936client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4937worker2 # [ 50.123972] nix-grpc-daemon[782]: ts=2026-09-22T14:51:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375264938worker2 # [ 50.129691] nix-grpc-daemon[884]: time=2026-09-22T14:51:22.156Z level=INFO msg="All 2 paths already cached"4939worker2 # [ 50.141930] nix-grpc-daemon[782]: ts=2026-09-22T14:51:22Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14940client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.16 seconds)4941client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4942client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.09 seconds)4943lb: must succeed: curl -sf localhost:9901/clusters4944lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4945lb: must succeed: curl -sf localhost:9901/clusters4946lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4947worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4948worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4949(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.73 seconds)4950subtest: inputs already in the cache are not sent to the scheduler4951lb: must succeed: curl -sf localhost:9901/clusters4952lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4953lb: must succeed: curl -sf localhost:9901/clusters4954lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4955worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4956worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.02 seconds)4957client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24958client # don't know how to build these paths:4959worker2 # [ 50.574909] nix-grpc-daemon[782]: ts=2026-09-22T14:51:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375264960client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4961worker2 # [ 50.578230] nix-grpc-daemon[782]: ts=2026-09-22T14:51:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375264962worker2 # [ 50.935563] nix-grpc-daemon[884]: time=2026-09-22T14:51:22.961Z level=INFO msg="All 2 paths already cached"4963worker2 # [ 50.952744] nix-grpc-daemon[782]: ts=2026-09-22T14:51:22Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04964worker2 # [ 50.957843] nix-grpc-daemon[782]: ts=2026-09-22T14:51:22Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14965worker2 # [ 50.997945] nix-grpc-daemon[782]: ts=2026-09-22T14:51:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375264966worker2 # [ 51.010672] nix-grpc-daemon[782]: ts=2026-09-22T14:51:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375264967worker2 # [ 51.027086] nix-grpc-daemon[884]: time=2026-09-22T14:51:23.053Z level=INFO msg="All 2 paths already cached"4968client # copying 1 paths to 'lb:50051'...4969client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4970worker2 # [ 51.036899] nix-grpc-daemon[782]: ts=2026-09-22T14:51:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375264971worker2 # [ 51.045842] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4972worker2 # [ 51.151072] nix-daemon[869]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4973worker2 # [ 51.165719] nix-daemon[869]: reaped child process 1034, status = succeeded4974worker2 # [ 51.252581] nix-daemon[869]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)4975worker2 # [ 51.267205] nix-daemon[869]: reaped child process 1039, status = succeeded4976lb # [ 51.266996] niks3-server[1059]: 2026/09/22 14:51:21 INFO Received uploads request method=POST path=/api/pending_closures4977worker2 # [ 51.293419] nix-grpc-daemon[884]: time=2026-09-22T14:51:23.319Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4978worker2 # [ 51.295083] nix-grpc-daemon[884]: time=2026-09-22T14:51:23.321Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4979lb # [ 51.305845] niks3-server[1059]: 2026/09/22 14:51:21 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4980lb # [ 51.316068] niks3-server[1059]: 2026/09/22 14:51:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4981lb # [ 51.323098] niks3-server[1059]: 2026/09/22 14:51:21 INFO Signed narinfos id=7 count=14982worker2 # [ 51.336600] nix-grpc-daemon[884]: time=2026-09-22T14:51:23.363Z level=INFO msg="Uploading 1 narinfos"4983lb # [ 51.330268] niks3-server[1059]: 2026/09/22 14:51:21 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4984lb # [ 51.342557] niks3-server[1059]: 2026/09/22 14:51:21 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4985lb # [ 51.349639] niks3-server[1059]: 2026/09/22 14:51:21 INFO Completed upload id=74986worker2 # [ 51.359314] nix-grpc-daemon[884]: time=2026-09-22T14:51:23.385Z level=INFO msg="Upload complete. (298ms)"4987lb # [ 51.353944] niks3-server[1059]: 2026/09/22 14:51:21 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4988worker2 # [ 51.364086] nix-grpc-daemon[782]: ts=2026-09-22T14:51:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_in=7524989worker2 # [ 51.370304] nix-grpc-daemon[782]: ts=2026-09-22T14:51:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375264990worker2 # [ 51.383428] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4991worker2 # [ 51.393314] nix-daemon[869]: reaped child process 1029, status = succeeded4992client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4993client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4994worker2 # [ 51.399610] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4995client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4996client # farm-top> LOG-farm-top-t1b4997worker2 # [ 52.615773] nix-daemon[869]: reaped child process 1043, status = succeeded4998worker2 # [ 52.698267] nix-daemon[869]: accepted connection from pid 1059, user nix-grpc-daemon (trusted)4999worker2 # [ 52.713145] nix-daemon[869]: reaped child process 1062, status = succeeded5000lb # [ 52.711693] niks3-server[1059]: 2026/09/22 14:51:23 INFO Received uploads request method=POST path=/api/pending_closures5001worker2 # [ 52.730161] nix-grpc-daemon[884]: time=2026-09-22T14:51:24.756Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5002worker2 # [ 52.731677] nix-grpc-daemon[884]: time=2026-09-22T14:51:24.758Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"5003lb # [ 52.754809] niks3-server[1059]: 2026/09/22 14:51:23 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst5004lb # [ 52.761916] niks3-server[1059]: 2026/09/22 14:51:23 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv5005lb # [ 52.771593] niks3-server[1059]: 2026/09/22 14:51:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign5006lb # [ 52.777526] niks3-server[1059]: 2026/09/22 14:51:23 INFO Signed narinfos id=8 count=15007worker2 # [ 52.791056] nix-grpc-daemon[884]: time=2026-09-22T14:51:24.817Z level=INFO msg="Uploading 1 narinfos"5008lb # [ 52.784135] niks3-server[1059]: 2026/09/22 14:51:23 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls5009lb # [ 52.791054] niks3-server[1059]: 2026/09/22 14:51:23 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete5010lb # [ 52.794769] niks3-server[1059]: 2026/09/22 14:51:23 INFO Completed upload id=85011lb # [ 52.796728] niks3-server[1059]: 2026/09/22 14:51:23 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo5012worker2 # [ 52.805850] nix-grpc-daemon[884]: time=2026-09-22T14:51:24.832Z level=INFO msg="Upload complete. (199ms)"5013worker2 # [ 52.815556] nix-grpc-daemon[782]: ts=2026-09-22T14:51:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=15014worker2 # [ 52.829274] nix-daemon[869]: reaped child process 1041, status = succeeded5015client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.42 seconds)5016client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath5017client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.12 seconds)5018lb: must succeed: curl -sf localhost:9901/clusters5019lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5020lb: must succeed: curl -sf localhost:9901/clusters5021lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5022worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true5023worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)5024(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.69 seconds)5025subtest: outputs read back through any worker carry the cache signature5026client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&25027client # don't know how to build these paths:5028client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5029worker1 # [ 53.236311] nix-grpc-daemon[781]: ts=2026-09-22T14:51:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575505030worker1 # [ 53.242155] nix-grpc-daemon[781]: ts=2026-09-22T14:51:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505031worker2 # [ 53.236208] nix-grpc-daemon[782]: ts=2026-09-22T14:51:25Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=05032worker2 # [ 53.240245] nix-grpc-daemon[782]: ts=2026-09-22T14:51:25Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=15033worker1 # [ 53.268161] nix-grpc-daemon[781]: ts=2026-09-22T14:51:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505034worker1 # [ 53.279897] nix-grpc-daemon[781]: ts=2026-09-22T14:51:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505035client # copying 1 paths to 'lb:50051'...5036client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...5037worker1 # [ 53.290881] nix-grpc-daemon[781]: ts=2026-09-22T14:51:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575505038worker1 # [ 53.302194] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5039worker1 # [ 53.322971] nix-grpc-daemon[781]: ts=2026-09-22T14:51:24Z level=info event=niks3_push_started pid=8745040worker1 # [ 53.450146] nix-grpc-daemon[874]: time=2026-09-22T14:51:24.647Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5041worker1 # [ 53.540135] nix-daemon[849]: accepted connection from pid 887, user nix-grpc-daemon (trusted)5042worker1 # [ 53.554516] nix-daemon[849]: reaped child process 890, status = succeeded5043worker1 # [ 53.636674] nix-daemon[849]: accepted connection from pid 892, user nix-grpc-daemon (trusted)5044worker1 # [ 53.650214] nix-daemon[849]: reaped child process 895, status = succeeded5045lb # [ 53.624354] niks3-server[1059]: 2026/09/22 14:51:24 INFO Received uploads request method=POST path=/api/pending_closures5046worker1 # [ 53.663329] nix-grpc-daemon[874]: time=2026-09-22T14:51:24.861Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5047worker1 # [ 53.665650] nix-grpc-daemon[874]: time=2026-09-22T14:51:24.862Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"5048lb # [ 53.657684] niks3-server[1059]: 2026/09/22 14:51:24 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst5049lb # [ 53.667424] niks3-server[1059]: 2026/09/22 14:51:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign5050lb # [ 53.672942] niks3-server[1059]: 2026/09/22 14:51:24 INFO Signed narinfos id=9 count=15051worker1 # [ 53.706694] nix-grpc-daemon[874]: time=2026-09-22T14:51:24.904Z level=INFO msg="Uploading 1 narinfos"5052lb # [ 53.689676] niks3-server[1059]: 2026/09/22 14:51:24 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete5053lb # [ 53.697375] niks3-server[1059]: 2026/09/22 14:51:24 INFO Completed upload id=95054worker1 # [ 53.730805] nix-grpc-daemon[874]: time=2026-09-22T14:51:24.928Z level=INFO msg="Upload complete. (276ms)"5055lb # [ 53.700683] niks3-server[1059]: 2026/09/22 14:51:24 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls5056worker1 # [ 53.732990] nix-grpc-daemon[781]: ts=2026-09-22T14:51:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=1 nar_bytes_in=5045057lb # [ 53.704321] niks3-server[1059]: 2026/09/22 14:51:24 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo5058worker1 # [ 53.739597] nix-daemon[849]: reaped child process 872, status = succeeded5059worker1 # [ 53.741328] nix-grpc-daemon[781]: ts=2026-09-22T14:51:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505060worker1 # [ 53.745332] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5061client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...5062client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv5063worker1 # [ 53.750715] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5064client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5065client # farm-a> LOG-farm-a-sig5066worker1 # [ 55.166889] nix-daemon[849]: reaped child process 899, status = succeeded5067worker1 # [ 55.240349] nix-daemon[849]: accepted connection from pid 915, user nix-grpc-daemon (trusted)5068worker1 # [ 55.252766] nix-daemon[849]: reaped child process 918, status = succeeded5069lb # [ 55.231793] niks3-server[1059]: 2026/09/22 14:51:25 INFO Received uploads request method=POST path=/api/pending_closures5070worker1 # [ 55.269864] nix-grpc-daemon[874]: time=2026-09-22T14:51:26.467Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5071worker1 # [ 55.271550] nix-grpc-daemon[874]: time=2026-09-22T14:51:26.469Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"5072lb # [ 55.270267] niks3-server[1059]: 2026/09/22 14:51:25 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv5073lb # [ 55.283456] niks3-server[1059]: 2026/09/22 14:51:25 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst5074lb # [ 55.298551] niks3-server[1059]: 2026/09/22 14:51:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign5075worker1 # [ 55.337164] nix-grpc-daemon[874]: time=2026-09-22T14:51:26.534Z level=INFO msg="Uploading 1 narinfos"5076lb # [ 55.305445] niks3-server[1059]: 2026/09/22 14:51:25 INFO Signed narinfos id=10 count=15077lb # [ 55.312431] niks3-server[1059]: 2026/09/22 14:51:25 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls5078lb # [ 55.318659] niks3-server[1059]: 2026/09/22 14:51:25 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete5079lb # [ 55.321551] niks3-server[1059]: 2026/09/22 14:51:25 INFO Completed upload id=105080worker1 # [ 55.353597] nix-grpc-daemon[874]: time=2026-09-22T14:51:26.551Z level=INFO msg="Upload complete. (195ms)"5081lb # [ 55.323745] niks3-server[1059]: 2026/09/22 14:51:25 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo5082worker1 # [ 55.356159] nix-grpc-daemon[781]: ts=2026-09-22T14:51:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=15083worker1 # [ 55.363213] nix-daemon[849]: reaped child process 897, status = succeeded5084worker2 # [ 55.344697] nix-grpc-daemon[782]: ts=2026-09-22T14:51:27Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=05085worker2 # [ 55.348696] nix-grpc-daemon[782]: ts=2026-09-22T14:51:27Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=15086worker1 # [ 55.410198] nix-grpc-daemon[781]: ts=2026-09-22T14:51:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505087worker1 # [ 55.422925] nix-grpc-daemon[781]: ts=2026-09-22T14:51:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505088worker1 # [ 55.434791] nix-grpc-daemon[874]: time=2026-09-22T14:51:26.632Z level=INFO msg="All 1 paths already cached"5089client # copying 1 paths to 'lb:50051'...5090client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...5091worker1 # [ 55.443442] nix-grpc-daemon[781]: ts=2026-09-22T14:51:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575505092worker1 # [ 55.457175] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5093worker1 # [ 55.557036] nix-daemon[849]: accepted connection from pid 922, user nix-grpc-daemon (trusted)5094worker1 # [ 55.571233] nix-daemon[849]: reaped child process 925, status = succeeded5095worker1 # [ 55.655971] nix-daemon[849]: accepted connection from pid 927, user nix-grpc-daemon (trusted)5096worker1 # [ 55.670744] nix-daemon[849]: reaped child process 930, status = succeeded5097lb # [ 55.644179] niks3-server[1059]: 2026/09/22 14:51:26 INFO Received uploads request method=POST path=/api/pending_closures5098worker1 # [ 55.683492] nix-grpc-daemon[874]: time=2026-09-22T14:51:26.881Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5099worker1 # [ 55.685488] nix-grpc-daemon[874]: time=2026-09-22T14:51:26.882Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"5100lb # [ 55.671461] niks3-server[1059]: 2026/09/22 14:51:26 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst5101lb # [ 55.684392] niks3-server[1059]: 2026/09/22 14:51:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign5102worker1 # [ 55.722431] nix-grpc-daemon[874]: time=2026-09-22T14:51:26.919Z level=INFO msg="Uploading 1 narinfos"5103lb # [ 55.691469] niks3-server[1059]: 2026/09/22 14:51:26 INFO Signed narinfos id=11 count=15104lb # [ 55.695696] niks3-server[1059]: 2026/09/22 14:51:26 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls5105lb # [ 55.707351] niks3-server[1059]: 2026/09/22 14:51:26 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete5106lb # [ 55.710295] niks3-server[1059]: 2026/09/22 14:51:26 INFO Completed upload id=115107worker1 # [ 55.741293] nix-grpc-daemon[874]: time=2026-09-22T14:51:26.938Z level=INFO msg="Upload complete. (272ms)"5108worker1 # [ 55.743760] nix-grpc-daemon[781]: ts=2026-09-22T14:51:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=1 nar_bytes_in=6245109lb # [ 55.713459] niks3-server[1059]: 2026/09/22 14:51:26 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo5110worker1 # [ 55.747135] nix-grpc-daemon[781]: ts=2026-09-22T14:51:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505111worker1 # [ 55.755441] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5112worker1 # [ 55.759736] nix-daemon[849]: reaped child process 920, status = succeeded5113worker1 # [ 55.763711] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5114client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...5115client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv5116client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5117client # farm-b> LOG-farm-b-sig5118worker1 # [ 57.000670] nix-daemon[849]: reaped child process 934, status = succeeded5119worker1 # [ 57.079197] nix-daemon[849]: accepted connection from pid 950, user nix-grpc-daemon (trusted)5120worker1 # [ 57.091817] nix-daemon[849]: reaped child process 953, status = succeeded5121lb # [ 57.065578] niks3-server[1059]: 2026/09/22 14:51:27 INFO Received uploads request method=POST path=/api/pending_closures5122worker1 # [ 57.105427] nix-grpc-daemon[874]: time=2026-09-22T14:51:28.302Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5123worker1 # [ 57.107167] nix-grpc-daemon[874]: time=2026-09-22T14:51:28.304Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"5124lb # [ 57.108753] niks3-server[1059]: 2026/09/22 14:51:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5125lb # [ 57.114703] niks3-server[1059]: 2026/09/22 14:51:27 INFO Signed narinfos id=12 count=15126worker1 # [ 57.149598] nix-grpc-daemon[874]: time=2026-09-22T14:51:28.347Z level=INFO msg="Uploading 1 narinfos"5127lb # [ 57.122071] niks3-server[1059]: 2026/09/22 14:51:27 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv5128lb # [ 57.128368] niks3-server[1059]: 2026/09/22 14:51:27 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst5129lb # [ 57.138137] niks3-server[1059]: 2026/09/22 14:51:27 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls5130lb # [ 57.145212] niks3-server[1059]: 2026/09/22 14:51:27 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5131lb # [ 57.152411] niks3-server[1059]: 2026/09/22 14:51:27 INFO Completed upload id=125132worker1 # [ 57.183384] nix-grpc-daemon[874]: time=2026-09-22T14:51:28.380Z level=INFO msg="Upload complete. (212ms)"5133lb # [ 57.156495] niks3-server[1059]: 2026/09/22 14:51:27 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo5134worker1 # [ 57.189672] nix-grpc-daemon[781]: ts=2026-09-22T14:51:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=15135worker2 # [ 57.174861] nix-grpc-daemon[782]: ts=2026-09-22T14:51:29Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=05136worker2 # [ 57.179356] nix-grpc-daemon[782]: ts=2026-09-22T14:51:29Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=15137worker1 # [ 57.204970] nix-daemon[849]: reaped child process 932, status = succeeded5138worker1 # [ 57.244394] nix-grpc-daemon[781]: ts=2026-09-22T14:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505139worker1 # [ 57.256224] nix-grpc-daemon[781]: ts=2026-09-22T14:51:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505140worker1 # [ 57.269083] nix-grpc-daemon[874]: time=2026-09-22T14:51:28.466Z level=INFO msg="All 2 paths already cached"5141client # copying 1 paths to 'lb:50051'...5142client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...5143worker1 # [ 57.276897] nix-grpc-daemon[781]: ts=2026-09-22T14:51:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575505144worker1 # [ 57.284647] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5145worker1 # [ 57.377259] nix-daemon[849]: accepted connection from pid 957, user nix-grpc-daemon (trusted)5146worker1 # [ 57.390601] nix-daemon[849]: reaped child process 960, status = succeeded5147worker1 # [ 57.462994] nix-daemon[849]: accepted connection from pid 962, user nix-grpc-daemon (trusted)5148worker1 # [ 57.476194] nix-daemon[849]: reaped child process 965, status = succeeded5149lb # [ 57.449256] niks3-server[1059]: 2026/09/22 14:51:27 INFO Received uploads request method=POST path=/api/pending_closures5150worker1 # [ 57.488381] nix-grpc-daemon[874]: time=2026-09-22T14:51:28.686Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5151worker1 # [ 57.490038] nix-grpc-daemon[874]: time=2026-09-22T14:51:28.687Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"5152lb # [ 57.479327] niks3-server[1059]: 2026/09/22 14:51:27 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst5153lb # [ 57.489402] niks3-server[1059]: 2026/09/22 14:51:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5154lb # [ 57.494660] niks3-server[1059]: 2026/09/22 14:51:27 INFO Signed narinfos id=13 count=15155worker1 # [ 57.528553] nix-grpc-daemon[874]: time=2026-09-22T14:51:28.726Z level=INFO msg="Uploading 1 narinfos"5156lb # [ 57.508092] niks3-server[1059]: 2026/09/22 14:51:27 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls5157lb # [ 57.515144] niks3-server[1059]: 2026/09/22 14:51:27 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5158worker1 # [ 57.554243] nix-grpc-daemon[874]: time=2026-09-22T14:51:28.751Z level=INFO msg="Upload complete. (256ms)"5159lb # [ 57.524043] niks3-server[1059]: 2026/09/22 14:51:27 INFO Completed upload id=135160lb # [ 57.526443] niks3-server[1059]: 2026/09/22 14:51:27 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo5161worker1 # [ 57.557642] nix-grpc-daemon[781]: ts=2026-09-22T14:51:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=1 nar_bytes_in=7525162worker1 # [ 57.562115] nix-grpc-daemon[781]: ts=2026-09-22T14:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505163worker1 # [ 57.572264] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5164worker1 # [ 57.579106] nix-daemon[849]: reaped child process 955, status = succeeded5165client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...5166worker1 # [ 57.582115] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5167client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5168lb # [ 57.645526] envoy[924]: conn peer=[2001:db8:1::1]:53582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4395 ms=57705169lb # [ 57.648469] envoy[924]: conn peer=[2001:db8:1::1]:53586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=385170lb # [ 57.651944] envoy[924]: conn peer=[2001:db8:1::1]:53602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4182 tx=1921 ms=22925171client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5172client # farm-top> LOG-farm-top-sig5173worker1 # [ 58.638098] nix-daemon[849]: reaped child process 969, status = succeeded5174worker1 # [ 58.684966] nix-daemon[849]: accepted connection from pid 985, user nix-grpc-daemon (trusted)5175worker1 # [ 58.697364] nix-daemon[849]: reaped child process 988, status = succeeded5176lb # [ 58.672787] niks3-server[1059]: 2026/09/22 14:51:29 INFO Received uploads request method=POST path=/api/pending_closures5177worker1 # [ 58.709100] nix-grpc-daemon[874]: time=2026-09-22T14:51:29.906Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5178worker1 # [ 58.710568] nix-grpc-daemon[874]: time=2026-09-22T14:51:29.908Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5179lb # [ 58.694136] niks3-server[1059]: 2026/09/22 14:51:29 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5180lb # [ 58.701801] niks3-server[1059]: 2026/09/22 14:51:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5181lb # [ 58.705303] niks3-server[1059]: 2026/09/22 14:51:29 INFO Signed narinfos id=14 count=15182worker1 # [ 58.736203] nix-grpc-daemon[874]: time=2026-09-22T14:51:29.933Z level=INFO msg="Uploading 1 narinfos"5183lb # [ 58.707070] niks3-server[1059]: 2026/09/22 14:51:29 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5184lb # [ 58.712828] niks3-server[1059]: 2026/09/22 14:51:29 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5185lb # [ 58.717338] niks3-server[1059]: 2026/09/22 14:51:29 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5186lb # [ 58.721155] niks3-server[1059]: 2026/09/22 14:51:29 INFO Completed upload id=145187worker1 # [ 58.751928] nix-grpc-daemon[874]: time=2026-09-22T14:51:29.949Z level=INFO msg="Upload complete. (162ms)"5188worker1 # [ 58.754455] nix-grpc-daemon[781]: ts=2026-09-22T14:51:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15189lb # [ 58.724731] niks3-server[1059]: 2026/09/22 14:51:29 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5190worker1 # [ 58.761326] nix-daemon[849]: reaped child process 967, status = succeeded5191client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.68 seconds)5192client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5193client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.11 seconds)5194client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5195worker1 # [ 58.996143] nix-grpc-daemon[781]: ts=2026-09-22T14:51:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:383125196worker1 # [ 59.002557] nix-grpc-daemon[781]: ts=2026-09-22T14:51:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:383125197client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.16 seconds)5198client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5199worker2 # [ 59.128617] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:347945200client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5201client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5202client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5203client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5204worker2 # [ 59.157050] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:347945205worker2 # [ 59.162478] nix-grpc-daemon[782]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5206worker2 # [ 59.174728] nix-grpc-daemon[782]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5207worker2 # [ 59.185326] nix-grpc-daemon[782]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5208client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.19 seconds)5209client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5210client # finding garbage collector roots...5211client # removing stale temporary roots file "/nix/var/nix/temproots/841"5212client # removing stale temporary roots file "/nix/var/nix/temproots/853"5213client # removing stale temporary roots file "/nix/var/nix/temproots/882"5214client # removing stale temporary roots file "/nix/var/nix/temproots/820"5215client # removing stale temporary roots file "/nix/var/nix/temproots/874"5216client # removing stale temporary roots file "/nix/var/nix/temproots/806"5217client # removing stale temporary roots file "/nix/var/nix/temproots/861"5218client # removing stale temporary roots file "/nix/var/nix/temproots/897"5219client # deleting specified paths...5220client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5221client # deleting unused links...5222client # note: hard linking is currently saving 0.0 KiB5223worker1 # [ 59.473283] nix-grpc-daemon[781]: ts=2026-09-22T14:51:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575505224worker1 # [ 59.480305] nix-grpc-daemon[781]: ts=2026-09-22T14:51:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505225worker1 # [ 59.488888] nix-grpc-daemon[874]: time=2026-09-22T14:51:30.686Z level=INFO msg="All 1 paths already cached"5226worker2 # [ 59.474274] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375265227worker1 # [ 59.500407] nix-grpc-daemon[781]: ts=2026-09-22T14:51:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575505228client # copying 3 paths...5229worker2 # [ 59.488902] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375265230client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5231worker1 # [ 59.526369] nix-grpc-daemon[781]: ts=2026-09-22T14:51:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:575505232worker1 # [ 59.531758] nix-grpc-daemon[781]: ts=2026-09-22T14:51:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=1 nar_bytes_out=2325233worker2 # [ 59.512140] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375265234worker2 # [ 59.517872] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_out=1765235worker2 # [ 59.524285] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375265236worker2 # [ 59.528096] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_out=1285237client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5238client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5239client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.39 seconds)5240(finished: subtest: outputs read back through any worker carry the cache signature, in 6.52 seconds)5241subtest: two clients wanting the same drv build it once5242client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5243client # [ 59.643679] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5244client # Running as unit: dedup1.service; invocation ID: 876f28ec45d042519fc4a8f06ed170505245client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)5246client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5247worker2 # [ 59.755825] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375265248client # [ 59.768846] nix[950]: don't know how to build these paths:5249client # [ 59.769781] nix[950]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5250worker2 # [ 59.762788] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05251worker2 # [ 59.764982] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=15252worker2 # [ 59.767749] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265253worker2 # [ 59.776433] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265254client # [ 59.797980] nix[950]: copying 1 paths to 'lb:50051'...5255client # [ 59.798821] nix[950]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5256worker2 # [ 59.787964] nix-grpc-daemon[782]: ts=2026-09-22T14:51:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375265257worker2 # [ 59.799408] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5258worker2 # [ 59.903478] nix-daemon[869]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)5259worker2 # [ 59.927727] nix-daemon[869]: reaped child process 1085, status = succeeded5260worker2 # [ 60.003668] nix-daemon[869]: accepted connection from pid 1087, user nix-grpc-daemon (trusted)5261worker2 # [ 60.022263] nix-daemon[869]: reaped child process 1090, status = succeeded5262lb # [ 60.018218] niks3-server[1059]: 2026/09/22 14:51:30 INFO Received uploads request method=POST path=/api/pending_closures5263worker2 # [ 60.035280] nix-grpc-daemon[884]: time=2026-09-22T14:51:32.061Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5264worker2 # [ 60.037136] nix-grpc-daemon[884]: time=2026-09-22T14:51:32.063Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5265lb # [ 60.049948] niks3-server[1059]: 2026/09/22 14:51:30 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5266lb # [ 60.058896] niks3-server[1059]: 2026/09/22 14:51:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5267lb # [ 60.063847] niks3-server[1059]: 2026/09/22 14:51:30 INFO Signed narinfos id=15 count=15268worker2 # [ 60.077302] nix-grpc-daemon[884]: time=2026-09-22T14:51:32.103Z level=INFO msg="Uploading 1 narinfos"5269lb # [ 60.073958] niks3-server[1059]: 2026/09/22 14:51:30 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5270lb # [ 60.084540] niks3-server[1059]: 2026/09/22 14:51:30 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5271lb # [ 60.090514] niks3-server[1059]: 2026/09/22 14:51:30 INFO Completed upload id=155272worker2 # [ 60.100110] nix-grpc-daemon[884]: time=2026-09-22T14:51:32.126Z level=INFO msg="Upload complete. (290ms)"5273worker2 # [ 60.103631] nix-grpc-daemon[782]: ts=2026-09-22T14:51:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_in=4325274lb # [ 60.096248] niks3-server[1059]: 2026/09/22 14:51:30 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5275worker2 # [ 60.109164] nix-grpc-daemon[782]: ts=2026-09-22T14:51:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265276worker2 # [ 60.121068] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5277worker2 # [ 60.125177] nix-daemon[869]: reaped child process 1080, status = succeeded5278worker2 # [ 60.130294] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5279client # [ 60.141531] nix[950]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5280client # [ 60.697628] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5281client # Running as unit: dedup2.service; invocation ID: a0c914d0c7ad4225b9e02fc9157b218f5282client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.06 seconds)5283worker1 # [ 60.872549] nix-grpc-daemon[781]: ts=2026-09-22T14:51:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575505284client # [ 60.868616] nix[966]: don't know how to build these paths:5285client # [ 60.869513] nix[966]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5286worker2 # [ 60.866578] nix-grpc-daemon[782]: ts=2026-09-22T14:51:32Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05287worker2 # [ 60.876444] nix-grpc-daemon[782]: ts=2026-09-22T14:51:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265288client # [ 60.893629] nix[966]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5289client: must succeed: sleep 25290client: (finished: must succeed: sleep 2, in 2.04 seconds)5291client: waiting for success: ! systemctl is-active dedup1 dedup25292worker2 # [ 64.172283] nix-daemon[869]: reaped child process 1094, status = succeeded5293worker2 # [ 64.251957] nix-daemon[869]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)5294worker2 # [ 64.265026] nix-daemon[869]: reaped child process 1130, status = succeeded5295lb # [ 64.263526] niks3-server[1059]: 2026/09/22 14:51:34 INFO Received uploads request method=POST path=/api/pending_closures5296worker2 # [ 64.281682] nix-grpc-daemon[884]: time=2026-09-22T14:51:36.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5297worker2 # [ 64.283206] nix-grpc-daemon[884]: time=2026-09-22T14:51:36.309Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5298lb # [ 64.315483] niks3-server[1059]: 2026/09/22 14:51:34 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5299lb # [ 64.320726] niks3-server[1059]: 2026/09/22 14:51:34 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5300lb # [ 64.333204] niks3-server[1059]: 2026/09/22 14:51:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5301lb # [ 64.338115] niks3-server[1059]: 2026/09/22 14:51:34 INFO Signed narinfos id=16 count=15302worker2 # [ 64.347923] nix-grpc-daemon[884]: time=2026-09-22T14:51:36.374Z level=INFO msg="Uploading 1 narinfos"5303lb # [ 64.347343] niks3-server[1059]: 2026/09/22 14:51:34 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5304lb # [ 64.351773] niks3-server[1059]: 2026/09/22 14:51:34 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5305lb # [ 64.357087] niks3-server[1059]: 2026/09/22 14:51:34 INFO Completed upload id=165306worker2 # [ 64.366776] nix-grpc-daemon[884]: time=2026-09-22T14:51:36.393Z level=INFO msg="Upload complete. (225ms)"5307lb # [ 64.358945] niks3-server[1059]: 2026/09/22 14:51:34 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5308worker2 # [ 64.374274] nix-grpc-daemon[782]: ts=2026-09-22T14:51:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15309worker2 # [ 64.382261] nix-grpc-daemon[782]: ts=2026-09-22T14:51:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15310worker2 # [ 64.389620] nix-daemon[869]: reaped child process 1092, status = succeeded5311client # [ 64.419571] systemd[1]: dedup2.service: Deactivated successfully.5312client # [ 64.422919] systemd[1]: dedup2.service: Consumed 142ms CPU time over 3.721s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5313client # [ 64.460076] systemd[1]: dedup1.service: Deactivated successfully.5314client # [ 64.462911] systemd[1]: dedup1.service: Consumed 115ms CPU time over 4.818s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.5315client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)5316client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5317client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5318worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5319worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.02 seconds)5320worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5321worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.03 seconds)5322worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5323worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5324worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5325worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5326(finished: subtest: two clients wanting the same drv build it once, in 5.64 seconds)5327subtest: --timeout and --max-silent-time reach the builder5328worker1 # [ 65.409482] nix-grpc-daemon[781]: ts=2026-09-22T14:51:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575505329worker2 # [ 65.401837] nix-grpc-daemon[782]: ts=2026-09-22T14:51:37Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05330worker2 # [ 65.406499] nix-grpc-daemon[782]: ts=2026-09-22T14:51:37Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=15331worker1 # [ 65.430181] nix-grpc-daemon[781]: ts=2026-09-22T14:51:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505332worker1 # [ 65.442365] nix-grpc-daemon[781]: ts=2026-09-22T14:51:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505333worker1 # [ 65.452874] nix-grpc-daemon[781]: ts=2026-09-22T14:51:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575505334worker1 # [ 65.463720] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5335worker1 # [ 65.563537] nix-daemon[849]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)5336worker1 # [ 65.576988] nix-daemon[849]: reaped child process 1016, status = succeeded5337worker1 # [ 65.659519] nix-daemon[849]: accepted connection from pid 1018, user nix-grpc-daemon (trusted)5338worker1 # [ 65.671895] nix-daemon[849]: reaped child process 1021, status = succeeded5339lb # [ 65.648645] niks3-server[1059]: 2026/09/22 14:51:36 INFO Received uploads request method=POST path=/api/pending_closures5340worker1 # [ 65.688435] nix-grpc-daemon[874]: time=2026-09-22T14:51:36.885Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5341worker1 # [ 65.689761] nix-grpc-daemon[874]: time=2026-09-22T14:51:36.885Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5342lb # [ 65.680919] niks3-server[1059]: 2026/09/22 14:51:36 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5343lb # [ 65.689070] niks3-server[1059]: 2026/09/22 14:51:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5344worker1 # [ 65.724863] nix-grpc-daemon[874]: time=2026-09-22T14:51:36.922Z level=INFO msg="Uploading 1 narinfos"5345lb # [ 65.694815] niks3-server[1059]: 2026/09/22 14:51:36 INFO Signed narinfos id=17 count=15346lb # [ 65.701133] niks3-server[1059]: 2026/09/22 14:51:36 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5347lb # [ 65.713839] niks3-server[1059]: 2026/09/22 14:51:36 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5348lb # [ 65.720221] niks3-server[1059]: 2026/09/22 14:51:36 INFO Completed upload id=175349worker1 # [ 65.754216] nix-grpc-daemon[874]: time=2026-09-22T14:51:36.951Z level=INFO msg="Upload complete. (279ms)"5350lb # [ 65.725665] niks3-server[1059]: 2026/09/22 14:51:36 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5351worker1 # [ 65.758552] nix-grpc-daemon[781]: ts=2026-09-22T14:51:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=1 nar_bytes_in=4245352worker1 # [ 65.763622] nix-grpc-daemon[781]: ts=2026-09-22T14:51:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505353worker1 # [ 65.771141] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5354worker1 # [ 65.780464] nix-daemon[849]: reaped child process 1011, status = succeeded5355worker1 # [ 65.785149] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5356lb # [ 67.646236] envoy[924]: conn peer=[2001:db8:1::1]:36542 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4373 ms=55355357lb # [ 67.648201] envoy[924]: conn peer=[2001:db8:1::1]:36556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1395358lb # [ 67.650072] envoy[924]: conn peer=[2001:db8:1::1]:36562 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=855359lb # [ 67.651774] envoy[924]: conn peer=[2001:db8:1::1]:36592 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=725360lb # [ 67.653437] envoy[924]: conn peer=[2001:db8:1::1]:36576 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=805361lb # [ 67.655134] envoy[924]: conn peer=[2001:db8:1::1]:36602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1194 ms=35465362lb # [ 67.656818] envoy[924]: conn peer=[2001:db8:1::1]:36594 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2998 tx=1538 ms=46765363worker1 # [ 71.997693] nix-grpc-daemon[781]: ts=2026-09-22T14:51:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05364worker1 # [ 72.008961] nix-daemon[849]: reaped child process 1023, status = succeeded5365worker1 # [ 72.009968] nix-daemon[849]: reaped child process 1025, status = succeeded5366worker1 # [ 72.169440] nix-grpc-daemon[781]: ts=2026-09-22T14:51:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575505367worker2 # [ 72.157946] nix-grpc-daemon[782]: ts=2026-09-22T14:51:44Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05368worker2 # [ 72.166753] nix-grpc-daemon[782]: ts=2026-09-22T14:51:44Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=15369worker2 # [ 72.171255] nix-grpc-daemon[782]: ts=2026-09-22T14:51:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265370worker2 # [ 72.182640] nix-grpc-daemon[782]: ts=2026-09-22T14:51:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265371worker2 # [ 72.193784] nix-grpc-daemon[782]: ts=2026-09-22T14:51:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375265372worker2 # [ 72.205239] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5373worker2 # [ 72.314892] nix-daemon[869]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)5374worker2 # [ 72.330536] nix-daemon[869]: reaped child process 1150, status = succeeded5375worker2 # [ 72.420958] nix-daemon[869]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)5376worker2 # [ 72.436366] nix-daemon[869]: reaped child process 1155, status = succeeded5377lb # [ 72.433407] niks3-server[1059]: 2026/09/22 14:51:42 INFO Received uploads request method=POST path=/api/pending_closures5378worker2 # [ 72.452278] nix-grpc-daemon[884]: time=2026-09-22T14:51:44.478Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5379worker2 # [ 72.453875] nix-grpc-daemon[884]: time=2026-09-22T14:51:44.480Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5380lb # [ 72.465911] niks3-server[1059]: 2026/09/22 14:51:42 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5381lb # [ 72.480864] niks3-server[1059]: 2026/09/22 14:51:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5382lb # [ 72.488846] niks3-server[1059]: 2026/09/22 14:51:42 INFO Signed narinfos id=18 count=15383worker2 # [ 72.502256] nix-grpc-daemon[884]: time=2026-09-22T14:51:44.528Z level=INFO msg="Uploading 1 narinfos"5384lb # [ 72.496651] niks3-server[1059]: 2026/09/22 14:51:42 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5385lb # [ 72.507594] niks3-server[1059]: 2026/09/22 14:51:42 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5386lb # [ 72.514086] niks3-server[1059]: 2026/09/22 14:51:42 INFO Completed upload id=185387worker2 # [ 72.523863] nix-grpc-daemon[884]: time=2026-09-22T14:51:44.550Z level=INFO msg="Upload complete. (307ms)"5388lb # [ 72.516668] niks3-server[1059]: 2026/09/22 14:51:42 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5389worker2 # [ 72.528709] nix-grpc-daemon[782]: ts=2026-09-22T14:51:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_in=4245390worker2 # [ 72.535059] nix-grpc-daemon[782]: ts=2026-09-22T14:51:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265391worker2 # [ 72.544879] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5392worker2 # [ 72.555917] nix-daemon[869]: reaped child process 1145, status = succeeded5393worker2 # [ 72.561326] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5394lb # [ 77.647226] envoy[924]: conn peer=[2001:db8:1::1]:44932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3009 tx=975 ms=66045395worker2 # [ 78.774224] nix-grpc-daemon[782]: ts=2026-09-22T14:51:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05396worker2 # [ 78.786618] nix-daemon[869]: reaped child process 1157, status = succeeded5397worker2 # [ 78.787573] nix-daemon[869]: reaped child process 1159, status = succeeded5398(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.65 seconds)5399subtest: the first client leaving does not fail the second5400client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5401client # [ 78.952897] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5402client # Running as unit: share1.service; invocation ID: 8c3489ca04d9467d99353f11e65885985403client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5404client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5405worker2 # [ 79.083300] nix-grpc-daemon[782]: ts=2026-09-22T14:51:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375265406client # [ 79.096119] nix[1038]: don't know how to build these paths:5407client # [ 79.096984] nix[1038]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5408worker2 # [ 79.089339] nix-grpc-daemon[782]: ts=2026-09-22T14:51:51Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05409worker2 # [ 79.091256] nix-grpc-daemon[782]: ts=2026-09-22T14:51:51Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=15410worker1 # [ 79.117809] nix-grpc-daemon[781]: ts=2026-09-22T14:51:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505411worker1 # [ 79.130665] nix-grpc-daemon[781]: ts=2026-09-22T14:51:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505412client # [ 79.130647] nix[1038]: copying 1 paths to 'lb:50051'...5413client # [ 79.131557] nix[1038]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5414worker1 # [ 79.141771] nix-grpc-daemon[781]: ts=2026-09-22T14:51:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575505415worker1 # [ 79.151970] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5416worker1 # [ 79.251750] nix-daemon[849]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)5417worker1 # [ 79.263955] nix-daemon[849]: reaped child process 1051, status = succeeded5418worker1 # [ 79.344352] nix-daemon[849]: accepted connection from pid 1053, user nix-grpc-daemon (trusted)5419worker1 # [ 79.356926] nix-daemon[849]: reaped child process 1056, status = succeeded5420lb # [ 79.333070] niks3-server[1059]: 2026/09/22 14:51:49 INFO Received uploads request method=POST path=/api/pending_closures5421worker1 # [ 79.372585] nix-grpc-daemon[874]: time=2026-09-22T14:51:50.569Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5422worker1 # [ 79.374025] nix-grpc-daemon[874]: time=2026-09-22T14:51:50.569Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5423lb # [ 79.361748] niks3-server[1059]: 2026/09/22 14:51:49 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5424lb # [ 79.377131] niks3-server[1059]: 2026/09/22 14:51:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5425worker1 # [ 79.412674] nix-grpc-daemon[874]: time=2026-09-22T14:51:50.610Z level=INFO msg="Uploading 1 narinfos"5426lb # [ 79.381637] niks3-server[1059]: 2026/09/22 14:51:49 INFO Signed narinfos id=19 count=15427lb # [ 79.392948] niks3-server[1059]: 2026/09/22 14:51:49 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5428lb # [ 79.399873] niks3-server[1059]: 2026/09/22 14:51:49 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5429worker1 # [ 79.437516] nix-grpc-daemon[874]: time=2026-09-22T14:51:50.635Z level=INFO msg="Upload complete. (274ms)"5430lb # [ 79.406877] niks3-server[1059]: 2026/09/22 14:51:49 INFO Completed upload id=195431worker1 # [ 79.439561] nix-grpc-daemon[781]: ts=2026-09-22T14:51:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=1 nar_bytes_in=4325432worker1 # [ 79.442670] nix-grpc-daemon[781]: ts=2026-09-22T14:51:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505433lb # [ 79.412659] niks3-server[1059]: 2026/09/22 14:51:49 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5434worker1 # [ 79.447379] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5435worker1 # [ 79.452977] nix-daemon[849]: reaped child process 1046, status = succeeded5436client # [ 79.446615] nix[1038]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5437worker1 # [ 79.457132] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5438client # [ 80.020510] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5439client # Running as unit: share2.service; invocation ID: 7ef78c3a61434a73962facecd65b14ef5440client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.06 seconds)5441worker1 # [ 80.166764] nix-grpc-daemon[781]: ts=2026-09-22T14:51:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575505442client # [ 80.166364] nix[1054]: this derivation will be built:5443worker1 # [ 80.177320] nix-grpc-daemon[781]: ts=2026-09-22T14:51:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575505444client # [ 80.176613] nix[1054]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5445worker2 # [ 80.170461] nix-grpc-daemon[782]: ts=2026-09-22T14:51:52Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05446worker1 # [ 80.194071] nix-grpc-daemon[781]: ts=2026-09-22T14:51:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505447client # [ 80.189324] nix[1054]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5448client: must succeed: sleep 2; systemctl kill -s INT share15449client # [ 83.255569] systemd[1]: share1.service: Sent signal SIGINT to main process 1038 (nix) on client request.5450client # [ 83.261658] nix[1038]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5451client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5452client: waiting for success: ! systemctl is-active share15453client # [ 83.269761] nix[1038]: error: interrupted by the user5454client # [ 83.312795] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5455client # [ 83.314431] systemd[1]: share1.service: Failed with result 'exit-code'.5456client # [ 83.316232] systemd[1]: share1.service: Consumed 114ms CPU time over 4.362s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic.5457client: (finished: waiting for success: ! systemctl is-active share1, in 1.06 seconds)5458client: waiting for success: ! systemctl is-active share25459worker1 # [ 84.677756] nix-daemon[849]: reaped child process 1060, status = succeeded5460worker1 # [ 84.750063] nix-daemon[849]: accepted connection from pid 1094, user nix-grpc-daemon (trusted)5461worker1 # [ 84.763403] nix-daemon[849]: reaped child process 1097, status = succeeded5462lb # [ 84.737747] niks3-server[1059]: 2026/09/22 14:51:55 INFO Received uploads request method=POST path=/api/pending_closures5463worker1 # [ 84.772767] nix-grpc-daemon[874]: time=2026-09-22T14:51:55.970Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5464worker1 # [ 84.774262] nix-grpc-daemon[874]: time=2026-09-22T14:51:55.971Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5465lb # [ 84.755736] niks3-server[1059]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5466lb # [ 84.760153] niks3-server[1059]: 2026/09/22 14:51:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5467lb # [ 84.762069] niks3-server[1059]: 2026/09/22 14:51:55 INFO Signed narinfos id=20 count=15468worker1 # [ 84.794086] nix-grpc-daemon[874]: time=2026-09-22T14:51:55.991Z level=INFO msg="Uploading 1 narinfos"5469lb # [ 84.766600] niks3-server[1059]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5470lb # [ 84.771326] niks3-server[1059]: 2026/09/22 14:51:55 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5471lb # [ 84.774088] niks3-server[1059]: 2026/09/22 14:51:55 INFO Completed upload id=205472worker1 # [ 84.804735] nix-grpc-daemon[874]: time=2026-09-22T14:51:56.002Z level=INFO msg="Upload complete. (136ms)"5473lb # [ 84.775555] niks3-server[1059]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5474worker1 # [ 84.807549] nix-grpc-daemon[781]: ts=2026-09-22T14:51:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15475lb # [ 84.781320] niks3-server[1059]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5476worker1 # [ 84.811128] nix-grpc-daemon[781]: ts=2026-09-22T14:51:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15477worker1 # [ 84.816618] nix-daemon[849]: reaped child process 1058, status = succeeded5478client # [ 84.857898] systemd[1]: share2.service: Deactivated successfully.5479client # [ 84.859072] systemd[1]: share2.service: Consumed 113ms CPU time over 4.838s wall clock time, 35.4M memory peak, 6.6K incoming IP traffic, 6K outgoing IP traffic.5480client: (finished: waiting for success: ! systemctl is-active share2, in 1.06 seconds)5481client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5482client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5483(finished: subtest: the first client leaving does not fail the second, in 6.68 seconds)5484subtest: build hook: nix-daemon with builders = grpc://lb5485client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5486client # [ 85.672074] systemd[1]: Started Nix Daemon.5487client # [ 85.750869] nix-daemon[1106]: accepted connection from pid 1103, user root (trusted)5488worker1 # [ 85.866459] nix-grpc-daemon[781]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:575505489worker2 # [ 85.848364] nix-grpc-daemon[782]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265490worker2 # [ 85.855839] nix-grpc-daemon[782]: ts=2026-09-22T14:51:57Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05491worker2 # [ 85.857970] nix-grpc-daemon[782]: ts=2026-09-22T14:51:57Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15492worker2 # [ 85.860654] nix-grpc-daemon[782]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265493worker2 # [ 85.867481] nix-grpc-daemon[782]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265494worker2 # [ 85.874389] nix-grpc-daemon[782]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375265495worker2 # [ 85.879510] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5496worker2 # [ 85.963462] nix-daemon[869]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5497worker2 # [ 85.976916] nix-daemon[869]: reaped child process 1207, status = succeeded5498worker2 # [ 86.059378] nix-daemon[869]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5499worker2 # [ 86.072369] nix-daemon[869]: reaped child process 1212, status = succeeded5500lb # [ 86.069983] niks3-server[1059]: 2026/09/22 14:51:56 INFO Received uploads request method=POST path=/api/pending_closures5501worker2 # [ 86.091160] nix-grpc-daemon[884]: time=2026-09-22T14:51:58.117Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5502worker2 # [ 86.092574] nix-grpc-daemon[884]: time=2026-09-22T14:51:58.117Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5503lb # [ 86.100599] niks3-server[1059]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5504lb # [ 86.117212] niks3-server[1059]: 2026/09/22 14:51:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5505lb # [ 86.123112] niks3-server[1059]: 2026/09/22 14:51:56 INFO Signed narinfos id=21 count=15506worker2 # [ 86.135580] nix-grpc-daemon[884]: time=2026-09-22T14:51:58.161Z level=INFO msg="Uploading 1 narinfos"5507lb # [ 86.136679] niks3-server[1059]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5508lb # [ 86.144257] niks3-server[1059]: 2026/09/22 14:51:56 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5509lb # [ 86.152175] niks3-server[1059]: 2026/09/22 14:51:56 INFO Completed upload id=215510lb # [ 86.154205] niks3-server[1059]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5511worker2 # [ 86.163310] nix-grpc-daemon[884]: time=2026-09-22T14:51:58.189Z level=INFO msg="Upload complete. (279ms)"5512worker2 # [ 86.167291] nix-grpc-daemon[782]: ts=2026-09-22T14:51:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_in=5125513worker2 # [ 86.172354] nix-grpc-daemon[782]: ts=2026-09-22T14:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265514worker2 # [ 86.182870] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5515worker2 # [ 86.193589] nix-daemon[869]: reaped child process 1202, status = succeeded5516worker2 # [ 86.198188] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5517worker2 # [ 87.242155] nix-daemon[869]: reaped child process 1216, status = succeeded5518worker2 # [ 87.321603] nix-daemon[869]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5519worker2 # [ 87.334704] nix-daemon[869]: reaped child process 1235, status = succeeded5520lb # [ 87.331966] niks3-server[1059]: 2026/09/22 14:51:57 INFO Received uploads request method=POST path=/api/pending_closures5521worker2 # [ 87.349349] nix-grpc-daemon[884]: time=2026-09-22T14:51:59.375Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5522worker2 # [ 87.350837] nix-grpc-daemon[884]: time=2026-09-22T14:51:59.377Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5523lb # [ 87.371147] niks3-server[1059]: 2026/09/22 14:51:57 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5524lb # [ 87.381931] niks3-server[1059]: 2026/09/22 14:51:57 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5525lb # [ 87.394363] niks3-server[1059]: 2026/09/22 14:51:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5526lb # [ 87.398648] niks3-server[1059]: 2026/09/22 14:51:57 INFO Signed narinfos id=22 count=15527worker2 # [ 87.408415] nix-grpc-daemon[884]: time=2026-09-22T14:51:59.434Z level=INFO msg="Uploading 1 narinfos"5528lb # [ 87.404396] niks3-server[1059]: 2026/09/22 14:51:57 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5529lb # [ 87.412408] niks3-server[1059]: 2026/09/22 14:51:57 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5530lb # [ 87.416473] niks3-server[1059]: 2026/09/22 14:51:57 INFO Completed upload id=225531lb # [ 87.418710] niks3-server[1059]: 2026/09/22 14:51:57 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5532worker2 # [ 87.427442] nix-grpc-daemon[884]: time=2026-09-22T14:51:59.453Z level=INFO msg="Upload complete. (221ms)"5533worker2 # [ 87.433861] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15534worker2 # [ 87.447865] nix-daemon[869]: reaped child process 1214, status = succeeded5535worker2 # [ 87.479959] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375265536worker2 # [ 87.498672] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375265537worker2 # [ 87.503652] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_out=1285538lb # [ 87.647439] envoy[924]: conn peer=[2001:db8:1::1]:44940 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=953 ms=66405539lb # [ 87.648957] envoy[924]: conn peer=[2001:db8:1::1]:51012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3008 tx=788 ms=41855540lb # [ 87.650366] envoy[924]: conn peer=[2001:db8:1::1]:51022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1602 tx=1596 ms=46945541lb # [ 87.651800] envoy[924]: conn peer=[2001:db8:1::1]:33458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=585542lb # [ 87.653282] envoy[924]: conn peer=[2001:db8:1::1]:33442 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3406 tx=1916 ms=17125543worker2 # [ 87.700791] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:375265544worker1 # [ 87.724636] nix-grpc-daemon[781]: ts=2026-09-22T14:51:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505545worker2 # [ 87.718655] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05546worker2 # [ 87.723301] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15547worker2 # [ 87.727984] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265548worker2 # [ 87.736481] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265549worker2 # [ 87.747575] nix-grpc-daemon[884]: time=2026-09-22T14:51:59.773Z level=INFO msg="All 1 paths already cached"5550worker2 # [ 87.754546] nix-grpc-daemon[782]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375265551worker2 # [ 87.763169] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5552worker2 # [ 87.861037] nix-daemon[869]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)5553worker2 # [ 87.875052] nix-daemon[869]: reaped child process 1244, status = succeeded5554worker2 # [ 87.958652] nix-daemon[869]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5555worker2 # [ 87.973194] nix-daemon[869]: reaped child process 1249, status = succeeded5556lb # [ 87.968252] niks3-server[1059]: 2026/09/22 14:51:58 INFO Received uploads request method=POST path=/api/pending_closures5557worker2 # [ 87.985690] nix-grpc-daemon[884]: time=2026-09-22T14:52:00.012Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5558worker2 # [ 87.987162] nix-grpc-daemon[884]: time=2026-09-22T14:52:00.013Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5559lb # [ 87.999594] niks3-server[1059]: 2026/09/22 14:51:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5560worker2 # [ 88.015135] nix-grpc-daemon[884]: time=2026-09-22T14:52:00.041Z level=INFO msg="Uploading 1 narinfos"5561lb # [ 88.005491] niks3-server[1059]: 2026/09/22 14:51:58 INFO Signed narinfos id=23 count=15562lb # [ 88.015529] niks3-server[1059]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5563lb # [ 88.022709] niks3-server[1059]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5564lb # [ 88.032830] niks3-server[1059]: 2026/09/22 14:51:58 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5565lb # [ 88.038893] niks3-server[1059]: 2026/09/22 14:51:58 INFO Completed upload id=235566worker2 # [ 88.048271] nix-grpc-daemon[884]: time=2026-09-22T14:52:00.074Z level=INFO msg="Upload complete. (273ms)"5567lb # [ 88.041832] niks3-server[1059]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5568worker2 # [ 88.051890] nix-grpc-daemon[782]: ts=2026-09-22T14:52:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_in=6245569worker2 # [ 88.057753] nix-grpc-daemon[782]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265570worker2 # [ 88.067290] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5571worker2 # [ 88.073900] nix-daemon[869]: reaped child process 1239, status = succeeded5572worker2 # [ 88.076899] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5573worker2 # [ 88.706662] nix-daemon[869]: reaped child process 1253, status = succeeded5574worker2 # [ 88.775608] nix-daemon[869]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)5575worker2 # [ 88.788732] nix-daemon[869]: reaped child process 1272, status = succeeded5576lb # [ 88.786958] niks3-server[1059]: 2026/09/22 14:51:59 INFO Received uploads request method=POST path=/api/pending_closures5577worker2 # [ 88.805478] nix-grpc-daemon[884]: time=2026-09-22T14:52:00.831Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5578worker2 # [ 88.808060] nix-grpc-daemon[884]: time=2026-09-22T14:52:00.833Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5579lb # [ 88.822174] niks3-server[1059]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5580lb # [ 88.840134] niks3-server[1059]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5581lb # [ 88.847644] niks3-server[1059]: 2026/09/22 14:51:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5582lb # [ 88.854955] niks3-server[1059]: 2026/09/22 14:51:59 INFO Signed narinfos id=24 count=15583worker2 # [ 88.867045] nix-grpc-daemon[884]: time=2026-09-22T14:52:00.893Z level=INFO msg="Uploading 1 narinfos"5584lb # [ 88.864394] niks3-server[1059]: 2026/09/22 14:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5585lb # [ 88.868322] niks3-server[1059]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5586lb # [ 88.872386] niks3-server[1059]: 2026/09/22 14:51:59 INFO Completed upload id=245587lb # [ 88.874156] niks3-server[1059]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5588worker2 # [ 88.883346] nix-grpc-daemon[884]: time=2026-09-22T14:52:00.909Z level=INFO msg="Upload complete. (199ms)"5589worker2 # [ 88.889969] nix-grpc-daemon[782]: ts=2026-09-22T14:52:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15590worker2 # [ 88.898743] nix-grpc-daemon[782]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375265591worker2 # [ 88.909804] nix-daemon[869]: reaped child process 1251, status = succeeded5592worker2 # [ 88.915203] nix-grpc-daemon[782]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375265593worker2 # [ 88.918919] nix-grpc-daemon[782]: ts=2026-09-22T14:52:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_out=1765594worker1 # [ 89.144592] nix-grpc-daemon[781]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:575505595worker2 # [ 89.125915] nix-grpc-daemon[782]: ts=2026-09-22T14:52:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265596worker2 # [ 89.135233] nix-grpc-daemon[884]: time=2026-09-22T14:52:01.161Z level=INFO msg="All 2 paths already cached"5597worker2 # [ 89.147544] nix-grpc-daemon[782]: ts=2026-09-22T14:52:01Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05598worker2 # [ 89.153371] nix-grpc-daemon[782]: ts=2026-09-22T14:52:01Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15599worker2 # [ 89.158754] nix-grpc-daemon[782]: ts=2026-09-22T14:52:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265600worker2 # [ 89.167672] nix-grpc-daemon[782]: ts=2026-09-22T14:52:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265601worker2 # [ 89.180287] nix-grpc-daemon[884]: time=2026-09-22T14:52:01.206Z level=INFO msg="All 2 paths already cached"5602worker2 # [ 89.186922] nix-grpc-daemon[782]: ts=2026-09-22T14:52:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375265603worker2 # [ 89.196334] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5604worker2 # [ 89.295740] nix-daemon[869]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5605worker2 # [ 89.311231] nix-daemon[869]: reaped child process 1281, status = succeeded5606worker2 # [ 89.390891] nix-daemon[869]: accepted connection from pid 1283, user nix-grpc-daemon (trusted)5607worker2 # [ 89.404127] nix-daemon[869]: reaped child process 1286, status = succeeded5608lb # [ 89.400691] niks3-server[1059]: 2026/09/22 14:51:59 INFO Received uploads request method=POST path=/api/pending_closures5609worker2 # [ 89.418293] nix-grpc-daemon[884]: time=2026-09-22T14:52:01.444Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5610worker2 # [ 89.419655] nix-grpc-daemon[884]: time=2026-09-22T14:52:01.444Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5611lb # [ 89.426578] niks3-server[1059]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5612lb # [ 89.436267] niks3-server[1059]: 2026/09/22 14:51:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5613lb # [ 89.441622] niks3-server[1059]: 2026/09/22 14:51:59 INFO Signed narinfos id=25 count=15614worker2 # [ 89.454083] nix-grpc-daemon[884]: time=2026-09-22T14:52:01.480Z level=INFO msg="Uploading 1 narinfos"5615lb # [ 89.453110] niks3-server[1059]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5616lb # [ 89.461507] niks3-server[1059]: 2026/09/22 14:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5617worker2 # [ 89.478377] nix-grpc-daemon[884]: time=2026-09-22T14:52:01.504Z level=INFO msg="Upload complete. (270ms)"5618lb # [ 89.469194] niks3-server[1059]: 2026/09/22 14:51:59 INFO Completed upload id=255619worker2 # [ 89.481199] nix-grpc-daemon[782]: ts=2026-09-22T14:52:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_in=7525620lb # [ 89.473396] niks3-server[1059]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5621worker2 # [ 89.486566] nix-grpc-daemon[782]: ts=2026-09-22T14:52:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265622worker2 # [ 89.495445] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5623worker2 # [ 89.499112] nix-daemon[869]: reaped child process 1276, status = succeeded5624worker2 # [ 89.502838] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5625worker2 # [ 90.319055] nix-daemon[869]: reaped child process 1290, status = succeeded5626worker2 # [ 90.390958] nix-daemon[869]: accepted connection from pid 1306, user nix-grpc-daemon (trusted)5627worker2 # [ 90.404296] nix-daemon[869]: reaped child process 1309, status = succeeded5628lb # [ 90.401046] niks3-server[1059]: 2026/09/22 14:52:00 INFO Received uploads request method=POST path=/api/pending_closures5629worker2 # [ 90.415282] nix-grpc-daemon[884]: time=2026-09-22T14:52:02.441Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5630worker2 # [ 90.416709] nix-grpc-daemon[884]: time=2026-09-22T14:52:02.443Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5631lb # [ 90.422506] niks3-server[1059]: 2026/09/22 14:52:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5632worker2 # [ 90.434272] nix-grpc-daemon[884]: time=2026-09-22T14:52:02.460Z level=INFO msg="Uploading 1 narinfos"5633lb # [ 90.425114] niks3-server[1059]: 2026/09/22 14:52:00 INFO Signed narinfos id=26 count=15634lb # [ 90.427200] niks3-server[1059]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5635lb # [ 90.432180] niks3-server[1059]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5636lb # [ 90.437206] niks3-server[1059]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5637lb # [ 90.442382] niks3-server[1059]: 2026/09/22 14:52:00 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5638lb # [ 90.445286] niks3-server[1059]: 2026/09/22 14:52:00 INFO Completed upload id=265639worker2 # [ 90.454661] nix-grpc-daemon[884]: time=2026-09-22T14:52:02.481Z level=INFO msg="Upload complete. (145ms)"5640worker2 # [ 90.457251] nix-grpc-daemon[782]: ts=2026-09-22T14:52:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15641lb # [ 90.449603] niks3-server[1059]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5642worker2 # [ 90.462557] nix-grpc-daemon[782]: ts=2026-09-22T14:52:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375265643worker2 # [ 90.467266] nix-daemon[869]: reaped child process 1288, status = succeeded5644worker1 # [ 90.494302] nix-grpc-daemon[781]: ts=2026-09-22T14:52:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:575505645worker1 # [ 90.504583] nix-grpc-daemon[781]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5646worker1 # [ 90.516273] nix-grpc-daemon[781]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5647worker1 # [ 90.525259] nix-grpc-daemon[781]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5648worker1 # [ 90.534993] nix-grpc-daemon[781]: ts=2026-09-22T14:52:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=1 nar_bytes_out=2325649client # [ 90.626252] nix-daemon[1106]: reaped child process 1108, status = succeeded5650client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.05 seconds)5651client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5652client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5653client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5654client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5655client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5656client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5657(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.10 seconds)5658subtest: an input only one worker has still reaches the builder5659client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5660client # this derivation will be built:5661client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5662client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5663client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.21 seconds)5664client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5665client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5666worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5667worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5668worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5669worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5670client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25671client # [ 91.251652] nix-daemon[1106]: accepted connection from pid 1189, user root (trusted)5672client # this derivation will be built:5673client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5674worker2 # [ 91.356380] nix-grpc-daemon[782]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:375265675client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5676worker1 # [ 91.382940] nix-grpc-daemon[781]: ts=2026-09-22T14:52:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505677worker1 # [ 91.487984] nix-daemon[849]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)5678worker1 # [ 91.500607] nix-daemon[849]: reaped child process 1110, status = succeeded5679lb # [ 91.476420] niks3-server[1059]: 2026/09/22 14:52:01 INFO Received uploads request method=POST path=/api/pending_closures5680worker1 # [ 91.516624] nix-grpc-daemon[874]: time=2026-09-22T14:52:02.714Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5681worker1 # [ 91.518383] nix-grpc-daemon[874]: time=2026-09-22T14:52:02.715Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5682lb # [ 91.504050] niks3-server[1059]: 2026/09/22 14:52:01 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5683lb # [ 91.515396] niks3-server[1059]: 2026/09/22 14:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5684worker1 # [ 91.553917] nix-grpc-daemon[874]: time=2026-09-22T14:52:02.751Z level=INFO msg="Uploading 1 narinfos"5685lb # [ 91.523126] niks3-server[1059]: 2026/09/22 14:52:01 INFO Signed narinfos id=27 count=15686lb # [ 91.530122] niks3-server[1059]: 2026/09/22 14:52:01 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5687lb # [ 91.542570] niks3-server[1059]: 2026/09/22 14:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5688worker1 # [ 91.579628] nix-grpc-daemon[874]: time=2026-09-22T14:52:02.777Z level=INFO msg="Upload complete. (192ms)"5689lb # [ 91.548938] niks3-server[1059]: 2026/09/22 14:52:01 INFO Completed upload id=275690lb # [ 91.553546] niks3-server[1059]: 2026/09/22 14:52:01 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5691worker2 # [ 91.572858] nix-grpc-daemon[782]: ts=2026-09-22T14:52:03Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05692worker2 # [ 91.574865] nix-grpc-daemon[782]: ts=2026-09-22T14:52:03Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15693worker1 # [ 91.638801] nix-grpc-daemon[781]: ts=2026-09-22T14:52:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505694worker1 # [ 91.649602] nix-grpc-daemon[781]: ts=2026-09-22T14:52:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505695client # copying 2 paths to 'lb:50051'...5696client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5697worker1 # [ 91.664910] nix-grpc-daemon[781]: ts=2026-09-22T14:52:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575505698client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5699worker1 # [ 91.673140] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5700worker1 # [ 91.768348] nix-daemon[849]: accepted connection from pid 1114, user nix-grpc-daemon (trusted)5701worker1 # [ 91.781062] nix-daemon[849]: reaped child process 1117, status = succeeded5702worker1 # [ 91.854744] nix-daemon[849]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5703worker1 # [ 91.871497] nix-daemon[849]: reaped child process 1122, status = succeeded5704lb # [ 91.840281] niks3-server[1059]: 2026/09/22 14:52:02 INFO Received uploads request method=POST path=/api/pending_closures5705lb # [ 91.851802] niks3-server[1059]: 2026/09/22 14:52:02 INFO Received uploads request method=POST path=/api/pending_closures5706worker1 # [ 91.891546] nix-grpc-daemon[874]: time=2026-09-22T14:52:03.089Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5707worker1 # [ 91.892992] nix-grpc-daemon[874]: time=2026-09-22T14:52:03.090Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5708worker1 # [ 91.894706] nix-grpc-daemon[874]: time=2026-09-22T14:52:03.092Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5709lb # [ 91.900092] niks3-server[1059]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5710lb # [ 91.903584] niks3-server[1059]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5711lb # [ 91.906630] niks3-server[1059]: 2026/09/22 14:52:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5712lb # [ 91.909431] niks3-server[1059]: 2026/09/22 14:52:02 INFO Signed narinfos id=29 count=25713lb # [ 91.911995] niks3-server[1059]: 2026/09/22 14:52:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5714lb # [ 91.914679] niks3-server[1059]: 2026/09/22 14:52:02 INFO Signed narinfos id=28 count=15715lb # [ 91.916292] niks3-server[1059]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5716worker1 # [ 91.947590] nix-grpc-daemon[874]: time=2026-09-22T14:52:03.144Z level=INFO msg="Uploading 3 narinfos"5717lb # [ 91.920916] niks3-server[1059]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5718lb # [ 91.930627] niks3-server[1059]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5719lb # [ 91.935443] niks3-server[1059]: 2026/09/22 14:52:02 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5720lb # [ 91.939171] niks3-server[1059]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5721lb # [ 91.942086] niks3-server[1059]: 2026/09/22 14:52:02 INFO Completed upload id=295722lb # [ 91.943796] niks3-server[1059]: 2026/09/22 14:52:02 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5723lb # [ 91.946870] niks3-server[1059]: 2026/09/22 14:52:02 INFO Completed upload id=285724lb # [ 91.949282] niks3-server[1059]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5725worker1 # [ 91.978965] nix-grpc-daemon[874]: time=2026-09-22T14:52:03.176Z level=INFO msg="Upload complete. (292ms)"5726worker1 # [ 91.983754] nix-grpc-daemon[781]: ts=2026-09-22T14:52:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=2 nar_bytes_in=9765727worker1 # [ 91.989855] nix-grpc-daemon[781]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505728worker1 # [ 92.001108] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5729worker1 # [ 92.009837] nix-daemon[849]: reaped child process 1112, status = succeeded5730worker1 # [ 92.016371] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5731client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5732client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5733client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5734worker1 # [ 93.232182] nix-daemon[849]: reaped child process 1126, status = succeeded5735worker1 # [ 93.300799] nix-daemon[849]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)5736worker1 # [ 93.313255] nix-daemon[849]: reaped child process 1145, status = succeeded5737lb # [ 93.287650] niks3-server[1059]: 2026/09/22 14:52:03 INFO Received uploads request method=POST path=/api/pending_closures5738worker1 # [ 93.323678] nix-grpc-daemon[874]: time=2026-09-22T14:52:04.521Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5739lb # [ 93.314864] niks3-server[1059]: 2026/09/22 14:52:03 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5740lb # [ 93.321070] niks3-server[1059]: 2026/09/22 14:52:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5741lb # [ 93.325992] niks3-server[1059]: 2026/09/22 14:52:03 INFO Signed narinfos id=30 count=15742worker1 # [ 93.359800] nix-grpc-daemon[874]: time=2026-09-22T14:52:04.557Z level=INFO msg="Uploading 1 narinfos"5743lb # [ 93.336585] niks3-server[1059]: 2026/09/22 14:52:03 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5744lb # [ 93.344876] niks3-server[1059]: 2026/09/22 14:52:03 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5745lb # [ 93.352121] niks3-server[1059]: 2026/09/22 14:52:03 INFO Completed upload id=305746worker1 # [ 93.385118] nix-grpc-daemon[874]: time=2026-09-22T14:52:04.582Z level=INFO msg="Upload complete. (160ms)"5747lb # [ 93.359153] niks3-server[1059]: 2026/09/22 14:52:03 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5748worker1 # [ 93.388821] nix-grpc-daemon[781]: ts=2026-09-22T14:52:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15749client # copying 1 paths...5750worker1 # [ 93.397557] nix-grpc-daemon[781]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575505751client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5752worker1 # [ 93.405599] nix-daemon[849]: reaped child process 1124, status = succeeded5753worker1 # [ 93.414319] nix-grpc-daemon[781]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:575505754worker1 # [ 93.416680] nix-grpc-daemon[781]: ts=2026-09-22T14:52:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=1 nar_bytes_out=1205755client # [ 93.515600] nix-daemon[1106]: reaped child process 1192, status = succeeded5756client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.38 seconds)5757client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25758client # [ 93.620114] nix-daemon[1106]: accepted connection from pid 1211, user root (trusted)5759client # this derivation will be built:5760client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5761client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5762worker2 # [ 93.726469] nix-grpc-daemon[782]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:375265763worker2 # [ 93.732586] nix-grpc-daemon[782]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265764worker2 # [ 93.748211] nix-grpc-daemon[782]: ts=2026-09-22T14:52:05Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05765worker2 # [ 93.754340] nix-grpc-daemon[782]: ts=2026-09-22T14:52:05Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15766worker2 # [ 93.759625] nix-grpc-daemon[782]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265767worker2 # [ 93.769627] nix-grpc-daemon[782]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265768client # copying 1 paths to 'lb:50051'...5769client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5770worker2 # [ 93.785769] nix-grpc-daemon[782]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375265771worker2 # [ 93.796977] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5772worker2 # [ 93.811414] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 1/5)5773worker2 # [ 93.826200] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 2/5)5774worker2 # [ 93.931803] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 363 ms (attempt 3/5)5775worker2 # [ 94.296438] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 513 ms (attempt 4/5)5776worker2 # [ 94.812875] nix-grpc-daemon[782]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5777worker2 # [ 94.816504] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5778worker2 # [ 94.821819] nix-grpc-daemon[782]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5779worker2 # [ 94.931431] nix-daemon[869]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5780worker2 # [ 94.947810] nix-daemon[869]: reaped child process 1328, status = succeeded5781worker2 # [ 95.029190] nix-daemon[869]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)5782worker2 # [ 95.046285] nix-daemon[869]: reaped child process 1333, status = succeeded5783lb # [ 95.041790] niks3-server[1059]: 2026/09/22 14:52:05 INFO Received uploads request method=POST path=/api/pending_closures5784worker2 # [ 95.055722] nix-grpc-daemon[884]: time=2026-09-22T14:52:07.081Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5785worker2 # [ 95.057143] nix-grpc-daemon[884]: time=2026-09-22T14:52:07.081Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5786lb # [ 95.053294] niks3-server[1059]: 2026/09/22 14:52:05 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5787lb # [ 95.058279] niks3-server[1059]: 2026/09/22 14:52:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5788lb # [ 95.061063] niks3-server[1059]: 2026/09/22 14:52:05 INFO Signed narinfos id=31 count=15789worker2 # [ 95.070606] nix-grpc-daemon[884]: time=2026-09-22T14:52:07.096Z level=INFO msg="Uploading 1 narinfos"5790lb # [ 95.067828] niks3-server[1059]: 2026/09/22 14:52:05 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5791lb # [ 95.071289] niks3-server[1059]: 2026/09/22 14:52:05 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5792lb # [ 95.073689] niks3-server[1059]: 2026/09/22 14:52:05 INFO Completed upload id=315793worker2 # [ 95.083259] nix-grpc-daemon[884]: time=2026-09-22T14:52:07.109Z level=INFO msg="Upload complete. (247ms)"5794worker2 # [ 95.085261] nix-grpc-daemon[782]: ts=2026-09-22T14:52:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=1 paths=1 nar_bytes_in=5605795lb # [ 95.076763] niks3-server[1059]: 2026/09/22 14:52:05 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5796worker2 # [ 95.088737] nix-grpc-daemon[782]: ts=2026-09-22T14:52:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375265797worker2 # [ 95.094950] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5798worker2 # [ 95.099064] nix-daemon[869]: reaped child process 1318, status = succeeded5799worker2 # [ 95.102109] nix-grpc-daemon[782]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5800worker2 # [ 95.109290] nix-daemon[869]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5801client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5802client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5803client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5804worker2 # [ 95.522857] nix-daemon[869]: reaped child process 1338, status = succeeded5805worker2 # [ 95.600657] nix-daemon[869]: accepted connection from pid 1354, user nix-grpc-daemon (trusted)5806worker2 # [ 95.615865] nix-daemon[869]: reaped child process 1357, status = succeeded5807lb # [ 95.610770] niks3-server[1059]: 2026/09/22 14:52:06 INFO Received uploads request method=POST path=/api/pending_closures5808worker2 # [ 95.630461] nix-grpc-daemon[884]: time=2026-09-22T14:52:07.656Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5809lb # [ 95.643176] niks3-server[1059]: 2026/09/22 14:52:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5810lb # [ 95.649139] niks3-server[1059]: 2026/09/22 14:52:06 INFO Signed narinfos id=32 count=15811worker2 # [ 95.661864] nix-grpc-daemon[884]: time=2026-09-22T14:52:07.688Z level=INFO msg="Uploading 1 narinfos"5812lb # [ 95.657125] niks3-server[1059]: 2026/09/22 14:52:06 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5813lb # [ 95.671819] niks3-server[1059]: 2026/09/22 14:52:06 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5814lb # [ 95.677974] niks3-server[1059]: 2026/09/22 14:52:06 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5815lb # [ 95.683422] niks3-server[1059]: 2026/09/22 14:52:06 INFO Completed upload id=325816worker2 # [ 95.693250] nix-grpc-daemon[884]: time=2026-09-22T14:52:07.719Z level=INFO msg="Upload complete. (177ms)"5817lb # [ 95.688415] niks3-server[1059]: 2026/09/22 14:52:06 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5818worker2 # [ 95.698787] nix-grpc-daemon[782]: ts=2026-09-22T14:52:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15819client # copying 1 paths...5820worker2 # [ 95.710333] nix-grpc-daemon[782]: ts=2026-09-22T14:52:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375265821client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5822worker2 # [ 95.719440] nix-daemon[869]: reaped child process 1335, status = succeeded5823worker2 # [ 95.729377] nix-grpc-daemon[782]: ts=2026-09-22T14:52:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375265824worker2 # [ 95.732202] nix-grpc-daemon[782]: ts=2026-09-22T14:52:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_out=1205825client # [ 95.851298] nix-daemon[1106]: reaped child process 1214, status = succeeded5826client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.34 seconds)5827client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25828client # [ 95.953163] nix-daemon[1106]: accepted connection from pid 1233, user root (trusted)5829client # this derivation will be built:5830client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5831worker1 # [ 96.080334] nix-grpc-daemon[781]: ts=2026-09-22T14:52:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:575505832client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5833worker2 # [ 96.064602] nix-grpc-daemon[782]: ts=2026-09-22T14:52:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375265834worker2 # [ 96.071628] nix-grpc-daemon[884]: time=2026-09-22T14:52:08.096Z level=INFO msg="All 1 paths already cached"5835worker2 # [ 96.081203] nix-grpc-daemon[782]: ts=2026-09-22T14:52:08Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05836worker2 # [ 96.086723] nix-grpc-daemon[782]: ts=2026-09-22T14:52:08Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15837worker1 # [ 96.111100] nix-grpc-daemon[781]: ts=2026-09-22T14:52:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505838worker1 # [ 96.123494] nix-grpc-daemon[781]: ts=2026-09-22T14:52:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575505839worker1 # [ 96.137348] nix-grpc-daemon[874]: time=2026-09-22T14:52:07.334Z level=INFO msg="All 1 paths already cached"5840client # copying 1 paths to 'lb:50051'...5841client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5842worker1 # [ 96.146144] nix-grpc-daemon[781]: ts=2026-09-22T14:52:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575505843worker1 # [ 96.155656] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5844worker1 # [ 96.269931] nix-daemon[849]: accepted connection from pid 1149, user nix-grpc-daemon (trusted)5845worker1 # [ 96.286706] nix-daemon[849]: reaped child process 1152, status = succeeded5846worker1 # [ 96.379929] nix-daemon[849]: accepted connection from pid 1154, user nix-grpc-daemon (trusted)5847worker1 # [ 96.396957] nix-daemon[849]: reaped child process 1157, status = succeeded5848lb # [ 96.371392] niks3-server[1059]: 2026/09/22 14:52:06 INFO Received uploads request method=POST path=/api/pending_closures5849worker1 # [ 96.406288] nix-grpc-daemon[874]: time=2026-09-22T14:52:07.603Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5850worker1 # [ 96.407989] nix-grpc-daemon[874]: time=2026-09-22T14:52:07.605Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5851lb # [ 96.388344] niks3-server[1059]: 2026/09/22 14:52:06 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5852lb # [ 96.392053] niks3-server[1059]: 2026/09/22 14:52:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5853lb # [ 96.394783] niks3-server[1059]: 2026/09/22 14:52:06 INFO Signed narinfos id=33 count=15854worker1 # [ 96.426830] nix-grpc-daemon[874]: time=2026-09-22T14:52:07.624Z level=INFO msg="Uploading 1 narinfos"5855lb # [ 96.397586] niks3-server[1059]: 2026/09/22 14:52:06 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5856lb # [ 96.403107] niks3-server[1059]: 2026/09/22 14:52:06 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5857lb # [ 96.406509] niks3-server[1059]: 2026/09/22 14:52:06 INFO Completed upload id=335858worker1 # [ 96.437407] nix-grpc-daemon[874]: time=2026-09-22T14:52:07.635Z level=INFO msg="Upload complete. (266ms)"5859lb # [ 96.409347] niks3-server[1059]: 2026/09/22 14:52:06 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5860worker1 # [ 96.440458] nix-grpc-daemon[781]: ts=2026-09-22T14:52:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=0 paths=1 nar_bytes_in=5605861worker1 # [ 96.445076] nix-grpc-daemon[781]: ts=2026-09-22T14:52:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575505862worker1 # [ 96.454567] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5863worker1 # [ 96.458253] nix-daemon[849]: reaped child process 1147, status = succeeded5864worker1 # [ 96.462877] nix-daemon[849]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5865client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5866client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5867client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5868worker1 # [ 97.292639] nix-daemon[849]: reaped child process 1161, status = succeeded5869worker1 # [ 97.378089] nix-daemon[849]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5870worker1 # [ 97.393405] nix-daemon[849]: reaped child process 1180, status = succeeded5871lb # [ 97.367833] niks3-server[1059]: 2026/09/22 14:52:07 INFO Received uploads request method=POST path=/api/pending_closures5872worker1 # [ 97.405284] nix-grpc-daemon[874]: time=2026-09-22T14:52:08.601Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5873lb # [ 97.393748] niks3-server[1059]: 2026/09/22 14:52:07 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5874lb # [ 97.402953] niks3-server[1059]: 2026/09/22 14:52:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5875worker1 # [ 97.438924] nix-grpc-daemon[874]: time=2026-09-22T14:52:08.636Z level=INFO msg="Uploading 1 narinfos"5876lb # [ 97.408495] niks3-server[1059]: 2026/09/22 14:52:07 INFO Signed narinfos id=34 count=15877lb # [ 97.415722] niks3-server[1059]: 2026/09/22 14:52:07 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5878lb # [ 97.428199] niks3-server[1059]: 2026/09/22 14:52:07 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5879lb # [ 97.434886] niks3-server[1059]: 2026/09/22 14:52:07 INFO Completed upload id=345880worker1 # [ 97.468195] nix-grpc-daemon[874]: time=2026-09-22T14:52:08.665Z level=INFO msg="Upload complete. (198ms)"5881lb # [ 97.439363] niks3-server[1059]: 2026/09/22 14:52:07 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5882client # copying 1 paths...5883worker1 # [ 97.474726] nix-grpc-daemon[781]: ts=2026-09-22T14:52:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57550 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15884worker2 # [ 97.460391] nix-grpc-daemon[782]: ts=2026-09-22T14:52:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:375265885worker1 # [ 97.487815] nix-daemon[849]: reaped child process 1159, status = succeeded5886worker2 # [ 97.468926] nix-grpc-daemon[782]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5887client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5888worker2 # [ 97.501740] nix-grpc-daemon[782]: ts=2026-09-22T14:52:09Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:375265889worker2 # [ 97.506494] nix-grpc-daemon[782]: ts=2026-09-22T14:52:09Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37526 duration_s=0 paths=1 nar_bytes_out=1205890client # [ 97.628895] nix-daemon[1106]: reaped child process 1236, status = succeeded5891client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.78 seconds)5892(finished: subtest: an input only one worker has still reaches the builder, in 6.94 seconds)5893subtest: upload whose reference the worker lacks is completed from the cache5894client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5895lb # [ 97.648788] envoy[924]: conn peer=[2001:db8:1::1]:33464 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3850 tx=2163 ms=12685896lb # [ 97.652423] envoy[924]: conn peer=[2001:db8:1::1]:33470 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=555897lb # [ 97.655976] envoy[924]: conn peer=[2001:db8:1::1]:33478 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4341 tx=2387 ms=14415898lb # [ 97.659469] envoy[924]: conn peer=[2001:db8:1::1]:33484 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=905899lb # [ 97.662827] envoy[924]: conn peer=[2001:db8:1::1]:33490 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3851 tx=2064 ms=20935900lb # [ 97.666882] envoy[924]: conn peer=[2001:db8:1::1]:35328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=525901lb # [ 97.670325] envoy[924]: conn peer=[2001:db8:1::1]:35336 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3724 tx=2086 ms=20585902lb # [ 97.673952] envoy[924]: conn peer=[2001:db8:1::1]:35338 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=545903lb # [ 97.677408] envoy[924]: conn peer=[2001:db8:1::1]:35354 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3729 tx=2098 ms=15025904lb # [ 97.680891] envoy[924]: conn peer=[2001:db8:1::1]:35370 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=645905client # this derivation will be built:5906client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5907client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5908client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5909client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5910client # this derivation will be built:5911client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5912client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5913client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.23 seconds)5914client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5915client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5916worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5917worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5918worker2 # [ 98.362677] systemd[1]: Stopping Nix worker-protocol over gRPC...5919worker2 # [ 98.901607] nix-grpc-daemon[782]: ts=2026-09-22T14:52:10Z level=info event=signal_exit5920lb # [ 99.613963] niks3-server[1059]: 2026/09/22 14:52:10 INFO lead: released remote=192.168.1.4:581965921lb # [ 99.615796] niks3-server[1059]: 2026/09/22 14:52:10 INFO lead: acquired remote=192.168.1.3:366565922worker1 # [ 99.648118] nix-grpc-daemon[781]: ts=2026-09-22T14:52:10Z level=info event=scheduler_take_over5923worker1 # [ 100.646233] nix-grpc-daemon[781]: ts=2026-09-22T14:52:11Z level=info event=scheduler_restarting addr=lb:500515924worker2 # [ 100.624792] nix-grpc-daemon[782]: ts=2026-09-22T14:52:12Z level=info event=scheduler_restarting5925worker2 # [ 100.633478] nix-grpc-daemon[782]: ts=2026-09-22T14:52:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:375145926worker2 # [ 100.637260] nix-grpc-daemon[782]: ts=2026-09-22T14:52:12Z level=info event=worker_gone addr=192.168.1.3:500515927worker2 # [ 100.640780] nix-grpc-daemon[782]: ts=2026-09-22T14:52:12Z level=info event=scheduler_restarting addr=lb:500515928worker2 # [ 100.643856] nix-grpc-daemon[782]: ts=2026-09-22T14:52:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:375145929worker2 # [ 100.647495] nix-grpc-daemon[782]: ts=2026-09-22T14:52:12Z level=info event=worker_gone addr=192.168.1.4:500515930worker2 # [ 100.664728] nix-daemon[869]: reaped child process 872, status = succeeded5931worker1 # [ 100.851610] nix-grpc-daemon[781]: ts=2026-09-22T14:52:12Z level=info event=scheduler_connected addr=lb:500515932worker1 # [ 100.855617] nix-grpc-daemon[781]: ts=2026-09-22T14:52:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5933worker2 # [ 100.845240] nix-grpc-daemon[782]: ts=2026-09-22T14:52:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5934worker2 # [ 101.027386] nix-grpc-daemon[782]: E0000 00:00:1790088733.053468 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5935worker2 # [ 101.032252] nix-grpc-daemon[782]: E0000 00:00:1790088733.053665 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5936worker1 # [ 101.056723] nix-grpc-daemon[781]: ts=2026-09-22T14:52:12Z level=info event=scheduler_connected addr=lb:500515937worker1 # [ 101.067128] nix-grpc-daemon[781]: ts=2026-09-22T14:52:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:550405938worker1 # [ 101.070679] nix-grpc-daemon[781]: ts=2026-09-22T14:52:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05939worker2 # [ 101.307746] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5940worker2 # [ 101.311416] systemd[1]: Stopped Nix worker-protocol over gRPC.5941worker2 # [ 101.315550] systemd[1]: nix-grpc-daemon.service: Consumed 3.220s CPU time over 1min 23.572s wall clock time, 70.6M memory peak, 117.3K incoming IP traffic, 324.6K outgoing IP traffic.5942worker2 # [ 101.327199] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5943worker2 # [ 101.331524] systemd[1]: Closed Nix worker-protocol over gRPC.5944worker2 # [ 101.334817] systemd[1]: nix-grpc-daemon.socket: Consumed 319us CPU time over 1min 33.606s wall clock time, 256K memory peak, 121.9K incoming IP traffic, 102.5K outgoing IP traffic.5945lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5946lb: must succeed: curl -sf localhost:9901/clusters5947lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5948lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.02 seconds)5949client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25950worker1 # [ 101.522115] nix-grpc-daemon[781]: ts=2026-09-22T14:52:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575505951worker1 # [ 101.528684] nix-grpc-daemon[781]: ts=2026-09-22T14:52:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575505952client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5953client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds)5954worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5955worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5956worker2 # [ 101.586862] systemd[1]: Listening on Nix worker-protocol over gRPC.5957worker1 # [ 101.655286] systemd[1]: Stopping Nix worker-protocol over gRPC...5958worker1 # [ 101.824002] nix-grpc-daemon[781]: ts=2026-09-22T14:52:13Z level=info event=signal_exit5959worker2 # [ 102.300913] systemd[1]: Starting scheduler-token.service...5960worker2 # [ 102.334755] systemd[1]: scheduler-token.service: Deactivated successfully.5961worker2 # [ 102.336332] systemd[1]: Finished scheduler-token.service.5962worker2 # [ 102.337537] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 35ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5963worker2 # [ 102.341722] systemd[1]: Starting Nix worker-protocol over gRPC...5964worker2 # [ 102.422171] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5965worker2 # [ 102.424208] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5966worker2 # [ 102.427553] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15967worker2 # [ 102.431882] nix-grpc-daemon[1384]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5968worker2 # [ 102.433251] nix-grpc-daemon[1384]: E0000 00:00:1790088734.458291 1384 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5969worker2 # [ 102.438856] nix-grpc-daemon[1384]: E0000 00:00:1790088734.463924 1384 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5970worker2 # [ 102.443546] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5971worker2 # [ 102.445213] systemd[1]: Started Nix worker-protocol over gRPC.5972worker2 # [ 102.451256] nix-grpc-daemon[1384]: E0000 00:00:1790088734.477747 1425 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5973worker2 # [ 102.465881] nix-grpc-daemon[1384]: E0000 00:00:1790088734.492316 1425 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5974worker2 # [ 102.470054] nix-grpc-daemon[1384]: E0000 00:00:1790088734.494604 1425 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5975worker2 # [ 102.480323] nix-grpc-daemon[1384]: E0000 00:00:1790088734.506765 1425 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5976worker2 # [ 102.489995] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=scheduler_connected addr=lb:500515977worker2 # [ 102.491560] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5978lb # [ 102.624541] niks3-server[1059]: 2026/09/22 14:52:13 INFO lead: released remote=192.168.1.3:366565979worker2 # [ 102.693087] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=scheduler_connected addr=lb:500515980worker2 # [ 102.697289] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5981worker2 # [ 102.897742] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=scheduler_connected addr=lb:500515982worker2 # [ 102.899512] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5983worker2 # [ 103.100440] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_connected addr=lb:500515984worker2 # [ 103.102172] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5985worker2 # [ 103.302858] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_connected addr=lb:500515986worker2 # [ 103.304682] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5987worker2 # [ 103.506966] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_connected addr=lb:500515988worker2 # [ 103.510255] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5989worker1 # [ 103.660354] nix-grpc-daemon[781]: ts=2026-09-22T14:52:14Z level=info event=scheduler_restarting5990worker1 # [ 103.664596] nix-grpc-daemon[781]: ts=2026-09-22T14:52:14Z level=info event=scheduler_restarting addr=lb:500515991worker1 # [ 103.668526] nix-grpc-daemon[781]: ts=2026-09-22T14:52:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:550405992worker1 # [ 103.671820] nix-grpc-daemon[781]: ts=2026-09-22T14:52:14Z level=info event=worker_gone addr=192.168.1.3:500515993worker1 # [ 103.694554] nix-daemon[849]: reaped child process 851, status = succeeded5994worker2 # [ 103.707994] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_connected addr=lb:500515995worker2 # [ 103.712117] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5996worker1 # [ 103.866080] nix-grpc-daemon[781]: ts=2026-09-22T14:52:15Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5997worker2 # [ 103.913103] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_connected addr=lb:500515998worker2 # [ 103.917207] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5999worker1 # [ 103.973634] nix-grpc-daemon[781]: E0000 00:00:1790088735.170250 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6000worker1 # [ 103.979296] nix-grpc-daemon[781]: E0000 00:00:1790088735.170482 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6001worker2 # [ 104.118201] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_connected addr=lb:500516002worker2 # [ 104.122178] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6003worker2 # [ 104.322755] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_connected addr=lb:500516004worker2 # [ 104.324581] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6005worker2 # [ 104.525495] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_connected addr=lb:500516006worker2 # [ 104.527268] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6007worker2 # [ 104.727948] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_connected addr=lb:500516008worker2 # [ 104.729667] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6009worker2 # [ 104.930856] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_connected addr=lb:500516010worker2 # [ 104.934809] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6011worker2 # [ 105.135821] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_connected addr=lb:500516012worker2 # [ 105.139859] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6013worker1 # [ 105.291401] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6014worker1 # [ 105.293801] systemd[1]: Stopped Nix worker-protocol over gRPC.6015worker1 # [ 105.296714] systemd[1]: nix-grpc-daemon.service: Consumed 2.369s CPU time over 1min 27.618s wall clock time, 60.7M memory peak, 90.9K incoming IP traffic, 170.8K outgoing IP traffic.6016worker1 # [ 105.302773] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6017worker1 # [ 105.304601] systemd[1]: Closed Nix worker-protocol over gRPC.6018worker1 # [ 105.305975] systemd[1]: nix-grpc-daemon.socket: Consumed 315us CPU time over 1min 37.709s wall clock time, 256K memory peak, 96.4K incoming IP traffic, 84.5K outgoing IP traffic.6019lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy6020lb: must succeed: curl -sf localhost:9901/clusters6021lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6022lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.03 seconds)6023client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&26024worker2 # [ 105.340436] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_connected addr=lb:500516025worker2 # [ 105.342489] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6026worker2 # [ 105.465255] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387586027worker2 # [ 105.476744] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6028worker2 # [ 105.492769] nix-grpc-daemon[1384]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 1/5)6029worker2 # [ 105.545295] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_connected addr=lb:500516030worker2 # [ 105.548409] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6031worker2 # [ 105.571231] nix-grpc-daemon[1384]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 2/5)6032worker2 # [ 105.630628] nix-grpc-daemon[1384]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 364 ms (attempt 3/5)6033worker2 # [ 105.746448] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_connected addr=lb:500516034worker2 # [ 105.750738] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6035worker2 # [ 105.951634] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_connected addr=lb:500516036worker2 # [ 105.955532] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6037worker2 # [ 105.996765] nix-grpc-daemon[1384]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 283 ms (attempt 4/5)6038worker2 # [ 106.156783] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=scheduler_connected addr=lb:500516039worker2 # [ 106.160654] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6040worker2 # [ 106.282314] nix-grpc-daemon[1384]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6041worker2 # [ 106.286145] nix-grpc-daemon[1384]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6042client # copying 2 paths...6043client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...6044worker2 # [ 106.308147] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387586045client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...6046worker2 # [ 106.318581] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6047worker2 # [ 106.343432] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=niks3_push_started pid=14446048worker2 # [ 106.361630] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=scheduler_connected addr=lb:500516049worker2 # [ 106.365427] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6050worker2 # [ 106.455830] nix-grpc-daemon[1444]: time=2026-09-22T14:52:18.481Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6051worker2 # [ 106.546340] nix-daemon[869]: accepted connection from pid 1457, user nix-grpc-daemon (trusted)6052worker2 # [ 106.561119] nix-daemon[869]: reaped child process 1460, status = succeeded6053worker2 # [ 106.566162] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=scheduler_connected addr=lb:500516054lb # [ 106.555649] niks3-server[1059]: 2026/09/22 14:52:16 INFO Received uploads request method=POST path=/api/pending_closures6055worker2 # [ 106.570250] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6056lb # [ 106.568145] niks3-server[1059]: 2026/09/22 14:52:16 INFO Received uploads request method=POST path=/api/pending_closures6057worker2 # [ 106.585075] nix-grpc-daemon[1444]: time=2026-09-22T14:52:18.610Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"6058worker2 # [ 106.586459] nix-grpc-daemon[1444]: time=2026-09-22T14:52:18.610Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"6059worker2 # [ 106.589780] nix-grpc-daemon[1444]: time=2026-09-22T14:52:18.610Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"6060lb # [ 106.617538] niks3-server[1059]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst6061lb # [ 106.631941] niks3-server[1059]: 2026/09/22 14:52:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign6062lb # [ 106.640312] niks3-server[1059]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls6063lb # [ 106.651252] niks3-server[1059]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls6064lb # [ 106.656134] niks3-server[1059]: 2026/09/22 14:52:17 INFO Signed narinfos id=35 count=16065lb # [ 106.657171] niks3-server[1059]: 2026/09/22 14:52:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign6066lb # [ 106.659143] niks3-server[1059]: 2026/09/22 14:52:17 INFO Signed narinfos id=36 count=26067worker2 # [ 106.670181] nix-grpc-daemon[1444]: time=2026-09-22T14:52:18.695Z level=INFO msg="Uploading 3 narinfos"6068lb # [ 106.663111] niks3-server[1059]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst6069lb # [ 106.678801] niks3-server[1059]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo6070lb # [ 106.684672] niks3-server[1059]: 2026/09/22 14:52:17 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete6071lb # [ 106.688993] niks3-server[1059]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo6072lb # [ 106.692623] niks3-server[1059]: 2026/09/22 14:52:17 INFO Completed upload id=356073lb # [ 106.694401] niks3-server[1059]: 2026/09/22 14:52:17 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete6074lb # [ 106.696531] niks3-server[1059]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo6075lb # [ 106.699493] niks3-server[1059]: 2026/09/22 14:52:17 INFO Completed upload id=366076worker2 # [ 106.709341] nix-grpc-daemon[1444]: time=2026-09-22T14:52:18.735Z level=INFO msg="Upload complete. (249ms)"6077worker2 # [ 106.714898] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38758 duration_s=0 paths=2 nar_bytes_in=3126078worker2 # [ 106.729475] nix-daemon[869]: reaped child process 1442, status = succeeded6079client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.43 seconds)6080worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache6081worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)6082worker2 # [ 106.771265] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=scheduler_connected addr=lb:500516083worker2 # [ 106.774298] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6084worker1 # [ 106.834118] systemd[1]: Listening on Nix worker-protocol over gRPC.6085worker1 # [ 106.846415] systemd[1]: Starting Nix worker-protocol over gRPC...6086worker1 # [ 106.932056] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6087worker1 # [ 106.934099] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6088worker1 # [ 106.937398] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16089worker1 # [ 106.941708] nix-grpc-daemon[1203]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6090worker1 # [ 106.943064] nix-grpc-daemon[1203]: E0000 00:00:1790088738.139230 1203 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6091worker1 # [ 106.949126] nix-grpc-daemon[1203]: E0000 00:00:1790088738.144598 1203 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6092worker1 # [ 106.952122] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6093worker1 # [ 106.953753] systemd[1]: Started Nix worker-protocol over gRPC.6094worker1 # [ 106.959429] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=scheduler_take_over6095lb # [ 106.928878] niks3-server[1059]: 2026/09/22 14:52:17 INFO lead: acquired remote=192.168.1.3:559606096worker1 # [ 106.962134] nix-grpc-daemon[1203]: E0000 00:00:1790088738.158461 1244 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6097lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6098lb: must succeed: curl -sf localhost:9901/clusters6099worker1 # [ 106.973740] nix-grpc-daemon[1203]: E0000 00:00:1790088738.171388 1244 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6100worker1 # [ 106.978055] nix-grpc-daemon[1203]: E0000 00:00:1790088738.173674 1244 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6101worker1 # [ 106.987744] nix-grpc-daemon[1203]: E0000 00:00:1790088738.185406 1244 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6102worker2 # [ 106.975389] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:19Z level=info event=scheduler_connected addr=lb:500516103worker1 # [ 106.999840] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=scheduler_connected addr=lb:500516104worker1 # [ 107.001450] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6105worker2 # [ 106.979754] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6106lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6107worker2 # [ 107.180889] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:19Z level=info event=scheduler_connected addr=lb:500516108worker1 # [ 107.202715] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=scheduler_connected addr=lb:500516109worker2 # [ 107.185179] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6110worker1 # [ 107.206761] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6111worker2 # [ 107.386323] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:19Z level=info event=scheduler_connected addr=lb:500516112worker1 # [ 107.407524] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=scheduler_connected addr=lb:500516113worker1 # [ 107.414739] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:503226114worker1 # [ 107.421093] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06115worker1 # [ 107.429319] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=oidc_jwks provider=mock keys=16116worker1 # [ 107.432410] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:503226117worker1 # [ 107.436406] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:18Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06118lb # [ 107.648921] envoy[924]: conn peer=[2001:db8:1::4]:58772 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=91306 tx=4830 ms=829216119lb # [ 107.652961] envoy[924]: conn peer=[2001:db8:1::1]:35384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=206120lb # [ 107.656507] envoy[924]: conn peer=[2001:db8:1::3]:55848 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=24041 tx=4525 ms=860136121lb # [ 107.660426] envoy[924]: conn peer=[2001:db8:1::1]:60182 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=281 ms=12666122lb: must succeed: curl -sf localhost:9901/clusters6123lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6124lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)6125(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.42 seconds)6126subtest: a niks3 restart does not cost the leader its role6127lb: must succeed: curl -sf localhost:9901/clusters6128lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6129lb: must succeed: curl -sf localhost:9901/clusters6130lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6131lb # [ 108.111224] niks3-server[1059]: 2026/09/22 14:52:18 INFO Shutdown signal received, draining in-flight requests timeout=10s6132lb # [ 108.112822] niks3-server[1059]: 2026/09/22 14:52:18 INFO lead: released remote=192.168.1.3:559606133worker1 # [ 108.142541] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:19Z level=info event=scheduler_lock_lost status=2006134worker2 # [ 108.121957] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:20Z level=info event=scheduler_lock_lost status=2006135lb # [ 108.118209] systemd[1]: Stopping niks3 server...6136lb # [ 108.130728] systemd[1]: niks3.service: Deactivated successfully.6137lb # [ 108.133083] systemd[1]: Stopped niks3 server.6138lb # [ 108.133741] systemd[1]: niks3.service: Consumed 688ms CPU time over 1min 18.744s wall clock time, 28.7M memory peak, 92.6K incoming IP traffic, 134.6K outgoing IP traffic.6139lb # [ 108.140768] systemd[1]: Starting niks3 server...6140lb # [ 108.189640] niks3-server[1194]: 2026/09/22 14:52:18 goose: no migrations to run. current version: 202609200000006141lb # [ 108.195576] niks3-server[1194]: 2026/09/22 14:52:18 OK 1_commit_pending_closure.sql (4.05ms)6142lb # [ 108.199069] niks3-server[1194]: 2026/09/22 14:52:18 OK 2_object_stats_trigger.sql (1.83ms)6143lb # [ 108.200157] niks3-server[1194]: 2026/09/22 14:52:18 goose: up to current file version: 26144lb # [ 108.201468] niks3-server[1194]: 2026/09/22 14:52:18 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key6145lb # [ 108.212618] niks3-server[1194]: 2026/09/22 14:52:18 INFO Read proxy enabled — serving cache objects from S36146lb # [ 108.214559] niks3-server[1194]: 2026/09/22 14:52:18 INFO Using socket-activated listener address=0.0.0.0:57516147lb # [ 108.216804] systemd[1]: Started niks3 server.6148lb # [ 108.218337] niks3-server[1194]: 2026/09/22 14:52:18 INFO Starting HTTP server address=0.0.0.0:57516149lb # [ 108.220452] niks3-server[1194]: 2026/09/22 14:52:18 INFO systemd watchdog enabled interval=15s6150lb: waiting for unit niks3.service6151lb: (finished: waiting for unit niks3.service, in 0.02 seconds)6152lb: must succeed: curl -sf localhost:9901/clusters6153lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6154worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6155worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6156lb: must succeed: curl -sf localhost:9901/clusters6157lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6158lb: must succeed: curl -sf localhost:9901/clusters6159lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6160worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield6161worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.05 seconds)6162client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&26163worker1 # [ 108.587148] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:503346164worker1 # [ 108.598534] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6165client # don't know how to build these paths:6166client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6167worker2 # [ 108.591541] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387586168worker1 # [ 108.626997] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:19Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=06169worker1 # [ 108.631713] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:19Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.3:50051 clients=16170worker1 # [ 108.675313] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503346171worker1 # [ 108.686145] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 1/5)6172worker1 # [ 108.709556] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 103 ms (attempt 2/5)6173worker1 # [ 108.815603] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 312 ms (attempt 3/5)6174worker1 # [ 109.129538] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 309 ms (attempt 4/5)6175lb # [ 109.116255] niks3-server[1194]: 2026/09/22 14:52:19 INFO lead: acquired remote=192.168.1.3:559726176worker1 # [ 109.442382] nix-grpc-daemon[1203]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6177worker1 # [ 109.445564] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6178worker1 # [ 109.462693] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503346179client # copying 1 paths to 'lb:50051'...6180client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...6181worker1 # [ 109.477087] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:503346182worker1 # [ 109.485142] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6183worker1 # [ 109.505604] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:20Z level=info event=niks3_push_started pid=12756184worker1 # [ 109.607668] nix-grpc-daemon[1275]: time=2026-09-22T14:52:20.805Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6185worker1 # [ 109.701957] nix-daemon[849]: accepted connection from pid 1288, user nix-grpc-daemon (trusted)6186worker1 # [ 109.713530] nix-daemon[849]: reaped child process 1291, status = succeeded6187worker1 # [ 109.796748] nix-daemon[849]: accepted connection from pid 1293, user nix-grpc-daemon (trusted)6188worker1 # [ 109.809081] nix-daemon[849]: reaped child process 1296, status = succeeded6189lb # [ 109.785339] niks3-server[1194]: 2026/09/22 14:52:20 INFO Received uploads request method=POST path=/api/pending_closures6190worker1 # [ 109.822366] nix-grpc-daemon[1275]: time=2026-09-22T14:52:21.020Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6191worker1 # [ 109.823893] nix-grpc-daemon[1275]: time=2026-09-22T14:52:21.021Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"6192lb # [ 109.809205] niks3-server[1194]: 2026/09/22 14:52:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6193lb # [ 109.816093] niks3-server[1194]: 2026/09/22 14:52:20 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst6194lb # [ 109.825311] niks3-server[1194]: 2026/09/22 14:52:20 INFO Signed narinfos id=37 count=16195worker1 # [ 109.857644] nix-grpc-daemon[1275]: time=2026-09-22T14:52:21.055Z level=INFO msg="Uploading 1 narinfos"6196lb # [ 109.834112] niks3-server[1194]: 2026/09/22 14:52:20 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls6197lb # [ 109.837784] niks3-server[1194]: 2026/09/22 14:52:20 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6198lb # [ 109.843823] niks3-server[1194]: 2026/09/22 14:52:20 INFO Completed upload id=376199worker1 # [ 109.874655] nix-grpc-daemon[1275]: time=2026-09-22T14:52:21.072Z level=INFO msg="Upload complete. (264ms)"6200worker1 # [ 109.876756] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50334 duration_s=0 paths=1 nar_bytes_in=5366201lb # [ 109.848074] niks3-server[1194]: 2026/09/22 14:52:20 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo6202worker1 # [ 109.881417] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503346203worker1 # [ 109.887978] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6204worker1 # [ 109.889360] nix-daemon[849]: reaped child process 1273, status = succeeded6205worker1 # [ 109.894568] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6206client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...6207client # farm-a> node-a: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6208client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6209client # farm-a> LOG-farm-a-after-niks3-restart6210worker1 # [ 111.329890] nix-daemon[849]: reaped child process 1300, status = succeeded6211worker1 # [ 111.395473] nix-daemon[849]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)6212worker1 # [ 111.409308] nix-daemon[849]: reaped child process 1319, status = succeeded6213lb # [ 111.384250] niks3-server[1194]: 2026/09/22 14:52:21 INFO Received uploads request method=POST path=/api/pending_closures6214worker1 # [ 111.420778] nix-grpc-daemon[1275]: time=2026-09-22T14:52:22.618Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6215worker1 # [ 111.422233] nix-grpc-daemon[1275]: time=2026-09-22T14:52:22.619Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"6216lb # [ 111.406167] niks3-server[1194]: 2026/09/22 14:52:21 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6217lb # [ 111.411758] niks3-server[1194]: 2026/09/22 14:52:21 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst6218lb # [ 111.415770] niks3-server[1194]: 2026/09/22 14:52:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6219lb # [ 111.417893] niks3-server[1194]: 2026/09/22 14:52:21 INFO Signed narinfos id=38 count=16220worker1 # [ 111.448607] nix-grpc-daemon[1275]: time=2026-09-22T14:52:22.646Z level=INFO msg="Uploading 1 narinfos"6221lb # [ 111.421398] niks3-server[1194]: 2026/09/22 14:52:21 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls6222lb # [ 111.426553] niks3-server[1194]: 2026/09/22 14:52:21 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6223lb # [ 111.430934] niks3-server[1194]: 2026/09/22 14:52:21 INFO Completed upload id=386224worker1 # [ 111.461692] nix-grpc-daemon[1275]: time=2026-09-22T14:52:22.659Z level=INFO msg="Upload complete. (159ms)"6225lb # [ 111.433134] niks3-server[1194]: 2026/09/22 14:52:21 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6226worker1 # [ 111.464325] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50334 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16227worker1 # [ 111.472201] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:22Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06228worker1 # [ 111.473976] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:22Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.3:50051 clients=16229worker1 # [ 111.478384] nix-daemon[849]: reaped child process 1298, status = succeeded6230worker1 # [ 111.516501] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503346231worker1 # [ 111.527186] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503346232worker1 # [ 111.539323] nix-grpc-daemon[1275]: time=2026-09-22T14:52:22.736Z level=INFO msg="All 1 paths already cached"6233client # copying 1 paths to 'lb:50051'...6234client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6235worker1 # [ 111.546533] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:503346236worker1 # [ 111.552811] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6237worker1 # [ 111.639868] nix-daemon[849]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)6238worker1 # [ 111.654179] nix-daemon[849]: reaped child process 1328, status = succeeded6239worker1 # [ 111.728460] nix-daemon[849]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)6240worker1 # [ 111.743675] nix-daemon[849]: reaped child process 1333, status = succeeded6241lb # [ 111.713915] niks3-server[1194]: 2026/09/22 14:52:22 INFO Received uploads request method=POST path=/api/pending_closures6242worker1 # [ 111.753726] nix-grpc-daemon[1275]: time=2026-09-22T14:52:22.951Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6243worker1 # [ 111.755082] nix-grpc-daemon[1275]: time=2026-09-22T14:52:22.951Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6244lb # [ 111.743868] niks3-server[1194]: 2026/09/22 14:52:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6245worker1 # [ 111.783724] nix-grpc-daemon[1275]: time=2026-09-22T14:52:22.981Z level=INFO msg="Uploading 1 narinfos"6246lb # [ 111.753737] niks3-server[1194]: 2026/09/22 14:52:22 INFO Signed narinfos id=39 count=16247lb # [ 111.762146] niks3-server[1194]: 2026/09/22 14:52:22 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6248lb # [ 111.766554] niks3-server[1194]: 2026/09/22 14:52:22 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6249lb # [ 111.776695] niks3-server[1194]: 2026/09/22 14:52:22 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6250lb # [ 111.782125] niks3-server[1194]: 2026/09/22 14:52:22 INFO Completed upload id=396251worker1 # [ 111.812928] nix-grpc-daemon[1275]: time=2026-09-22T14:52:23.010Z level=INFO msg="Upload complete. (249ms)"6252lb # [ 111.784703] niks3-server[1194]: 2026/09/22 14:52:22 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6253worker1 # [ 111.816539] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50334 duration_s=0 paths=1 nar_bytes_in=6566254worker1 # [ 111.821365] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503346255worker1 # [ 111.829264] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6256worker1 # [ 111.834910] nix-daemon[849]: reaped child process 1323, status = succeeded6257worker1 # [ 111.839201] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6258client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6259client # farm-b> node-a: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6260client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6261client # farm-b> LOG-farm-b-after-niks3-restart6262worker1 # [ 112.853817] nix-daemon[849]: reaped child process 1337, status = succeeded6263worker1 # [ 112.922302] nix-daemon[849]: accepted connection from pid 1353, user nix-grpc-daemon (trusted)6264worker1 # [ 112.936288] nix-daemon[849]: reaped child process 1356, status = succeeded6265lb # [ 112.905494] niks3-server[1194]: 2026/09/22 14:52:23 INFO Received uploads request method=POST path=/api/pending_closures6266worker1 # [ 112.951829] nix-grpc-daemon[1275]: time=2026-09-22T14:52:24.149Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6267worker1 # [ 112.953432] nix-grpc-daemon[1275]: time=2026-09-22T14:52:24.151Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6268lb # [ 112.943125] niks3-server[1194]: 2026/09/22 14:52:23 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6269lb # [ 112.954922] niks3-server[1194]: 2026/09/22 14:52:23 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6270lb # [ 112.965290] niks3-server[1194]: 2026/09/22 14:52:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6271lb # [ 112.968081] niks3-server[1194]: 2026/09/22 14:52:23 INFO Signed narinfos id=40 count=16272worker1 # [ 112.999094] nix-grpc-daemon[1275]: time=2026-09-22T14:52:24.196Z level=INFO msg="Uploading 1 narinfos"6273lb # [ 112.971240] niks3-server[1194]: 2026/09/22 14:52:23 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6274lb # [ 112.975238] niks3-server[1194]: 2026/09/22 14:52:23 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6275lb # [ 112.981621] niks3-server[1194]: 2026/09/22 14:52:23 INFO Completed upload id=406276worker1 # [ 113.012417] nix-grpc-daemon[1275]: time=2026-09-22T14:52:24.210Z level=INFO msg="Upload complete. (167ms)"6277lb # [ 112.983313] niks3-server[1194]: 2026/09/22 14:52:23 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6278worker1 # [ 113.015092] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50334 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16279worker1 # [ 113.022425] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:24Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06280worker1 # [ 113.024460] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:24Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.3:50051 clients=16281worker1 # [ 113.028244] nix-daemon[849]: reaped child process 1335, status = succeeded6282worker1 # [ 113.065990] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503346283worker1 # [ 113.072454] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:503346284worker1 # [ 113.078818] nix-grpc-daemon[1275]: time=2026-09-22T14:52:24.276Z level=INFO msg="All 2 paths already cached"6285client # copying 1 paths to 'lb:50051'...6286worker1 # [ 113.084369] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:503346287client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6288worker1 # [ 113.090654] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6289worker1 # [ 113.172171] nix-daemon[849]: accepted connection from pid 1362, user nix-grpc-daemon (trusted)6290worker1 # [ 113.185092] nix-daemon[849]: reaped child process 1365, status = succeeded6291worker1 # [ 113.264347] nix-daemon[849]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)6292worker1 # [ 113.279446] nix-daemon[849]: reaped child process 1370, status = succeeded6293lb # [ 113.247991] niks3-server[1194]: 2026/09/22 14:52:23 INFO Received uploads request method=POST path=/api/pending_closures6294worker1 # [ 113.287621] nix-grpc-daemon[1275]: time=2026-09-22T14:52:24.485Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6295worker1 # [ 113.289446] nix-grpc-daemon[1275]: time=2026-09-22T14:52:24.486Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6296lb # [ 113.281508] niks3-server[1194]: 2026/09/22 14:52:23 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6297lb # [ 113.287859] niks3-server[1194]: 2026/09/22 14:52:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6298lb # [ 113.291735] niks3-server[1194]: 2026/09/22 14:52:23 INFO Signed narinfos id=41 count=16299worker1 # [ 113.325351] nix-grpc-daemon[1275]: time=2026-09-22T14:52:24.522Z level=INFO msg="Uploading 1 narinfos"6300lb # [ 113.304369] niks3-server[1194]: 2026/09/22 14:52:23 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6301lb # [ 113.309749] niks3-server[1194]: 2026/09/22 14:52:23 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6302lb # [ 113.313859] niks3-server[1194]: 2026/09/22 14:52:23 INFO Completed upload id=416303worker1 # [ 113.344626] nix-grpc-daemon[1275]: time=2026-09-22T14:52:24.542Z level=INFO msg="Upload complete. (249ms)"6304worker1 # [ 113.346578] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50334 duration_s=0 paths=1 nar_bytes_in=7846305lb # [ 113.316466] niks3-server[1194]: 2026/09/22 14:52:23 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6306worker1 # [ 113.352363] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:503346307worker1 # [ 113.356434] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6308worker1 # [ 113.358636] nix-daemon[849]: reaped child process 1360, status = succeeded6309worker1 # [ 113.363623] nix-daemon[849]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6310client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6311client # farm-top> node-a: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6312client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6313client # farm-top> LOG-farm-top-after-niks3-restart6314worker1 # [ 114.604786] nix-daemon[849]: reaped child process 1374, status = succeeded6315worker1 # [ 114.684513] nix-daemon[849]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)6316worker1 # [ 114.695762] nix-daemon[849]: reaped child process 1393, status = succeeded6317lb # [ 114.672959] niks3-server[1194]: 2026/09/22 14:52:25 INFO Received uploads request method=POST path=/api/pending_closures6318worker1 # [ 114.713162] nix-grpc-daemon[1275]: time=2026-09-22T14:52:25.910Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6319worker1 # [ 114.714569] nix-grpc-daemon[1275]: time=2026-09-22T14:52:25.912Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6320lb # [ 114.711313] niks3-server[1194]: 2026/09/22 14:52:25 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6321lb # [ 114.724149] niks3-server[1194]: 2026/09/22 14:52:25 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6322lb # [ 114.733276] niks3-server[1194]: 2026/09/22 14:52:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6323worker1 # [ 114.770370] nix-grpc-daemon[1275]: time=2026-09-22T14:52:25.967Z level=INFO msg="Uploading 1 narinfos"6324lb # [ 114.739109] niks3-server[1194]: 2026/09/22 14:52:25 INFO Signed narinfos id=42 count=16325lb # [ 114.744062] niks3-server[1194]: 2026/09/22 14:52:25 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6326lb # [ 114.753662] niks3-server[1194]: 2026/09/22 14:52:25 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6327lb # [ 114.757297] niks3-server[1194]: 2026/09/22 14:52:25 INFO Completed upload id=426328worker1 # [ 114.788362] nix-grpc-daemon[1275]: time=2026-09-22T14:52:25.985Z level=INFO msg="Upload complete. (217ms)"6329lb # [ 114.760535] niks3-server[1194]: 2026/09/22 14:52:25 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6330worker1 # [ 114.795754] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50334 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16331worker1 # [ 114.810107] nix-daemon[849]: reaped child process 1372, status = succeeded6332client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 6.41 seconds)6333client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6334client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.13 seconds)6335(finished: subtest: a niks3 restart does not cost the leader its role, in 6.91 seconds)6336subtest: scheduler down: the other node takes the lock and keeps it6337lb: must succeed: curl -sf localhost:9901/clusters6338lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6339lb: must succeed: curl -sf localhost:9901/clusters6340lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6341lb: must succeed: curl -sf localhost:9901/clusters6342lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6343lb: must succeed: curl -sf localhost:9901/clusters6344lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6345worker1 # [ 115.131748] systemd[1]: Stopping Nix worker-protocol over gRPC...6346worker1 # [ 115.952288] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:27Z level=info event=signal_exit6347lb # [ 116.119972] niks3-server[1194]: 2026/09/22 14:52:26 INFO lead: released remote=192.168.1.3:559726348worker2 # [ 117.132899] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:29Z level=info event=scheduler_restarting addr=lb:500516349worker1 # [ 117.151976] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:28Z level=info event=scheduler_restarting6350worker1 # [ 117.161537] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:28Z level=info event=scheduler_restarting addr=lb:500516351worker1 # [ 117.164748] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503226352worker1 # [ 117.168328] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:28Z level=info event=worker_gone addr=192.168.1.4:500516353worker1 # [ 117.171590] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:503226354worker1 # [ 117.175183] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:28Z level=info event=worker_gone addr=192.168.1.3:500516355worker1 # [ 117.188979] nix-daemon[849]: reaped child process 1263, status = succeeded6356worker1 # [ 117.318861] nix-grpc-daemon[1203]: E0000 00:00:1790088748.516398 1248 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true6357worker1 # [ 117.324348] nix-grpc-daemon[1203]: E0000 00:00:1790088748.521746 1248 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6358worker2 # [ 117.337621] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:29Z level=info event=scheduler_connected addr=lb:500516359worker2 # [ 117.341480] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6360worker1 # [ 117.364969] nix-grpc-daemon[1203]: ts=2026-09-22T14:52:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6361worker2 # [ 117.542529] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:29Z level=info event=scheduler_connected addr=lb:500516362worker2 # [ 117.546373] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6363lb # [ 117.648104] envoy[924]: conn peer=[2001:db8:1::1]:60198 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9709 tx=4471 ms=62306364lb # [ 117.650399] envoy[924]: conn peer=[2001:db8:1::3]:47382 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1527 tx=541 ms=103636365worker2 # [ 117.744468] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:29Z level=info event=scheduler_connected addr=lb:500516366worker2 # [ 117.748309] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6367worker2 # [ 117.949248] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:29Z level=info event=scheduler_connected addr=lb:500516368worker2 # [ 117.953171] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6369worker2 # [ 118.155584] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:30Z level=info event=scheduler_connected addr=lb:500516370worker2 # [ 118.159118] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6371worker1 # [ 118.363944] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6372worker1 # [ 118.366290] systemd[1]: Stopped Nix worker-protocol over gRPC.6373worker1 # [ 118.367211] systemd[1]: nix-grpc-daemon.service: Consumed 929ms CPU time over 11.523s wall clock time, 38.8M memory peak, 35.4K incoming IP traffic, 59K outgoing IP traffic.6374worker1 # [ 118.371562] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6375worker1 # [ 118.373302] systemd[1]: Closed Nix worker-protocol over gRPC.6376worker1 # [ 118.374490] systemd[1]: nix-grpc-daemon.socket: Consumed 615us CPU time over 11.545s wall clock time, 256K memory peak, 33.7K incoming IP traffic, 24.6K outgoing IP traffic.6377worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6378worker2 # [ 118.359593] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:30Z level=info event=scheduler_connected addr=lb:500516379worker2 # [ 118.362079] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6380worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.03 seconds)6381worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6382worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6383worker2 # [ 118.562130] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:30Z level=info event=scheduler_connected addr=lb:500516384worker2 # [ 118.563811] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6385worker2 # [ 118.765238] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:30Z level=info event=scheduler_connected addr=lb:500516386worker2 # [ 118.769324] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6387worker2 # [ 118.970219] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:30Z level=info event=scheduler_connected addr=lb:500516388worker2 # [ 118.974229] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6389worker2 # [ 119.132554] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=scheduler_take_over6390lb # [ 119.123434] niks3-server[1194]: 2026/09/22 14:52:29 INFO lead: acquired remote=192.168.1.4:470666391worker2 # [ 119.176455] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=scheduler_connected addr=lb:500516392worker2 # [ 119.179844] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6393worker2 # [ 119.380825] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=scheduler_connected addr=lb:500516394worker2 # [ 119.384774] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6395worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6396worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6397worker2 # [ 119.585778] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=scheduler_connected addr=lb:500516398worker2 # [ 119.603138] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=oidc_jwks provider=mock keys=16399worker2 # [ 119.607154] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:370506400worker2 # [ 119.611079] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06401worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6402worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6403client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26404client # don't know how to build these paths:6405client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6406worker2 # [ 120.631163] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:387586407worker2 # [ 120.634617] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387586408worker2 # [ 120.656878] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:32Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06409worker2 # [ 120.661612] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:32Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=16410worker2 # [ 120.701919] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387586411worker2 # [ 120.714728] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387586412client # copying 1 paths to 'lb:50051'...6413client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6414worker2 # [ 120.727570] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387586415worker2 # [ 120.739113] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6416worker2 # [ 120.843151] nix-daemon[869]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6417worker2 # [ 120.856401] nix-daemon[869]: reaped child process 1489, status = succeeded6418worker2 # [ 120.940071] nix-daemon[869]: accepted connection from pid 1491, user nix-grpc-daemon (trusted)6419worker2 # [ 120.953299] nix-daemon[869]: reaped child process 1494, status = succeeded6420lb # [ 120.950975] niks3-server[1194]: 2026/09/22 14:52:31 INFO Received uploads request method=POST path=/api/pending_closures6421worker2 # [ 120.969655] nix-grpc-daemon[1444]: time=2026-09-22T14:52:32.995Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6422worker2 # [ 120.971215] nix-grpc-daemon[1444]: time=2026-09-22T14:52:32.995Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6423lb # [ 120.977543] niks3-server[1194]: 2026/09/22 14:52:31 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6424lb # [ 120.986354] niks3-server[1194]: 2026/09/22 14:52:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6425lb # [ 120.992572] niks3-server[1194]: 2026/09/22 14:52:31 INFO Signed narinfos id=43 count=16426worker2 # [ 121.005311] nix-grpc-daemon[1444]: time=2026-09-22T14:52:33.031Z level=INFO msg="Uploading 1 narinfos"6427lb # [ 121.008429] niks3-server[1194]: 2026/09/22 14:52:31 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6428lb # [ 121.015587] niks3-server[1194]: 2026/09/22 14:52:31 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6429lb # [ 121.023756] niks3-server[1194]: 2026/09/22 14:52:31 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6430lb # [ 121.026267] niks3-server[1194]: 2026/09/22 14:52:31 INFO Completed upload id=436431worker2 # [ 121.035922] nix-grpc-daemon[1444]: time=2026-09-22T14:52:33.062Z level=INFO msg="Upload complete. (285ms)"6432worker2 # [ 121.037980] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38758 duration_s=0 paths=1 nar_bytes_in=5206433worker2 # [ 121.041858] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387586434worker2 # [ 121.053239] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6435worker2 # [ 121.054615] nix-daemon[869]: reaped child process 1484, status = succeeded6436worker2 # [ 121.059933] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6437client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6438client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6439client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6440client # farm-a> LOG-farm-a-on-standby6441worker2 # [ 122.676480] nix-daemon[869]: reaped child process 1498, status = succeeded6442worker2 # [ 122.748248] nix-daemon[869]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)6443worker2 # [ 122.760852] nix-daemon[869]: reaped child process 1517, status = succeeded6444lb # [ 122.757602] niks3-server[1194]: 2026/09/22 14:52:33 INFO Received uploads request method=POST path=/api/pending_closures6445worker2 # [ 122.781279] nix-grpc-daemon[1444]: time=2026-09-22T14:52:34.807Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6446worker2 # [ 122.782989] nix-grpc-daemon[1444]: time=2026-09-22T14:52:34.809Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6447lb # [ 122.795570] niks3-server[1194]: 2026/09/22 14:52:33 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6448lb # [ 122.809601] niks3-server[1194]: 2026/09/22 14:52:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6449worker2 # [ 122.824494] nix-grpc-daemon[1444]: time=2026-09-22T14:52:34.850Z level=INFO msg="Uploading 1 narinfos"6450lb # [ 122.815343] niks3-server[1194]: 2026/09/22 14:52:33 INFO Signed narinfos id=44 count=16451lb # [ 122.825103] niks3-server[1194]: 2026/09/22 14:52:33 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6452lb # [ 122.833127] niks3-server[1194]: 2026/09/22 14:52:33 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6453lb # [ 122.839450] niks3-server[1194]: 2026/09/22 14:52:33 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6454lb # [ 122.843698] niks3-server[1194]: 2026/09/22 14:52:33 INFO Completed upload id=446455worker2 # [ 122.853407] nix-grpc-daemon[1444]: time=2026-09-22T14:52:34.879Z level=INFO msg="Upload complete. (185ms)"6456lb # [ 122.845944] niks3-server[1194]: 2026/09/22 14:52:33 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6457worker2 # [ 122.865515] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38758 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16458worker2 # [ 122.872184] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:34Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06459worker2 # [ 122.876259] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:34Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16460worker2 # [ 122.885405] nix-daemon[869]: reaped child process 1496, status = succeeded6461worker2 # [ 122.915550] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387586462worker2 # [ 122.928374] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387586463worker2 # [ 122.941328] nix-grpc-daemon[1444]: time=2026-09-22T14:52:34.967Z level=INFO msg="All 1 paths already cached"6464client # copying 1 paths to 'lb:50051'...6465client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6466worker2 # [ 122.949869] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387586467worker2 # [ 122.959330] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6468worker2 # [ 123.054982] nix-daemon[869]: accepted connection from pid 1523, user nix-grpc-daemon (trusted)6469worker2 # [ 123.067416] nix-daemon[869]: reaped child process 1526, status = succeeded6470worker2 # [ 123.149928] nix-daemon[869]: accepted connection from pid 1528, user nix-grpc-daemon (trusted)6471worker2 # [ 123.162346] nix-daemon[869]: reaped child process 1531, status = succeeded6472lb # [ 123.160371] niks3-server[1194]: 2026/09/22 14:52:33 INFO Received uploads request method=POST path=/api/pending_closures6473worker2 # [ 123.179134] nix-grpc-daemon[1444]: time=2026-09-22T14:52:35.205Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6474worker2 # [ 123.180489] nix-grpc-daemon[1444]: time=2026-09-22T14:52:35.205Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6475lb # [ 123.192242] niks3-server[1194]: 2026/09/22 14:52:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6476lb # [ 123.195823] niks3-server[1194]: 2026/09/22 14:52:33 INFO Signed narinfos id=45 count=16477worker2 # [ 123.209177] nix-grpc-daemon[1444]: time=2026-09-22T14:52:35.235Z level=INFO msg="Uploading 1 narinfos"6478lb # [ 123.203401] niks3-server[1194]: 2026/09/22 14:52:33 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6479lb # [ 123.208654] niks3-server[1194]: 2026/09/22 14:52:33 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6480lb # [ 123.221112] niks3-server[1194]: 2026/09/22 14:52:33 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6481lb # [ 123.227706] niks3-server[1194]: 2026/09/22 14:52:33 INFO Completed upload id=456482worker2 # [ 123.238429] nix-grpc-daemon[1444]: time=2026-09-22T14:52:35.264Z level=INFO msg="Upload complete. (267ms)"6483lb # [ 123.229519] niks3-server[1194]: 2026/09/22 14:52:33 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6484worker2 # [ 123.240677] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38758 duration_s=0 paths=1 nar_bytes_in=6406485worker2 # [ 123.243247] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387586486worker2 # [ 123.250907] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6487worker2 # [ 123.253312] nix-daemon[869]: reaped child process 1521, status = succeeded6488client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6489client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6490worker2 # [ 123.258449] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6491client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6492client # farm-b> LOG-farm-b-on-standby6493worker2 # [ 124.692367] nix-daemon[869]: reaped child process 1535, status = succeeded6494worker2 # [ 124.774833] nix-daemon[869]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6495worker2 # [ 124.787880] nix-daemon[869]: reaped child process 1554, status = succeeded6496lb # [ 124.784558] niks3-server[1194]: 2026/09/22 14:52:35 INFO Received uploads request method=POST path=/api/pending_closures6497worker2 # [ 124.801740] nix-grpc-daemon[1444]: time=2026-09-22T14:52:36.828Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6498worker2 # [ 124.803989] nix-grpc-daemon[1444]: time=2026-09-22T14:52:36.829Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6499lb # [ 124.806205] niks3-server[1194]: 2026/09/22 14:52:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6500lb # [ 124.808404] niks3-server[1194]: 2026/09/22 14:52:35 INFO Signed narinfos id=46 count=16501worker2 # [ 124.817815] nix-grpc-daemon[1444]: time=2026-09-22T14:52:36.844Z level=INFO msg="Uploading 1 narinfos"6502lb # [ 124.812276] niks3-server[1194]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6503lb # [ 124.816557] niks3-server[1194]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6504lb # [ 124.820348] niks3-server[1194]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6505lb # [ 124.821728] niks3-server[1194]: 2026/09/22 14:52:35 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6506lb # [ 124.825053] niks3-server[1194]: 2026/09/22 14:52:35 INFO Completed upload id=466507worker2 # [ 124.834459] nix-grpc-daemon[1444]: time=2026-09-22T14:52:36.860Z level=INFO msg="Upload complete. (166ms)"6508lb # [ 124.828207] niks3-server[1194]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6509worker2 # [ 124.836990] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38758 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16510worker2 # [ 124.843447] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:36Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06511worker2 # [ 124.845475] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:36Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16512worker2 # [ 124.850243] nix-daemon[869]: reaped child process 1533, status = succeeded6513worker2 # [ 124.885926] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387586514worker2 # [ 124.894793] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387586515worker2 # [ 124.903259] nix-grpc-daemon[1444]: time=2026-09-22T14:52:36.929Z level=INFO msg="All 2 paths already cached"6516client # copying 1 paths to 'lb:50051'...6517client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6518worker2 # [ 124.910320] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387586519worker2 # [ 124.918192] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6520worker2 # [ 125.003905] nix-daemon[869]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6521worker2 # [ 125.017306] nix-daemon[869]: reaped child process 1563, status = succeeded6522worker2 # [ 125.094786] nix-daemon[869]: accepted connection from pid 1565, user nix-grpc-daemon (trusted)6523worker2 # [ 125.107448] nix-daemon[869]: reaped child process 1568, status = succeeded6524lb # [ 125.105536] niks3-server[1194]: 2026/09/22 14:52:35 INFO Received uploads request method=POST path=/api/pending_closures6525worker2 # [ 125.122483] nix-grpc-daemon[1444]: time=2026-09-22T14:52:37.148Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6526worker2 # [ 125.124514] nix-grpc-daemon[1444]: time=2026-09-22T14:52:37.150Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6527lb # [ 125.130117] niks3-server[1194]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6528lb # [ 125.142702] niks3-server[1194]: 2026/09/22 14:52:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6529lb # [ 125.147696] niks3-server[1194]: 2026/09/22 14:52:35 INFO Signed narinfos id=47 count=16530worker2 # [ 125.160131] nix-grpc-daemon[1444]: time=2026-09-22T14:52:37.186Z level=INFO msg="Uploading 1 narinfos"6531lb # [ 125.155800] niks3-server[1194]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6532lb # [ 125.167325] niks3-server[1194]: 2026/09/22 14:52:35 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6533lb # [ 125.174268] niks3-server[1194]: 2026/09/22 14:52:35 INFO Completed upload id=476534worker2 # [ 125.183712] nix-grpc-daemon[1444]: time=2026-09-22T14:52:37.210Z level=INFO msg="Upload complete. (257ms)"6535lb # [ 125.179122] niks3-server[1194]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6536worker2 # [ 125.187450] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38758 duration_s=0 paths=1 nar_bytes_in=7686537worker2 # [ 125.192614] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387586538worker2 # [ 125.201848] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6539worker2 # [ 125.209881] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6540client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6541worker2 # [ 125.212299] nix-daemon[869]: reaped child process 1558, status = succeeded6542client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6543client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6544client # farm-top> LOG-farm-top-on-standby6545worker2 # [ 126.227805] nix-daemon[869]: reaped child process 1572, status = succeeded6546worker2 # [ 126.300489] nix-daemon[869]: accepted connection from pid 1588, user nix-grpc-daemon (trusted)6547worker2 # [ 126.313532] nix-daemon[869]: reaped child process 1591, status = succeeded6548lb # [ 126.311736] niks3-server[1194]: 2026/09/22 14:52:36 INFO Received uploads request method=POST path=/api/pending_closures6549worker2 # [ 126.329178] nix-grpc-daemon[1444]: time=2026-09-22T14:52:38.355Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6550worker2 # [ 126.330736] nix-grpc-daemon[1444]: time=2026-09-22T14:52:38.357Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6551lb # [ 126.344206] niks3-server[1194]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6552lb # [ 126.358698] niks3-server[1194]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6553lb # [ 126.365707] niks3-server[1194]: 2026/09/22 14:52:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6554lb # [ 126.372341] niks3-server[1194]: 2026/09/22 14:52:36 INFO Signed narinfos id=48 count=16555worker2 # [ 126.384731] nix-grpc-daemon[1444]: time=2026-09-22T14:52:38.410Z level=INFO msg="Uploading 1 narinfos"6556lb # [ 126.381099] niks3-server[1194]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6557lb # [ 126.391049] niks3-server[1194]: 2026/09/22 14:52:36 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6558lb # [ 126.394928] niks3-server[1194]: 2026/09/22 14:52:36 INFO Completed upload id=486559worker2 # [ 126.404482] nix-grpc-daemon[1444]: time=2026-09-22T14:52:38.430Z level=INFO msg="Upload complete. (186ms)"6560lb # [ 126.396696] niks3-server[1194]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6561worker2 # [ 126.410594] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38758 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16562worker2 # [ 126.426460] nix-daemon[869]: reaped child process 1570, status = succeeded6563client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 6.01 seconds)6564client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6565client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6566worker1 # [ 126.671639] systemd[1]: Listening on Nix worker-protocol over gRPC.6567worker1 # [ 126.678604] systemd[1]: Starting Nix worker-protocol over gRPC...6568worker1 # [ 126.760326] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6569worker1 # [ 126.767138] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6570worker1 # [ 126.769806] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16571worker1 # [ 126.774046] nix-grpc-daemon[1411]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6572worker1 # [ 126.775428] nix-grpc-daemon[1411]: E0000 00:00:1790088757.963315 1411 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6573worker1 # [ 126.777929] nix-grpc-daemon[1411]: E0000 00:00:1790088757.965589 1411 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6574worker1 # [ 126.780059] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6575worker1 # [ 126.783821] systemd[1]: Started Nix worker-protocol over gRPC.6576worker1 # [ 126.787146] nix-grpc-daemon[1411]: E0000 00:00:1790088757.983217 1450 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6577lb: must succeed: curl -sf localhost:9901/clusters6578worker1 # [ 126.798521] nix-grpc-daemon[1411]: E0000 00:00:1790088757.996218 1450 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6579worker1 # [ 126.801167] nix-grpc-daemon[1411]: E0000 00:00:1790088757.998263 1450 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6580worker1 # [ 126.812291] nix-grpc-daemon[1411]: E0000 00:00:1790088758.009939 1450 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6581worker1 # [ 126.821688] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:38Z level=info event=scheduler_connected addr=lb:500516582worker2 # [ 126.802726] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:38Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:370506583worker2 # [ 126.806466] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:38Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06584lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6585worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6586worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6587lb: must succeed: curl -sf localhost:9901/clusters6588lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6589lb: must succeed: curl -sf localhost:9901/clusters6590lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6591worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6592worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6593(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.02 seconds)6594subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6595client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6596client # [ 127.028887] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6597client # Running as unit: rs.service; invocation ID: 143d73cacc8d4b7aa313153267f020876598client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6599client # [ 127.166852] nix[1359]: don't know how to build these paths:6600client # [ 127.167774] nix[1359]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6601worker2 # [ 127.154169] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:387586602worker2 # [ 127.164816] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:39Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06603worker2 # [ 127.169562] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:39Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16604worker2 # [ 127.209427] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387586605worker2 # [ 127.221820] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:387586606client # [ 127.242634] nix[1359]: copying 1 paths to 'lb:50051'...6607client # [ 127.246205] nix[1359]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6608worker2 # [ 127.235421] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:387586609worker2 # [ 127.243985] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6610worker2 # [ 127.344845] nix-daemon[869]: accepted connection from pid 1604, user nix-grpc-daemon (trusted)6611worker2 # [ 127.358950] nix-daemon[869]: reaped child process 1607, status = succeeded6612worker2 # [ 127.440106] nix-daemon[869]: accepted connection from pid 1609, user nix-grpc-daemon (trusted)6613worker2 # [ 127.454106] nix-daemon[869]: reaped child process 1612, status = succeeded6614lb # [ 127.450396] niks3-server[1194]: 2026/09/22 14:52:37 INFO Received uploads request method=POST path=/api/pending_closures6615worker2 # [ 127.467932] nix-grpc-daemon[1444]: time=2026-09-22T14:52:39.493Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6616worker2 # [ 127.469322] nix-grpc-daemon[1444]: time=2026-09-22T14:52:39.493Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6617lb # [ 127.476812] niks3-server[1194]: 2026/09/22 14:52:37 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6618lb # [ 127.486728] niks3-server[1194]: 2026/09/22 14:52:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6619lb # [ 127.491749] niks3-server[1194]: 2026/09/22 14:52:37 INFO Signed narinfos id=49 count=16620worker2 # [ 127.505080] nix-grpc-daemon[1444]: time=2026-09-22T14:52:39.531Z level=INFO msg="Uploading 1 narinfos"6621lb # [ 127.507344] niks3-server[1194]: 2026/09/22 14:52:37 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6622lb # [ 127.511474] niks3-server[1194]: 2026/09/22 14:52:37 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6623lb # [ 127.519924] niks3-server[1194]: 2026/09/22 14:52:37 INFO Completed upload id=496624lb # [ 127.522059] niks3-server[1194]: 2026/09/22 14:52:37 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6625worker2 # [ 127.531751] nix-grpc-daemon[1444]: time=2026-09-22T14:52:39.558Z level=INFO msg="Upload complete. (274ms)"6626worker2 # [ 127.537123] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38758 duration_s=0 paths=1 nar_bytes_in=4166627worker2 # [ 127.541574] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:387586628worker2 # [ 127.552191] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6629worker2 # [ 127.554498] nix-daemon[869]: reaped child process 1602, status = succeeded6630worker2 # [ 127.560355] nix-daemon[869]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)6631client # [ 127.570780] nix[1359]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6632lb # [ 127.648981] envoy[924]: conn peer=[2001:db8:1::1]:55190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9573 tx=4381 ms=58376633lb: must succeed: curl -sf localhost:9901/clusters6634lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6635lb: must succeed: curl -sf localhost:9901/clusters6636lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6637lb: must succeed: curl -sf localhost:9901/clusters6638lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6639lb: must succeed: curl -sf localhost:9901/clusters6640lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6641worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6642worker1 # [ 129.589745] systemd-journald[283]: Received client request to rotate journal, rotating.6643worker1 # [ 129.601466] systemd-journald[283]: Vacuuming done, freed 0B of archived journals from /var/log/journal/198c814fd8ef4e29ae1f5caf28ce282c.6644worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.03 seconds)6645worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6646worker2 # [ 129.477203] systemd[1]: Stopping Nix worker-protocol over gRPC...6647worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)6648client: must succeed: sleep 5.06649worker2 # [ 130.450953] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:42Z level=info event=draining6650client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6651worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6652worker2 # [ 134.806978] nix-daemon[869]: reaped child process 1616, status = succeeded6653worker2 # [ 134.879167] nix-daemon[869]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6654worker2 # [ 134.891610] nix-daemon[869]: reaped child process 1660, status = succeeded6655lb # [ 134.888708] niks3-server[1194]: 2026/09/22 14:52:45 INFO Received uploads request method=POST path=/api/pending_closures6656worker2 # [ 134.907209] nix-grpc-daemon[1444]: time=2026-09-22T14:52:46.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6657worker2 # [ 134.908683] nix-grpc-daemon[1444]: time=2026-09-22T14:52:46.935Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6658lb # [ 134.923994] niks3-server[1194]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6659lb # [ 134.940115] niks3-server[1194]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6660lb # [ 134.952179] niks3-server[1194]: 2026/09/22 14:52:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6661worker2 # [ 134.968276] nix-grpc-daemon[1444]: time=2026-09-22T14:52:46.994Z level=INFO msg="Uploading 1 narinfos"6662lb # [ 134.958403] niks3-server[1194]: 2026/09/22 14:52:45 INFO Signed narinfos id=50 count=16663lb # [ 134.965108] niks3-server[1194]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6664lb # [ 134.972075] niks3-server[1194]: 2026/09/22 14:52:45 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6665lb # [ 134.976525] niks3-server[1194]: 2026/09/22 14:52:45 INFO Completed upload id=506666lb # [ 134.978636] niks3-server[1194]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6667worker2 # [ 134.987581] nix-grpc-daemon[1444]: time=2026-09-22T14:52:47.013Z level=INFO msg="Upload complete. (215ms)"6668worker2 # [ 134.996469] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38758 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16669worker2 # [ 135.009951] nix-daemon[869]: reaped child process 1614, status = succeeded6670client # [ 135.037085] systemd[1]: rs.service: Deactivated successfully.6671client # [ 135.040794] systemd[1]: rs.service: Consumed 130ms CPU time over 8.011s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.6672worker2 # [ 135.454951] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:47Z level=info event=signal_exit6673lb # [ 136.127507] niks3-server[1194]: 2026/09/22 14:52:46 INFO lead: released remote=192.168.1.4:470666674lb # [ 136.754815] niks3-server[1194]: 2026/09/22 14:52:47 INFO lead: acquired remote=192.168.1.3:418746675worker1 # [ 136.789927] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:47Z level=info event=scheduler_take_over6676worker1 # [ 137.158251] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:48Z level=info event=scheduler_restarting addr=lb:500516677worker2 # [ 137.141971] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:49Z level=info event=scheduler_restarting6678worker2 # [ 137.145462] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:49Z level=info event=scheduler_restarting addr=lb:500516679worker2 # [ 137.148985] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:370506680worker2 # [ 137.152925] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:49Z level=info event=worker_gone addr=192.168.1.3:500516681worker2 # [ 137.156571] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:370506682worker2 # [ 137.160186] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:49Z level=info event=worker_gone addr=192.168.1.4:500516683worker2 # [ 137.175625] nix-daemon[869]: reaped child process 1436, status = succeeded6684worker1 # [ 137.358819] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:48Z level=info event=scheduler_connected addr=lb:500516685worker1 # [ 137.360681] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6686worker2 # [ 137.350351] nix-grpc-daemon[1384]: ts=2026-09-22T14:52:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6687worker2 # [ 137.521191] nix-grpc-daemon[1384]: E0000 00:00:1790088769.547108 1429 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6688worker2 # [ 137.526067] nix-grpc-daemon[1384]: E0000 00:00:1790088769.547418 1429 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6689worker1 # [ 137.561248] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:48Z level=info event=scheduler_connected addr=lb:500516690worker1 # [ 137.564898] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:482226691worker1 # [ 137.566499] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06692lb # [ 137.649811] envoy[924]: conn peer=[2001:db8:1::1]:33336 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=1517 ms=78526693lb # [ 137.653714] envoy[924]: conn peer=[2001:db8:1::4]:47518 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=31766 tx=1863 ms=348556694worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.10 seconds)6695client: waiting for success: ! systemctl is-active rs6696client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6697client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6698client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6699client: must fail: journalctl -u rs | grep -q 'reconnecting'6700client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)6701lb: must succeed: curl -sf localhost:9901/clusters6702lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6703worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6704worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6705worker2 # [ 138.894543] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6706worker2 # [ 138.898235] systemd[1]: Stopped Nix worker-protocol over gRPC.6707worker2 # [ 138.901540] systemd[1]: nix-grpc-daemon.service: Consumed 1.424s CPU time over 36.552s wall clock time, 39.6M memory peak, 62.1K incoming IP traffic, 159.5K outgoing IP traffic.6708worker2 # [ 138.915986] systemd[1]: Starting scheduler-token.service...6709lb: must succeed: curl -sf localhost:9901/clusters6710worker2 # [ 138.972081] systemd[1]: scheduler-token.service: Deactivated successfully.6711worker2 # [ 138.973677] systemd[1]: Finished scheduler-token.service.6712worker2 # [ 138.974786] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 57ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6713lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6714worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6715worker2 # [ 138.979395] systemd[1]: Starting Nix worker-protocol over gRPC...6716worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6717worker2 # [ 139.058686] nix-grpc-daemon[1670]: ts=2026-09-22T14:52:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6718worker2 # [ 139.063398] nix-grpc-daemon[1670]: ts=2026-09-22T14:52:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6719worker2 # [ 139.067123] nix-grpc-daemon[1670]: ts=2026-09-22T14:52:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16720worker2 # [ 139.069074] nix-grpc-daemon[1670]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6721worker2 # [ 139.070359] nix-grpc-daemon[1670]: E0000 00:00:1790088771.090842 1670 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6722worker2 # [ 139.077386] nix-grpc-daemon[1670]: E0000 00:00:1790088771.099628 1670 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6723worker2 # [ 139.080687] nix-grpc-daemon[1670]: ts=2026-09-22T14:52:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6724worker2 # [ 139.083102] systemd[1]: Started Nix worker-protocol over gRPC.6725worker2 # [ 139.083948] nix-grpc-daemon[1670]: E0000 00:00:1790088771.109103 1711 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6726worker2 # [ 139.095887] nix-grpc-daemon[1670]: E0000 00:00:1790088771.122310 1711 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6727worker2 # [ 139.100056] nix-grpc-daemon[1670]: E0000 00:00:1790088771.124584 1711 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6728worker2 # [ 139.110176] nix-grpc-daemon[1670]: E0000 00:00:1790088771.136592 1711 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6729worker2 # [ 139.119492] nix-grpc-daemon[1670]: ts=2026-09-22T14:52:51Z level=info event=scheduler_connected addr=lb:500516730worker1 # [ 139.148777] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:50Z level=info event=oidc_jwks provider=mock keys=16731worker1 # [ 139.152407] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:50Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:482226732worker1 # [ 139.156255] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:50Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06733lb: must succeed: curl -sf localhost:9901/clusters6734lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6735worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6736worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6737(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.13 seconds)6738subtest: scheduler restart mid-build: build finishes, no double build6739client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6740client # [ 140.168645] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6741client # Running as unit: mid.service; invocation ID: 380c25c85019413eba9ad444bf7f43f16742client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.07 seconds)6743worker1 # [ 140.333432] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:482366744worker1 # [ 140.344429] nix-daemon[849]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)6745client # [ 140.347067] nix[1395]: don't know how to build these paths:6746client # [ 140.347941] nix[1395]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6747worker1 # [ 140.365801] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:51Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06748worker1 # [ 140.371496] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:51Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16749worker1 # [ 140.413893] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482366750worker1 # [ 140.424588] nix-grpc-daemon[1411]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)6751worker1 # [ 140.459622] nix-grpc-daemon[1411]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 2/5)6752worker1 # [ 140.516692] nix-grpc-daemon[1411]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 321 ms (attempt 3/5)6753worker1 # [ 140.839728] nix-grpc-daemon[1411]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 4/5)6754worker1 # [ 140.940098] nix-grpc-daemon[1411]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6755worker1 # [ 140.942927] nix-grpc-daemon[1411]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6756worker1 # [ 140.958733] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482366757client # [ 140.958710] nix[1395]: copying 1 paths to 'lb:50051'...6758client # [ 140.960674] nix[1395]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6759worker1 # [ 140.971383] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:482366760worker1 # [ 140.979575] nix-daemon[849]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)6761worker1 # [ 140.999379] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:52Z level=info event=niks3_push_started pid=15426762worker1 # [ 141.092548] nix-grpc-daemon[1542]: time=2026-09-22T14:52:52.289Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6763worker1 # [ 141.171618] nix-daemon[849]: accepted connection from pid 1554, user nix-grpc-daemon (trusted)6764worker1 # [ 141.183595] nix-daemon[849]: reaped child process 1557, status = succeeded6765worker1 # [ 141.257994] nix-daemon[849]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)6766worker1 # [ 141.270075] nix-daemon[849]: reaped child process 1562, status = succeeded6767lb # [ 141.245561] niks3-server[1194]: 2026/09/22 14:52:51 INFO Received uploads request method=POST path=/api/pending_closures6768worker1 # [ 141.284410] nix-grpc-daemon[1542]: time=2026-09-22T14:52:52.482Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6769worker1 # [ 141.287706] nix-grpc-daemon[1542]: time=2026-09-22T14:52:52.483Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6770lb # [ 141.281418] niks3-server[1194]: 2026/09/22 14:52:51 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6771lb # [ 141.288659] niks3-server[1194]: 2026/09/22 14:52:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6772lb # [ 141.298225] niks3-server[1194]: 2026/09/22 14:52:51 INFO Signed narinfos id=51 count=16773worker1 # [ 141.332121] nix-grpc-daemon[1542]: time=2026-09-22T14:52:52.529Z level=INFO msg="Uploading 1 narinfos"6774lb # [ 141.318485] niks3-server[1194]: 2026/09/22 14:52:51 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6775lb # [ 141.326912] niks3-server[1194]: 2026/09/22 14:52:51 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6776lb # [ 141.332978] niks3-server[1194]: 2026/09/22 14:52:51 INFO Completed upload id=516777worker1 # [ 141.365044] nix-grpc-daemon[1542]: time=2026-09-22T14:52:52.562Z level=INFO msg="Upload complete. (269ms)"6778worker1 # [ 141.367512] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48236 duration_s=0 paths=1 nar_bytes_in=4246779lb # [ 141.338955] niks3-server[1194]: 2026/09/22 14:52:51 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6780worker1 # [ 141.371184] nix-grpc-daemon[1411]: ts=2026-09-22T14:52:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482366781worker1 # [ 141.377645] nix-daemon[849]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)6782worker1 # [ 141.380876] nix-daemon[849]: reaped child process 1540, status = succeeded6783client # [ 141.374771] nix[1395]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6784worker1 # [ 141.385092] nix-daemon[849]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)6785lb: must succeed: curl -sf localhost:9901/clusters6786lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6787lb: must succeed: curl -sf localhost:9901/clusters6788lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6789worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6790worker1 # [ 143.723443] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1411 (nix-grpc-daemon) on client request.6791worker1 # [ 143.728374] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6792lb # [ 143.704394] niks3-server[1194]: 2026/09/22 14:52:54 INFO lead: released remote=192.168.1.3:418746793client # [ 143.731150] nix[1395]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6794worker1 # [ 143.741564] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6795worker1 # [ 143.743058] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6796worker1 # [ 143.744311] systemd[1]: nix-grpc-daemon.service: Consumed 410ms CPU time over 17.059s wall clock time, 35.5M memory peak, 15.1K incoming IP traffic, 31.6K outgoing IP traffic.6797worker2 # [ 143.720095] nix-grpc-daemon[1670]: ts=2026-09-22T14:52:55Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6798client # [ 143.733753] nix[1395]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6799worker1 # [ 143.758110] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6800worker1 # [ 143.759637] systemd[1]: Starting Nix worker-protocol over gRPC...6801worker1 # [ 143.788563] nix-daemon[849]: reaped child process 1568, status = succeeded6802worker1 # [ 143.790729] nix-daemon[849]: reaped child process 1532, status = succeeded6803worker1 # [ 143.796892] nix-daemon[1570]: unexpected Nix daemon error: error: interrupted by the user6804worker1 # [ 143.803456] nix-daemon[849]: reaped child process 1570, status = failed with exit code 16805worker1 # [ 143.849625] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6806worker1 # [ 143.851590] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6807worker1 # [ 143.854691] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16808worker1 # [ 143.859045] nix-grpc-daemon[1606]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6809worker1 # [ 143.860426] nix-grpc-daemon[1606]: E0000 00:00:1790088775.056465 1606 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6810worker1 # [ 143.867630] nix-grpc-daemon[1606]: E0000 00:00:1790088775.061832 1606 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6811worker1 # [ 143.869777] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6812worker1 # [ 143.871341] systemd[1]: Started Nix worker-protocol over gRPC.6813worker1 # [ 143.876228] nix-grpc-daemon[1606]: E0000 00:00:1790088775.073136 1648 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6814lb # [ 143.845785] niks3-server[1194]: 2026/09/22 14:52:54 INFO lead: acquired remote=192.168.1.3:398826815worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.18 seconds)6816client: must succeed: sleep 5.06817worker1 # [ 143.882067] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=scheduler_take_over6818worker1 # [ 143.889417] nix-grpc-daemon[1606]: E0000 00:00:1790088775.087071 1648 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6819worker1 # [ 143.894053] nix-grpc-daemon[1606]: E0000 00:00:1790088775.089391 1648 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6820worker1 # [ 143.903678] nix-grpc-daemon[1606]: E0000 00:00:1790088775.101332 1648 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6821worker1 # [ 143.912899] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=scheduler_connected addr=lb:500516822worker1 # [ 143.919159] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:384646823worker1 # [ 143.920665] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06824worker2 # [ 143.920448] nix-grpc-daemon[1670]: ts=2026-09-22T14:52:55Z level=info event=scheduler_connected addr=lb:500516825worker1 # [ 143.946399] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=oidc_jwks provider=mock keys=16826worker1 # [ 143.947825] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:384646827worker1 # [ 143.949487] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06828worker1 # [ 144.253907] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06829worker1 # [ 144.258895] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16830worker1 # [ 144.307144] nix-grpc-daemon[1606]: ts=2026-09-22T14:52:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384806831worker1 # [ 144.317318] nix-daemon[849]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6832worker1 # [ 144.332091] nix-daemon[849]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6833worker1 # [ 144.347394] nix-daemon[849]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6834client # [ 144.336174] nix[1395]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6835lb # [ 147.649385] envoy[924]: conn peer=[2001:db8:1::3]:59464 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=993 tx=376 ms=169236836client: (finished: must succeed: sleep 5.0, in 5.02 seconds)6837client: waiting for success: ! systemctl is-active mid6838worker1 # [ 149.170667] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:00Z level=info event=niks3_push_started pid=16906839worker1 # [ 149.213625] nix-daemon[849]: reaped child process 1665, status = succeeded6840worker1 # [ 149.296643] nix-grpc-daemon[1690]: time=2026-09-22T14:53:00.492Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6841worker1 # [ 149.388765] nix-daemon[849]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)6842worker1 # [ 149.403936] nix-daemon[849]: reaped child process 1706, status = succeeded6843lb # [ 149.382489] niks3-server[1194]: 2026/09/22 14:52:59 INFO Received uploads request method=POST path=/api/pending_closures6844worker1 # [ 149.423807] nix-grpc-daemon[1690]: time=2026-09-22T14:53:00.621Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6845worker1 # [ 149.426607] nix-grpc-daemon[1690]: time=2026-09-22T14:53:00.624Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6846lb # [ 149.435590] niks3-server[1194]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6847lb # [ 149.443970] niks3-server[1194]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6848lb # [ 149.459808] niks3-server[1194]: 2026/09/22 14:52:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6849lb # [ 149.462653] niks3-server[1194]: 2026/09/22 14:52:59 INFO Signed narinfos id=52 count=16850worker1 # [ 149.494914] nix-grpc-daemon[1690]: time=2026-09-22T14:53:00.691Z level=INFO msg="Uploading 1 narinfos"6851lb # [ 149.467839] niks3-server[1194]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6852lb # [ 149.471058] niks3-server[1194]: 2026/09/22 14:52:59 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6853lb # [ 149.477443] niks3-server[1194]: 2026/09/22 14:52:59 INFO Completed upload id=526854lb # [ 149.479372] niks3-server[1194]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6855worker1 # [ 149.509560] nix-grpc-daemon[1690]: time=2026-09-22T14:53:00.707Z level=INFO msg="Upload complete. (213ms)"6856worker1 # [ 149.515965] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38480 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16857worker1 # [ 149.531109] nix-daemon[849]: reaped child process 1663, status = succeeded6858client # [ 149.544247] systemd[1]: mid.service: Deactivated successfully.6859client # [ 149.548887] systemd[1]: mid.service: Consumed 153ms CPU time over 9.379s wall clock time, 39.5M memory peak, 8.6K incoming IP traffic, 10.4K outgoing IP traffic.6860client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6861client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6862client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6863(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.02 seconds)6864subtest: interrupting the client stops the build on the worker6865client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6866client # [ 150.162374] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6867client # Running as unit: intr.service; invocation ID: cb7b6e750d384d66868eff9797b1213c6868client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6869worker2 # [ 150.289499] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:499306870worker2 # [ 150.300784] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6871client # [ 150.324516] nix[1431]: don't know how to build these paths:6872client # [ 150.325501] nix[1431]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6873worker1 # [ 150.346508] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:01Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06874worker1 # [ 150.350682] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:01Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16875worker2 # [ 150.368290] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499306876worker2 # [ 150.379478] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5)6877worker2 # [ 150.409938] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 2/5)6878worker2 # [ 150.428665] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 249 ms (attempt 3/5)6879worker2 # [ 150.685408] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 4/5)6880worker2 # [ 150.850833] nix-grpc-daemon[1670]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6881worker2 # [ 150.854321] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6882worker2 # [ 150.871178] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499306883client # [ 150.893272] nix[1431]: copying 1 paths to 'lb:50051'...6884worker2 # [ 150.882607] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499306885client # [ 150.894845] nix[1431]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6886worker2 # [ 150.894157] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6887worker2 # [ 150.915078] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:02Z level=info event=niks3_push_started pid=17626888worker2 # [ 151.020692] nix-grpc-daemon[1762]: time=2026-09-22T14:53:03.046Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6889worker2 # [ 151.107391] nix-daemon[869]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6890worker2 # [ 151.118733] nix-daemon[869]: reaped child process 1778, status = succeeded6891worker2 # [ 151.202189] nix-daemon[869]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)6892worker2 # [ 151.215046] nix-daemon[869]: reaped child process 1783, status = succeeded6893lb # [ 151.213072] niks3-server[1194]: 2026/09/22 14:53:01 INFO Received uploads request method=POST path=/api/pending_closures6894worker2 # [ 151.231066] nix-grpc-daemon[1762]: time=2026-09-22T14:53:03.257Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6895worker2 # [ 151.234091] nix-grpc-daemon[1762]: time=2026-09-22T14:53:03.259Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6896lb # [ 151.249562] niks3-server[1194]: 2026/09/22 14:53:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6897lb # [ 151.257770] niks3-server[1194]: 2026/09/22 14:53:01 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6898lb # [ 151.264217] niks3-server[1194]: 2026/09/22 14:53:01 INFO Signed narinfos id=53 count=16899worker2 # [ 151.277355] nix-grpc-daemon[1762]: time=2026-09-22T14:53:03.303Z level=INFO msg="Uploading 1 narinfos"6900lb # [ 151.286925] niks3-server[1194]: 2026/09/22 14:53:01 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6901lb # [ 151.291185] niks3-server[1194]: 2026/09/22 14:53:01 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6902lb # [ 151.294156] niks3-server[1194]: 2026/09/22 14:53:01 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6903lb # [ 151.296613] niks3-server[1194]: 2026/09/22 14:53:01 INFO Completed upload id=536904worker2 # [ 151.306170] nix-grpc-daemon[1762]: time=2026-09-22T14:53:03.332Z level=INFO msg="Upload complete. (282ms)"6905worker2 # [ 151.308588] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=0 paths=1 nar_bytes_in=4246906worker2 # [ 151.312188] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499306907worker2 # [ 151.318714] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6908worker2 # [ 151.321552] nix-daemon[869]: reaped child process 1760, status = succeeded6909worker2 # [ 151.326091] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6910client # [ 151.338323] nix[1431]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6911client: must succeed: systemctl kill -s INT intr6912client # [ 152.421246] systemd[1]: intr.service: Sent signal SIGINT to main process 1431 (nix) on client request.6913worker2 # [ 152.412320] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:04Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6914client # [ 152.423752] nix[1431]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6915client # [ 152.425819] nix[1431]: error: interrupted by the user6916client: (finished: must succeed: systemctl kill -s INT intr, in 0.02 seconds)6917client # [ 152.442945] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6918client # [ 152.444355] systemd[1]: intr.service: Failed with result 'exit-code'.6919client # [ 152.446206] systemd[1]: intr.service: Consumed 117ms CPU time over 2.283s wall clock time, 43.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.6920worker2 # [ 152.544103] nix-daemon[869]: reaped child process 1789, status = succeeded6921worker2 # [ 152.545315] nix-daemon[1791]: unexpected Nix daemon error: error: interrupted by the user6922worker2 # [ 152.549610] nix-daemon[869]: reaped child process 1791, status = failed with exit code 16923(finished: subtest: interrupting the client stops the build on the worker, in 3.46 seconds)6924subtest: a deploy mid-build drains: the build finishes, the new generation takes over6925client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6926client # [ 153.649562] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6927client # Running as unit: sw.service; invocation ID: 310a985c55c0437b8513a0799ac8cdfb6928client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6929client # [ 153.788352] nix[1452]: don't know how to build these paths:6930client # [ 153.789223] nix[1452]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6931worker1 # [ 153.797152] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:384806932worker1 # [ 153.808107] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:05Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06933worker1 # [ 153.812788] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:05Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16934worker1 # [ 153.852087] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384806935worker1 # [ 153.863800] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5)6936worker1 # [ 153.956792] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 112 ms (attempt 2/5)6937worker1 # [ 154.070931] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 3/5)6938worker1 # [ 154.239831] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 648 ms (attempt 4/5)6939worker1 # [ 154.889723] nix-grpc-daemon[1606]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6940worker1 # [ 154.891718] nix-grpc-daemon[1606]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6941worker1 # [ 154.903865] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:384806942client # [ 154.901651] nix[1452]: copying 1 paths to 'lb:50051'...6943worker1 # [ 154.915452] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:384806944client # [ 154.905453] nix[1452]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6945worker1 # [ 154.923817] nix-daemon[849]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6946worker1 # [ 155.013097] nix-daemon[849]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6947worker1 # [ 155.027076] nix-daemon[849]: reaped child process 1749, status = succeeded6948worker1 # [ 155.109736] nix-daemon[849]: accepted connection from pid 1751, user nix-grpc-daemon (trusted)6949worker1 # [ 155.123244] nix-daemon[849]: reaped child process 1754, status = succeeded6950lb # [ 155.098183] niks3-server[1194]: 2026/09/22 14:53:05 INFO Received uploads request method=POST path=/api/pending_closures6951worker1 # [ 155.136183] nix-grpc-daemon[1690]: time=2026-09-22T14:53:06.333Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6952worker1 # [ 155.137550] nix-grpc-daemon[1690]: time=2026-09-22T14:53:06.333Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6953lb # [ 155.113676] niks3-server[1194]: 2026/09/22 14:53:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6954lb # [ 155.116236] niks3-server[1194]: 2026/09/22 14:53:05 INFO Signed narinfos id=54 count=16955worker1 # [ 155.148423] nix-grpc-daemon[1690]: time=2026-09-22T14:53:06.345Z level=INFO msg="Uploading 1 narinfos"6956lb # [ 155.120821] niks3-server[1194]: 2026/09/22 14:53:05 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6957lb # [ 155.126059] niks3-server[1194]: 2026/09/22 14:53:05 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6958lb # [ 155.129889] niks3-server[1194]: 2026/09/22 14:53:05 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6959lb # [ 155.133232] niks3-server[1194]: 2026/09/22 14:53:05 INFO Completed upload id=546960worker1 # [ 155.164108] nix-grpc-daemon[1690]: time=2026-09-22T14:53:06.361Z level=INFO msg="Upload complete. (234ms)"6961worker1 # [ 155.166220] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38480 duration_s=0 paths=1 nar_bytes_in=4166962lb # [ 155.136025] niks3-server[1194]: 2026/09/22 14:53:05 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6963worker1 # [ 155.169176] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:384806964worker1 # [ 155.175754] nix-daemon[849]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6965worker1 # [ 155.177995] nix-daemon[849]: reaped child process 1744, status = succeeded6966worker1 # [ 155.182887] nix-daemon[849]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)6967client # [ 155.172877] nix[1452]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6968worker1: must succeed: systemctl reload nix-grpc-daemon.service6969worker1 # [ 157.172264] systemd[1]: Reloading Nix worker-protocol over gRPC...6970worker1 # [ 157.198321] systemd[1]: Reloaded Nix worker-protocol over gRPC.6971worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6972worker1: must succeed: pgrep -f 'read -t [9]0 x'6973worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6974client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26975worker2 # [ 157.355632] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:499306976client # don't know how to build these paths:6977client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6978worker2 # [ 157.364085] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499306979worker1 # [ 157.407413] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:08Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06980worker1 # [ 157.409111] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:08Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16981worker2 # [ 157.435785] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499306982worker2 # [ 157.448605] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499306983client # copying 1 paths to 'lb:50051'...6984worker2 # [ 157.461519] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499306985client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6986worker2 # [ 157.483114] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6987worker2 # [ 157.588358] nix-daemon[869]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6988worker2 # [ 157.600638] nix-daemon[869]: reaped child process 1845, status = succeeded6989lb # [ 157.650387] envoy[924]: conn peer=[2001:db8:1::1]:59030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3908 tx=1939 ms=92116990lb # [ 157.654179] envoy[924]: conn peer=[2001:db8:1::1]:44274 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2967 tx=827 ms=21446991worker2 # [ 157.683049] nix-daemon[869]: accepted connection from pid 1847, user nix-grpc-daemon (trusted)6992worker2 # [ 157.696120] nix-daemon[869]: reaped child process 1850, status = succeeded6993lb # [ 157.692730] niks3-server[1194]: 2026/09/22 14:53:08 INFO Received uploads request method=POST path=/api/pending_closures6994worker2 # [ 157.711142] nix-grpc-daemon[1762]: time=2026-09-22T14:53:09.737Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6995worker2 # [ 157.712553] nix-grpc-daemon[1762]: time=2026-09-22T14:53:09.737Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6996lb # [ 157.724167] niks3-server[1194]: 2026/09/22 14:53:08 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6997lb # [ 157.735128] niks3-server[1194]: 2026/09/22 14:53:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6998worker2 # [ 157.751535] nix-grpc-daemon[1762]: time=2026-09-22T14:53:09.777Z level=INFO msg="Uploading 1 narinfos"6999lb # [ 157.741793] niks3-server[1194]: 2026/09/22 14:53:08 INFO Signed narinfos id=55 count=17000lb # [ 157.753257] niks3-server[1194]: 2026/09/22 14:53:08 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls7001lb # [ 157.762158] niks3-server[1194]: 2026/09/22 14:53:08 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete7002lb # [ 157.766063] niks3-server[1194]: 2026/09/22 14:53:08 INFO Completed upload id=557003worker2 # [ 157.775614] nix-grpc-daemon[1762]: time=2026-09-22T14:53:09.802Z level=INFO msg="Upload complete. (280ms)"7004lb # [ 157.768422] niks3-server[1194]: 2026/09/22 14:53:08 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo7005worker2 # [ 157.777961] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=0 paths=1 nar_bytes_in=5287006worker2 # [ 157.782668] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307007worker2 # [ 157.792347] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7008worker2 # [ 157.795269] nix-daemon[869]: reaped child process 1840, status = succeeded7009client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...7010worker2 # [ 157.801098] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7011client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv7012worker1 # [ 157.870908] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:09Z level=info event=draining7013client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7014client # farm-a> LOG-farm-a-during-drain7015worker2 # [ 159.040769] nix-daemon[869]: reaped child process 1854, status = succeeded7016worker2 # [ 159.122093] nix-daemon[869]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)7017worker2 # [ 159.133821] nix-daemon[869]: reaped child process 1873, status = succeeded7018lb # [ 159.132434] niks3-server[1194]: 2026/09/22 14:53:09 INFO Received uploads request method=POST path=/api/pending_closures7019worker2 # [ 159.150441] nix-grpc-daemon[1762]: time=2026-09-22T14:53:11.176Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7020worker2 # [ 159.151969] nix-grpc-daemon[1762]: time=2026-09-22T14:53:11.178Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"7021lb # [ 159.164733] niks3-server[1194]: 2026/09/22 14:53:09 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv7022lb # [ 159.182304] niks3-server[1194]: 2026/09/22 14:53:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign7023worker2 # [ 159.200162] nix-grpc-daemon[1762]: time=2026-09-22T14:53:11.226Z level=INFO msg="Uploading 1 narinfos"7024lb # [ 159.190172] niks3-server[1194]: 2026/09/22 14:53:09 INFO Signed narinfos id=56 count=17025lb # [ 159.195951] niks3-server[1194]: 2026/09/22 14:53:09 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst7026lb # [ 159.204556] niks3-server[1194]: 2026/09/22 14:53:09 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls7027lb # [ 159.210660] niks3-server[1194]: 2026/09/22 14:53:09 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete7028lb # [ 159.215500] niks3-server[1194]: 2026/09/22 14:53:09 INFO Completed upload id=567029worker2 # [ 159.224833] nix-grpc-daemon[1762]: time=2026-09-22T14:53:11.251Z level=INFO msg="Upload complete. (216ms)"7030lb # [ 159.218450] niks3-server[1194]: 2026/09/22 14:53:09 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo7031worker2 # [ 159.227404] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=17032worker2 # [ 159.234866] nix-daemon[869]: reaped child process 1852, status = succeeded7033worker1 # [ 159.256480] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:10Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=07034worker1 # [ 159.258190] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:10Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=17035worker2 # [ 159.278103] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307036worker2 # [ 159.288714] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499307037worker2 # [ 159.299920] nix-grpc-daemon[1762]: time=2026-09-22T14:53:11.326Z level=INFO msg="All 1 paths already cached"7038client # copying 1 paths to 'lb:50051'...7039worker2 # [ 159.307804] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499307040client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...7041worker2 # [ 159.325518] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7042worker2 # [ 159.427183] nix-daemon[869]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)7043worker2 # [ 159.439475] nix-daemon[869]: reaped child process 1880, status = succeeded7044worker2 # [ 159.523668] nix-daemon[869]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)7045worker2 # [ 159.536184] nix-daemon[869]: reaped child process 1885, status = succeeded7046lb # [ 159.534037] niks3-server[1194]: 2026/09/22 14:53:09 INFO Received uploads request method=POST path=/api/pending_closures7047worker2 # [ 159.548411] nix-grpc-daemon[1762]: time=2026-09-22T14:53:11.574Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7048worker2 # [ 159.550472] nix-grpc-daemon[1762]: time=2026-09-22T14:53:11.576Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"7049lb # [ 159.548608] niks3-server[1194]: 2026/09/22 14:53:09 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst7050lb # [ 159.552697] niks3-server[1194]: 2026/09/22 14:53:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign7051lb # [ 159.555066] niks3-server[1194]: 2026/09/22 14:53:09 INFO Signed narinfos id=57 count=17052worker2 # [ 159.566055] nix-grpc-daemon[1762]: time=2026-09-22T14:53:11.592Z level=INFO msg="Uploading 1 narinfos"7053lb # [ 159.558586] niks3-server[1194]: 2026/09/22 14:53:09 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls7054lb # [ 159.563523] niks3-server[1194]: 2026/09/22 14:53:09 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete7055lb # [ 159.566599] niks3-server[1194]: 2026/09/22 14:53:09 INFO Completed upload id=577056worker2 # [ 159.576122] nix-grpc-daemon[1762]: time=2026-09-22T14:53:11.602Z level=INFO msg="Upload complete. (236ms)"7057worker2 # [ 159.578217] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=0 paths=1 nar_bytes_in=6407058lb # [ 159.570030] niks3-server[1194]: 2026/09/22 14:53:09 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo7059worker2 # [ 159.581874] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307060worker2 # [ 159.587980] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7061worker2 # [ 159.590231] nix-daemon[869]: reaped child process 1875, status = succeeded7062worker2 # [ 159.595445] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7063client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...7064client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv7065client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7066client # farm-b> LOG-farm-b-during-drain7067worker2 # [ 161.034487] nix-daemon[869]: reaped child process 1889, status = succeeded7068worker2 # [ 161.116747] nix-daemon[869]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)7069worker2 # [ 161.128267] nix-daemon[869]: reaped child process 1908, status = succeeded7070lb # [ 161.126910] niks3-server[1194]: 2026/09/22 14:53:11 INFO Received uploads request method=POST path=/api/pending_closures7071worker2 # [ 161.144549] nix-grpc-daemon[1762]: time=2026-09-22T14:53:13.170Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7072worker2 # [ 161.146142] nix-grpc-daemon[1762]: time=2026-09-22T14:53:13.172Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"7073lb # [ 161.163078] niks3-server[1194]: 2026/09/22 14:53:11 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv7074lb # [ 161.171933] niks3-server[1194]: 2026/09/22 14:53:11 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst7075lb # [ 161.185866] niks3-server[1194]: 2026/09/22 14:53:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign7076lb # [ 161.191431] niks3-server[1194]: 2026/09/22 14:53:11 INFO Signed narinfos id=58 count=17077worker2 # [ 161.204421] nix-grpc-daemon[1762]: time=2026-09-22T14:53:13.230Z level=INFO msg="Uploading 1 narinfos"7078lb # [ 161.195805] niks3-server[1194]: 2026/09/22 14:53:11 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls7079lb # [ 161.206973] niks3-server[1194]: 2026/09/22 14:53:11 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete7080lb # [ 161.210845] niks3-server[1194]: 2026/09/22 14:53:11 INFO Completed upload id=587081lb # [ 161.213148] niks3-server[1194]: 2026/09/22 14:53:11 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo7082worker2 # [ 161.221712] nix-grpc-daemon[1762]: time=2026-09-22T14:53:13.248Z level=INFO msg="Upload complete. (218ms)"7083worker2 # [ 161.227807] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=17084worker1 # [ 161.253376] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:12Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=07085worker1 # [ 161.256314] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:12Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=17086worker2 # [ 161.243947] nix-daemon[869]: reaped child process 1887, status = succeeded7087worker2 # [ 161.276864] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307088worker2 # [ 161.294855] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499307089worker2 # [ 161.307254] nix-grpc-daemon[1762]: time=2026-09-22T14:53:13.333Z level=INFO msg="All 2 paths already cached"7090client # copying 1 paths to 'lb:50051'...7091worker2 # [ 161.316754] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499307092client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...7093worker2 # [ 161.337139] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7094worker2 # [ 161.438331] nix-daemon[869]: accepted connection from pid 1912, user nix-grpc-daemon (trusted)7095worker2 # [ 161.451367] nix-daemon[869]: reaped child process 1915, status = succeeded7096worker2 # [ 161.535409] nix-daemon[869]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7097worker2 # [ 161.547871] nix-daemon[869]: reaped child process 1920, status = succeeded7098lb # [ 161.546226] niks3-server[1194]: 2026/09/22 14:53:11 INFO Received uploads request method=POST path=/api/pending_closures7099worker2 # [ 161.563379] nix-grpc-daemon[1762]: time=2026-09-22T14:53:13.589Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7100worker2 # [ 161.565175] nix-grpc-daemon[1762]: time=2026-09-22T14:53:13.591Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"7101lb # [ 161.572560] niks3-server[1194]: 2026/09/22 14:53:11 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst7102lb # [ 161.584314] niks3-server[1194]: 2026/09/22 14:53:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign7103worker2 # [ 161.599470] nix-grpc-daemon[1762]: time=2026-09-22T14:53:13.625Z level=INFO msg="Uploading 1 narinfos"7104lb # [ 161.589729] niks3-server[1194]: 2026/09/22 14:53:12 INFO Signed narinfos id=59 count=17105lb # [ 161.600843] niks3-server[1194]: 2026/09/22 14:53:12 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls7106lb # [ 161.610383] niks3-server[1194]: 2026/09/22 14:53:12 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete7107lb # [ 161.616231] niks3-server[1194]: 2026/09/22 14:53:12 INFO Completed upload id=597108worker2 # [ 161.628158] nix-grpc-daemon[1762]: time=2026-09-22T14:53:13.654Z level=INFO msg="Upload complete. (278ms)"7109lb # [ 161.620772] niks3-server[1194]: 2026/09/22 14:53:12 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo7110worker2 # [ 161.632239] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=0 paths=1 nar_bytes_in=7687111worker2 # [ 161.636880] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307112worker2 # [ 161.648108] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7113worker2 # [ 161.652200] nix-daemon[869]: reaped child process 1910, status = succeeded7114worker2 # [ 161.657148] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7115client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...7116client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv7117client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7118client # farm-top> LOG-farm-top-during-drain7119worker2 # [ 162.694945] nix-daemon[869]: reaped child process 1924, status = succeeded7120worker2 # [ 162.781081] nix-daemon[869]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)7121worker2 # [ 162.793360] nix-daemon[869]: reaped child process 1943, status = succeeded7122lb # [ 162.792130] niks3-server[1194]: 2026/09/22 14:53:13 INFO Received uploads request method=POST path=/api/pending_closures7123worker2 # [ 162.809936] nix-grpc-daemon[1762]: time=2026-09-22T14:53:14.836Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7124worker2 # [ 162.812742] nix-grpc-daemon[1762]: time=2026-09-22T14:53:14.837Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"7125lb # [ 162.831732] niks3-server[1194]: 2026/09/22 14:53:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign7126worker2 # [ 162.846669] nix-grpc-daemon[1762]: time=2026-09-22T14:53:14.872Z level=INFO msg="Uploading 1 narinfos"7127lb # [ 162.837256] niks3-server[1194]: 2026/09/22 14:53:13 INFO Signed narinfos id=60 count=17128lb # [ 162.850136] niks3-server[1194]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst7129lb # [ 162.854163] niks3-server[1194]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv7130lb # [ 162.865589] niks3-server[1194]: 2026/09/22 14:53:13 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete7131lb # [ 162.870773] niks3-server[1194]: 2026/09/22 14:53:13 INFO Completed upload id=607132worker2 # [ 162.882201] nix-grpc-daemon[1762]: time=2026-09-22T14:53:14.908Z level=INFO msg="Upload complete. (217ms)"7133lb # [ 162.873812] niks3-server[1194]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls7134lb # [ 162.877767] niks3-server[1194]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo7135worker2 # [ 162.887892] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=17136worker2 # [ 162.899526] nix-daemon[869]: reaped child process 1922, status = succeeded7137client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.71 seconds)7138client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath7139client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)7140worker1: must succeed: pkill -f 'read -t [9]0 x'7141worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)7142client: waiting for success: ! systemctl is-active sw7143worker1 # [ 163.206562] nix-daemon[849]: reaped child process 1758, status = succeeded7144worker1 # [ 163.279783] nix-daemon[849]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)7145worker1 # [ 163.291714] nix-daemon[849]: reaped child process 1808, status = succeeded7146lb # [ 163.269379] niks3-server[1194]: 2026/09/22 14:53:13 INFO Received uploads request method=POST path=/api/pending_closures7147worker1 # [ 163.308198] nix-grpc-daemon[1690]: time=2026-09-22T14:53:14.505Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7148worker1 # [ 163.309800] nix-grpc-daemon[1690]: time=2026-09-22T14:53:14.507Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"7149lb # [ 163.298081] niks3-server[1194]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv7150lb # [ 163.314759] niks3-server[1194]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst7151lb # [ 163.330292] niks3-server[1194]: 2026/09/22 14:53:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign7152worker1 # [ 163.366155] nix-grpc-daemon[1690]: time=2026-09-22T14:53:14.563Z level=INFO msg="Uploading 1 narinfos"7153lb # [ 163.334826] niks3-server[1194]: 2026/09/22 14:53:13 INFO Signed narinfos id=61 count=17154lb # [ 163.339538] niks3-server[1194]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls7155lb # [ 163.348084] niks3-server[1194]: 2026/09/22 14:53:13 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete7156lb # [ 163.352061] niks3-server[1194]: 2026/09/22 14:53:13 INFO Completed upload id=617157worker1 # [ 163.382748] nix-grpc-daemon[1690]: time=2026-09-22T14:53:14.580Z level=INFO msg="Upload complete. (185ms)"7158lb # [ 163.353878] niks3-server[1194]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo7159worker1 # [ 163.385331] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38480 duration_s=8 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=17160worker1 # [ 163.393257] nix-daemon[849]: reaped child process 1756, status = succeeded7161client # [ 163.409605] systemd[1]: sw.service: Deactivated successfully.7162client # [ 163.413885] systemd[1]: sw.service: Consumed 129ms CPU time over 9.764s wall clock time, 43.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.7163worker1 # [ 163.873233] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:15Z level=info event=signal_exit7164lb # [ 163.847460] niks3-server[1194]: 2026/09/22 14:53:14 INFO lead: released remote=192.168.1.3:398827165lb # [ 164.071478] niks3-server[1194]: 2026/09/22 14:53:14 INFO lead: acquired remote=192.168.1.4:394327166worker2 # [ 164.082382] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:16Z level=info event=scheduler_take_over7167client: (finished: waiting for success: ! systemctl is-active sw, in 1.04 seconds)7168client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }7169client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.02 seconds)7170worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service7171worker2 # [ 164.858137] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:16Z level=info event=scheduler_restarting addr=lb:500517172worker1 # [ 164.884420] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:16Z level=info event=scheduler_restarting7173worker1 # [ 164.888709] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:384647174worker1 # [ 164.892634] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:16Z level=info event=worker_gone addr=192.168.1.4:500517175worker1 # [ 164.896258] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:16Z level=info event=scheduler_restarting addr=lb:500517176worker1 # [ 164.899835] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:384647177worker1 # [ 164.903516] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:16Z level=info event=worker_gone addr=192.168.1.3:500517178worker1 # [ 164.918114] nix-daemon[849]: reaped child process 1661, status = succeeded7179worker2 # [ 165.059346] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_connected addr=lb:500517180worker2 # [ 165.063399] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7181worker1 # [ 165.093802] nix-grpc-daemon[1606]: ts=2026-09-22T14:53:16Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"7182worker2 # [ 165.265968] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_connected addr=lb:500517183worker2 # [ 165.269323] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7184worker1 # [ 165.455292] nix-grpc-daemon[1606]: E0000 00:00:1790088796.652613 1652 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true7185worker1 # [ 165.460318] nix-grpc-daemon[1606]: E0000 00:00:1790088796.652906 1652 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true7186worker2 # [ 165.467115] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_connected addr=lb:500517187worker2 # [ 165.471256] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7188worker2 # [ 165.672448] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_connected addr=lb:500517189worker2 # [ 165.676518] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7190worker1 # [ 165.895133] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.7191worker2 # [ 165.877712] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_connected addr=lb:500517192worker1 # [ 165.898994] systemd[1]: nix-grpc-daemon.service: Consumed 640ms CPU time over 22.156s wall clock time, 36.5M memory peak, 24.4K incoming IP traffic, 31.5K outgoing IP traffic.7193worker2 # [ 165.881451] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"7194worker1 # [ 165.914405] systemd[1]: Starting Nix worker-protocol over gRPC...7195worker1 # [ 166.003736] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:17Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"7196worker1 # [ 166.005708] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc7197worker1 # [ 166.009114] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=17198worker1 # [ 166.013559] nix-grpc-daemon[1822]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR7199worker1 # [ 166.016158] nix-grpc-daemon[1822]: E0000 00:00:1790088797.210885 1822 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7200worker1 # [ 166.020779] nix-grpc-daemon[1822]: E0000 00:00:1790088797.216305 1822 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7201worker1 # [ 166.024519] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket7202worker1 # [ 166.026751] systemd[1]: Started Nix worker-protocol over gRPC.7203worker1 # [ 166.032058] nix-grpc-daemon[1822]: E0000 00:00:1790088797.227699 1863 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7204worker1 # [ 166.043576] nix-grpc-daemon[1822]: E0000 00:00:1790088797.241176 1863 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7205worker1 # [ 166.047128] nix-grpc-daemon[1822]: E0000 00:00:1790088797.243436 1863 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7206worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.90 seconds)7207lb: waiting until exactly no endpoints of x86_64-linux are unhealthy7208lb: must succeed: curl -sf localhost:9901/clusters7209worker1 # [ 166.058514] nix-grpc-daemon[1822]: E0000 00:00:1790088797.256170 1863 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7210worker1 # [ 166.069196] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:17Z level=info event=scheduler_connected addr=lb:500517211worker2 # [ 166.056072] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:18Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:406207212worker2 # [ 166.057902] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07213worker2 # [ 166.082732] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:18Z level=info event=scheduler_connected addr=lb:500517214lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7215worker2 # [ 166.091133] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:18Z level=info event=oidc_jwks provider=mock keys=17216worker2 # [ 166.095309] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:18Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:406207217worker2 # [ 166.099193] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:18Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07218lb: must succeed: curl -sf localhost:9901/clusters7219lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7220lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.08 seconds)7221lb: must succeed: curl -sf localhost:9901/clusters7222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7223worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7224worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7225(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.58 seconds)7226subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7227client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17228worker1 # [ 167.344401] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:440507229worker1 # [ 167.349594] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7230worker1 # [ 167.355834] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507231worker1 # [ 167.360647] nix-grpc-daemon[1822]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 1/5)7232worker1 # [ 167.404996] nix-grpc-daemon[1822]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 2/5)7233worker1 # [ 167.418643] nix-grpc-daemon[1822]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 303 ms (attempt 3/5)7234lb # [ 167.650658] envoy[924]: conn peer=[2001:db8:1::1]:41082 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9598 tx=4405 ms=55487235lb # [ 167.654553] envoy[924]: conn peer=[2001:db8:1::1]:41074 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1558 ms=96047236lb # [ 167.658147] envoy[924]: conn peer=[2001:db8:1::3]:55152 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=958 tx=344 ms=211767237worker1 # [ 167.724133] nix-grpc-daemon[1822]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 575 ms (attempt 4/5)7238worker1 # [ 168.300357] nix-grpc-daemon[1822]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7239worker1 # [ 168.301810] nix-grpc-daemon[1822]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7240worker2 # [ 168.321745] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:20Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07241worker2 # [ 168.326270] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:20Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17242worker2 # [ 168.330631] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:20Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07243worker2 # [ 168.334491] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:20Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07244worker1 # [ 168.387231] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507245worker1 # [ 168.395336] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507246worker1 # [ 168.402710] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440507247worker1 # [ 168.406560] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7248worker1 # [ 168.418551] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:19Z level=info event=niks3_push_started pid=18827249worker1 # [ 168.509643] nix-grpc-daemon[1882]: time=2026-09-22T14:53:19.707Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""7250worker1 # [ 168.597755] nix-daemon[849]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)7251worker1 # [ 168.608505] nix-daemon[849]: reaped child process 1898, status = succeeded7252worker1 # [ 168.693206] nix-daemon[849]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)7253worker1 # [ 168.705391] nix-daemon[849]: reaped child process 1903, status = succeeded7254lb # [ 168.683200] niks3-server[1194]: 2026/09/22 14:53:19 INFO Received uploads request method=POST path=/api/pending_closures7255worker1 # [ 168.726854] nix-grpc-daemon[1882]: time=2026-09-22T14:53:19.924Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7256worker1 # [ 168.728911] nix-grpc-daemon[1882]: time=2026-09-22T14:53:19.926Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7257lb # [ 168.727283] niks3-server[1194]: 2026/09/22 14:53:19 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7258lb # [ 168.734170] niks3-server[1194]: 2026/09/22 14:53:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7259worker1 # [ 168.776138] nix-grpc-daemon[1882]: time=2026-09-22T14:53:19.973Z level=INFO msg="Uploading 1 narinfos"7260lb # [ 168.744412] niks3-server[1194]: 2026/09/22 14:53:19 INFO Signed narinfos id=62 count=17261lb # [ 168.750824] niks3-server[1194]: 2026/09/22 14:53:19 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7262lb # [ 168.764058] niks3-server[1194]: 2026/09/22 14:53:19 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7263lb # [ 168.772273] niks3-server[1194]: 2026/09/22 14:53:19 INFO Completed upload id=627264lb # [ 168.773896] niks3-server[1194]: 2026/09/22 14:53:19 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7265worker1 # [ 168.803692] nix-grpc-daemon[1882]: time=2026-09-22T14:53:20.001Z level=INFO msg="Upload complete. (290ms)"7266worker1 # [ 168.808593] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=0 paths=1 nar_bytes_in=5047267worker1 # [ 168.814308] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507268worker1 # [ 168.824671] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7269worker1 # [ 168.832255] nix-daemon[849]: reaped child process 1880, status = succeeded7270worker1 # [ 168.837174] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7271worker1 # [ 169.852878] nix-daemon[849]: reaped child process 1907, status = succeeded7272worker1 # [ 169.928755] nix-daemon[849]: accepted connection from pid 1923, user nix-grpc-daemon (trusted)7273worker1 # [ 169.941960] nix-daemon[849]: reaped child process 1926, status = succeeded7274lb # [ 169.918665] niks3-server[1194]: 2026/09/22 14:53:20 INFO Received uploads request method=POST path=/api/pending_closures7275worker1 # [ 169.958238] nix-grpc-daemon[1882]: time=2026-09-22T14:53:21.155Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7276worker1 # [ 169.959706] nix-grpc-daemon[1882]: time=2026-09-22T14:53:21.157Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7277lb # [ 169.958171] niks3-server[1194]: 2026/09/22 14:53:20 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7278lb # [ 169.964154] niks3-server[1194]: 2026/09/22 14:53:20 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7279lb # [ 169.973399] niks3-server[1194]: 2026/09/22 14:53:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7280lb # [ 169.978282] niks3-server[1194]: 2026/09/22 14:53:20 INFO Signed narinfos id=63 count=17281worker1 # [ 170.009737] nix-grpc-daemon[1882]: time=2026-09-22T14:53:21.206Z level=INFO msg="Uploading 1 narinfos"7282lb # [ 169.986345] niks3-server[1194]: 2026/09/22 14:53:20 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7283lb # [ 169.989243] niks3-server[1194]: 2026/09/22 14:53:20 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7284lb # [ 169.992773] niks3-server[1194]: 2026/09/22 14:53:20 INFO Completed upload id=637285worker1 # [ 170.023571] nix-grpc-daemon[1882]: time=2026-09-22T14:53:21.221Z level=INFO msg="Upload complete. (179ms)"7286lb # [ 169.994921] niks3-server[1194]: 2026/09/22 14:53:20 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7287worker1 # [ 170.026218] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=17288worker1 # [ 170.033338] nix-daemon[849]: reaped child process 1905, status = succeeded7289worker2 # [ 170.011783] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:22Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17290worker2 # [ 170.019170] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:22Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07291worker1 # [ 170.081725] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507292worker1 # [ 170.088363] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507293worker1 # [ 170.094600] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440507294worker1 # [ 170.098346] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7295worker1 # [ 170.183885] nix-daemon[849]: accepted connection from pid 1930, user nix-grpc-daemon (trusted)7296worker1 # [ 170.196546] nix-daemon[849]: reaped child process 1933, status = succeeded7297worker1 # [ 170.279448] nix-daemon[849]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)7298worker1 # [ 170.293128] nix-daemon[849]: reaped child process 1938, status = succeeded7299lb # [ 170.267359] niks3-server[1194]: 2026/09/22 14:53:20 INFO Received uploads request method=POST path=/api/pending_closures7300worker1 # [ 170.306613] nix-grpc-daemon[1882]: time=2026-09-22T14:53:21.504Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7301worker1 # [ 170.307964] nix-grpc-daemon[1882]: time=2026-09-22T14:53:21.504Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7302lb # [ 170.291784] niks3-server[1194]: 2026/09/22 14:53:20 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7303lb # [ 170.305561] niks3-server[1194]: 2026/09/22 14:53:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7304lb # [ 170.313378] niks3-server[1194]: 2026/09/22 14:53:20 INFO Signed narinfos id=64 count=17305worker1 # [ 170.347155] nix-grpc-daemon[1882]: time=2026-09-22T14:53:21.544Z level=INFO msg="Uploading 1 narinfos"7306lb # [ 170.318136] niks3-server[1194]: 2026/09/22 14:53:20 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7307lb # [ 170.328961] niks3-server[1194]: 2026/09/22 14:53:20 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7308worker1 # [ 170.366252] nix-grpc-daemon[1882]: time=2026-09-22T14:53:21.563Z level=INFO msg="Upload complete. (262ms)"7309lb # [ 170.335738] niks3-server[1194]: 2026/09/22 14:53:20 INFO Completed upload id=647310worker1 # [ 170.368270] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=0 paths=1 nar_bytes_in=5047311worker1 # [ 170.372152] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507312lb # [ 170.341479] niks3-server[1194]: 2026/09/22 14:53:20 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7313worker1 # [ 170.378487] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7314worker1 # [ 170.380753] nix-daemon[849]: reaped child process 1928, status = succeeded7315worker1 # [ 170.385708] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7316worker1 # [ 171.423651] nix-daemon[849]: reaped child process 1942, status = succeeded7317worker1 # [ 171.507646] nix-daemon[849]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)7318worker1 # [ 171.518870] nix-daemon[849]: reaped child process 1961, status = succeeded7319lb # [ 171.496261] niks3-server[1194]: 2026/09/22 14:53:21 INFO Received uploads request method=POST path=/api/pending_closures7320worker1 # [ 171.535453] nix-grpc-daemon[1882]: time=2026-09-22T14:53:22.732Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7321worker1 # [ 171.537105] nix-grpc-daemon[1882]: time=2026-09-22T14:53:22.734Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7322lb # [ 171.533603] niks3-server[1194]: 2026/09/22 14:53:21 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7323lb # [ 171.542907] niks3-server[1194]: 2026/09/22 14:53:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7324lb # [ 171.548559] niks3-server[1194]: 2026/09/22 14:53:21 INFO Signed narinfos id=65 count=17325worker1 # [ 171.582300] nix-grpc-daemon[1882]: time=2026-09-22T14:53:22.779Z level=INFO msg="Uploading 1 narinfos"7326lb # [ 171.558585] niks3-server[1194]: 2026/09/22 14:53:21 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7327lb # [ 171.573522] niks3-server[1194]: 2026/09/22 14:53:21 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7328lb # [ 171.577486] niks3-server[1194]: 2026/09/22 14:53:21 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7329lb # [ 171.582753] niks3-server[1194]: 2026/09/22 14:53:22 INFO Completed upload id=657330worker1 # [ 171.613743] nix-grpc-daemon[1882]: time=2026-09-22T14:53:22.811Z level=INFO msg="Upload complete. (220ms)"7331lb # [ 171.584693] niks3-server[1194]: 2026/09/22 14:53:22 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7332worker1 # [ 171.619708] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=17333worker2 # [ 171.599628] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:23Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17334worker2 # [ 171.608875] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:23Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07335worker1 # [ 171.635311] nix-daemon[849]: reaped child process 1940, status = succeeded7336worker1 # [ 171.669311] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507337worker1 # [ 171.679985] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507338worker1 # [ 171.689934] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440507339worker1 # [ 171.697902] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7340worker1 # [ 171.796220] nix-daemon[849]: accepted connection from pid 1965, user nix-grpc-daemon (trusted)7341worker1 # [ 171.808943] nix-daemon[849]: reaped child process 1968, status = succeeded7342worker1 # [ 171.891634] nix-daemon[849]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)7343worker1 # [ 171.905622] nix-daemon[849]: reaped child process 1973, status = succeeded7344lb # [ 171.881506] niks3-server[1194]: 2026/09/22 14:53:22 INFO Received uploads request method=POST path=/api/pending_closures7345worker1 # [ 171.920517] nix-grpc-daemon[1882]: time=2026-09-22T14:53:23.118Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7346worker1 # [ 171.922382] nix-grpc-daemon[1882]: time=2026-09-22T14:53:23.119Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7347lb # [ 171.917100] niks3-server[1194]: 2026/09/22 14:53:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7348worker1 # [ 171.960680] nix-grpc-daemon[1882]: time=2026-09-22T14:53:23.158Z level=INFO msg="Uploading 1 narinfos"7349lb # [ 171.930343] niks3-server[1194]: 2026/09/22 14:53:22 INFO Signed narinfos id=66 count=17350lb # [ 171.933820] niks3-server[1194]: 2026/09/22 14:53:22 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7351lb # [ 171.953752] niks3-server[1194]: 2026/09/22 14:53:22 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7352lb # [ 171.959251] niks3-server[1194]: 2026/09/22 14:53:22 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7353lb # [ 171.962073] niks3-server[1194]: 2026/09/22 14:53:22 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7354lb # [ 171.964889] niks3-server[1194]: 2026/09/22 14:53:22 INFO Completed upload id=667355worker1 # [ 171.995873] nix-grpc-daemon[1882]: time=2026-09-22T14:53:23.193Z level=INFO msg="Upload complete. (284ms)"7356worker1 # [ 172.000702] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=0 paths=1 nar_bytes_in=5047357worker1 # [ 172.007301] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507358worker1 # [ 172.016668] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7359worker1 # [ 172.026341] nix-daemon[849]: reaped child process 1963, status = succeeded7360worker1 # [ 172.034493] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7361worker1 # [ 172.877413] nix-daemon[849]: reaped child process 1977, status = succeeded7362worker1 # [ 172.958991] nix-daemon[849]: accepted connection from pid 1993, user nix-grpc-daemon (trusted)7363worker1 # [ 172.972775] nix-daemon[849]: reaped child process 1996, status = succeeded7364lb # [ 172.947579] niks3-server[1194]: 2026/09/22 14:53:23 INFO Received uploads request method=POST path=/api/pending_closures7365worker1 # [ 172.986230] nix-grpc-daemon[1882]: time=2026-09-22T14:53:24.183Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7366worker1 # [ 172.988233] nix-grpc-daemon[1882]: time=2026-09-22T14:53:24.185Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7367lb # [ 172.987263] niks3-server[1194]: 2026/09/22 14:53:23 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7368lb # [ 172.996724] niks3-server[1194]: 2026/09/22 14:53:23 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7369lb # [ 173.004559] niks3-server[1194]: 2026/09/22 14:53:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7370lb # [ 173.012421] niks3-server[1194]: 2026/09/22 14:53:23 INFO Signed narinfos id=67 count=17371worker1 # [ 173.043688] nix-grpc-daemon[1882]: time=2026-09-22T14:53:24.241Z level=INFO msg="Uploading 1 narinfos"7372lb # [ 173.015627] niks3-server[1194]: 2026/09/22 14:53:23 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7373lb # [ 173.028390] niks3-server[1194]: 2026/09/22 14:53:23 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7374lb # [ 173.032057] niks3-server[1194]: 2026/09/22 14:53:23 INFO Completed upload id=677375worker1 # [ 173.062909] nix-grpc-daemon[1882]: time=2026-09-22T14:53:24.260Z level=INFO msg="Upload complete. (221ms)"7376lb # [ 173.034267] niks3-server[1194]: 2026/09/22 14:53:23 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7377worker1 # [ 173.068840] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=17378worker2 # [ 173.049344] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:25Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17379worker1 # [ 173.077517] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507380worker2 # [ 173.058666] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:25Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07381worker1 # [ 173.088628] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507382worker1 # [ 173.097466] nix-daemon[849]: reaped child process 1975, status = succeeded7383worker1 # [ 173.100478] nix-grpc-daemon[1882]: time=2026-09-22T14:53:24.297Z level=INFO msg="All 1 paths already cached"7384worker1 # [ 173.104526] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440507385worker1 # [ 173.109440] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7386worker1 # [ 173.194557] nix-daemon[849]: accepted connection from pid 2000, user nix-grpc-daemon (trusted)7387worker1 # [ 173.206959] nix-daemon[849]: reaped child process 2003, status = succeeded7388worker1 # [ 173.290330] nix-daemon[849]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)7389worker1 # [ 173.302784] nix-daemon[849]: reaped child process 2008, status = succeeded7390lb # [ 173.279367] niks3-server[1194]: 2026/09/22 14:53:23 INFO Received uploads request method=POST path=/api/pending_closures7391worker1 # [ 173.318317] nix-grpc-daemon[1882]: time=2026-09-22T14:53:24.515Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7392worker1 # [ 173.319695] nix-grpc-daemon[1882]: time=2026-09-22T14:53:24.515Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7393lb # [ 173.304134] niks3-server[1194]: 2026/09/22 14:53:23 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7394lb # [ 173.317511] niks3-server[1194]: 2026/09/22 14:53:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7395lb # [ 173.323143] niks3-server[1194]: 2026/09/22 14:53:23 INFO Signed narinfos id=68 count=17396worker1 # [ 173.356909] nix-grpc-daemon[1882]: time=2026-09-22T14:53:24.554Z level=INFO msg="Uploading 1 narinfos"7397lb # [ 173.329377] niks3-server[1194]: 2026/09/22 14:53:23 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7398lb # [ 173.341987] niks3-server[1194]: 2026/09/22 14:53:23 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7399worker1 # [ 173.378901] nix-grpc-daemon[1882]: time=2026-09-22T14:53:24.576Z level=INFO msg="Upload complete. (264ms)"7400lb # [ 173.347986] niks3-server[1194]: 2026/09/22 14:53:23 INFO Completed upload id=687401worker1 # [ 173.381983] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=0 paths=1 nar_bytes_in=6247402lb # [ 173.354068] niks3-server[1194]: 2026/09/22 14:53:23 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7403worker1 # [ 173.388116] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507404worker1 # [ 173.396689] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7405worker1 # [ 173.402124] nix-daemon[849]: reaped child process 1998, status = succeeded7406worker1 # [ 173.406195] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7407worker1 # [ 174.222404] nix-daemon[849]: reaped child process 2012, status = succeeded7408worker1 # [ 174.291945] nix-daemon[849]: accepted connection from pid 2028, user nix-grpc-daemon (trusted)7409worker1 # [ 174.304882] nix-daemon[849]: reaped child process 2031, status = succeeded7410lb # [ 174.280677] niks3-server[1194]: 2026/09/22 14:53:24 INFO Received uploads request method=POST path=/api/pending_closures7411worker1 # [ 174.319753] nix-grpc-daemon[1882]: time=2026-09-22T14:53:25.517Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7412worker1 # [ 174.321239] nix-grpc-daemon[1882]: time=2026-09-22T14:53:25.518Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7413lb # [ 174.318462] niks3-server[1194]: 2026/09/22 14:53:24 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7414lb # [ 174.335185] niks3-server[1194]: 2026/09/22 14:53:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7415worker1 # [ 174.372689] nix-grpc-daemon[1882]: time=2026-09-22T14:53:25.570Z level=INFO msg="Uploading 1 narinfos"7416lb # [ 174.341743] niks3-server[1194]: 2026/09/22 14:53:24 INFO Signed narinfos id=69 count=17417lb # [ 174.348170] niks3-server[1194]: 2026/09/22 14:53:24 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7418lb # [ 174.353210] niks3-server[1194]: 2026/09/22 14:53:24 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7419lb # [ 174.362769] niks3-server[1194]: 2026/09/22 14:53:24 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7420lb # [ 174.365377] niks3-server[1194]: 2026/09/22 14:53:24 INFO Completed upload id=697421worker1 # [ 174.397542] nix-grpc-daemon[1882]: time=2026-09-22T14:53:25.595Z level=INFO msg="Upload complete. (185ms)"7422lb # [ 174.368854] niks3-server[1194]: 2026/09/22 14:53:24 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7423worker1 # [ 174.403770] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=17424worker2 # [ 174.383717] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:26Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17425worker1 # [ 174.411716] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507426worker2 # [ 174.391285] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:26Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07427worker1 # [ 174.422247] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507428worker1 # [ 174.429782] nix-daemon[849]: reaped child process 2010, status = succeeded7429worker1 # [ 174.435606] nix-grpc-daemon[1882]: time=2026-09-22T14:53:25.633Z level=INFO msg="All 1 paths already cached"7430worker1 # [ 174.441483] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440507431worker1 # [ 174.449375] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7432worker1 # [ 174.528175] nix-daemon[849]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)7433worker1 # [ 174.540437] nix-daemon[849]: reaped child process 2038, status = succeeded7434worker1 # [ 174.617398] nix-daemon[849]: accepted connection from pid 2040, user nix-grpc-daemon (trusted)7435worker1 # [ 174.631088] nix-daemon[849]: reaped child process 2043, status = succeeded7436lb # [ 174.603966] niks3-server[1194]: 2026/09/22 14:53:25 INFO Received uploads request method=POST path=/api/pending_closures7437worker1 # [ 174.640762] nix-grpc-daemon[1882]: time=2026-09-22T14:53:25.838Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7438worker1 # [ 174.642120] nix-grpc-daemon[1882]: time=2026-09-22T14:53:25.838Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7439lb # [ 174.630868] niks3-server[1194]: 2026/09/22 14:53:25 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7440lb # [ 174.641114] niks3-server[1194]: 2026/09/22 14:53:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7441worker1 # [ 174.677712] nix-grpc-daemon[1882]: time=2026-09-22T14:53:25.875Z level=INFO msg="Uploading 1 narinfos"7442lb # [ 174.647309] niks3-server[1194]: 2026/09/22 14:53:25 INFO Signed narinfos id=70 count=17443lb # [ 174.652955] niks3-server[1194]: 2026/09/22 14:53:25 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7444lb # [ 174.668241] niks3-server[1194]: 2026/09/22 14:53:25 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7445lb # [ 174.676084] niks3-server[1194]: 2026/09/22 14:53:25 INFO Completed upload id=707446worker1 # [ 174.706601] nix-grpc-daemon[1882]: time=2026-09-22T14:53:25.904Z level=INFO msg="Upload complete. (252ms)"7447lb # [ 174.680926] niks3-server[1194]: 2026/09/22 14:53:25 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7448worker1 # [ 174.710978] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=0 paths=1 nar_bytes_in=6247449worker1 # [ 174.716245] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507450worker1 # [ 174.724641] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7451worker1 # [ 174.732405] nix-daemon[849]: reaped child process 2033, status = succeeded7452worker1 # [ 174.737125] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7453worker1 # [ 176.173972] nix-daemon[849]: reaped child process 2047, status = succeeded7454worker1 # [ 176.249818] nix-daemon[849]: accepted connection from pid 2063, user nix-grpc-daemon (trusted)7455worker1 # [ 176.262133] nix-daemon[849]: reaped child process 2066, status = succeeded7456lb # [ 176.238994] niks3-server[1194]: 2026/09/22 14:53:26 INFO Received uploads request method=POST path=/api/pending_closures7457worker1 # [ 176.278480] nix-grpc-daemon[1882]: time=2026-09-22T14:53:27.475Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7458worker1 # [ 176.280056] nix-grpc-daemon[1882]: time=2026-09-22T14:53:27.477Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7459lb # [ 176.282434] niks3-server[1194]: 2026/09/22 14:53:26 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7460lb # [ 176.293154] niks3-server[1194]: 2026/09/22 14:53:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7461worker1 # [ 176.327594] nix-grpc-daemon[1882]: time=2026-09-22T14:53:27.525Z level=INFO msg="Uploading 1 narinfos"7462lb # [ 176.296693] niks3-server[1194]: 2026/09/22 14:53:26 INFO Signed narinfos id=71 count=17463lb # [ 176.301869] niks3-server[1194]: 2026/09/22 14:53:26 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7464lb # [ 176.309072] niks3-server[1194]: 2026/09/22 14:53:26 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7465lb # [ 176.313847] niks3-server[1194]: 2026/09/22 14:53:26 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7466lb # [ 176.317061] niks3-server[1194]: 2026/09/22 14:53:26 INFO Completed upload id=717467worker1 # [ 176.348094] nix-grpc-daemon[1882]: time=2026-09-22T14:53:27.545Z level=INFO msg="Upload complete. (203ms)"7468lb # [ 176.319823] niks3-server[1194]: 2026/09/22 14:53:26 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7469worker1 # [ 176.355286] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=17470worker2 # [ 176.334870] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:28Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17471worker1 # [ 176.362789] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507472worker2 # [ 176.344498] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:28Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07473worker1 # [ 176.374501] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507474worker1 # [ 176.382476] nix-daemon[849]: reaped child process 2045, status = succeeded7475worker1 # [ 176.385580] nix-grpc-daemon[1882]: time=2026-09-22T14:53:27.582Z level=INFO msg="All 1 paths already cached"7476worker1 # [ 176.389626] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440507477worker1 # [ 176.394186] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7478worker1 # [ 176.475345] nix-daemon[849]: accepted connection from pid 2070, user nix-grpc-daemon (trusted)7479worker1 # [ 176.487439] nix-daemon[849]: reaped child process 2073, status = succeeded7480worker1 # [ 176.568941] nix-daemon[849]: accepted connection from pid 2075, user nix-grpc-daemon (trusted)7481worker1 # [ 176.581529] nix-daemon[849]: reaped child process 2078, status = succeeded7482lb # [ 176.557209] niks3-server[1194]: 2026/09/22 14:53:26 INFO Received uploads request method=POST path=/api/pending_closures7483worker1 # [ 176.596644] nix-grpc-daemon[1882]: time=2026-09-22T14:53:27.794Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7484worker1 # [ 176.598266] nix-grpc-daemon[1882]: time=2026-09-22T14:53:27.795Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7485lb # [ 176.586939] niks3-server[1194]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7486lb # [ 176.598074] niks3-server[1194]: 2026/09/22 14:53:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7487worker1 # [ 176.634564] nix-grpc-daemon[1882]: time=2026-09-22T14:53:27.832Z level=INFO msg="Uploading 1 narinfos"7488lb # [ 176.603677] niks3-server[1194]: 2026/09/22 14:53:27 INFO Signed narinfos id=72 count=17489lb # [ 176.609258] niks3-server[1194]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7490lb # [ 176.621523] niks3-server[1194]: 2026/09/22 14:53:27 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7491lb # [ 176.624906] niks3-server[1194]: 2026/09/22 14:53:27 INFO Completed upload id=727492worker1 # [ 176.655797] nix-grpc-daemon[1882]: time=2026-09-22T14:53:27.853Z level=INFO msg="Upload complete. (256ms)"7493lb # [ 176.626405] niks3-server[1194]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7494worker1 # [ 176.658762] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=0 paths=1 nar_bytes_in=6247495worker1 # [ 176.663577] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507496worker1 # [ 176.673287] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7497worker1 # [ 176.678053] nix-daemon[849]: reaped child process 2068, status = succeeded7498worker1 # [ 176.682648] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7499worker1 # [ 177.722614] nix-daemon[849]: reaped child process 2082, status = succeeded7500worker1 # [ 177.804536] nix-daemon[849]: accepted connection from pid 2098, user nix-grpc-daemon (trusted)7501worker1 # [ 177.815674] nix-daemon[849]: reaped child process 2101, status = succeeded7502lb # [ 177.792683] niks3-server[1194]: 2026/09/22 14:53:28 INFO Received uploads request method=POST path=/api/pending_closures7503worker1 # [ 177.831681] nix-grpc-daemon[1882]: time=2026-09-22T14:53:29.029Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7504worker1 # [ 177.833177] nix-grpc-daemon[1882]: time=2026-09-22T14:53:29.030Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7505lb # [ 177.827173] niks3-server[1194]: 2026/09/22 14:53:28 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7506lb # [ 177.840214] niks3-server[1194]: 2026/09/22 14:53:28 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7507lb # [ 177.852079] niks3-server[1194]: 2026/09/22 14:53:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7508lb # [ 177.856606] niks3-server[1194]: 2026/09/22 14:53:28 INFO Signed narinfos id=73 count=17509worker1 # [ 177.890297] nix-grpc-daemon[1882]: time=2026-09-22T14:53:29.087Z level=INFO msg="Uploading 1 narinfos"7510lb # [ 177.863281] niks3-server[1194]: 2026/09/22 14:53:28 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7511lb # [ 177.873130] niks3-server[1194]: 2026/09/22 14:53:28 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7512lb # [ 177.876952] niks3-server[1194]: 2026/09/22 14:53:28 INFO Completed upload id=737513lb # [ 177.878269] niks3-server[1194]: 2026/09/22 14:53:28 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7514worker1 # [ 177.907584] nix-grpc-daemon[1882]: time=2026-09-22T14:53:29.105Z level=INFO msg="Upload complete. (216ms)"7515worker1 # [ 177.914387] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=17516worker2 # [ 177.893931] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:29Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17517worker2 # [ 177.903834] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:29Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07518worker1 # [ 177.930243] nix-daemon[849]: reaped child process 2080, status = succeeded7519worker1 # [ 177.962956] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507520worker1 # [ 177.968590] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507521worker1 # [ 177.974218] nix-grpc-daemon[1882]: time=2026-09-22T14:53:29.171Z level=INFO msg="All 2 paths already cached"7522worker1 # [ 177.977527] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440507523worker1 # [ 177.982229] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7524worker1 # [ 178.063859] nix-daemon[849]: accepted connection from pid 2105, user nix-grpc-daemon (trusted)7525worker1 # [ 178.076519] nix-daemon[849]: reaped child process 2108, status = succeeded7526worker1 # [ 178.156524] nix-daemon[849]: accepted connection from pid 2110, user nix-grpc-daemon (trusted)7527lb # [ 178.139948] niks3-server[1194]: 2026/09/22 14:53:28 INFO Received uploads request method=POST path=/api/pending_closures7528worker1 # [ 178.171590] nix-daemon[849]: reaped child process 2113, status = succeeded7529worker1 # [ 178.175535] nix-grpc-daemon[1882]: time=2026-09-22T14:53:29.373Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7530worker1 # [ 178.176939] nix-grpc-daemon[1882]: time=2026-09-22T14:53:29.373Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7531lb # [ 178.154571] niks3-server[1194]: 2026/09/22 14:53:28 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7532lb # [ 178.158252] niks3-server[1194]: 2026/09/22 14:53:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7533lb # [ 178.160806] niks3-server[1194]: 2026/09/22 14:53:28 INFO Signed narinfos id=74 count=17534worker1 # [ 178.191645] nix-grpc-daemon[1882]: time=2026-09-22T14:53:29.389Z level=INFO msg="Uploading 1 narinfos"7535lb # [ 178.164151] niks3-server[1194]: 2026/09/22 14:53:28 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7536lb # [ 178.169408] niks3-server[1194]: 2026/09/22 14:53:28 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7537lb # [ 178.172659] niks3-server[1194]: 2026/09/22 14:53:28 INFO Completed upload id=747538worker1 # [ 178.203388] nix-grpc-daemon[1882]: time=2026-09-22T14:53:29.401Z level=INFO msg="Upload complete. (216ms)"7539worker1 # [ 178.205360] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=0 paths=1 nar_bytes_in=7527540lb # [ 178.176274] niks3-server[1194]: 2026/09/22 14:53:28 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7541worker1 # [ 178.209507] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507542worker1 # [ 178.216058] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7543worker1 # [ 178.217161] nix-daemon[849]: reaped child process 2103, status = succeeded7544worker1 # [ 178.222205] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7545worker1 # [ 179.074631] nix-daemon[849]: reaped child process 2117, status = succeeded7546worker1 # [ 179.139275] nix-daemon[849]: accepted connection from pid 2133, user nix-grpc-daemon (trusted)7547worker1 # [ 179.151574] nix-daemon[849]: reaped child process 2136, status = succeeded7548lb # [ 179.128678] niks3-server[1194]: 2026/09/22 14:53:29 INFO Received uploads request method=POST path=/api/pending_closures7549worker1 # [ 179.167821] nix-grpc-daemon[1882]: time=2026-09-22T14:53:30.365Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7550worker1 # [ 179.169449] nix-grpc-daemon[1882]: time=2026-09-22T14:53:30.367Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7551lb # [ 179.164662] niks3-server[1194]: 2026/09/22 14:53:29 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7552lb # [ 179.181652] niks3-server[1194]: 2026/09/22 14:53:29 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7553lb # [ 179.190056] niks3-server[1194]: 2026/09/22 14:53:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7554lb # [ 179.194145] niks3-server[1194]: 2026/09/22 14:53:29 INFO Signed narinfos id=75 count=17555worker1 # [ 179.225579] nix-grpc-daemon[1882]: time=2026-09-22T14:53:30.422Z level=INFO msg="Uploading 1 narinfos"7556lb # [ 179.201828] niks3-server[1194]: 2026/09/22 14:53:29 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7557lb # [ 179.210313] niks3-server[1194]: 2026/09/22 14:53:29 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7558lb # [ 179.213487] niks3-server[1194]: 2026/09/22 14:53:29 INFO Completed upload id=757559worker1 # [ 179.244154] nix-grpc-daemon[1882]: time=2026-09-22T14:53:30.441Z level=INFO msg="Upload complete. (214ms)"7560worker1 # [ 179.247154] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=17561lb # [ 179.218646] niks3-server[1194]: 2026/09/22 14:53:29 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7562worker2 # [ 179.227713] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:31Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17563worker1 # [ 179.255791] nix-daemon[849]: reaped child process 2115, status = succeeded7564worker1 # [ 179.298136] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507565worker1 # [ 179.306810] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507566worker1 # [ 179.315915] nix-grpc-daemon[1882]: time=2026-09-22T14:53:30.513Z level=INFO msg="All 2 paths already cached"7567worker1 # [ 179.321079] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440507568worker1 # [ 179.325228] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7569worker1 # [ 179.410857] nix-daemon[849]: accepted connection from pid 2140, user nix-grpc-daemon (trusted)7570worker1 # [ 179.424076] nix-daemon[849]: reaped child process 2143, status = succeeded7571worker1 # [ 179.502815] nix-daemon[849]: accepted connection from pid 2145, user nix-grpc-daemon (trusted)7572worker1 # [ 179.514939] nix-daemon[849]: reaped child process 2148, status = succeeded7573lb # [ 179.491589] niks3-server[1194]: 2026/09/22 14:53:29 INFO Received uploads request method=POST path=/api/pending_closures7574worker1 # [ 179.530380] nix-grpc-daemon[1882]: time=2026-09-22T14:53:30.728Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7575worker1 # [ 179.532249] nix-grpc-daemon[1882]: time=2026-09-22T14:53:30.729Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7576lb # [ 179.524471] niks3-server[1194]: 2026/09/22 14:53:29 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7577lb # [ 179.534365] niks3-server[1194]: 2026/09/22 14:53:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7578worker1 # [ 179.570825] nix-grpc-daemon[1882]: time=2026-09-22T14:53:30.768Z level=INFO msg="Uploading 1 narinfos"7579lb # [ 179.540386] niks3-server[1194]: 2026/09/22 14:53:29 INFO Signed narinfos id=76 count=17580lb # [ 179.546351] niks3-server[1194]: 2026/09/22 14:53:29 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7581lb # [ 179.558960] niks3-server[1194]: 2026/09/22 14:53:29 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7582worker1 # [ 179.598271] nix-grpc-daemon[1882]: time=2026-09-22T14:53:30.795Z level=INFO msg="Upload complete. (265ms)"7583lb # [ 179.567590] niks3-server[1194]: 2026/09/22 14:53:29 INFO Completed upload id=767584worker1 # [ 179.600545] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=0 paths=1 nar_bytes_in=7527585lb # [ 179.571240] niks3-server[1194]: 2026/09/22 14:53:29 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7586worker1 # [ 179.604516] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507587worker1 # [ 179.611582] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7588worker1 # [ 179.613090] nix-daemon[849]: reaped child process 2138, status = succeeded7589worker1 # [ 179.618189] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7590worker1 # [ 180.644626] nix-daemon[849]: reaped child process 2152, status = succeeded7591worker1 # [ 180.720920] nix-daemon[849]: accepted connection from pid 2168, user nix-grpc-daemon (trusted)7592worker1 # [ 180.732403] nix-daemon[849]: reaped child process 2171, status = succeeded7593lb # [ 180.709503] niks3-server[1194]: 2026/09/22 14:53:31 INFO Received uploads request method=POST path=/api/pending_closures7594worker1 # [ 180.749029] nix-grpc-daemon[1882]: time=2026-09-22T14:53:31.946Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7595worker1 # [ 180.751459] nix-grpc-daemon[1882]: time=2026-09-22T14:53:31.948Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7596lb # [ 180.746594] niks3-server[1194]: 2026/09/22 14:53:31 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7597lb # [ 180.754587] niks3-server[1194]: 2026/09/22 14:53:31 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7598lb # [ 180.766425] niks3-server[1194]: 2026/09/22 14:53:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7599lb # [ 180.770693] niks3-server[1194]: 2026/09/22 14:53:31 INFO Signed narinfos id=77 count=17600worker1 # [ 180.804527] nix-grpc-daemon[1882]: time=2026-09-22T14:53:32.002Z level=INFO msg="Uploading 1 narinfos"7601lb # [ 180.777217] niks3-server[1194]: 2026/09/22 14:53:31 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7602lb # [ 180.787505] niks3-server[1194]: 2026/09/22 14:53:31 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7603lb # [ 180.790918] niks3-server[1194]: 2026/09/22 14:53:31 INFO Completed upload id=777604worker1 # [ 180.821790] nix-grpc-daemon[1882]: time=2026-09-22T14:53:32.019Z level=INFO msg="Upload complete. (197ms)"7605lb # [ 180.793252] niks3-server[1194]: 2026/09/22 14:53:31 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7606worker1 # [ 180.827832] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=17607worker2 # [ 180.808753] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:32Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17608worker1 # [ 180.843072] nix-daemon[849]: reaped child process 2150, status = succeeded7609worker1 # [ 180.872909] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507610worker1 # [ 180.883356] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507611worker1 # [ 180.893245] nix-grpc-daemon[1882]: time=2026-09-22T14:53:32.090Z level=INFO msg="All 2 paths already cached"7612worker1 # [ 180.900594] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:440507613worker1 # [ 180.910159] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7614worker1 # [ 181.004728] nix-daemon[849]: accepted connection from pid 2175, user nix-grpc-daemon (trusted)7615worker1 # [ 181.017610] nix-daemon[849]: reaped child process 2178, status = succeeded7616worker1 # [ 181.098350] nix-daemon[849]: accepted connection from pid 2180, user nix-grpc-daemon (trusted)7617worker1 # [ 181.111075] nix-daemon[849]: reaped child process 2183, status = succeeded7618lb # [ 181.086157] niks3-server[1194]: 2026/09/22 14:53:31 INFO Received uploads request method=POST path=/api/pending_closures7619worker1 # [ 181.121140] nix-grpc-daemon[1882]: time=2026-09-22T14:53:32.318Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7620worker1 # [ 181.122491] nix-grpc-daemon[1882]: time=2026-09-22T14:53:32.318Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7621lb # [ 181.098973] niks3-server[1194]: 2026/09/22 14:53:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7622lb # [ 181.102539] niks3-server[1194]: 2026/09/22 14:53:31 INFO Signed narinfos id=78 count=17623worker1 # [ 181.133188] nix-grpc-daemon[1882]: time=2026-09-22T14:53:32.330Z level=INFO msg="Uploading 1 narinfos"7624lb # [ 181.104774] niks3-server[1194]: 2026/09/22 14:53:31 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7625lb # [ 181.108556] niks3-server[1194]: 2026/09/22 14:53:31 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7626lb # [ 181.117737] niks3-server[1194]: 2026/09/22 14:53:31 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7627lb # [ 181.120731] niks3-server[1194]: 2026/09/22 14:53:31 INFO Completed upload id=787628worker1 # [ 181.151605] nix-grpc-daemon[1882]: time=2026-09-22T14:53:32.349Z level=INFO msg="Upload complete. (229ms)"7629worker1 # [ 181.153823] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=0 paths=1 nar_bytes_in=7527630lb # [ 181.123778] niks3-server[1194]: 2026/09/22 14:53:31 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7631worker1 # [ 181.156798] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:440507632worker1 # [ 181.163437] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7633worker1 # [ 181.165720] nix-daemon[849]: reaped child process 2173, status = succeeded7634worker1 # [ 181.170698] nix-daemon[849]: accepted connection from pid 1822, user nix-grpc-daemon (trusted)7635worker1 # [ 182.185481] nix-daemon[849]: reaped child process 2187, status = succeeded7636worker1 # [ 182.258591] nix-daemon[849]: accepted connection from pid 2203, user nix-grpc-daemon (trusted)7637worker1 # [ 182.270771] nix-daemon[849]: reaped child process 2206, status = succeeded7638lb # [ 182.248621] niks3-server[1194]: 2026/09/22 14:53:32 INFO Received uploads request method=POST path=/api/pending_closures7639worker1 # [ 182.288131] nix-grpc-daemon[1882]: time=2026-09-22T14:53:33.485Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7640worker1 # [ 182.289819] nix-grpc-daemon[1882]: time=2026-09-22T14:53:33.487Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7641lb # [ 182.287350] niks3-server[1194]: 2026/09/22 14:53:32 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7642lb # [ 182.299140] niks3-server[1194]: 2026/09/22 14:53:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7643worker1 # [ 182.340826] nix-grpc-daemon[1882]: time=2026-09-22T14:53:33.538Z level=INFO msg="Uploading 1 narinfos"7644lb # [ 182.310223] niks3-server[1194]: 2026/09/22 14:53:32 INFO Signed narinfos id=79 count=17645lb # [ 182.314989] niks3-server[1194]: 2026/09/22 14:53:32 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7646lb # [ 182.320388] niks3-server[1194]: 2026/09/22 14:53:32 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7647lb # [ 182.331593] niks3-server[1194]: 2026/09/22 14:53:32 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7648lb # [ 182.338326] niks3-server[1194]: 2026/09/22 14:53:32 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7649lb # [ 182.342359] niks3-server[1194]: 2026/09/22 14:53:32 INFO Completed upload id=797650worker1 # [ 182.373409] nix-grpc-daemon[1882]: time=2026-09-22T14:53:33.571Z level=INFO msg="Upload complete. (195ms)"7651worker1 # [ 182.379537] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44050 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=17652worker1 # [ 182.396253] nix-daemon[849]: reaped child process 2185, status = succeeded7653client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.23 seconds)7654worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7655worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7656client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7657client # [ 182.508988] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7658client # Running as unit: gpu.service; invocation ID: 75890cb453ff4d61a8b9022c908048757659client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7660lb: must succeed: curl -sf localhost:9901/clusters7661lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7662lb: must succeed: curl -sf localhost:9901/clusters7663lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7664worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7665worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7666worker1 # [ 182.650245] nix-grpc-daemon[1822]: ts=2026-09-22T14:53:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:440507667client # [ 182.646067] nix[1531]: don't know how to build these paths:7668client # [ 182.646914] nix[1531]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7669worker2 # [ 182.636689] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499307670client # [ 182.667094] nix[1531]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7671worker2 # [ 182.655137] nix-grpc-daemon[1670]: ts=2026-09-22T14:53:34Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07672lb: must succeed: curl -sf localhost:9901/clusters7673lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7674lb: must succeed: curl -sf localhost:9901/clusters7675lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7676worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7677worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7678client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7679lb # [ 187.651542] envoy[924]: conn peer=[2001:db8:1::1]:33958 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27098 tx=12337 ms=150497680client # [ 212.668226] nix[1531]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7681client # [ 212.677783] nix[1531]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7682client # [ 212.703976] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7683client # [ 212.707651] systemd[1]: gpu.service: Failed with result 'exit-code'.7684client # [ 212.711681] systemd[1]: gpu.service: Consumed 141ms CPU time over 30.196s wall clock time, 35.6M memory peak, 5.2K incoming IP traffic, 5K outgoing IP traffic.7685client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.21 seconds)7686client: must succeed: journalctl -u gpu -o cat7687client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7688lb: must succeed: curl -sf localhost:9901/clusters7689lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7690lb: must succeed: curl -sf localhost:9901/clusters7691lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7692worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7693worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7694(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.89 seconds)7695subtest: low disk drains a worker and builds go to the other7696worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7697worker1 # [ 214.033958] nix-grpc-daemon[1822]: ts=2026-09-22T14:54:05Z level=info event=unhealthy reason=min_free7698worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.96 seconds)7699worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7700worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.02 seconds)7701client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27702worker2 # [ 214.190403] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:499307703client # don't know how to build these paths:7704client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7705worker1 # [ 214.220700] nix-grpc-daemon[1822]: ts=2026-09-22T14:54:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:440507706worker2 # [ 214.221149] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:06Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07707worker2 # [ 214.225066] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:06Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17708worker2 # [ 214.273833] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307709worker2 # [ 214.284365] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499307710client # copying 1 paths to 'lb:50051'...7711client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7712worker2 # [ 214.294976] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499307713worker2 # [ 214.303836] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7714worker2 # [ 214.410387] nix-daemon[869]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)7715worker2 # [ 214.423989] nix-daemon[869]: reaped child process 1980, status = succeeded7716worker2 # [ 214.508678] nix-daemon[869]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)7717worker2 # [ 214.519770] nix-daemon[869]: reaped child process 1985, status = succeeded7718lb # [ 214.518596] niks3-server[1194]: 2026/09/22 14:54:04 INFO Received uploads request method=POST path=/api/pending_closures7719worker2 # [ 214.534143] nix-grpc-daemon[1762]: time=2026-09-22T14:54:06.560Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7720worker2 # [ 214.536105] nix-grpc-daemon[1762]: time=2026-09-22T14:54:06.562Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7721lb # [ 214.542650] niks3-server[1194]: 2026/09/22 14:54:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7722worker2 # [ 214.556651] nix-grpc-daemon[1762]: time=2026-09-22T14:54:06.583Z level=INFO msg="Uploading 1 narinfos"7723lb # [ 214.547548] niks3-server[1194]: 2026/09/22 14:54:04 INFO Signed narinfos id=80 count=17724lb # [ 214.558497] niks3-server[1194]: 2026/09/22 14:54:04 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7725lb # [ 214.563258] niks3-server[1194]: 2026/09/22 14:54:04 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7726lb # [ 214.570386] niks3-server[1194]: 2026/09/22 14:54:04 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7727lb # [ 214.576928] niks3-server[1194]: 2026/09/22 14:54:04 INFO Completed upload id=807728worker2 # [ 214.586564] nix-grpc-daemon[1762]: time=2026-09-22T14:54:06.613Z level=INFO msg="Upload complete. (269ms)"7729worker2 # [ 214.588781] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=0 paths=1 nar_bytes_in=5127730lb # [ 214.581262] niks3-server[1194]: 2026/09/22 14:54:04 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7731worker2 # [ 214.593036] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307732worker2 # [ 214.599257] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7733worker2 # [ 214.602283] nix-daemon[869]: reaped child process 1975, status = succeeded7734client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7735client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7736worker2 # [ 214.607263] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7737client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7738client # farm-a> LOG-farm-a-drain7739worker2 # [ 215.427380] nix-daemon[869]: reaped child process 1989, status = succeeded7740worker2 # [ 215.496515] nix-daemon[869]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)7741worker2 # [ 215.508236] nix-daemon[869]: reaped child process 2008, status = succeeded7742lb # [ 215.505066] niks3-server[1194]: 2026/09/22 14:54:05 INFO Received uploads request method=POST path=/api/pending_closures7743worker2 # [ 215.519447] nix-grpc-daemon[1762]: time=2026-09-22T14:54:07.545Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7744worker2 # [ 215.521095] nix-grpc-daemon[1762]: time=2026-09-22T14:54:07.547Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7745lb # [ 215.527254] niks3-server[1194]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7746lb # [ 215.532193] niks3-server[1194]: 2026/09/22 14:54:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7747lb # [ 215.534615] niks3-server[1194]: 2026/09/22 14:54:05 INFO Signed narinfos id=81 count=17748worker2 # [ 215.543905] nix-grpc-daemon[1762]: time=2026-09-22T14:54:07.570Z level=INFO msg="Uploading 1 narinfos"7749lb # [ 215.537131] niks3-server[1194]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7750lb # [ 215.542074] niks3-server[1194]: 2026/09/22 14:54:05 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7751lb # [ 215.545359] niks3-server[1194]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7752lb # [ 215.548936] niks3-server[1194]: 2026/09/22 14:54:05 INFO Completed upload id=817753worker2 # [ 215.558450] nix-grpc-daemon[1762]: time=2026-09-22T14:54:07.584Z level=INFO msg="Upload complete. (144ms)"7754lb # [ 215.551089] niks3-server[1194]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7755worker2 # [ 215.561207] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=0 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=17756worker2 # [ 215.568351] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:07Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07757worker2 # [ 215.570963] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:07Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17758worker2 # [ 215.574631] nix-daemon[869]: reaped child process 1987, status = succeeded7759worker2 # [ 215.611826] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307760worker2 # [ 215.620091] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499307761worker2 # [ 215.630475] nix-grpc-daemon[1762]: time=2026-09-22T14:54:07.656Z level=INFO msg="All 1 paths already cached"7762client # copying 1 paths to 'lb:50051'...7763client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7764worker2 # [ 215.636517] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499307765worker2 # [ 215.644126] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7766worker2 # [ 215.729788] nix-daemon[869]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)7767worker2 # [ 215.742393] nix-daemon[869]: reaped child process 2017, status = succeeded7768worker2 # [ 215.824616] nix-daemon[869]: accepted connection from pid 2019, user nix-grpc-daemon (trusted)7769worker2 # [ 215.837591] nix-daemon[869]: reaped child process 2022, status = succeeded7770lb # [ 215.834379] niks3-server[1194]: 2026/09/22 14:54:06 INFO Received uploads request method=POST path=/api/pending_closures7771worker2 # [ 215.854059] nix-grpc-daemon[1762]: time=2026-09-22T14:54:07.880Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7772worker2 # [ 215.855434] nix-grpc-daemon[1762]: time=2026-09-22T14:54:07.880Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7773lb # [ 215.864496] niks3-server[1194]: 2026/09/22 14:54:06 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7774lb # [ 215.874313] niks3-server[1194]: 2026/09/22 14:54:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7775lb # [ 215.879252] niks3-server[1194]: 2026/09/22 14:54:06 INFO Signed narinfos id=82 count=17776worker2 # [ 215.891170] nix-grpc-daemon[1762]: time=2026-09-22T14:54:07.917Z level=INFO msg="Uploading 1 narinfos"7777lb # [ 215.887853] niks3-server[1194]: 2026/09/22 14:54:06 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7778lb # [ 215.896728] niks3-server[1194]: 2026/09/22 14:54:06 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7779lb # [ 215.900144] niks3-server[1194]: 2026/09/22 14:54:06 INFO Completed upload id=827780worker2 # [ 215.909882] nix-grpc-daemon[1762]: time=2026-09-22T14:54:07.936Z level=INFO msg="Upload complete. (258ms)"7781lb # [ 215.901986] niks3-server[1194]: 2026/09/22 14:54:06 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7782worker2 # [ 215.913966] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=0 paths=1 nar_bytes_in=6247783worker2 # [ 215.918741] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307784worker2 # [ 215.927978] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7785worker2 # [ 215.934924] nix-daemon[869]: reaped child process 2012, status = succeeded7786worker2 # [ 215.937841] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7787client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7788client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7789client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7790client # farm-b> LOG-farm-b-drain7791worker2 # [ 217.374746] nix-daemon[869]: reaped child process 2026, status = succeeded7792worker2 # [ 217.457583] nix-daemon[869]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)7793worker2 # [ 217.469177] nix-daemon[869]: reaped child process 2045, status = succeeded7794lb # [ 217.467580] niks3-server[1194]: 2026/09/22 14:54:07 INFO Received uploads request method=POST path=/api/pending_closures7795worker2 # [ 217.485440] nix-grpc-daemon[1762]: time=2026-09-22T14:54:09.511Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7796worker2 # [ 217.486990] nix-grpc-daemon[1762]: time=2026-09-22T14:54:09.513Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7797lb # [ 217.508258] niks3-server[1194]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7798lb # [ 217.513917] niks3-server[1194]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7799lb # [ 217.528550] niks3-server[1194]: 2026/09/22 14:54:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7800lb # [ 217.534806] niks3-server[1194]: 2026/09/22 14:54:07 INFO Signed narinfos id=83 count=17801worker2 # [ 217.547561] nix-grpc-daemon[1762]: time=2026-09-22T14:54:09.573Z level=INFO msg="Uploading 1 narinfos"7802lb # [ 217.541745] niks3-server[1194]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7803lb # [ 217.550967] niks3-server[1194]: 2026/09/22 14:54:07 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7804lb # [ 217.554693] niks3-server[1194]: 2026/09/22 14:54:07 INFO Completed upload id=837805worker2 # [ 217.564394] nix-grpc-daemon[1762]: time=2026-09-22T14:54:09.590Z level=INFO msg="Upload complete. (218ms)"7806lb # [ 217.557224] niks3-server[1194]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7807worker2 # [ 217.572670] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=17808worker2 # [ 217.582192] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:09Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07809worker2 # [ 217.588306] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:09Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17810worker2 # [ 217.597874] nix-daemon[869]: reaped child process 2024, status = succeeded7811worker2 # [ 217.626412] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307812worker2 # [ 217.639103] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499307813worker2 # [ 217.652097] nix-grpc-daemon[1762]: time=2026-09-22T14:54:09.678Z level=INFO msg="All 2 paths already cached"7814client # copying 1 paths to 'lb:50051'...7815client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7816worker2 # [ 217.657301] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499307817worker2 # [ 217.662543] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7818lb # [ 217.652749] envoy[924]: conn peer=[2001:db8:1::1]:40340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=562 ms=300457819worker2 # [ 217.748792] nix-daemon[869]: accepted connection from pid 2051, user nix-grpc-daemon (trusted)7820worker2 # [ 217.762416] nix-daemon[869]: reaped child process 2054, status = succeeded7821worker2 # [ 217.846501] nix-daemon[869]: accepted connection from pid 2056, user nix-grpc-daemon (trusted)7822worker2 # [ 217.860201] nix-daemon[869]: reaped child process 2059, status = succeeded7823lb # [ 217.857610] niks3-server[1194]: 2026/09/22 14:54:08 INFO Received uploads request method=POST path=/api/pending_closures7824worker2 # [ 217.874851] nix-grpc-daemon[1762]: time=2026-09-22T14:54:09.901Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7825worker2 # [ 217.876238] nix-grpc-daemon[1762]: time=2026-09-22T14:54:09.901Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7826lb # [ 217.881746] niks3-server[1194]: 2026/09/22 14:54:08 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7827lb # [ 217.895245] niks3-server[1194]: 2026/09/22 14:54:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7828worker2 # [ 217.909242] nix-grpc-daemon[1762]: time=2026-09-22T14:54:09.935Z level=INFO msg="Uploading 1 narinfos"7829lb # [ 217.899676] niks3-server[1194]: 2026/09/22 14:54:08 INFO Signed narinfos id=84 count=17830lb # [ 217.906871] niks3-server[1194]: 2026/09/22 14:54:08 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7831lb # [ 217.917238] niks3-server[1194]: 2026/09/22 14:54:08 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7832lb # [ 217.923799] niks3-server[1194]: 2026/09/22 14:54:08 INFO Completed upload id=847833worker2 # [ 217.935450] nix-grpc-daemon[1762]: time=2026-09-22T14:54:09.961Z level=INFO msg="Upload complete. (267ms)"7834lb # [ 217.928742] niks3-server[1194]: 2026/09/22 14:54:08 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7835worker2 # [ 217.939069] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=0 paths=1 nar_bytes_in=7527836worker2 # [ 217.943821] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499307837worker2 # [ 217.954377] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7838worker2 # [ 217.960767] nix-daemon[869]: reaped child process 2049, status = succeeded7839worker2 # [ 217.963681] nix-daemon[869]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7840client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7841client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7842client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7843client # farm-top> LOG-farm-top-drain7844worker2 # [ 218.979323] nix-daemon[869]: reaped child process 2063, status = succeeded7845worker2 # [ 219.052535] nix-daemon[869]: accepted connection from pid 2079, user nix-grpc-daemon (trusted)7846worker2 # [ 219.065975] nix-daemon[869]: reaped child process 2082, status = succeeded7847lb # [ 219.060938] niks3-server[1194]: 2026/09/22 14:54:09 INFO Received uploads request method=POST path=/api/pending_closures7848worker2 # [ 219.079979] nix-grpc-daemon[1762]: time=2026-09-22T14:54:11.106Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7849worker2 # [ 219.082362] nix-grpc-daemon[1762]: time=2026-09-22T14:54:11.108Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7850lb # [ 219.094592] niks3-server[1194]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7851lb # [ 219.107707] niks3-server[1194]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7852lb # [ 219.120201] niks3-server[1194]: 2026/09/22 14:54:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7853worker2 # [ 219.135209] nix-grpc-daemon[1762]: time=2026-09-22T14:54:11.161Z level=INFO msg="Uploading 1 narinfos"7854lb # [ 219.125594] niks3-server[1194]: 2026/09/22 14:54:09 INFO Signed narinfos id=85 count=17855lb # [ 219.132147] niks3-server[1194]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7856lb # [ 219.140143] niks3-server[1194]: 2026/09/22 14:54:09 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7857lb # [ 219.144420] niks3-server[1194]: 2026/09/22 14:54:09 INFO Completed upload id=857858worker2 # [ 219.153989] nix-grpc-daemon[1762]: time=2026-09-22T14:54:11.180Z level=INFO msg="Upload complete. (183ms)"7859lb # [ 219.146239] niks3-server[1194]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7860worker2 # [ 219.156779] nix-grpc-daemon[1670]: ts=2026-09-22T14:54:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49930 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=17861worker2 # [ 219.169611] nix-daemon[869]: reaped child process 2061, status = succeeded7862client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.15 seconds)7863client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7864client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.10 seconds)7865worker1: must succeed: rm /nix/.rw-store/fill7866worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.11 seconds)7867worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7868worker1 # [ 220.035603] nix-grpc-daemon[1822]: ts=2026-09-22T14:54:11Z level=info event=healthy reason=min_free7869worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7870(finished: subtest: low disk drains a worker and builds go to the other, in 7.45 seconds)7871(finished: run the VM test script, in 221.45 seconds)7872test script finished in 221.63s7873cleanup7874kill QemuMachine (pid 49)7875client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7876client # [2026-09-22T14:54:12Z INFO virtiofsd] Client disconnected, shutting down7877client # [2026-09-22T14:54:12Z INFO virtiofsd] Client disconnected, shutting down7878client # [2026-09-22T14:54:12Z INFO virtiofsd] Client disconnected, shutting down7879kill QemuMachine (pid 50)7880lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7881lb # [2026-09-22T14:54:12Z INFO virtiofsd] Client disconnected, shutting down7882lb # [2026-09-22T14:54:12Z INFO virtiofsd] Client disconnected, shutting down7883lb # [2026-09-22T14:54:12Z INFO virtiofsd] Client disconnected, shutting down7884kill QemuMachine (pid 51)7885worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7886worker1 # [2026-09-22T14:54:12Z INFO virtiofsd] Client disconnected, shutting down7887worker1 # [2026-09-22T14:54:12Z INFO virtiofsd] Client disconnected, shutting down7888worker1 # [2026-09-22T14:54:12Z INFO virtiofsd] Client disconnected, shutting down7889kill QemuMachine (pid 52)7890worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7891worker2 # [2026-09-22T14:54:13Z INFO virtiofsd] Client disconnected, shutting down7892worker2 # [2026-09-22T14:54:13Z INFO virtiofsd] Client disconnected, shutting down7893worker2 # [2026-09-22T14:54:13Z INFO virtiofsd] Client disconnected, shutting down7894(finished: cleanup, in 1.08 seconds)7895additionally exposed symbols:7896 client, lb, worker1, worker2,7897 vlan1,7898 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh