vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #532
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.7ND1ZAinQ2', fmt=raw size=107374182414worker2 # mke2fs 1.47.4 (6-Mar-2025)15worker2: QEMU running (pid 52)16worker2 # Discarding device blocks: 0/262144 done17worker1 # Disk image does not exist, creating the virtualisation disk image...18client: QEMU running (pid 50)19worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes20worker1 # Formatting '/build/vm-state-worker1/tmp.knK14uUqmZ', fmt=raw size=107374182421worker1: QEMU running (pid 51)22client # Disk image does not exist, creating the virtualisation disk image...23worker2 # Filesystem UUID: c78d41dc-5684-49b7-adc2-e6a090d67ada24lb # Disk image does not exist, creating the virtualisation disk image...25lb: QEMU running (pid 49)26worker2 # Superblock backups stored on blocks:27lb # Formatting '/build/vm-state-lb/tmp.4qabcTta6p', fmt=raw size=107374182428client # Formatting '/build/vm-state-client/tmp.ctckxly6Dv', fmt=raw size=107374182429worker1 # mke2fs 1.47.4 (6-Mar-2025)30worker2 # 32768, 98304, 163840, 22937631lb # mke2fs 1.47.4 (6-Mar-2025)32worker1 # Discarding device blocks: 0/262144 done33worker2 # 34lb # Discarding device blocks: 0/262144 done35worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes36worker2 # Allocating group tables: 0/8 done37lb # Creating filesystem with 262144 4k blocks and 65536 inodes38worker1 # Filesystem UUID: 70146035-bf5b-448d-8907-857e8c6e25a339worker2 # Writing inode tables: 0/8 done40lb # Filesystem UUID: f6a17f58-4013-4783-9a94-b59379a6066241worker1 # Superblock backups stored on blocks:42worker2 # Creating journal (8192 blocks): done43lb # Superblock backups stored on blocks:44worker1 # 32768, 98304, 163840, 22937645worker2 # Writing superblocks and filesystem accounting information: 0/8 done46lb # 32768, 98304, 163840, 22937647worker1 # 48worker2 # 49lb # 50worker1 # Allocating group tables: 0/8 done51worker2 # Virtualisation disk image created.52lb # Allocating group tables: 0/8 done53worker1 # Writing inode tables: 0/8 done54worker2 # Starting virtiofs daemons...55lb # Writing inode tables: 0/8 done56worker1 # Creating journal (8192 blocks): done57worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)58lb # Creating journal (8192 blocks): done59worker1 # Writing superblocks and filesystem accounting information: 0/8 done60worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether61lb # Writing superblocks and filesystem accounting information: 0/8 done62worker1 # 63worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...64lb # 65worker1 # Virtualisation disk image created.66worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67lb # Virtualisation disk image created.68worker1 # Starting virtiofs daemons...69worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70lb # Starting virtiofs daemons...71worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)72worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...73lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether75worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...78worker2 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79lb # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...80worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)81worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...82lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)83worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests85lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether86worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...87worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests88lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)89worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90worker2 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests91lb # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker1 # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93(finished: start all VMs, in 0.24 seconds)94client # mke2fs 1.47.4 (6-Mar-2025)95lb # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...96worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...97client: waiting for unit network-addresses-eth1.service98client # Discarding device blocks: 0/262144 done99lb # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...100worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests101client: waiting for the VM to finish booting102client # Creating filesystem with 262144 4k blocks and 65536 inodes103lb # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests104client # Filesystem UUID: d8967c73-c804-4787-9148-a44ac641fad0105lb # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests106client # Superblock backups stored on blocks:107lb # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests108client # 32768, 98304, 163840, 229376109worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests110client # 111worker1 # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests112client # Allocating group tables: 0/8 done113client # Writing inode tables: 0/8 done114client # Creating journal (8192 blocks): done115client # Writing superblocks and filesystem accounting information: 0/8 done116client # 117client # Virtualisation disk image created.118client # Starting virtiofs daemons...119client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...122client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)125client # [2026-09-23T05:32:52Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether126client # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...127client # [2026-09-23T05:32:52Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-23T05:32:52Z INFO virtiofsd] Client connected, servicing requests131lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135lb # 136lb # 137client # 138client # 139lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00140client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141worker1 # 142worker1 # 143worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00144lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...145lb # 146lb # 147lb # 148lb # 149client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...150client # 151client # 152client # 153client # 154lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00155client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...157worker1 # 158worker1 # 159worker1 # 160worker1 # 161worker2 # 162worker2 # 163worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00164worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00165lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...166lb # 167lb # 168client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169client # 170client # 171worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...172worker2 # 173worker2 # 174worker2 # 175worker2 # 176worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...177worker1 # 178worker1 # 179worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker2 # 182worker2 # 183lb # Booting from ROM...184client # Booting from ROM...185worker1 # Booting from ROM...186worker2 # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026189worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026190client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test/init regInfo=/nix/store/lkz8fx5m8vmi31dy1j6w4dxp7rzq8kxy-closure-info/registration console=ttyS0,115200n8 console=tty0191client # [ 0.000000] BIOS-provided physical RAM map:192client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable193lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n0idzb2a5795zcdcs33p9240n19lall0-nixos-system-lb-test/init regInfo=/nix/store/2bxws5jbs8f1q8hlcfqpb1bb7xljw1g0-closure-info/registration console=ttyS0,115200n8 console=tty0194client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved195worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgp1w89psmkzqxph18mrrd4l45zq8icp-nixos-system-worker1-test/init regInfo=/nix/store/4vn7n0cqh1fw60k9fdivbpaggw9ifflq-closure-info/registration console=ttyS0,115200n8 console=tty0196lb # [ 0.000000] BIOS-provided physical RAM map:197worker1 # [ 0.000000] BIOS-provided physical RAM map:198client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved199lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable200worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable201client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable202lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved203worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved204client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved205lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved206worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved207client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved208lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable209worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable210client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved211worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026212lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved213worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved214client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved215lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved216worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved217client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved218lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved219worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved220client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved221lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved222client # [ 0.000000] NX (Execute Disable) protection: active223worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved224lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved225client # [ 0.000000] APIC: Static calls initialized226client # [ 0.000000] SMBIOS 2.8 present.227worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kknbsz59pyihvz42vvc4xnin6z1984g8-nixos-system-worker2-test/init regInfo=/nix/store/d30hd8qj8b373nfwj9wkrwf1qqzjbbdk-closure-info/registration console=ttyS0,115200n8 console=tty0228worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved229lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved230worker2 # [ 0.000000] BIOS-provided physical RAM map:231worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved232lb # [ 0.000000] NX (Execute Disable) protection: active233client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014234worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable235worker1 # [ 0.000000] NX (Execute Disable) protection: active236lb # [ 0.000000] APIC: Static calls initialized237client # [ 0.000000] DMI: Memory slots populated: 1/1238worker1 # [ 0.000000] APIC: Static calls initialized239lb # [ 0.000000] SMBIOS 2.8 present.240client # [ 0.000000] Hypervisor detected: KVM241worker1 # [ 0.000000] SMBIOS 2.8 present.242worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved243client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000244worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved245lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014246worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014247client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00248lb # [ 0.000000] DMI: Memory slots populated: 1/1249worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable250worker1 # [ 0.000000] DMI: Memory slots populated: 1/1251client # [ 0.000000] kvm-clock: using sched offset of 481706363 cycles252lb # [ 0.000000] Hypervisor detected: KVM253worker1 # [ 0.000000] Hypervisor detected: KVM254worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved255lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000256worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000257client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns258lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00259worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved260worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00261client # [ 0.000005] tsc: Detected 2400.012 MHz processor262lb # [ 0.000001] kvm-clock: using sched offset of 476503539 cycles263worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved264worker1 # [ 0.000000] kvm-clock: using sched offset of 486640087 cycles265client # [ 0.000809] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000266worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved267lb # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns268client # [ 0.000836] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs269worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns270lb # [ 0.000005] tsc: Detected 2400.012 MHz processor271worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved272worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor273client # [ 0.000838] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT274lb # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000275worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000276worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved277client # [ 0.002741] found SMP MP-table at [mem 0x000f5450-0x000f545f]278worker2 # [ 0.000000] NX (Execute Disable) protection: active279lb # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs280client # [ 0.002752] Using GB pages for direct mapping281worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs282worker2 # [ 0.000000] APIC: Static calls initialized283client # [ 0.002822] RAMDISK: [mem 0x5e370000-0x5ffcffff]284worker2 # [ 0.000000] SMBIOS 2.8 present.285lb # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT286worker1 # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT287client # [ 0.002831] ACPI: Early table checksum verification disabled288lb # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]289worker1 # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f]290client # [ 0.002834] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )291lb # [ 0.002740] Using GB pages for direct mapping292worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014293worker1 # [ 0.002741] Using GB pages for direct mapping294worker2 # [ 0.000000] DMI: Memory slots populated: 1/1295lb # [ 0.002885] RAMDISK: [mem 0x5e36e000-0x5ffcffff]296worker1 # [ 0.002868] RAMDISK: [mem 0x5e370000-0x5ffcffff]297worker2 # [ 0.000000] Hypervisor detected: KVM298client # [ 0.002838] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)299lb # [ 0.002894] ACPI: Early table checksum verification disabled300worker1 # [ 0.002876] ACPI: Early table checksum verification disabled301worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000302client # [ 0.002842] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)303lb # [ 0.002896] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )304worker1 # [ 0.002879] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )305worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00306lb # [ 0.002900] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)307worker2 # [ 0.000000] kvm-clock: using sched offset of 494355129 cycles308client # [ 0.002849] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)309worker1 # [ 0.002883] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)310client # [ 0.002851] ACPI: FACS 0x000000005FFE0000 000040311lb # [ 0.002904] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)312worker1 # [ 0.002887] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)313worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns314client # [ 0.002853] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)315worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor316lb # [ 0.002911] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)317worker1 # [ 0.002893] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)318client # [ 0.002854] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)319worker2 # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000320lb # [ 0.002913] ACPI: FACS 0x000000005FFE0000 000040321worker1 # [ 0.002895] ACPI: FACS 0x000000005FFE0000 000040322lb # [ 0.002914] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)323client # [ 0.002856] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)324worker2 # [ 0.000833] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs325worker1 # [ 0.002897] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)326worker2 # [ 0.000836] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT327client # [ 0.002857] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)328lb # [ 0.002916] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)329worker1 # [ 0.002898] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)330worker2 # [ 0.002721] found SMP MP-table at [mem 0x000f5450-0x000f545f]331client # [ 0.002859] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]332lb # [ 0.002917] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)333worker2 # [ 0.002732] Using GB pages for direct mapping334worker1 # [ 0.002900] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)335worker2 # [ 0.002825] RAMDISK: [mem 0x5e36f000-0x5ffcffff]336client # [ 0.002860] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]337lb # [ 0.002919] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)338worker2 # [ 0.002834] ACPI: Early table checksum verification disabled339worker1 # [ 0.002901] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)340client # [ 0.002860] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]341lb # [ 0.002920] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]342worker2 # [ 0.002837] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )343worker1 # [ 0.002903] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]344client # [ 0.002861] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]345lb # [ 0.002921] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]346worker1 # [ 0.002904] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]347worker2 # [ 0.002841] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)348client # [ 0.002861] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]349lb # [ 0.002922] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]350worker1 # [ 0.002904] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]351client # [ 0.002862] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]352worker2 # [ 0.002845] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)353lb # [ 0.002922] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]354worker1 # [ 0.002905] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]355client # [ 0.002862] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]356worker2 # [ 0.002851] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)357client # [ 0.003095] No NUMA configuration found358lb # [ 0.002923] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]359worker1 # [ 0.002905] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]360worker2 # [ 0.002853] ACPI: FACS 0x000000005FFE0000 000040361client # [ 0.003096] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]362lb # [ 0.002923] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]363worker1 # [ 0.002906] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]364worker2 # [ 0.002855] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)365client # [ 0.003099] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]366lb # [ 0.002923] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]367client # [ 0.005254] Zone ranges:368worker1 # [ 0.002906] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]369lb # [ 0.003160] No NUMA configuration found370worker2 # [ 0.002856] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)371worker1 # [ 0.003136] No NUMA configuration found372client # [ 0.005254] DMA [mem 0x0000000000001000-0x0000000000ffffff]373lb # [ 0.003161] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]374worker1 # [ 0.003137] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]375client # [ 0.005256] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]376worker2 # [ 0.002858] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)377client # [ 0.005257] Normal empty378lb # [ 0.003163] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]379worker1 # [ 0.003140] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]380lb # [ 0.005360] Zone ranges:381client # [ 0.005258] Device empty382worker1 # [ 0.005373] Zone ranges:383client # [ 0.005258] Movable zone start for each node384worker2 # [ 0.002859] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)385lb # [ 0.005361] DMA [mem 0x0000000000001000-0x0000000000ffffff]386client # [ 0.005259] Early memory node ranges387worker1 # [ 0.005374] DMA [mem 0x0000000000001000-0x0000000000ffffff]388worker2 # [ 0.002861] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]389lb # [ 0.005363] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]390client # [ 0.005259] node 0: [mem 0x0000000000001000-0x000000000009efff]391worker1 # [ 0.005375] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]392lb # [ 0.005364] Normal empty393worker1 # [ 0.005376] Normal empty394lb # [ 0.005365] Device empty395worker2 # [ 0.002862] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]396worker1 # [ 0.005377] Device empty397client # [ 0.005261] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]398lb # [ 0.005365] Movable zone start for each node399worker1 # [ 0.005377] Movable zone start for each node400worker2 # [ 0.002862] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]401lb # [ 0.005366] Early memory node ranges402worker1 # [ 0.005378] Early memory node ranges403client # [ 0.005262] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]404lb # [ 0.005366] node 0: [mem 0x0000000000001000-0x000000000009efff]405client # [ 0.005282] On node 0, zone DMA: 1 pages in unavailable ranges406worker2 # [ 0.002863] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]407worker1 # [ 0.005379] node 0: [mem 0x0000000000001000-0x000000000009efff]408lb # [ 0.005367] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]409client # [ 0.005547] On node 0, zone DMA: 97 pages in unavailable ranges410worker1 # [ 0.005380] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]411worker2 # [ 0.002863] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]412client # [ 0.030819] On node 0, zone DMA32: 40 pages in unavailable ranges413lb # [ 0.005368] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]414worker1 # [ 0.005381] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]415worker2 # [ 0.002864] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]416client # [ 0.031284] ACPI: PM-Timer IO Port: 0x608417lb # [ 0.005392] On node 0, zone DMA: 1 pages in unavailable ranges418worker1 # [ 0.005399] On node 0, zone DMA: 1 pages in unavailable ranges419worker2 # [ 0.002864] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]420client # [ 0.031296] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])421lb # [ 0.005653] On node 0, zone DMA: 97 pages in unavailable ranges422worker2 # [ 0.003101] No NUMA configuration found423worker1 # [ 0.005653] On node 0, zone DMA: 97 pages in unavailable ranges424client # [ 0.031325] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23425lb # [ 0.031191] On node 0, zone DMA32: 40 pages in unavailable ranges426worker1 # [ 0.031082] On node 0, zone DMA32: 40 pages in unavailable ranges427worker2 # [ 0.003102] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]428lb # [ 0.031651] ACPI: PM-Timer IO Port: 0x608429client # [ 0.031328] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)430worker1 # [ 0.031551] ACPI: PM-Timer IO Port: 0x608431worker2 # [ 0.003104] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]432lb # [ 0.031663] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])433worker2 # [ 0.005302] Zone ranges:434client # [ 0.031329] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)435worker1 # [ 0.031563] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])436worker2 # [ 0.005303] DMA [mem 0x0000000000001000-0x0000000000ffffff]437lb # [ 0.031692] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23438client # [ 0.031331] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)439worker1 # [ 0.031594] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23440lb # [ 0.031694] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)441worker2 # [ 0.005305] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]442client # [ 0.031331] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)443worker1 # [ 0.031596] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)444worker2 # [ 0.005306] Normal empty445worker2 # [ 0.005307] Device empty446lb # [ 0.031696] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)447client # [ 0.031332] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)448worker1 # [ 0.031598] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)449worker2 # [ 0.005308] Movable zone start for each node450lb # [ 0.031697] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)451client # [ 0.031334] ACPI: Using ACPI (MADT) for SMP configuration information452worker2 # [ 0.005308] Early memory node ranges453worker1 # [ 0.031599] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)454client # [ 0.031335] ACPI: HPET id: 0x8086a201 base: 0xfed00000455worker2 # [ 0.005309] node 0: [mem 0x0000000000001000-0x000000000009efff]456lb # [ 0.031698] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)457worker1 # [ 0.031600] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)458client # [ 0.031338] TSC deadline timer available459worker2 # [ 0.005310] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]460lb # [ 0.031698] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)461client # [ 0.031342] CPU topo: Max. logical packages: 1462worker1 # [ 0.031601] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)463client # [ 0.031343] CPU topo: Max. logical dies: 1464lb # [ 0.031701] ACPI: Using ACPI (MADT) for SMP configuration information465worker2 # [ 0.005311] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]466worker1 # [ 0.031603] ACPI: Using ACPI (MADT) for SMP configuration information467client # [ 0.031343] CPU topo: Max. dies per package: 1468lb # [ 0.031702] ACPI: HPET id: 0x8086a201 base: 0xfed00000469worker2 # [ 0.005332] On node 0, zone DMA: 1 pages in unavailable ranges470worker1 # [ 0.031604] ACPI: HPET id: 0x8086a201 base: 0xfed00000471client # [ 0.031347] CPU topo: Max. threads per core: 1472lb # [ 0.031704] TSC deadline timer available473worker1 # [ 0.031607] TSC deadline timer available474worker2 # [ 0.005584] On node 0, zone DMA: 97 pages in unavailable ranges475lb # [ 0.031709] CPU topo: Max. logical packages: 1476worker1 # [ 0.031611] CPU topo: Max. logical packages: 1477client # [ 0.031347] CPU topo: Num. cores per package: 1478worker2 # [ 0.030842] On node 0, zone DMA32: 40 pages in unavailable ranges479lb # [ 0.031709] CPU topo: Max. logical dies: 1480client # [ 0.031348] CPU topo: Num. threads per package: 1481worker1 # [ 0.031612] CPU topo: Max. logical dies: 1482worker2 # [ 0.031311] ACPI: PM-Timer IO Port: 0x608483lb # [ 0.031709] CPU topo: Max. dies per package: 1484worker1 # [ 0.031612] CPU topo: Max. dies per package: 1485client # [ 0.031348] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs486lb # [ 0.031713] CPU topo: Max. threads per core: 1487worker2 # [ 0.031322] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])488worker1 # [ 0.031616] CPU topo: Max. threads per core: 1489client # [ 0.031365] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()490lb # [ 0.031713] CPU topo: Num. cores per package: 1491worker1 # [ 0.031616] CPU topo: Num. cores per package: 1492worker2 # [ 0.031350] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23493lb # [ 0.031714] CPU topo: Num. threads per package: 1494client # [ 0.031396] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]495worker1 # [ 0.031617] CPU topo: Num. threads per package: 1496worker2 # [ 0.031353] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)497lb # [ 0.031714] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs498worker1 # [ 0.031617] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs499client # [ 0.031397] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]500worker2 # [ 0.031354] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)501lb # [ 0.031732] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()502worker1 # [ 0.031634] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()503client # [ 0.031399] [mem 0x60000000-0xafffffff] available for PCI devices504worker2 # [ 0.031355] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)505client # [ 0.031400] Booting paravirtualized kernel on KVM506lb # [ 0.031763] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]507worker1 # [ 0.031665] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]508worker2 # [ 0.031356] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)509lb # [ 0.031765] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]510client # [ 0.031403] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns511worker1 # [ 0.031667] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]512worker2 # [ 0.031357] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)513lb # [ 0.031766] [mem 0x60000000-0xafffffff] available for PCI devices514worker1 # [ 0.031668] [mem 0x60000000-0xafffffff] available for PCI devices515client # [ 0.035866] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1516worker2 # [ 0.031360] ACPI: Using ACPI (MADT) for SMP configuration information517lb # [ 0.031768] Booting paravirtualized kernel on KVM518worker1 # [ 0.031669] Booting paravirtualized kernel on KVM519worker2 # [ 0.031361] ACPI: HPET id: 0x8086a201 base: 0xfed00000520client # [ 0.037949] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152521worker2 # [ 0.031364] TSC deadline timer available522client # [ 0.037996] kvm-guest: PV spinlocks disabled, single CPU523lb # [ 0.031770] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns524worker1 # [ 0.031672] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns525worker2 # [ 0.031368] CPU topo: Max. logical packages: 1526worker2 # [ 0.031369] CPU topo: Max. logical dies: 1527lb # [ 0.036244] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1528worker1 # [ 0.036147] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1529worker2 # [ 0.031369] CPU topo: Max. dies per package: 1530lb # [ 0.038333] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152531worker1 # [ 0.038229] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152532worker2 # [ 0.031372] CPU topo: Max. threads per core: 1533lb # [ 0.038381] kvm-guest: PV spinlocks disabled, single CPU534worker1 # [ 0.038281] kvm-guest: PV spinlocks disabled, single CPU535worker2 # [ 0.031373] CPU topo: Num. cores per package: 1536worker2 # [ 0.031373] CPU topo: Num. threads per package: 1537client # [ 0.037997] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qgir8rpawilvsh39a93frcz2w73apwjr-nixos-system-client-test/init regInfo=/nix/store/lkz8fx5m8vmi31dy1j6w4dxp7rzq8kxy-closure-info/registration console=ttyS0,115200n8 console=tty0538worker2 # [ 0.031374] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs539worker2 # [ 0.031393] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()540client # [ 0.038099] Unknown kernel command line parameters "regInfo=/nix/store/lkz8fx5m8vmi31dy1j6w4dxp7rzq8kxy-closure-info/registration", will be passed to user space.541worker2 # [ 0.031426] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]542client # [ 0.038112] random: crng init done543lb # [ 0.038382] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n0idzb2a5795zcdcs33p9240n19lall0-nixos-system-lb-test/init regInfo=/nix/store/2bxws5jbs8f1q8hlcfqpb1bb7xljw1g0-closure-info/registration console=ttyS0,115200n8 console=tty0544worker1 # [ 0.038282] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgp1w89psmkzqxph18mrrd4l45zq8icp-nixos-system-worker1-test/init regInfo=/nix/store/4vn7n0cqh1fw60k9fdivbpaggw9ifflq-closure-info/registration console=ttyS0,115200n8 console=tty0545client # [ 0.038113] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes546worker2 # [ 0.031427] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]547worker2 # [ 0.031429] [mem 0x60000000-0xafffffff] available for PCI devices548client # [ 0.040164] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)549lb # [ 0.038474] Unknown kernel command line parameters "regInfo=/nix/store/2bxws5jbs8f1q8hlcfqpb1bb7xljw1g0-closure-info/registration", will be passed to user space.550worker2 # [ 0.031430] Booting paravirtualized kernel on KVM551worker1 # [ 0.038375] Unknown kernel command line parameters "regInfo=/nix/store/4vn7n0cqh1fw60k9fdivbpaggw9ifflq-closure-info/registration", will be passed to user space.552lb # [ 0.038486] random: crng init done553worker1 # [ 0.038387] random: crng init done554client # [ 0.041337] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)555lb # [ 0.038487] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes556client # [ 0.041368] Fallback order for Node 0: 0557worker1 # [ 0.038388] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes558worker2 # [ 0.031433] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns559client # [ 0.041371] Built 1 zonelists, mobility grouping on. Total pages: 393078560lb # [ 0.040614] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)561worker2 # [ 0.035916] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1562client # [ 0.041371] Policy zone: DMA32563worker1 # [ 0.040453] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)564lb # [ 0.041801] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)565client # [ 0.043776] mem auto-init: stack:all(zero), heap alloc:on, heap free:off566worker2 # [ 0.038171] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152567worker1 # [ 0.041498] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)568lb # [ 0.041830] Fallback order for Node 0: 0569worker2 # [ 0.038215] kvm-guest: PV spinlocks disabled, single CPU570worker1 # [ 0.041528] Fallback order for Node 0: 0571client # [ 0.046749] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1572lb # [ 0.041832] Built 1 zonelists, mobility grouping on. Total pages: 393078573client # [ 0.049947] allocated 3145728 bytes of page_ext574worker1 # [ 0.041531] Built 1 zonelists, mobility grouping on. Total pages: 393078575lb # [ 0.041833] Policy zone: DMA32576worker1 # [ 0.041532] Policy zone: DMA32577client # [ 0.059788] ftrace: allocating 48787 entries in 192 pages578lb # [ 0.044268] mem auto-init: stack:all(zero), heap alloc:on, heap free:off579client # [ 0.059789] ftrace: allocated 192 pages with 2 groups580worker1 # [ 0.044081] mem auto-init: stack:all(zero), heap alloc:on, heap free:off581client # [ 0.060596] Dynamic Preempt: lazy582lb # [ 0.047305] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1583worker1 # [ 0.047092] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1584client # [ 0.060759] rcu: Preemptible hierarchical RCU implementation.585lb # [ 0.050535] allocated 3145728 bytes of page_ext586worker1 # [ 0.050299] allocated 3145728 bytes of page_ext587client # [ 0.060759] rcu: RCU event tracing is enabled.588worker2 # [ 0.038217] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kknbsz59pyihvz42vvc4xnin6z1984g8-nixos-system-worker2-test/init regInfo=/nix/store/d30hd8qj8b373nfwj9wkrwf1qqzjbbdk-closure-info/registration console=ttyS0,115200n8 console=tty0589lb # [ 0.060370] ftrace: allocating 48787 entries in 192 pages590worker1 # [ 0.060241] ftrace: allocating 48787 entries in 192 pages591lb # [ 0.060372] ftrace: allocated 192 pages with 2 groups592client # [ 0.060760] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.593worker1 # [ 0.060243] ftrace: allocated 192 pages with 2 groups594lb # [ 0.061217] Dynamic Preempt: lazy595client # [ 0.060761] Trampoline variant of Tasks RCU enabled.596worker1 # [ 0.061056] Dynamic Preempt: lazy597worker2 # [ 0.038310] Unknown kernel command line parameters "regInfo=/nix/store/d30hd8qj8b373nfwj9wkrwf1qqzjbbdk-closure-info/registration", will be passed to user space.598client # [ 0.060762] Rude variant of Tasks RCU enabled.599worker1 # [ 0.061215] rcu: Preemptible hierarchical RCU implementation.600lb # [ 0.061368] rcu: Preemptible hierarchical RCU implementation.601worker2 # [ 0.038327] random: crng init done602client # [ 0.060762] Tracing variant of Tasks RCU enabled.603lb # [ 0.061369] rcu: RCU event tracing is enabled.604worker1 # [ 0.061216] rcu: RCU event tracing is enabled.605worker2 # [ 0.038327] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes606lb # [ 0.061370] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.607client # [ 0.060763] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.608worker1 # [ 0.061217] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.609lb # [ 0.061371] Trampoline variant of Tasks RCU enabled.610worker1 # [ 0.061218] Trampoline variant of Tasks RCU enabled.611worker2 # [ 0.040391] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)612client # [ 0.060763] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1613lb # [ 0.061372] Rude variant of Tasks RCU enabled.614worker1 # [ 0.061219] Rude variant of Tasks RCU enabled.615lb # [ 0.061372] Tracing variant of Tasks RCU enabled.616worker1 # [ 0.061219] Tracing variant of Tasks RCU enabled.617worker2 # [ 0.041413] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)618client # [ 0.060783] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.619worker2 # [ 0.041443] Fallback order for Node 0: 0620lb # [ 0.061373] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.621worker1 # [ 0.061219] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.622client # [ 0.060784] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.623worker2 # [ 0.041447] Built 1 zonelists, mobility grouping on. Total pages: 393078624lb # [ 0.061373] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1625worker2 # [ 0.041448] Policy zone: DMA32626worker1 # [ 0.061220] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1627client # [ 0.060785] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.628worker2 # [ 0.043835] mem auto-init: stack:all(zero), heap alloc:on, heap free:off629lb # [ 0.061389] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.630worker1 # [ 0.061241] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.631client # [ 0.065109] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16632worker2 # [ 0.046773] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1633lb # [ 0.061390] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.634client # [ 0.065550] rcu: srcu_init: Setting srcu_struct sizes based on contention.635worker2 # [ 0.050111] allocated 3145728 bytes of page_ext636worker1 # [ 0.061242] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.637worker2 # [ 0.059754] ftrace: allocating 48787 entries in 192 pages638lb # [ 0.061391] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.639worker1 # [ 0.061243] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.640client # [ 0.065556] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns641worker2 # [ 0.059756] ftrace: allocated 192 pages with 2 groups642lb # [ 0.065722] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16643worker2 # [ 0.060563] Dynamic Preempt: lazy644worker1 # [ 0.065627] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16645worker2 # [ 0.060683] rcu: Preemptible hierarchical RCU implementation.646client # [ 0.065657] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)647lb # [ 0.066208] rcu: srcu_init: Setting srcu_struct sizes based on contention.648worker1 # [ 0.065901] rcu: srcu_init: Setting srcu_struct sizes based on contention.649client # [ 0.069222] Console: colour VGA+ 80x25650worker2 # [ 0.060684] rcu: RCU event tracing is enabled.651client # [ 0.069225] printk: legacy console [tty0] enabled652lb # [ 0.066215] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns653worker2 # [ 0.060685] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.654client # [ 0.098659] printk: legacy console [ttyS0] enabled655worker1 # [ 0.065908] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns656worker2 # [ 0.060686] Trampoline variant of Tasks RCU enabled.657client # [ 0.193319] ACPI: Core revision 20250807658lb # [ 0.066314] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)659worker2 # [ 0.060686] Rude variant of Tasks RCU enabled.660worker1 # [ 0.066010] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)661lb # [ 0.069838] Console: colour VGA+ 80x25662worker1 # [ 0.069531] Console: colour VGA+ 80x25663worker2 # [ 0.060687] Tracing variant of Tasks RCU enabled.664lb # [ 0.069841] printk: legacy console [tty0] enabled665worker1 # [ 0.069533] printk: legacy console [tty0] enabled666client # [ 0.194204] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns667worker2 # [ 0.060687] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.668lb # [ 0.099400] printk: legacy console [ttyS0] enabled669worker1 # [ 0.098955] printk: legacy console [ttyS0] enabled670client # [ 0.195695] APIC: Switch to symmetric I/O mode setup671worker2 # [ 0.060688] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1672worker1 # [ 0.194168] ACPI: Core revision 20250807673lb # [ 0.194785] ACPI: Core revision 20250807674client # [ 0.196624] x2apic enabled675worker2 # [ 0.060743] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.676lb # [ 0.195653] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns677worker1 # [ 0.195037] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns678client # [ 0.197367] APIC: Switched APIC routing to: physical x2apic679worker2 # [ 0.060745] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.680lb # [ 0.197160] APIC: Switch to symmetric I/O mode setup681worker1 # [ 0.196524] APIC: Switch to symmetric I/O mode setup682worker2 # [ 0.060745] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.683lb # [ 0.198154] x2apic enabled684worker1 # [ 0.197469] x2apic enabled685client # [ 0.199175] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1686worker2 # [ 0.065045] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16687worker1 # [ 0.198198] APIC: Switched APIC routing to: physical x2apic688lb # [ 0.198926] APIC: Switched APIC routing to: physical x2apic689worker2 # [ 0.065314] rcu: srcu_init: Setting srcu_struct sizes based on contention.690client # [ 0.200125] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns691worker2 # [ 0.065321] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns692worker1 # [ 0.200020] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1693lb # [ 0.200745] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1694client # [ 0.201718] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)695worker2 # [ 0.065418] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)696worker2 # [ 0.068920] Console: colour VGA+ 80x25697client # [ 0.202813] x86/cpu: User Mode Instruction Prevention (UMIP) activated698worker1 # [ 0.200954] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns699lb # [ 0.201708] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns700worker2 # [ 0.068923] printk: legacy console [tty0] enabled701client # [ 0.203859] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127702worker2 # [ 0.098340] printk: legacy console [ttyS0] enabled703worker1 # [ 0.202563] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)704lb # [ 0.203351] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)705worker2 # [ 0.195157] ACPI: Core revision 20250807706client # [ 0.204715] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0707worker1 # [ 0.203648] x86/cpu: User Mode Instruction Prevention (UMIP) activated708lb # [ 0.204436] x86/cpu: User Mode Instruction Prevention (UMIP) activated709worker2 # [ 0.196066] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns710client # [ 0.205718] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto711worker1 # [ 0.204736] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127712lb # [ 0.205523] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127713worker2 # [ 0.197586] APIC: Switch to symmetric I/O mode setup714worker1 # [ 0.205561] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0715lb # [ 0.207014] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0716client # [ 0.206715] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl717worker2 # [ 0.198538] x2apic enabled718client # [ 0.207715] Transient Scheduler Attacks: Vulnerable: No microcode719worker2 # [ 0.199307] APIC: Switched APIC routing to: physical x2apic720worker1 # [ 0.206564] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto721lb # [ 0.207351] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto722client # [ 0.208715] Spectre V2 : Mitigation: Enhanced / Automatic IBRS723worker1 # [ 0.207560] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl724client # [ 0.209715] Speculative Return Stack Overflow: Mitigation: Safe RET725lb # [ 0.209348] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl726worker2 # [ 0.201097] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1727worker1 # [ 0.208560] Transient Scheduler Attacks: Vulnerable: No microcode728lb # [ 0.210348] Transient Scheduler Attacks: Vulnerable: No microcode729worker1 # [ 0.209560] Spectre V2 : Mitigation: Enhanced / Automatic IBRS730worker2 # [ 0.202057] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns731lb # [ 0.211347] Spectre V2 : Mitigation: Enhanced / Automatic IBRS732worker1 # [ 0.210560] Speculative Return Stack Overflow: Mitigation: Safe RET733lb # [ 0.212348] Speculative Return Stack Overflow: Mitigation: Safe RET734worker2 # [ 0.203705] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)735client # [ 0.210715] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization736worker2 # [ 0.204789] x86/cpu: User Mode Instruction Prevention (UMIP) activated737client # [ 0.212722] Spectre V2 : Enabling IBPB for BPF738worker2 # [ 0.205873] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127739client # [ 0.213440] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier740worker2 # [ 0.207422] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0741client # [ 0.214716] active return thunk: srso_alias_return_thunk742worker1 # [ 0.211560] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization743lb # [ 0.213347] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization744worker1 # [ 0.213568] Spectre V2 : Enabling IBPB for BPF745lb # [ 0.215355] Spectre V2 : Enabling IBPB for BPF746client # [ 0.215546] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'747worker2 # [ 0.207705] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto748worker1 # [ 0.214282] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier749client # [ 0.216715] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'750lb # [ 0.216348] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier751worker2 # [ 0.209702] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl752worker1 # [ 0.215561] active return thunk: srso_alias_return_thunk753client # [ 0.217715] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'754lb # [ 0.217349] active return thunk: srso_alias_return_thunk755worker2 # [ 0.210702] Transient Scheduler Attacks: Vulnerable: No microcode756worker1 # [ 0.216581] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'757client # [ 0.218641] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'758lb # [ 0.218369] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'759worker2 # [ 0.211702] Spectre V2 : Mitigation: Enhanced / Automatic IBRS760client # [ 0.219439] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'761worker1 # [ 0.217560] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'762lb # [ 0.219348] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'763worker2 # [ 0.212702] Speculative Return Stack Overflow: Mitigation: Safe RET764worker1 # [ 0.218560] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'765client # [ 0.220430] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'766lb # [ 0.220347] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'767worker1 # [ 0.219560] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'768client # [ 0.221462] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'769lb # [ 0.221347] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'770worker1 # [ 0.220560] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'771client # [ 0.222715] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256772lb # [ 0.222347] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'773worker1 # [ 0.221560] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'774client # [ 0.223624] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64775lb # [ 0.224095] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'776worker2 # [ 0.213702] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization777client # [ 0.224414] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512778worker1 # [ 0.222560] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'779worker2 # [ 0.215709] Spectre V2 : Enabling IBPB for BPF780lb # [ 0.225110] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'781client # [ 0.225411] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024782worker1 # [ 0.223560] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256783lb # [ 0.226348] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256784worker2 # [ 0.216703] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier785client # [ 0.225715] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8786worker1 # [ 0.224560] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64787lb # [ 0.227257] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64788worker2 # [ 0.217702] active return thunk: srso_alias_return_thunk789worker1 # [ 0.225560] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512790client # [ 0.226715] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.791lb # [ 0.228036] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512792worker2 # [ 0.218723] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'793worker1 # [ 0.226561] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024794lb # [ 0.229041] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024795worker2 # [ 0.219702] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'796worker1 # [ 0.227560] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8797lb # [ 0.230065] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8798worker2 # [ 0.220702] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'799worker1 # [ 0.228560] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.800worker2 # [ 0.221702] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'801lb # [ 0.231067] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.802worker2 # [ 0.222702] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'803worker2 # [ 0.223702] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'804worker2 # [ 0.224702] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'805worker2 # [ 0.225702] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256806worker2 # [ 0.227463] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807worker2 # [ 0.228458] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808worker2 # [ 0.229463] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809worker2 # [ 0.230460] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810worker2 # [ 0.231469] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811client # [ 0.260428] Freeing SMP alternatives memory: 44K812client # [ 0.260718] pid_max: default: 32768 minimum: 301813client # [ 0.261872] LSM: initializing lsm=capability,landlock,yama,bpf,ima814worker1 # [ 0.262135] Freeing SMP alternatives memory: 44K815client # [ 0.262821] landlock: Up and running.816client # [ 0.263716] Yama: becoming mindful.817worker1 # [ 0.262563] pid_max: default: 32768 minimum: 301818lb # [ 0.265380] Freeing SMP alternatives memory: 44K819client # [ 0.264499] LSM support for eBPF active820worker1 # [ 0.263876] LSM: initializing lsm=capability,landlock,yama,bpf,ima821lb # [ 0.266176] pid_max: default: 32768 minimum: 301822worker1 # [ 0.264657] landlock: Up and running.823client # [ 0.265263] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)824lb # [ 0.266971] LSM: initializing lsm=capability,landlock,yama,bpf,ima825worker1 # [ 0.265561] Yama: becoming mindful.826lb # [ 0.267448] landlock: Up and running.827client # [ 0.265756] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)828worker1 # [ 0.266569] LSM support for eBPF active829lb # [ 0.268349] Yama: becoming mindful.830lb # [ 0.269564] LSM support for eBPF active831worker1 # [ 0.267301] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)832client # [ 0.268627] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)833lb # [ 0.270422] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)834worker1 # [ 0.268495] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)835client # [ 0.269202] Performance Events: Fam17h+ core perfctr, AMD PMU driver.836lb # [ 0.271383] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)837client # [ 0.269719] ... version: 2838worker1 # [ 0.270747] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)839client # [ 0.270392] ... bit width: 48840client # [ 0.270717] ... generic counters: 6841worker2 # [ 0.265396] Freeing SMP alternatives memory: 44K842lb # [ 0.273900] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)843client # [ 0.271394] ... generic bitmap: 000000000000003f844worker1 # [ 0.272102] Performance Events: Fam17h+ core perfctr, AMD PMU driver.845worker2 # [ 0.265705] pid_max: default: 32768 minimum: 301846worker1 # [ 0.272570] ... version: 2847client # [ 0.271760] ... fixed-purpose counters: 0848lb # [ 0.274879] Performance Events: Fam17h+ core perfctr, AMD PMU driver.849worker1 # [ 0.273258] ... bit width: 48850worker2 # [ 0.266815] LSM: initializing lsm=capability,landlock,yama,bpf,ima851client # [ 0.272448] ... fixed-purpose bitmap: 0000000000000000852lb # [ 0.275352] ... version: 2853worker1 # [ 0.273562] ... generic counters: 6854worker2 # [ 0.267806] landlock: Up and running.855client # [ 0.272718] ... value mask: 0000ffffffffffff856lb # [ 0.276061] ... bit width: 48857worker2 # [ 0.268703] Yama: becoming mindful.858worker1 # [ 0.274259] ... generic bitmap: 000000000000003f859client # [ 0.273568] ... max period: 00007fffffffffff860lb # [ 0.276350] ... generic counters: 6861worker1 # [ 0.274563] ... fixed-purpose counters: 0862worker2 # [ 0.269496] LSM support for eBPF active863client # [ 0.274353] ... global_ctrl mask: 000000000000003f864lb # [ 0.277017] ... generic bitmap: 000000000000003f865worker1 # [ 0.275232] ... fixed-purpose bitmap: 0000000000000000866client # [ 0.274840] signal: max sigframe size: 3376867worker2 # [ 0.270266] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)868lb # [ 0.277362] ... fixed-purpose counters: 0869worker1 # [ 0.275563] ... value mask: 0000ffffffffffff870client # [ 0.275573] rcu: Hierarchical SRCU implementation.871lb # [ 0.278068] ... fixed-purpose bitmap: 0000000000000000872worker1 # [ 0.276426] ... max period: 00007fffffffffff873worker2 # [ 0.270738] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)874client # [ 0.276289] rcu: Max phase no-delay instances is 400.875lb # [ 0.278350] ... value mask: 0000ffffffffffff876worker1 # [ 0.277214] ... global_ctrl mask: 000000000000003f877lb # [ 0.279232] ... max period: 00007fffffffffff878worker1 # [ 0.277674] signal: max sigframe size: 3376879lb # [ 0.280071] ... global_ctrl mask: 000000000000003f880worker2 # [ 0.273666] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)881worker1 # [ 0.278464] rcu: Hierarchical SRCU implementation.882lb # [ 0.280463] signal: max sigframe size: 3376883worker1 # [ 0.279114] rcu: Max phase no-delay instances is 400.884worker2 # [ 0.274203] Performance Events: Fam17h+ core perfctr, AMD PMU driver.885lb # [ 0.281263] rcu: Hierarchical SRCU implementation.886worker2 # [ 0.274707] ... version: 2887lb # [ 0.281928] rcu: Max phase no-delay instances is 400.888worker2 # [ 0.275414] ... bit width: 48889client # [ 0.281177] smp: Bringing up secondary CPUs ...890client # [ 0.281731] smp: Brought up 1 node, 1 CPU891worker2 # [ 0.275705] ... generic counters: 6892worker2 # [ 0.276425] ... generic bitmap: 000000000000003f893client # [ 0.282387] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)894worker2 # [ 0.276752] ... fixed-purpose counters: 0895worker1 # [ 0.284075] smp: Bringing up secondary CPUs ...896worker2 # [ 0.277469] ... fixed-purpose bitmap: 0000000000000000897worker1 # [ 0.284576] smp: Brought up 1 node, 1 CPU898client # [ 0.282913] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)899worker2 # [ 0.277705] ... value mask: 0000ffffffffffff900lb # [ 0.286868] smp: Bringing up secondary CPUs ...901client # [ 0.283967] devtmpfs: initialized902worker1 # [ 0.285229] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)903worker2 # [ 0.278587] ... max period: 00007fffffffffff904lb # [ 0.287364] smp: Brought up 1 node, 1 CPU905client # [ 0.284821] x86/mm: Memory block size: 128MB906worker2 # [ 0.279362] ... global_ctrl mask: 000000000000003f907lb # [ 0.288060] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)908worker2 # [ 0.279815] signal: max sigframe size: 3376909worker1 # [ 0.285768] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)910worker2 # [ 0.280592] rcu: Hierarchical SRCU implementation.911client # [ 0.286509] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)912worker1 # [ 0.286820] devtmpfs: initialized913worker2 # [ 0.281287] rcu: Max phase no-delay instances is 400.914lb # [ 0.288547] Memory: 1454588K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)915worker1 # [ 0.287651] x86/mm: Memory block size: 128MB916client # [ 0.287592] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).917lb # [ 0.289611] devtmpfs: initialized918client # [ 0.288794] pinctrl core: initialized pinctrl subsystem919lb # [ 0.290466] x86/mm: Memory block size: 128MB920client # [ 0.289868] PM: RTC time: 05:32:52, date: 2026-09-23921worker1 # [ 0.289371] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)922worker1 # [ 0.290461] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).923lb # [ 0.292202] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)924worker1 # [ 0.291649] pinctrl core: initialized pinctrl subsystem925worker2 # [ 0.286213] smp: Bringing up secondary CPUs ...926lb # [ 0.293302] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).927worker2 # [ 0.286717] smp: Brought up 1 node, 1 CPU928worker1 # [ 0.292711] PM: RTC time: 05:32:52, date: 2026-09-23929client # [ 0.293325] NET: Registered PF_NETLINK/PF_ROUTE protocol family930lb # [ 0.294431] pinctrl core: initialized pinctrl subsystem931worker2 # [ 0.287371] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)932lb # [ 0.295530] PM: RTC time: 05:32:52, date: 2026-09-23933client # [ 0.294201] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations934client # [ 0.294738] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations935worker2 # [ 0.287828] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)936worker1 # [ 0.296078] NET: Registered PF_NETLINK/PF_ROUTE protocol family937worker2 # [ 0.288958] devtmpfs: initialized938client # [ 0.295986] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations939worker2 # [ 0.289820] x86/mm: Memory block size: 128MB940worker1 # [ 0.297027] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations941lb # [ 0.299019] NET: Registered PF_NETLINK/PF_ROUTE protocol family942client # [ 0.296727] audit: initializing netlink subsys (disabled)943worker1 # [ 0.297583] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations944client # [ 0.297719] audit: type=2000 audit(1790141573.148:1): state=initialized audit_enabled=0 res=1945lb # [ 0.299846] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations946worker2 # [ 0.291554] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)947client # [ 0.298840] thermal_sys: Registered thermal governor 'fair_share'948worker1 # [ 0.298831] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations949lb # [ 0.300378] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations950worker2 # [ 0.292614] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).951worker1 # [ 0.299573] audit: initializing netlink subsys (disabled)952client # [ 0.298842] thermal_sys: Registered thermal governor 'bang_bang'953worker2 # [ 0.293793] pinctrl core: initialized pinctrl subsystem954client # [ 0.299718] thermal_sys: Registered thermal governor 'step_wise'955lb # [ 0.301625] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations956worker1 # [ 0.300715] thermal_sys: Registered thermal governor 'fair_share'957worker2 # [ 0.294878] PM: RTC time: 05:32:52, date: 2026-09-23958lb # [ 0.302360] audit: initializing netlink subsys (disabled)959client # [ 0.300618] thermal_sys: Registered thermal governor 'user_space'960worker1 # [ 0.300717] thermal_sys: Registered thermal governor 'bang_bang'961client # [ 0.301395] thermal_sys: Registered thermal governor 'power_allocator'962lb # [ 0.303467] thermal_sys: Registered thermal governor 'fair_share'963worker1 # [ 0.301564] thermal_sys: Registered thermal governor 'step_wise'964client # [ 0.301730] cpuidle: using governor menu965lb # [ 0.303469] thermal_sys: Registered thermal governor 'bang_bang'966worker1 # [ 0.302478] thermal_sys: Registered thermal governor 'user_space'967lb # [ 0.304351] thermal_sys: Registered thermal governor 'step_wise'968client # [ 0.303735] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5969worker1 # [ 0.303283] audit: type=2000 audit(1790141573.152:1): state=initialized audit_enabled=0 res=1970worker2 # [ 0.298492] NET: Registered PF_NETLINK/PF_ROUTE protocol family971lb # [ 0.305326] thermal_sys: Registered thermal governor 'user_space'972worker1 # [ 0.304565] thermal_sys: Registered thermal governor 'power_allocator'973client # [ 0.304943] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]974worker1 # [ 0.304579] cpuidle: using governor menu975worker2 # [ 0.299926] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations976lb # [ 0.306096] audit: type=2000 audit(1790141573.153:1): state=initialized audit_enabled=0 res=1977client # [ 0.305720] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry978lb # [ 0.307353] thermal_sys: Registered thermal governor 'power_allocator'979worker2 # [ 0.300723] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations980client # [ 0.306729] PCI: Using configuration type 1 for base access981worker1 # [ 0.306756] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5982lb # [ 0.307366] cpuidle: using governor menu983worker2 # [ 0.301974] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations984client # [ 0.307796] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.985worker1 # [ 0.307806] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]986lb # [ 0.309599] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5987worker2 # [ 0.302715] audit: initializing netlink subsys (disabled)988worker1 # [ 0.308566] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry989worker1 # [ 0.309575] PCI: Using configuration type 1 for base access990worker2 # [ 0.303706] audit: type=2000 audit(1790141573.161:1): state=initialized audit_enabled=0 res=1991lb # [ 0.310601] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]992lb # [ 0.311353] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry993worker2 # [ 0.304878] thermal_sys: Registered thermal governor 'fair_share'994worker1 # [ 0.310635] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.995lb # [ 0.312361] PCI: Using configuration type 1 for base access996worker2 # [ 0.304880] thermal_sys: Registered thermal governor 'bang_bang'997worker2 # [ 0.305705] thermal_sys: Registered thermal governor 'step_wise'998lb # [ 0.313487] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.999worker2 # [ 0.306663] thermal_sys: Registered thermal governor 'user_space'1000client # [ 0.314732] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1001worker2 # [ 0.307463] thermal_sys: Registered thermal governor 'power_allocator'1002worker2 # [ 0.307717] cpuidle: using governor menu1003client # [ 0.315717] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1004worker1 # [ 0.315783] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1005client # [ 0.316703] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1006worker2 # [ 0.309935] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51007worker1 # [ 0.316564] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1008worker2 # [ 0.310943] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1009worker2 # [ 0.311707] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1010worker2 # [ 0.312716] PCI: Using configuration type 1 for base access1011lb # [ 0.320612] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1012client # [ 0.321718] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1013lb # [ 0.321351] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1014worker2 # [ 0.313809] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1015worker1 # [ 0.321564] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1016worker1 # [ 0.322562] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1017lb # [ 0.326352] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1018lb # [ 0.327351] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1019worker2 # [ 0.320719] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1020client # [ 0.330046] ACPI: Added _OSI(Module Device)1021worker2 # [ 0.321705] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1022client # [ 0.330720] ACPI: Added _OSI(Processor Device)1023client # [ 0.331422] ACPI: Added _OSI(Processor Aggregator Device)1024worker1 # [ 0.332914] ACPI: Added _OSI(Module Device)1025worker1 # [ 0.333565] ACPI: Added _OSI(Processor Device)1026worker1 # [ 0.334330] ACPI: Added _OSI(Processor Aggregator Device)1027worker2 # [ 0.326707] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1028lb # [ 0.335948] ACPI: Added _OSI(Module Device)1029worker2 # [ 0.327705] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1030lb # [ 0.336352] ACPI: Added _OSI(Processor Device)1031lb # [ 0.337105] ACPI: Added _OSI(Processor Aggregator Device)1032client # [ 0.339247] ACPI: 1 ACPI AML tables successfully acquired and loaded1033client # [ 0.343130] ACPI: Interpreter enabled1034client # [ 0.343734] ACPI: PM: (supports S0 S3 S4 S5)1035worker1 # [ 0.343103] ACPI: 1 ACPI AML tables successfully acquired and loaded1036client # [ 0.344431] ACPI: Using IOAPIC for interrupt routing1037lb # [ 0.343860] ACPI: 1 ACPI AML tables successfully acquired and loaded1038worker1 # [ 0.344452] ACPI: Interpreter enabled1039worker1 # [ 0.345069] ACPI: PM: (supports S0 S3 S4 S5)1040worker2 # [ 0.338946] ACPI: Added _OSI(Module Device)1041worker2 # [ 0.339665] ACPI: Added _OSI(Processor Device)1042client # [ 0.346757] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1043lb # [ 0.347253] ACPI: Interpreter enabled1044worker2 # [ 0.340250] ACPI: Added _OSI(Processor Aggregator Device)1045client # [ 0.347718] PCI: Using E820 reservations for host bridge windows1046lb # [ 0.347812] ACPI: PM: (supports S0 S3 S4 S5)1047worker1 # [ 0.347563] ACPI: Using IOAPIC for interrupt routing1048lb # [ 0.348351] ACPI: Using IOAPIC for interrupt routing1049worker1 # [ 0.348448] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1050lb # [ 0.349258] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1051client # [ 0.350800] ACPI: Enabled 2 GPEs in block 00 to 3F1052worker1 # [ 0.351563] PCI: Using E820 reservations for host bridge windows1053worker1 # [ 0.352683] ACPI: Enabled 2 GPEs in block 00 to 3F1054worker2 # [ 0.346069] ACPI: 1 ACPI AML tables successfully acquired and loaded1055lb # [ 0.354351] PCI: Using E820 reservations for host bridge windows1056lb # [ 0.355490] ACPI: Enabled 2 GPEs in block 00 to 3F1057worker2 # [ 0.350118] ACPI: Interpreter enabled1058worker2 # [ 0.350721] ACPI: PM: (supports S0 S3 S4 S5)1059client # [ 0.358474] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1060worker2 # [ 0.351431] ACPI: Using IOAPIC for interrupt routing1061client # [ 0.358724] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1062client # [ 0.359797] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1063worker1 # [ 0.361294] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1064client # [ 0.360839] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1065worker2 # [ 0.353780] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1066client # [ 0.362076] PCI host bridge to bus 0000:001067worker2 # [ 0.354705] PCI: Using E820 reservations for host bridge windows1068worker1 # [ 0.362130] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1069client # [ 0.362722] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1070lb # [ 0.364249] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1071client # [ 0.363718] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1072worker1 # [ 0.362642] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1073worker2 # [ 0.357801] ACPI: Enabled 2 GPEs in block 00 to 3F1074client # [ 0.364705] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1075worker1 # [ 0.363685] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1076lb # [ 0.365465] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1077client # [ 0.365588] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1078worker1 # [ 0.364944] PCI host bridge to bus 0000:001079lb # [ 0.366431] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1080worker1 # [ 0.365567] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1081client # [ 0.366587] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1082lb # [ 0.367474] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1083worker1 # [ 0.366563] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1084client # [ 0.367587] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1085lb # [ 0.368733] PCI host bridge to bus 0000:001086client # [ 0.368598] pci_bus 0000:00: root bus resource [bus 00-ff]1087worker1 # [ 0.367563] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1088lb # [ 0.369355] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1089worker1 # [ 0.368563] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1090client # [ 0.369453] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1091lb # [ 0.370351] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1092worker1 # [ 0.369563] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1093worker2 # [ 0.364947] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1094lb # [ 0.371351] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1095client # [ 0.371112] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1096worker1 # [ 0.370563] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1097lb # [ 0.372350] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1098worker2 # [ 0.365711] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1099worker1 # [ 0.371563] pci_bus 0000:00: root bus resource [bus 00-ff]1100lb # [ 0.373350] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1101worker2 # [ 0.366793] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1102worker1 # [ 0.372544] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1103lb # [ 0.374350] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1104worker2 # [ 0.367827] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1105lb # [ 0.375351] pci_bus 0000:00: root bus resource [bus 00-ff]1106worker1 # [ 0.374010] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1107worker2 # [ 0.369063] PCI host bridge to bus 0000:001108worker2 # [ 0.369709] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1109lb # [ 0.376390] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1110worker2 # [ 0.370705] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1111client # [ 0.373887] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1112lb # [ 0.377792] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1113client # [ 0.374731] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1114worker2 # [ 0.371705] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1115client # [ 0.375657] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1116worker2 # [ 0.372704] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1117worker2 # [ 0.373705] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1118client # [ 0.376821] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1119worker2 # [ 0.374705] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1120worker1 # [ 0.376623] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1121worker2 # [ 0.375705] pci_bus 0000:00: root bus resource [bus 00-ff]1122worker1 # [ 0.377575] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1123client # [ 0.378444] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1124worker1 # [ 0.378536] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1125worker2 # [ 0.376667] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1126lb # [ 0.380436] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1127client # [ 0.380737] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1128worker2 # [ 0.378146] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1129worker1 # [ 0.379626] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1130lb # [ 0.381363] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1131client # [ 0.381569] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1132lb # [ 0.382330] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1133client # [ 0.382443] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1134worker1 # [ 0.381303] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1135client # [ 0.382724] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1136lb # [ 0.383451] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1137client # [ 0.384223] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1138worker1 # [ 0.384581] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1139lb # [ 0.385267] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1140worker1 # [ 0.385445] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1141worker1 # [ 0.386342] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1142worker2 # [ 0.380769] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1143worker1 # [ 0.387442] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1144client # [ 0.386735] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1145worker2 # [ 0.381718] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1146lb # [ 0.388393] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1147client # [ 0.387544] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1148worker2 # [ 0.382668] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1149lb # [ 0.389215] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1150client # [ 0.388423] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1151lb # [ 0.390044] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1152worker1 # [ 0.388858] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1153worker2 # [ 0.383608] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1154lb # [ 0.390357] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1155client # [ 0.389279] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1156worker1 # [ 0.391509] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1157worker2 # [ 0.385651] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1158lb # [ 0.391917] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1159worker1 # [ 0.392204] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1160client # [ 0.391742] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1161worker1 # [ 0.392584] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1162client # [ 0.392561] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1163worker2 # [ 0.388724] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1164lb # [ 0.393390] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1165worker1 # [ 0.394138] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1166lb # [ 0.394206] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1167worker2 # [ 0.389580] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1168client # [ 0.393443] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1169worker2 # [ 0.390438] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1170lb # [ 0.395048] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1171client # [ 0.394270] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1172worker2 # [ 0.390711] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1173lb # [ 0.395936] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1174client # [ 0.395726] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1175worker1 # [ 0.396582] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1176worker2 # [ 0.392272] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1177client # [ 0.396542] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1178worker1 # [ 0.397570] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1179client # [ 0.397440] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1180worker1 # [ 0.398552] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1181client # [ 0.397723] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1182worker2 # [ 0.394603] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1183lb # [ 0.398393] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1184worker2 # [ 0.395335] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1185worker1 # [ 0.399982] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1186lb # [ 0.400003] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1187client # [ 0.399221] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1188worker2 # [ 0.395726] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1189lb # [ 0.400374] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1190worker1 # [ 0.402434] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1191worker2 # [ 0.397259] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1192lb # [ 0.401900] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1193client # [ 0.400742] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1194worker1 # [ 0.403174] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1195client # [ 0.401676] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1196worker1 # [ 0.403584] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1197lb # [ 0.403357] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1198worker1 # [ 0.404569] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1199lb # [ 0.404189] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1200client # [ 0.403097] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1201lb # [ 0.405119] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1202worker2 # [ 0.399722] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1203client # [ 0.404724] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1204worker2 # [ 0.400712] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1205worker1 # [ 0.406123] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1206lb # [ 0.406246] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1207client # [ 0.405640] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1208worker2 # [ 0.401640] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1209worker1 # [ 0.407586] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1210lb # [ 0.407925] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1211client # [ 0.407098] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1212worker2 # [ 0.403141] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1213worker1 # [ 0.408510] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1214lb # [ 0.409363] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1215worker1 # [ 0.409942] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1216worker2 # [ 0.405441] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1217lb # [ 0.410372] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1218client # [ 0.409074] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1219worker2 # [ 0.405712] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1220client # [ 0.410438] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1221worker1 # [ 0.411588] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1222worker2 # [ 0.406656] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1223lb # [ 0.411945] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1224worker2 # [ 0.407508] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1225client # [ 0.411259] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1226worker1 # [ 0.412531] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1227lb # [ 0.413363] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1228client # [ 0.412740] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1229worker2 # [ 0.409681] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1230worker1 # [ 0.414184] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1231lb # [ 0.414372] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1232client # [ 0.413664] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1233worker1 # [ 0.415589] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1234worker2 # [ 0.411717] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1235lb # [ 0.415933] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1236client # [ 0.415062] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1237worker1 # [ 0.416532] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1238worker2 # [ 0.412652] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1239lb # [ 0.417363] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1240worker1 # [ 0.417961] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1241worker2 # [ 0.414113] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1242lb # [ 0.418372] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1243client # [ 0.417411] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1244client # [ 0.418353] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1245worker1 # [ 0.419587] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1246lb # [ 0.419893] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1247worker2 # [ 0.415751] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1248client # [ 0.418743] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1249worker1 # [ 0.420514] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1250worker2 # [ 0.416727] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1251lb # [ 0.421363] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1252client # [ 0.420268] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1253worker1 # [ 0.422108] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1254lb # [ 0.422320] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1255worker2 # [ 0.418293] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1256client # [ 0.421726] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1257client # [ 0.422606] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1258lb # [ 0.423894] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1259worker2 # [ 0.419753] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1260client # [ 0.423444] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1261worker1 # [ 0.424452] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1262worker1 # [ 0.425173] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1263worker2 # [ 0.420727] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1264lb # [ 0.425370] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1265worker1 # [ 0.425584] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1266client # [ 0.424287] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1267lb # [ 0.426197] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1268worker2 # [ 0.422273] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1269lb # [ 0.427051] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1270worker1 # [ 0.427117] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1271client # [ 0.425432] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1272worker2 # [ 0.423717] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1273client # [ 0.425913] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1274lb # [ 0.427935] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1275client # [ 0.427360] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1276worker2 # [ 0.424715] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1277worker1 # [ 0.429476] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1278client # [ 0.427915] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1279worker1 # [ 0.430210] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1280worker2 # [ 0.427156] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1281lb # [ 0.430351] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1282worker1 # [ 0.430585] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1283client # [ 0.429379] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1284lb # [ 0.431267] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1285lb # [ 0.432128] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1286worker1 # [ 0.432129] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1287worker2 # [ 0.429519] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1288worker2 # [ 0.430378] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1289lb # [ 0.433848] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1290worker1 # [ 0.433376] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1291worker2 # [ 0.430726] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1292worker1 # [ 0.434431] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1293lb # [ 0.435101] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1294worker2 # [ 0.432284] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1295worker1 # [ 0.436282] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1296lb # [ 0.435546] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1297lb # [ 0.437004] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1298worker1 # [ 0.436753] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1299worker2 # [ 0.434758] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1300lb # [ 0.437552] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1301worker1 # [ 0.438355] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1302worker2 # [ 0.435643] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1303worker2 # [ 0.436458] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1304lb # [ 0.440041] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1305client # [ 0.429937] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1306worker2 # [ 0.437258] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1307worker2 # [ 0.438453] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1308client # [ 0.432753] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1309worker2 # [ 0.438955] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1310client # [ 0.433970] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1311worker2 # [ 0.440433] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1312client # [ 0.435017] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1313worker2 # [ 0.440898] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1314client # [ 0.435969] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1315worker2 # [ 0.442551] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1316worker1 # [ 0.439395] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1317client # [ 0.437750] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1318lb # [ 0.440570] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1319worker1 # [ 0.441280] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1320client # [ 0.438587] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1321lb # [ 0.443005] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1322worker1 # [ 0.441849] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1323client # [ 0.439733] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1324lb # [ 0.443633] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1325worker1 # [ 0.442851] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1326client # [ 0.441417] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1327lb # [ 0.444673] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1328worker1 # [ 0.443824] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1329lb # [ 0.445634] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1330worker1 # [ 0.445591] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1331client # [ 0.444772] ACPI: PCI: Interrupt link LNKA configured for IRQ 101332worker1 # [ 0.446448] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1333client # [ 0.445807] ACPI: PCI: Interrupt link LNKB configured for IRQ 101334lb # [ 0.447378] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1335worker1 # [ 0.447629] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1336lb # [ 0.448267] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1337client # [ 0.446813] ACPI: PCI: Interrupt link LNKC configured for IRQ 111338client # [ 0.447823] ACPI: PCI: Interrupt link LNKD configured for IRQ 111339worker1 # [ 0.449225] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1340worker2 # [ 0.443635] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1341lb # [ 0.449440] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1342client # [ 0.448828] ACPI: PCI: Interrupt link LNKE configured for IRQ 101343worker2 # [ 0.445436] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1344lb # [ 0.451013] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1345client # [ 0.449817] ACPI: PCI: Interrupt link LNKF configured for IRQ 101346client # [ 0.450818] ACPI: PCI: Interrupt link LNKG configured for IRQ 111347worker2 # [ 0.445992] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1348client # [ 0.451820] ACPI: PCI: Interrupt link LNKH configured for IRQ 111349worker1 # [ 0.453137] ACPI: PCI: Interrupt link LNKA configured for IRQ 101350client # [ 0.452755] ACPI: PCI: Interrupt link GSIA configured for IRQ 161351worker2 # [ 0.447209] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1352client # [ 0.453687] ACPI: PCI: Interrupt link GSIB configured for IRQ 171353worker1 # [ 0.453673] ACPI: PCI: Interrupt link LNKB configured for IRQ 101354lb # [ 0.455100] ACPI: PCI: Interrupt link LNKA configured for IRQ 101355client # [ 0.454434] ACPI: PCI: Interrupt link GSIC configured for IRQ 181356worker2 # [ 0.447950] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1357worker1 # [ 0.454660] ACPI: PCI: Interrupt link LNKC configured for IRQ 111358lb # [ 0.455458] ACPI: PCI: Interrupt link LNKB configured for IRQ 101359client # [ 0.454729] ACPI: PCI: Interrupt link GSID configured for IRQ 191360worker1 # [ 0.455663] ACPI: PCI: Interrupt link LNKD configured for IRQ 111361client # [ 0.455660] ACPI: PCI: Interrupt link GSIE configured for IRQ 201362lb # [ 0.456450] ACPI: PCI: Interrupt link LNKC configured for IRQ 111363worker2 # [ 0.449776] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1364worker1 # [ 0.456666] ACPI: PCI: Interrupt link LNKE configured for IRQ 101365client # [ 0.456428] ACPI: PCI: Interrupt link GSIF configured for IRQ 211366lb # [ 0.457446] ACPI: PCI: Interrupt link LNKD configured for IRQ 111367worker2 # [ 0.450641] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1368worker1 # [ 0.457667] ACPI: PCI: Interrupt link LNKF configured for IRQ 101369client # [ 0.456748] ACPI: PCI: Interrupt link GSIG configured for IRQ 221370lb # [ 0.458446] ACPI: PCI: Interrupt link LNKE configured for IRQ 101371worker1 # [ 0.458670] ACPI: PCI: Interrupt link LNKG configured for IRQ 111372worker2 # [ 0.451759] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1373client # [ 0.457658] ACPI: PCI: Interrupt link GSIH configured for IRQ 231374lb # [ 0.459447] ACPI: PCI: Interrupt link LNKF configured for IRQ 101375worker1 # [ 0.459668] ACPI: PCI: Interrupt link LNKH configured for IRQ 111376lb # [ 0.460458] ACPI: PCI: Interrupt link LNKG configured for IRQ 111377worker1 # [ 0.460599] ACPI: PCI: Interrupt link GSIA configured for IRQ 161378worker2 # [ 0.453413] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1379client # [ 0.459567] iommu: Default domain type: Translated1380lb # [ 0.461450] ACPI: PCI: Interrupt link LNKH configured for IRQ 111381worker1 # [ 0.461539] ACPI: PCI: Interrupt link GSIB configured for IRQ 171382client # [ 0.460279] iommu: DMA domain TLB invalidation policy: lazy mode1383client # [ 0.460992] ACPI: bus type USB registered1384lb # [ 0.462397] ACPI: PCI: Interrupt link GSIA configured for IRQ 161385worker1 # [ 0.462579] ACPI: PCI: Interrupt link GSIC configured for IRQ 181386client # [ 0.461703] usbcore: registered new interface driver usbfs1387lb # [ 0.463366] ACPI: PCI: Interrupt link GSIB configured for IRQ 171388worker1 # [ 0.463579] ACPI: PCI: Interrupt link GSID configured for IRQ 191389client # [ 0.462373] usbcore: registered new interface driver hub1390lb # [ 0.464364] ACPI: PCI: Interrupt link GSIC configured for IRQ 181391worker1 # [ 0.464575] ACPI: PCI: Interrupt link GSIE configured for IRQ 201392client # [ 0.462727] usbcore: registered new device driver usb1393worker2 # [ 0.457503] ACPI: PCI: Interrupt link LNKA configured for IRQ 101394worker1 # [ 0.465574] ACPI: PCI: Interrupt link GSIF configured for IRQ 211395lb # [ 0.465362] ACPI: PCI: Interrupt link GSID configured for IRQ 191396worker2 # [ 0.458561] ACPI: PCI: Interrupt link LNKB configured for IRQ 101397worker1 # [ 0.466526] ACPI: PCI: Interrupt link GSIG configured for IRQ 221398lb # [ 0.466352] ACPI: PCI: Interrupt link GSIE configured for IRQ 201399client # [ 0.464480] NetLabel: Initializing1400client # [ 0.464719] NetLabel: domain hash size = 1281401worker2 # [ 0.459572] ACPI: PCI: Interrupt link LNKC configured for IRQ 111402worker1 # [ 0.467266] ACPI: PCI: Interrupt link GSIH configured for IRQ 231403lb # [ 0.467333] ACPI: PCI: Interrupt link GSIF configured for IRQ 211404client # [ 0.465417] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1405lb # [ 0.468082] ACPI: PCI: Interrupt link GSIG configured for IRQ 221406worker2 # [ 0.460535] ACPI: PCI: Interrupt link LNKD configured for IRQ 111407worker1 # [ 0.468633] iommu: Default domain type: Translated1408client # [ 0.465776] NetLabel: unlabeled traffic allowed by default1409lb # [ 0.468363] ACPI: PCI: Interrupt link GSIH configured for IRQ 231410worker2 # [ 0.461564] ACPI: PCI: Interrupt link LNKE configured for IRQ 101411client # [ 0.466731] PCI: Using ACPI for IRQ routing1412worker1 # [ 0.469382] iommu: DMA domain TLB invalidation policy: lazy mode1413worker2 # [ 0.462539] ACPI: PCI: Interrupt link LNKF configured for IRQ 101414worker1 # [ 0.470567] ACPI: bus type USB registered1415lb # [ 0.470319] iommu: Default domain type: Translated1416worker2 # [ 0.463687] ACPI: PCI: Interrupt link LNKG configured for IRQ 111417worker1 # [ 0.471293] usbcore: registered new interface driver usbfs1418lb # [ 0.470928] iommu: DMA domain TLB invalidation policy: lazy mode1419worker1 # [ 0.471577] usbcore: registered new interface driver hub1420lb # [ 0.471629] ACPI: bus type USB registered1421worker2 # [ 0.464515] ACPI: PCI: Interrupt link LNKH configured for IRQ 111422worker1 # [ 0.472425] usbcore: registered new device driver usb1423lb # [ 0.472358] usbcore: registered new interface driver usbfs1424worker2 # [ 0.465642] ACPI: PCI: Interrupt link GSIA configured for IRQ 161425lb # [ 0.473279] usbcore: registered new interface driver hub1426worker2 # [ 0.466554] ACPI: PCI: Interrupt link GSIB configured for IRQ 171427worker1 # [ 0.474123] NetLabel: Initializing1428lb # [ 0.474048] usbcore: registered new device driver usb1429worker1 # [ 0.474564] NetLabel: domain hash size = 1281430worker2 # [ 0.467500] ACPI: PCI: Interrupt link GSIC configured for IRQ 181431worker1 # [ 0.475270] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1432lb # [ 0.475190] NetLabel: Initializing1433worker2 # [ 0.468497] ACPI: PCI: Interrupt link GSID configured for IRQ 191434worker1 # [ 0.475618] NetLabel: unlabeled traffic allowed by default1435lb # [ 0.475787] NetLabel: domain hash size = 1281436worker2 # [ 0.469477] ACPI: PCI: Interrupt link GSIE configured for IRQ 201437worker1 # [ 0.476492] PCI: Using ACPI for IRQ routing1438lb # [ 0.476351] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1439worker2 # [ 0.469721] ACPI: PCI: Interrupt link GSIF configured for IRQ 211440lb # [ 0.477335] NetLabel: unlabeled traffic allowed by default1441worker2 # [ 0.470711] ACPI: PCI: Interrupt link GSIG configured for IRQ 221442lb # [ 0.478084] PCI: Using ACPI for IRQ routing1443worker2 # [ 0.471717] ACPI: PCI: Interrupt link GSIH configured for IRQ 231444worker2 # [ 0.473740] iommu: Default domain type: Translated1445worker2 # [ 0.474641] iommu: DMA domain TLB invalidation policy: lazy mode1446worker2 # [ 0.475762] ACPI: bus type USB registered1447worker2 # [ 0.476560] usbcore: registered new interface driver usbfs1448worker2 # [ 0.477440] usbcore: registered new interface driver hub1449worker2 # [ 0.477713] usbcore: registered new device driver usb1450worker2 # [ 0.479445] NetLabel: Initializing1451worker2 # [ 0.479710] NetLabel: domain hash size = 1281452worker2 # [ 0.480484] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453worker2 # [ 0.480767] NetLabel: unlabeled traffic allowed by default1454worker2 # [ 0.481716] PCI: Using ACPI for IRQ routing1455client # [ 0.510163] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456client # [ 0.510714] pci 0000:00:01.0: vgaarb: bridge control possible1457client # [ 0.510714] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458client # [ 0.510722] vgaarb: loaded1459client # [ 0.511449] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460client # [ 0.511717] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461client # [ 0.514810] clocksource: Switched to clocksource kvm-clock1462worker1 # [ 0.519910] pci 0000:00:01.0: vgaarb: setting as boot VGA device1463lb # [ 0.521402] pci 0000:00:01.0: vgaarb: setting as boot VGA device1464worker1 # [ 0.520559] pci 0000:00:01.0: vgaarb: bridge control possible1465lb # [ 0.522346] pci 0000:00:01.0: vgaarb: bridge control possible1466worker1 # [ 0.520559] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1467worker1 # [ 0.520568] vgaarb: loaded1468client # [ 0.518267] VFS: Disk quotas dquot_6.6.01469worker1 # [ 0.521322] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01470client # [ 0.518947] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1471lb # [ 0.522346] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1472worker1 # [ 0.521565] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1473client # [ 0.520272] pnp: PnP ACPI init1474lb # [ 0.522356] vgaarb: loaded1475client # [ 0.521001] ACPI: IRQ 4 override to edge(!), high(!)1476lb # [ 0.523169] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01477client # [ 0.521928] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1478lb # [ 0.523996] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1479client # [ 0.523439] pnp: PnP ACPI: found 5 devices1480worker1 # [ 0.524669] clocksource: Switched to clocksource kvm-clock1481worker1 # [ 0.526397] VFS: Disk quotas dquot_6.6.01482lb # [ 0.527439] clocksource: Switched to clocksource kvm-clock1483worker1 # [ 0.528943] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1484worker1 # [ 0.530305] pnp: PnP ACPI init1485lb # [ 0.531015] VFS: Disk quotas dquot_6.6.01486worker1 # [ 0.531176] ACPI: IRQ 4 override to edge(!), high(!)1487worker2 # [ 0.526335] pci 0000:00:01.0: vgaarb: setting as boot VGA device1488lb # [ 0.531738] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1489worker2 # [ 0.526701] pci 0000:00:01.0: vgaarb: bridge control possible1490client # [ 0.531011] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1491worker1 # [ 0.532100] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1492lb # [ 0.533016] pnp: PnP ACPI init1493worker2 # [ 0.526701] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1494worker1 # [ 0.533564] pnp: PnP ACPI: found 5 devices1495client # [ 0.532396] clocksource: Switched to clocksource acpi_pm1496lb # [ 0.533776] ACPI: IRQ 4 override to edge(!), high(!)1497worker2 # [ 0.526710] vgaarb: loaded1498client # [ 0.533319] NET: Registered PF_INET protocol family1499lb # [ 0.534658] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1500worker2 # [ 0.527471] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01501client # [ 0.534429] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1502lb # [ 0.536222] pnp: PnP ACPI: found 5 devices1503worker2 # [ 0.527705] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1504worker2 # [ 0.531794] clocksource: Switched to clocksource kvm-clock1505worker1 # [ 0.541117] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1506worker1 # [ 0.542616] clocksource: Switched to clocksource acpi_pm1507worker1 # [ 0.543627] NET: Registered PF_INET protocol family1508lb # [ 0.543790] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1509worker2 # [ 0.535327] VFS: Disk quotas dquot_6.6.01510lb # [ 0.545230] clocksource: Switched to clocksource acpi_pm1511worker1 # [ 0.544834] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1512worker2 # [ 0.536061] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1513lb # [ 0.546200] NET: Registered PF_INET protocol family1514worker2 # [ 0.537444] pnp: PnP ACPI init1515worker2 # [ 0.538247] ACPI: IRQ 4 override to edge(!), high(!)1516lb # [ 0.547389] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1517worker2 # [ 0.539265] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1518worker2 # [ 0.540864] pnp: PnP ACPI: found 5 devices1519client # [ 0.550405] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1520client # [ 0.551884] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1521client # [ 0.553141] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1522client # [ 0.554452] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1523client # [ 0.555649] TCP: Hash tables configured (established 16384 bind 16384)1524client # [ 0.556808] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1525worker2 # [ 0.548491] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1526client # [ 0.558086] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1527worker2 # [ 0.550023] clocksource: Switched to clocksource acpi_pm1528worker2 # [ 0.550970] NET: Registered PF_INET protocol family1529client # [ 0.559215] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1530client # [ 0.560354] NET: Registered PF_UNIX/PF_LOCAL protocol family1531worker2 # [ 0.552127] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1532worker1 # [ 0.561240] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1533client # [ 0.561266] NET: Registered PF_XDP protocol family1534client # [ 0.562037] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1535worker1 # [ 0.562762] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1536client # [ 0.563000] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1537lb # [ 0.563720] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1538worker1 # [ 0.564066] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1539client # [ 0.563957] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1540lb # [ 0.565288] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1541worker1 # [ 0.565383] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1542client # [ 0.565001] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1543worker1 # [ 0.566603] TCP: Hash tables configured (established 16384 bind 16384)1544lb # [ 0.566577] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1545client # [ 0.566043] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1546lb # [ 0.567914] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1547worker1 # [ 0.567691] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1548client # [ 0.567082] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1549worker1 # [ 0.568927] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1550lb # [ 0.569175] TCP: Hash tables configured (established 16384 bind 16384)1551client # [ 0.568714] ACPI: \_SB_.GSIA: Enabled at IRQ 161552worker1 # [ 0.569997] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1553lb # [ 0.570303] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1554worker1 # [ 0.571195] NET: Registered PF_UNIX/PF_LOCAL protocol family1555lb # [ 0.571589] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1556client # [ 0.571294] ACPI: \_SB_.GSIB: Enabled at IRQ 171557worker1 # [ 0.572157] NET: Registered PF_XDP protocol family1558lb # [ 0.572705] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1559worker1 # [ 0.572996] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1560lb # [ 0.573912] NET: Registered PF_UNIX/PF_LOCAL protocol family1561client # [ 0.573190] ACPI: \_SB_.GSIC: Enabled at IRQ 181562worker1 # [ 0.574022] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1563lb # [ 0.574844] NET: Registered PF_XDP protocol family1564worker1 # [ 0.575018] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1565lb # [ 0.575684] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1566client # [ 0.575005] ACPI: \_SB_.GSID: Enabled at IRQ 191567worker1 # [ 0.576114] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1568lb # [ 0.576689] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1569client # [ 0.576638] PCI: CLS 0 bytes, default 641570worker1 # [ 0.577222] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1571lb # [ 0.577736] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1572worker2 # [ 0.568216] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1573worker1 # [ 0.578289] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1574lb # [ 0.578767] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1575client # [ 0.577489] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1576worker2 # [ 0.569751] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1577lb # [ 0.579788] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1578worker1 # [ 0.579952] ACPI: \_SB_.GSIA: Enabled at IRQ 161579client # [ 0.579126] Trying to unpack rootfs image as initramfs...1580worker2 # [ 0.571034] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1581lb # [ 0.580824] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1582worker2 # [ 0.572404] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1583worker1 # [ 0.582453] ACPI: \_SB_.GSIB: Enabled at IRQ 171584lb # [ 0.582746] ACPI: \_SB_.GSIA: Enabled at IRQ 161585worker2 # [ 0.573682] TCP: Hash tables configured (established 16384 bind 16384)1586worker1 # [ 0.584388] ACPI: \_SB_.GSIC: Enabled at IRQ 181587worker2 # [ 0.574835] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1588lb # [ 0.585195] ACPI: \_SB_.GSIB: Enabled at IRQ 171589worker2 # [ 0.576120] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1590worker1 # [ 0.586289] ACPI: \_SB_.GSID: Enabled at IRQ 191591worker2 # [ 0.577254] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1592lb # [ 0.587093] ACPI: \_SB_.GSIC: Enabled at IRQ 181593worker1 # [ 0.587950] PCI: CLS 0 bytes, default 641594worker2 # [ 0.578504] NET: Registered PF_UNIX/PF_LOCAL protocol family1595worker2 # [ 0.579502] NET: Registered PF_XDP protocol family1596lb # [ 0.589112] ACPI: \_SB_.GSID: Enabled at IRQ 191597worker1 # [ 0.588819] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1598worker2 # [ 0.580372] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1599lb # [ 0.590750] PCI: CLS 0 bytes, default 641600worker1 # [ 0.590487] Trying to unpack rootfs image as initramfs...1601worker2 # [ 0.581449] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1602worker2 # [ 0.582529] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1603lb # [ 0.591587] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1604worker2 # [ 0.583661] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1605lb # [ 0.593387] Trying to unpack rootfs image as initramfs...1606worker2 # [ 0.584814] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1607worker2 # [ 0.585987] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1608worker2 # [ 0.587765] ACPI: \_SB_.GSIA: Enabled at IRQ 161609worker2 # [ 0.590583] ACPI: \_SB_.GSIB: Enabled at IRQ 171610worker2 # [ 0.592519] ACPI: \_SB_.GSIC: Enabled at IRQ 181611worker2 # [ 0.594529] ACPI: \_SB_.GSID: Enabled at IRQ 191612worker2 # [ 0.596178] PCI: CLS 0 bytes, default 641613worker2 # [ 0.597074] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614worker2 # [ 0.598892] Trying to unpack rootfs image as initramfs...1615client # [ 0.623090] Initialise system trusted keyrings1616client # [ 0.626237] workingset: timestamp_bits=40 max_order=19 bucket_order=01617worker1 # [ 0.634575] Initialise system trusted keyrings1618worker1 # [ 0.637728] workingset: timestamp_bits=40 max_order=19 bucket_order=01619lb # [ 0.638255] Initialise system trusted keyrings1620lb # [ 0.641206] workingset: timestamp_bits=40 max_order=19 bucket_order=01621client # [ 0.645455] Key type asymmetric registered1622client # [ 0.648193] Asymmetric key parser 'x509' registered1623client # [ 0.649058] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1624worker2 # [ 0.643790] Initialise system trusted keyrings1625client # [ 0.652288] io scheduler mq-deadline registered1626client # [ 0.653033] io scheduler kyber registered1627worker2 # [ 0.644738] workingset: timestamp_bits=40 max_order=19 bucket_order=01628client # [ 0.655765] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1629client # [ 0.657056] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630worker1 # [ 0.660691] Key type asymmetric registered1631worker1 # [ 0.661439] Asymmetric key parser 'x509' registered1632lb # [ 0.662418] Key type asymmetric registered1633worker1 # [ 0.662338] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1634client # [ 0.662948] Linux agpgart interface v0.1031635client # [ 0.663750] ACPI: bus type drm_connector registered1636lb # [ 0.665167] Asymmetric key parser 'x509' registered1637worker1 # [ 0.665814] io scheduler mq-deadline registered1638lb # [ 0.666052] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1639worker1 # [ 0.666546] io scheduler kyber registered1640client # [ 0.666653] usbcore: registered new interface driver usbserial_generic1641client # [ 0.667799] usbserial: USB Serial support registered for generic1642lb # [ 0.669261] io scheduler mq-deadline registered1643worker1 # [ 0.669248] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1644lb # [ 0.670052] io scheduler kyber registered1645worker1 # [ 0.670538] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1646lb # [ 0.672722] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1647client # [ 0.671203] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1648client # [ 0.671203] Please enable it if your BIOS has the CPPC option.1649lb # [ 0.673998] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1650client # [ 0.673438] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1651worker1 # [ 0.675960] Linux agpgart interface v0.1031652worker1 # [ 0.676751] ACPI: bus type drm_connector registered1653worker2 # [ 0.667608] Key type asymmetric registered1654client # [ 0.676301] drop_monitor: Initializing network drop monitor service1655worker2 # [ 0.668347] Asymmetric key parser 'x509' registered1656client # [ 0.677443] NET: Registered PF_INET6 protocol family1657worker2 # [ 0.669256] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1658lb # [ 0.679770] Linux agpgart interface v0.1031659worker1 # [ 0.679999] usbcore: registered new interface driver usbserial_generic1660lb # [ 0.680544] ACPI: bus type drm_connector registered1661client # [ 0.680636] Segment Routing with IPv61662worker1 # [ 0.681090] usbserial: USB Serial support registered for generic1663lb # [ 0.683593] usbcore: registered new interface driver usbserial_generic1664worker2 # [ 0.674550] io scheduler mq-deadline registered1665client # [ 0.683191] In-situ OAM (IOAM) with IPv61666worker2 # [ 0.675329] io scheduler kyber registered1667lb # [ 0.684694] usbserial: USB Serial support registered for generic1668client # [ 0.684216] IPI shorthand broadcast: enabled1669worker1 # [ 0.684691] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1670worker1 # [ 0.684691] Please enable it if your BIOS has the CPPC option.1671worker1 # [ 0.686948] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1672worker2 # [ 0.678934] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1673lb # [ 0.688173] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1674lb # [ 0.688173] Please enable it if your BIOS has the CPPC option.1675worker2 # [ 0.680209] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1676worker1 # [ 0.689853] drop_monitor: Initializing network drop monitor service1677lb # [ 0.690421] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1678worker1 # [ 0.691003] NET: Registered PF_INET6 protocol family1679lb # [ 0.693297] drop_monitor: Initializing network drop monitor service1680worker2 # [ 0.684229] Linux agpgart interface v0.1031681client # [ 0.692579] sched_clock: Marking stable (560014811, 132158099)->(767518152, -75345242)1682worker1 # [ 0.694199] Segment Routing with IPv61683worker2 # [ 0.685000] ACPI: bus type drm_connector registered1684lb # [ 0.694476] NET: Registered PF_INET6 protocol family1685lb # [ 0.697602] Segment Routing with IPv61686worker1 # [ 0.697722] In-situ OAM (IOAM) with IPv61687client # [ 0.697424] registered taskstats version 11688worker1 # [ 0.698777] IPI shorthand broadcast: enabled1689client # [ 0.698395] Loading compiled-in X.509 certificates1690worker2 # [ 0.689810] usbcore: registered new interface driver usbserial_generic1691lb # [ 0.700187] In-situ OAM (IOAM) with IPv61692worker2 # [ 0.691062] usbserial: USB Serial support registered for generic1693lb # [ 0.701232] IPI shorthand broadcast: enabled1694worker2 # [ 0.693426] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1695worker2 # [ 0.693426] Please enable it if your BIOS has the CPPC option.1696worker2 # [ 0.695743] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1697worker1 # [ 0.707087] sched_clock: Marking stable (574014956, 132651397)->(779452883, -72786530)1698lb # [ 0.708457] sched_clock: Marking stable (575015610, 133132214)->(781409903, -73262079)1699worker2 # [ 0.699552] drop_monitor: Initializing network drop monitor service1700worker2 # [ 0.700734] NET: Registered PF_INET6 protocol family1701worker1 # [ 0.711779] registered taskstats version 11702worker1 # [ 0.712778] Loading compiled-in X.509 certificates1703worker2 # [ 0.703819] Segment Routing with IPv61704lb # [ 0.713386] registered taskstats version 11705lb # [ 0.714363] Loading compiled-in X.509 certificates1706worker2 # [ 0.706442] In-situ OAM (IOAM) with IPv61707worker2 # [ 0.707564] IPI shorthand broadcast: enabled1708client # [ 0.716186] Demotion targets for Node 0: null1709client # [ 0.717044] Key type .fscrypt registered1710client # [ 0.720178] Key type fscrypt-provisioning registered1711client # [ 0.721102] ima: No TPM chip found, activating TPM-bypass!1712client # [ 0.722045] ima: Allocated hash algorithm: sha11713client # [ 0.724208] ima: No architecture policies found1714worker2 # [ 0.715782] sched_clock: Marking stable (581015032, 134380829)->(789086384, -73690523)1715client # [ 0.725124] PM: Magic number: 10:752:5171716client # [ 0.728751] RAS: Correctable Errors collector initialized.1717worker2 # [ 0.720505] registered taskstats version 11718worker1 # [ 0.730678] Demotion targets for Node 0: null1719worker2 # [ 0.721472] Loading compiled-in X.509 certificates1720worker1 # [ 0.731543] Key type .fscrypt registered1721lb # [ 0.733166] Demotion targets for Node 0: null1722lb # [ 0.734101] Key type .fscrypt registered1723worker1 # [ 0.734671] Key type fscrypt-provisioning registered1724worker1 # [ 0.735636] ima: No TPM chip found, activating TPM-bypass!1725worker1 # [ 0.736542] ima: Allocated hash algorithm: sha11726lb # [ 0.737152] Key type fscrypt-provisioning registered1727lb # [ 0.738117] ima: No TPM chip found, activating TPM-bypass!1728client # [ 0.737382] clk: Disabling unused clocks1729worker1 # [ 0.738698] ima: No architecture policies found1730lb # [ 0.739058] ima: Allocated hash algorithm: sha11731client # [ 0.738045] PM: genpd: Disabling unused power domains1732worker1 # [ 0.739614] PM: Magic number: 10:752:5171733lb # [ 0.741192] ima: No architecture policies found1734worker1 # [ 0.743231] RAS: Correctable Errors collector initialized.1735lb # [ 0.744167] PM: Magic number: 10:752:5171736lb # [ 0.745733] RAS: Correctable Errors collector initialized.1737worker2 # [ 0.739410] Demotion targets for Node 0: null1738worker2 # [ 0.740334] Key type .fscrypt registered1739worker1 # [ 0.751932] clk: Disabling unused clocks1740worker2 # [ 0.743400] Key type fscrypt-provisioning registered1741worker1 # [ 0.752627] PM: genpd: Disabling unused power domains1742lb # [ 0.753849] clk: Disabling unused clocks1743worker2 # [ 0.744372] ima: No TPM chip found, activating TPM-bypass!1744worker2 # [ 0.745335] ima: Allocated hash algorithm: sha11745worker2 # [ 0.747427] ima: No architecture policies found1746worker2 # [ 0.748433] PM: Magic number: 10:752:5171747lb # [ 0.758166] PM: genpd: Disabling unused power domains1748worker2 # [ 0.750014] RAS: Correctable Errors collector initialized.1749worker2 # [ 0.761109] clk: Disabling unused clocks1750worker2 # [ 0.763410] PM: genpd: Disabling unused power domains1751client # [ 0.898523] Freeing initrd memory: 29056K1752client # [ 0.901706] Freeing unused decrypted memory: 2028K1753client # [ 0.904287] Freeing unused kernel image (initmem) memory: 3652K1754client # [ 0.905405] Write protecting the kernel read-only data: 32768k1755client # [ 0.907275] Freeing unused kernel image (text/rodata gap) memory: 1184K1756client # [ 0.908824] Freeing unused kernel image (rodata/data gap) memory: 720K1757worker1 # [ 0.912289] Freeing initrd memory: 29056K1758worker1 # [ 0.915486] Freeing unused decrypted memory: 2028K1759lb # [ 0.916258] Freeing initrd memory: 29064K1760worker1 # [ 0.918080] Freeing unused kernel image (initmem) memory: 3652K1761lb # [ 0.919416] Freeing unused decrypted memory: 2028K1762worker1 # [ 0.919179] Write protecting the kernel read-only data: 32768k1763worker1 # [ 0.921014] Freeing unused kernel image (text/rodata gap) memory: 1184K1764lb # [ 0.922004] Freeing unused kernel image (initmem) memory: 3652K1765worker1 # [ 0.922531] Freeing unused kernel image (rodata/data gap) memory: 720K1766lb # [ 0.923153] Write protecting the kernel read-only data: 32768k1767lb # [ 0.924991] Freeing unused kernel image (text/rodata gap) memory: 1184K1768lb # [ 0.926518] Freeing unused kernel image (rodata/data gap) memory: 720K1769worker2 # [ 0.922317] Freeing initrd memory: 29060K1770worker2 # [ 0.925567] Freeing unused decrypted memory: 2028K1771worker2 # [ 0.928126] Freeing unused kernel image (initmem) memory: 3652K1772worker2 # [ 0.929258] Write protecting the kernel read-only data: 32768k1773worker2 # [ 0.931125] Freeing unused kernel image (text/rodata gap) memory: 1184K1774worker2 # [ 0.932711] Freeing unused kernel image (rodata/data gap) memory: 720K1775client # [ 0.959504] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776client # [ 0.960601] Run /init as init process1777client # [ 0.971108] systemd[1]: Inserted module 'autofs4'1778worker1 # [ 0.973478] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779worker1 # [ 0.974585] Run /init as init process1780lb # [ 0.977284] x86/mm: Checked W+X mappings: passed, no W+X pages found.1781lb # [ 0.978410] Run /init as init process1782worker1 # [ 0.985271] systemd[1]: Inserted module 'autofs4'1783lb # [ 0.989046] systemd[1]: Inserted module 'autofs4'1784client # [ 0.990817] fuse: init (API version 7.45)1785worker2 # [ 0.983490] x86/mm: Checked W+X mappings: passed, no W+X pages found.1786worker2 # [ 0.984609] Run /init as init process1787client # [ 0.998152] ACPI: \_SB_.GSIG: Enabled at IRQ 221788client # [ 1.000750] ACPI: \_SB_.GSIH: Enabled at IRQ 231789worker2 # [ 0.995209] systemd[1]: Inserted module 'autofs4'1790worker1 # [ 1.005271] fuse: init (API version 7.45)1791client # [ 1.004794] ACPI: \_SB_.GSIE: Enabled at IRQ 201792client # [ 1.006936] ACPI: \_SB_.GSIF: Enabled at IRQ 211793lb # [ 1.008792] fuse: init (API version 7.45)1794worker1 # [ 1.012780] ACPI: \_SB_.GSIG: Enabled at IRQ 221795client # [ 1.012495] virtiofs virtio5: discovered new tag: nix-store1796client # [ 1.014134] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1797worker1 # [ 1.015276] ACPI: \_SB_.GSIH: Enabled at IRQ 231798lb # [ 1.016253] ACPI: \_SB_.GSIG: Enabled at IRQ 221799worker1 # [ 1.018375] ACPI: \_SB_.GSIE: Enabled at IRQ 201800lb # [ 1.018887] ACPI: \_SB_.GSIH: Enabled at IRQ 231801worker1 # [ 1.020546] ACPI: \_SB_.GSIF: Enabled at IRQ 211802lb # [ 1.022839] ACPI: \_SB_.GSIE: Enabled at IRQ 201803worker2 # [ 1.015243] fuse: init (API version 7.45)1804client # [ 1.023762] virtiofs virtio6: discovered new tag: shared1805lb # [ 1.024988] ACPI: \_SB_.GSIF: Enabled at IRQ 211806worker1 # [ 1.026313] virtiofs virtio5: discovered new tag: nix-store1807client # [ 1.025397] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1808worker1 # [ 1.027872] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1809lb # [ 1.030408] virtiofs virtio5: discovered new tag: nix-store1810client # [ 1.029617] virtiofs virtio7: discovered new tag: xchg1811client # [ 1.031031] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1812lb # [ 1.032176] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1813worker2 # [ 1.023369] ACPI: \_SB_.GSIG: Enabled at IRQ 221814worker2 # [ 1.026546] ACPI: \_SB_.GSIH: Enabled at IRQ 231815worker1 # [ 1.036274] virtiofs virtio6: discovered new tag: shared1816worker1 # [ 1.037895] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1817worker2 # [ 1.030060] ACPI: \_SB_.GSIE: Enabled at IRQ 201818lb # [ 1.039803] virtiofs virtio6: discovered new tag: shared1819lb # [ 1.041337] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1820worker1 # [ 1.041268] virtiofs virtio7: discovered new tag: xchg1821worker2 # [ 1.032870] ACPI: \_SB_.GSIF: Enabled at IRQ 211822worker1 # [ 1.042858] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1823lb # [ 1.044732] virtiofs virtio7: discovered new tag: xchg1824lb # [ 1.046280] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1825worker2 # [ 1.040093] virtiofs virtio5: discovered new tag: nix-store1826worker2 # [ 1.042063] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1827client # [ 1.054019] systemd[1]: Successfully made /usr/ read-only.1828worker2 # [ 1.051984] virtiofs virtio6: discovered new tag: shared1829worker2 # [ 1.053518] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1830worker2 # [ 1.057129] virtiofs virtio7: discovered new tag: xchg1831worker1 # [ 1.066421] systemd[1]: Successfully made /usr/ read-only.1832worker2 # [ 1.058622] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1833lb # [ 1.068932] systemd[1]: Successfully made /usr/ read-only.1834worker2 # [ 1.082159] systemd[1]: Successfully made /usr/ read-only.1835client # [ 1.389585] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836client # [ 1.394794] systemd[1]: Detected virtualization kvm.1837client # [ 1.395691] systemd[1]: Detected architecture x86-64.1838client # [ 1.396600] systemd[1]: Running in initrd.1839client # [ 1.397698] systemd[1]: Initializing machine ID from random generator.1840client # [ 1.399003] systemd[1]: Hostname set to <client>.1841worker1 # [ 1.403066] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842worker1 # [ 1.415169] systemd[1]: Detected virtualization kvm.1843lb # [ 1.405596] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1844worker1 # [ 1.417332] systemd[1]: Detected architecture x86-64.1845lb # [ 1.417627] systemd[1]: Detected virtualization kvm.1846worker1 # [ 1.419493] systemd[1]: Running in initrd.1847lb # [ 1.419773] systemd[1]: Detected architecture x86-64.1848lb # [ 1.421918] systemd[1]: Running in initrd.1849worker1 # [ 1.421948] systemd[1]: Initializing machine ID from random generator.1850lb # [ 1.424391] systemd[1]: Initializing machine ID from random generator.1851worker1 # [ 1.424838] systemd[1]: Hostname set to <worker1>.1852lb # [ 1.427328] systemd[1]: Hostname set to <lb>.1853worker2 # [ 1.418812] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854worker2 # [ 1.430929] systemd[1]: Detected virtualization kvm.1855worker2 # [ 1.433129] systemd[1]: Detected architecture x86-64.1856worker2 # [ 1.435348] systemd[1]: Running in initrd.1857worker2 # [ 1.437846] systemd[1]: Initializing machine ID from random generator.1858worker2 # [ 1.440935] systemd[1]: Hostname set to <worker2>.1859client # [ 1.584487] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860client # [ 1.621701] systemd[1]: Queued start job for default target Initrd Default Target.1861client # [ 1.625466] systemd[1]: Created slice Slice /system/modprobe.1862client # [ 1.626717] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1863client # [ 1.628183] systemd[1]: Expecting device /dev/disk/by-label/nixos...1864client # [ 1.629313] systemd[1]: Reached target Path Units.1865client # [ 1.630235] systemd[1]: Reached target Slice Units.1866client # [ 1.631093] systemd[1]: Reached target Swaps.1867client # [ 1.631933] systemd[1]: Reached target Timer Units.1868client # [ 1.632936] systemd[1]: Listening on D-Bus System Message Bus Socket.1869client # [ 1.634213] systemd[1]: Listening on Journal Socket (/dev/log).1870client # [ 1.635374] systemd[1]: Listening on Journal Sockets.1871client # [ 1.636363] systemd[1]: Listening on udev Control Socket.1872client # [ 1.637413] systemd[1]: Listening on udev Kernel Socket.1873client # [ 1.638371] systemd[1]: Reached target Socket Units.1874client # [ 1.640094] systemd[1]: Starting Create List of Static Device Nodes...1875client # [ 1.643982] systemd[1]: Starting Load Kernel Module configfs...1876worker1 # [ 1.646756] systemd[1]: bpf-restrict-fs: LSM BPF program attached1877lb # [ 1.648290] systemd[1]: bpf-restrict-fs: LSM BPF program attached1878client # [ 1.652353] systemd[1]: Starting Journal Service...1879worker2 # [ 1.664541] systemd[1]: bpf-restrict-fs: LSM BPF program attached1880client # [ 1.677239] systemd[1]: Starting Load Kernel Modules...1881client # [ 1.681277] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1882client # [ 1.690243] systemd[1]: Starting Coldplug All udev Devices...1883worker1 # [ 1.699128] systemd[1]: Queued start job for default target Initrd Default Target.1884lb # [ 1.700899] systemd[1]: Queued start job for default target Initrd Default Target.1885client # [ 1.701507] systemd-journald[66]: Collecting audit messages is disabled.1886worker1 # [ 1.702911] systemd[1]: Created slice Slice /system/modprobe.1887client # [ 1.702972] systemd[1]: Finished Create List of Static Device Nodes.1888worker1 # [ 1.704109] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1889lb # [ 1.705399] systemd[1]: Created slice Slice /system/modprobe.1890worker1 # [ 1.705534] systemd[1]: Expecting device /dev/disk/by-label/nixos...1891worker1 # [ 1.706671] systemd[1]: Reached target Path Units.1892lb # [ 1.706598] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1893worker1 # [ 1.707504] systemd[1]: Reached target Slice Units.1894client # [ 1.706824] systemd[1]: modprobe@configfs.service: Deactivated successfully.1895lb # [ 1.708003] systemd[1]: Expecting device /dev/disk/by-label/nixos...1896worker1 # [ 1.708373] systemd[1]: Reached target Swaps.1897lb # [ 1.709062] systemd[1]: Reached target Path Units.1898worker1 # [ 1.709185] systemd[1]: Reached target Timer Units.1899lb # [ 1.709934] systemd[1]: Reached target Slice Units.1900worker1 # [ 1.710168] systemd[1]: Listening on D-Bus System Message Bus Socket.1901lb # [ 1.710814] systemd[1]: Reached target Swaps.1902lb # [ 1.711646] systemd[1]: Reached target Timer Units.1903worker1 # [ 1.711411] systemd[1]: Listening on Journal Socket (/dev/log).1904lb # [ 1.712639] systemd[1]: Listening on D-Bus System Message Bus Socket.1905worker1 # [ 1.712605] systemd[1]: Listening on Journal Sockets.1906worker1 # [ 1.713589] systemd[1]: Listening on udev Control Socket.1907lb # [ 1.713869] systemd[1]: Listening on Journal Socket (/dev/log).1908worker1 # [ 1.714611] systemd[1]: Listening on udev Kernel Socket.1909lb # [ 1.715079] systemd[1]: Listening on Journal Sockets.1910worker1 # [ 1.715559] systemd[1]: Reached target Socket Units.1911lb # [ 1.716119] systemd[1]: Listening on udev Control Socket.1912client # [ 1.715553] systemd[1]: Finished Load Kernel Module configfs.1913lb # [ 1.717195] systemd[1]: Listening on udev Kernel Socket.1914worker1 # [ 1.717361] systemd[1]: Starting Create List of Static Device Nodes...1915lb # [ 1.718169] systemd[1]: Reached target Socket Units.1916lb # [ 1.719954] systemd[1]: Starting Create List of Static Device Nodes...1917worker1 # [ 1.721736] systemd[1]: Starting Load Kernel Module configfs...1918client # [ 1.721610] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1919lb # [ 1.723961] systemd[1]: Starting Load Kernel Module configfs...1920worker2 # [ 1.717337] systemd[1]: Queued start job for default target Initrd Default Target.1921worker2 # [ 1.721698] systemd[1]: Created slice Slice /system/modprobe.1922worker2 # [ 1.722872] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1923worker2 # [ 1.724405] systemd[1]: Expecting device /dev/disk/by-label/nixos...1924client # [ 1.733256] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1925worker2 # [ 1.725537] systemd[1]: Reached target Path Units.1926worker2 # [ 1.726444] systemd[1]: Reached target Slice Units.1927worker2 # [ 1.727354] systemd[1]: Reached target Swaps.1928worker2 # [ 1.728197] systemd[1]: Reached target Timer Units.1929worker2 # [ 1.729195] systemd[1]: Listening on D-Bus System Message Bus Socket.1930worker2 # [ 1.730480] systemd[1]: Listening on Journal Socket (/dev/log).1931worker2 # [ 1.731729] systemd[1]: Listening on Journal Sockets.1932worker2 # [ 1.732750] systemd[1]: Listening on udev Control Socket.1933lb # [ 1.730767] systemd[1]: Starting Journal Service...1934worker2 # [ 1.733875] systemd[1]: Listening on udev Kernel Socket.1935worker1 # [ 1.731770] systemd[1]: Starting Journal Service...1936client # [ 1.741787] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1937worker2 # [ 1.734814] systemd[1]: Reached target Socket Units.1938worker2 # [ 1.736564] systemd[1]: Starting Create List of Static Device Nodes...1939worker2 # [ 1.740469] systemd[1]: Starting Load Kernel Module configfs...1940client # [ 1.752347] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1941worker1 # [ 1.755230] systemd[1]: Starting Load Kernel Modules...1942lb # [ 1.758325] systemd[1]: Starting Load Kernel Modules...1943worker1 # [ 1.759791] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1944lb # [ 1.762267] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1945worker1 # [ 1.766623] systemd[1]: Starting Coldplug All udev Devices...1946client # [ 1.766257] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1947lb # [ 1.768920] systemd[1]: Starting Coldplug All udev Devices...1948worker2 # [ 1.749831] systemd[1]: Starting Journal Service...1949client # [ 1.776432] systemd[1]: Starting Create Static Device Nodes in /dev...1950worker1 # [ 1.778990] systemd-journald[66]: Collecting audit messages is disabled.1951lb # [ 1.780798] systemd-journald[66]: Collecting audit messages is disabled.1952worker1 # [ 1.782964] systemd[1]: Finished Create List of Static Device Nodes.1953worker2 # [ 1.773997] systemd[1]: Starting Load Kernel Modules...1954lb # [ 1.783598] systemd[1]: Finished Create List of Static Device Nodes.1955worker1 # [ 1.786512] systemd[1]: modprobe@configfs.service: Deactivated successfully.1956worker2 # [ 1.778527] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1957lb # [ 1.789744] systemd[1]: modprobe@configfs.service: Deactivated successfully.1958worker1 # [ 1.793055] systemd[1]: Finished Load Kernel Module configfs.1959client # [ 1.793231] systemd[1]: Finished Load Kernel Modules.1960worker2 # [ 1.787579] systemd[1]: Starting Coldplug All udev Devices...1961lb # [ 1.797625] systemd[1]: Finished Load Kernel Module configfs.1962worker1 # [ 1.799087] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1963lb # [ 1.801561] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1964client # [ 1.802359] systemd[1]: Starting Apply Kernel Variables...1965worker2 # [ 1.798836] systemd-journald[66]: Collecting audit messages is disabled.1966worker1 # [ 1.808759] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1967worker2 # [ 1.800519] systemd[1]: Finished Create List of Static Device Nodes.1968lb # [ 1.814223] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1969client # [ 1.814317] systemd[1]: Finished Create Static Device Nodes in /dev.1970worker2 # [ 1.805839] systemd[1]: modprobe@configfs.service: Deactivated successfully.1971lb # [ 1.819896] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1972client # [ 1.820409] systemd[1]: Reached target Preparation for Local File Systems.1973worker1 # [ 1.821293] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1974worker2 # [ 1.813756] systemd[1]: Finished Load Kernel Module configfs.1975client # [ 1.825273] systemd[1]: Reached target Local File Systems.1976worker2 # [ 1.818025] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1977worker1 # [ 1.829825] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1978lb # [ 1.831333] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1979client # [ 1.696807] systemd-modules-load[67]: Inserted module 'dm_mod'1980client # [ 1.700271] systemd-modules-load[67]: Inserted module 'virtio_balloon'1981client # [ 1.701236] systemd-modules-load[67]: Inserted module 'virtio_gpu'1982client # [ 1.835143] systemd[1]: Starting Rule-based Manager for Device Events and Files...1983worker2 # [ 1.828575] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1984client # [ 1.839413] systemd[1]: Started Journal Service.1985client # [ 1.716174] systemd[1]: Finished Apply Kernel Variables.1986worker1 # [ 1.851200] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1987worker2 # [ 1.841610] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1988lb # [ 1.853694] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1989worker2 # [ 1.849591] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1990worker1 # [ 1.859851] systemd[1]: Starting Create Static Device Nodes in /dev...1991client # [ 1.727549] systemd[1]: Starting Create System Files and Directories...1992lb # [ 1.862328] systemd[1]: Starting Create Static Device Nodes in /dev...1993worker1 # [ 1.871747] systemd[1]: Finished Load Kernel Modules.1994lb # [ 1.874685] systemd[1]: Finished Load Kernel Modules.1995worker2 # [ 1.869960] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1996worker1 # [ 1.879916] systemd[1]: Starting Apply Kernel Variables...1997lb # [ 1.883226] systemd[1]: Starting Apply Kernel Variables...1998client # [ 1.753299] systemd[1]: Finished Create System Files and Directories.1999worker2 # [ 1.877599] systemd[1]: Starting Create Static Device Nodes in /dev...2000client # [ 1.759581] systemd-udevd[74]: Using default interface naming scheme 'v261'.
loading 2,555 hidden lines…
4556client # farm-top> LOG-farm-top-t1b4557worker1 # [ 46.547814] nix-daemon[847]: reaped child process 984, status = succeeded4558worker1 # [ 46.626153] nix-daemon[847]: accepted connection from pid 1000, user nix-grpc-daemon (trusted)4559worker1 # [ 46.639615] nix-daemon[847]: reaped child process 1003, status = succeeded4560lb # [ 46.646839] niks3-server[969]: 2026/09/23 05:33:38 INFO Received uploads request method=POST path=/api/pending_closures4561worker1 # [ 46.655884] nix-grpc-daemon[860]: time=2026-09-23T05:33:38.863Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4562worker1 # [ 46.657878] nix-grpc-daemon[860]: time=2026-09-23T05:33:38.864Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4563lb # [ 46.682057] niks3-server[969]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4564lb # [ 46.694184] niks3-server[969]: 2026/09/23 05:33:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4565lb # [ 46.698890] niks3-server[969]: 2026/09/23 05:33:38 INFO Signed narinfos id=8 count=14566worker1 # [ 46.704457] nix-grpc-daemon[860]: time=2026-09-23T05:33:38.911Z level=INFO msg="Uploading 1 narinfos"4567lb # [ 46.708076] niks3-server[969]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4568lb # [ 46.712148] niks3-server[969]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4569lb # [ 46.722238] niks3-server[969]: 2026/09/23 05:33:38 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4570lb # [ 46.727178] niks3-server[969]: 2026/09/23 05:33:38 INFO Completed upload id=84571worker1 # [ 46.727771] nix-grpc-daemon[860]: time=2026-09-23T05:33:38.935Z level=INFO msg="Upload complete. (210ms)"4572lb # [ 46.730079] niks3-server[969]: 2026/09/23 05:33:38 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4573worker1 # [ 46.733705] nix-grpc-daemon[780]: ts=2026-09-23T05:33:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14574worker1 # [ 46.748120] nix-daemon[847]: reaped child process 982, status = succeeded4575client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.48 seconds)4576client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4577client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.12 seconds)4578lb: must succeed: curl -sf localhost:9901/clusters4579lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4580lb: must succeed: curl -sf localhost:9901/clusters4581lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4582worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4583worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4584(finished: subtest: inputs already in the cache are not sent to the scheduler, in 1.80 seconds)4585subtest: outputs read back through any worker carry the cache signature4586client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24587client # don't know how to build these paths:4588client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4589worker2 # [ 47.103535] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480464590worker2 # [ 47.109305] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480464591worker2 # [ 47.126088] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04592worker2 # [ 47.132566] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.4:50051 clients=14593worker2 # [ 47.137348] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464594worker2 # [ 47.147714] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480464595client # copying 1 paths to 'lb:50051'...4596client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4597worker2 # [ 47.158827] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480464598worker2 # [ 47.169972] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4599worker2 # [ 47.189942] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=info event=niks3_push_started pid=9224600worker2 # [ 47.311889] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.661Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4601worker2 # [ 47.403548] nix-daemon[870]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4602worker2 # [ 47.415700] nix-daemon[870]: reaped child process 938, status = succeeded4603lb # [ 47.434543] niks3-server[969]: 2026/09/23 05:33:39 INFO Received uploads request method=POST path=/api/pending_closures4604worker2 # [ 47.431911] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.782Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4605worker2 # [ 47.433401] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.783Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4606lb # [ 47.471116] niks3-server[969]: 2026/09/23 05:33:39 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4607lb # [ 47.477389] niks3-server[969]: 2026/09/23 05:33:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4608lb # [ 47.484123] niks3-server[969]: 2026/09/23 05:33:39 INFO Signed narinfos id=9 count=14609worker2 # [ 47.476944] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.826Z level=INFO msg="Uploading 1 narinfos"4610lb # [ 47.498151] niks3-server[969]: 2026/09/23 05:33:39 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4611lb # [ 47.504943] niks3-server[969]: 2026/09/23 05:33:39 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4612lb # [ 47.511143] niks3-server[969]: 2026/09/23 05:33:39 INFO Completed upload id=94613worker2 # [ 47.500878] nix-grpc-daemon[922]: time=2026-09-23T05:33:39.851Z level=INFO msg="Upload complete. (184ms)"4614lb # [ 47.516463] niks3-server[969]: 2026/09/23 05:33:39 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4615worker2 # [ 47.504954] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=5044616worker2 # [ 47.510208] nix-grpc-daemon[783]: ts=2026-09-23T05:33:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464617worker2 # [ 47.518505] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4618worker2 # [ 47.523865] nix-daemon[870]: reaped child process 920, status = succeeded4619worker2 # [ 47.528458] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4620client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4621client # farm-a> worker2: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4622lb # [ 47.977509] envoy[929]: conn peer=[2001:db8:1::1]:39270 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=26504623lb # [ 47.981360] envoy[929]: conn peer=[2001:db8:1::1]:39286 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=644624lb # [ 47.984733] envoy[929]: conn peer=[2001:db8:1::1]:39288 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=74625lb # [ 47.988687] envoy[929]: conn peer=[2001:db8:1::1]:39292 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=84626lb # [ 47.992441] envoy[929]: conn peer=[2001:db8:1::1]:39302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4381 ms=51544627lb # [ 47.996063] envoy[929]: conn peer=[2001:db8:1::1]:59614 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=494628lb # [ 47.999445] envoy[929]: conn peer=[2001:db8:1::1]:59624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4178 tx=1872 ms=13174629worker1 # [ 48.166756] nix-grpc-daemon[780]: E0000 00:00:1790141620.372780 822 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4630worker1 # [ 48.196635] nix-grpc-daemon[780]: E0000 00:00:1790141620.403724 822 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4631worker1 # [ 48.207134] nix-grpc-daemon[780]: E0000 00:00:1790141620.409309 822 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4632worker2 # [ 48.213741] nix-grpc-daemon[783]: E0000 00:00:1790141620.562877 825 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4633worker1 # [ 48.228063] nix-grpc-daemon[780]: E0000 00:00:1790141620.435355 822 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4634worker2 # [ 48.245131] nix-grpc-daemon[783]: E0000 00:00:1790141620.594632 825 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4635worker2 # [ 48.254131] nix-grpc-daemon[783]: E0000 00:00:1790141620.601276 825 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4636worker2 # [ 48.276077] nix-grpc-daemon[783]: E0000 00:00:1790141620.626393 825 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4637client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4638client # farm-a> LOG-farm-a-sig4639worker2 # [ 48.948910] nix-daemon[870]: reaped child process 942, status = succeeded4640worker2 # [ 49.029042] nix-daemon[870]: accepted connection from pid 958, user nix-grpc-daemon (trusted)4641worker2 # [ 49.043165] nix-daemon[870]: reaped child process 961, status = succeeded4642lb # [ 49.063251] niks3-server[969]: 2026/09/23 05:33:41 INFO Received uploads request method=POST path=/api/pending_closures4643worker2 # [ 49.061615] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.411Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4644worker2 # [ 49.063128] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.413Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4645lb # [ 49.107770] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4646lb # [ 49.113368] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4647lb # [ 49.118533] niks3-server[969]: 2026/09/23 05:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4648lb # [ 49.123618] niks3-server[969]: 2026/09/23 05:33:41 INFO Signed narinfos id=10 count=14649worker2 # [ 49.113753] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.463Z level=INFO msg="Uploading 1 narinfos"4650lb # [ 49.129118] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4651lb # [ 49.139195] niks3-server[969]: 2026/09/23 05:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4652lb # [ 49.142955] niks3-server[969]: 2026/09/23 05:33:41 INFO Completed upload id=104653worker2 # [ 49.132385] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.482Z level=INFO msg="Upload complete. (195ms)"4654lb # [ 49.144576] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4655worker2 # [ 49.138262] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14656worker2 # [ 49.147874] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04657worker2 # [ 49.151652] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=14658worker2 # [ 49.157312] nix-daemon[870]: reaped child process 940, status = succeeded4659worker2 # [ 49.191778] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464660worker2 # [ 49.203952] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480464661worker2 # [ 49.216310] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.566Z level=INFO msg="All 1 paths already cached"4662client # copying 1 paths to 'lb:50051'...4663client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4664worker2 # [ 49.225523] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480464665worker2 # [ 49.235136] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4666worker2 # [ 49.337881] nix-daemon[870]: accepted connection from pid 967, user nix-grpc-daemon (trusted)4667worker2 # [ 49.354064] nix-daemon[870]: reaped child process 970, status = succeeded4668lb # [ 49.371642] niks3-server[969]: 2026/09/23 05:33:41 INFO Received uploads request method=POST path=/api/pending_closures4669worker2 # [ 49.369056] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.719Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4670worker2 # [ 49.371162] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.720Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4671lb # [ 49.402128] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4672lb # [ 49.410476] niks3-server[969]: 2026/09/23 05:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4673lb # [ 49.415495] niks3-server[969]: 2026/09/23 05:33:41 INFO Signed narinfos id=11 count=14674worker2 # [ 49.408349] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.758Z level=INFO msg="Uploading 1 narinfos"4675lb # [ 49.431059] niks3-server[969]: 2026/09/23 05:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4676lb # [ 49.438220] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4677lb # [ 49.447108] niks3-server[969]: 2026/09/23 05:33:41 INFO Completed upload id=114678worker2 # [ 49.436778] nix-grpc-daemon[922]: time=2026-09-23T05:33:41.787Z level=INFO msg="Upload complete. (190ms)"4679lb # [ 49.449231] niks3-server[969]: 2026/09/23 05:33:41 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4680worker2 # [ 49.442144] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=6244681worker2 # [ 49.448309] nix-grpc-daemon[783]: ts=2026-09-23T05:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464682worker2 # [ 49.459640] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4683worker2 # [ 49.467484] nix-daemon[870]: reaped child process 965, status = succeeded4684worker2 # [ 49.472674] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4685client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4686client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4687client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4688client # farm-b> LOG-farm-b-sig4689worker2 # [ 50.713670] nix-daemon[870]: reaped child process 974, status = succeeded4690worker2 # [ 50.796905] nix-daemon[870]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4691worker2 # [ 50.809817] nix-daemon[870]: reaped child process 993, status = succeeded4692lb # [ 50.829258] niks3-server[969]: 2026/09/23 05:33:43 INFO Received uploads request method=POST path=/api/pending_closures4693worker2 # [ 50.827869] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.178Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4694worker2 # [ 50.829305] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.179Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4695lb # [ 50.863076] niks3-server[969]: 2026/09/23 05:33:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4696lb # [ 50.865661] niks3-server[969]: 2026/09/23 05:33:43 INFO Signed narinfos id=12 count=14697worker2 # [ 50.855163] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.205Z level=INFO msg="Uploading 1 narinfos"4698lb # [ 50.870614] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4699lb # [ 50.873479] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4700lb # [ 50.882044] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4701lb # [ 50.884439] niks3-server[969]: 2026/09/23 05:33:43 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4702lb # [ 50.888556] niks3-server[969]: 2026/09/23 05:33:43 INFO Completed upload id=124703worker2 # [ 50.879319] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.229Z level=INFO msg="Upload complete. (198ms)"4704lb # [ 50.891514] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4705worker2 # [ 50.883841] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14706worker2 # [ 50.889783] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04707worker2 # [ 50.891835] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=14708worker2 # [ 50.897399] nix-daemon[870]: reaped child process 972, status = succeeded4709worker2 # [ 50.933655] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464710worker2 # [ 50.942742] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480464711worker2 # [ 50.951411] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.301Z level=INFO msg="All 2 paths already cached"4712client # copying 1 paths to 'lb:50051'...4713client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4714worker2 # [ 50.960524] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480464715worker2 # [ 50.967467] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4716worker2 # [ 51.066430] nix-daemon[870]: accepted connection from pid 999, user nix-grpc-daemon (trusted)4717worker2 # [ 51.081804] nix-daemon[870]: reaped child process 1002, status = succeeded4718lb # [ 51.098889] niks3-server[969]: 2026/09/23 05:33:43 INFO Received uploads request method=POST path=/api/pending_closures4719worker2 # [ 51.097283] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.447Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4720worker2 # [ 51.098797] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.449Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4721lb # [ 51.131859] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4722lb # [ 51.137109] niks3-server[969]: 2026/09/23 05:33:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4723lb # [ 51.141316] niks3-server[969]: 2026/09/23 05:33:43 INFO Signed narinfos id=13 count=14724worker2 # [ 51.133311] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.483Z level=INFO msg="Uploading 1 narinfos"4725lb # [ 51.152500] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4726lb # [ 51.156056] niks3-server[969]: 2026/09/23 05:33:43 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4727lb # [ 51.159385] niks3-server[969]: 2026/09/23 05:33:43 INFO Completed upload id=134728worker2 # [ 51.150069] nix-grpc-daemon[922]: time=2026-09-23T05:33:43.500Z level=INFO msg="Upload complete. (171ms)"4729lb # [ 51.164274] niks3-server[969]: 2026/09/23 05:33:43 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4730worker2 # [ 51.152645] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=7524731worker2 # [ 51.157525] nix-grpc-daemon[783]: ts=2026-09-23T05:33:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464732worker2 # [ 51.166060] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4733worker2 # [ 51.170229] nix-daemon[870]: reaped child process 997, status = succeeded4734worker2 # [ 51.175683] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4735client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4736client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4737client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4738client # farm-top> LOG-farm-top-sig4739worker2 # [ 52.620269] nix-daemon[870]: reaped child process 1006, status = succeeded4740worker2 # [ 52.704099] nix-daemon[870]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)4741worker2 # [ 52.718169] nix-daemon[870]: reaped child process 1025, status = succeeded4742lb # [ 52.736296] niks3-server[969]: 2026/09/23 05:33:44 INFO Received uploads request method=POST path=/api/pending_closures4743worker2 # [ 52.734847] nix-grpc-daemon[922]: time=2026-09-23T05:33:45.085Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4744worker2 # [ 52.737957] nix-grpc-daemon[922]: time=2026-09-23T05:33:45.086Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4745lb # [ 52.777985] niks3-server[969]: 2026/09/23 05:33:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4746worker2 # [ 52.779451] nix-grpc-daemon[922]: time=2026-09-23T05:33:45.129Z level=INFO msg="Uploading 1 narinfos"4747lb # [ 52.789829] niks3-server[969]: 2026/09/23 05:33:45 INFO Signed narinfos id=14 count=14748lb # [ 52.793311] niks3-server[969]: 2026/09/23 05:33:45 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4749lb # [ 52.799695] niks3-server[969]: 2026/09/23 05:33:45 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4750lb # [ 52.803820] niks3-server[969]: 2026/09/23 05:33:45 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4751lb # [ 52.815513] niks3-server[969]: 2026/09/23 05:33:45 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4752lb # [ 52.820414] niks3-server[969]: 2026/09/23 05:33:45 INFO Completed upload id=144753worker2 # [ 52.810221] nix-grpc-daemon[922]: time=2026-09-23T05:33:45.160Z level=INFO msg="Upload complete. (225ms)"4754lb # [ 52.822820] niks3-server[969]: 2026/09/23 05:33:45 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo4755worker2 # [ 52.818812] nix-grpc-daemon[783]: ts=2026-09-23T05:33:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=14756worker2 # [ 52.833559] nix-daemon[870]: reaped child process 1004, status = succeeded4757client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.87 seconds)4758client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath4759client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds)4760client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4761worker1 # [ 53.080804] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:589084762client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4763client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a4764client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b4765client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4766worker1 # [ 53.124864] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:589084767worker1 # [ 53.131626] nix-grpc-daemon[780]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...4768worker1 # [ 53.146464] nix-grpc-daemon[780]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...4769worker1 # [ 53.158547] nix-grpc-daemon[780]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...4770client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.21 seconds)4771client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4772worker2 # [ 53.299250] nix-grpc-daemon[783]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:472384773worker2 # [ 53.341903] nix-grpc-daemon[783]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:472384774client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.22 seconds)4775client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4776client # finding garbage collector roots...4777client # removing stale temporary roots file "/nix/var/nix/temproots/804"4778client # removing stale temporary roots file "/nix/var/nix/temproots/841"4779client # removing stale temporary roots file "/nix/var/nix/temproots/897"4780client # removing stale temporary roots file "/nix/var/nix/temproots/820"4781client # removing stale temporary roots file "/nix/var/nix/temproots/861"4782client # removing stale temporary roots file "/nix/var/nix/temproots/882"4783client # removing stale temporary roots file "/nix/var/nix/temproots/874"4784client # removing stale temporary roots file "/nix/var/nix/temproots/853"4785client # deleting specified paths...4786client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'4787client # deleting unused links...4788client # note: hard linking is currently saving 0.0 KiB4789worker1 # [ 53.664758] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569364790worker1 # [ 53.671075] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569364791worker1 # [ 53.676277] nix-grpc-daemon[860]: time=2026-09-23T05:33:45.882Z level=INFO msg="All 1 paths already cached"4792worker1 # [ 53.682672] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:569364793worker1 # [ 53.689333] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:569364794worker1 # [ 53.695989] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:569364795client # copying 3 paths...4796worker2 # [ 53.696725] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:480464797client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...4798worker2 # [ 53.712793] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:480464799worker2 # [ 53.718215] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=2324800worker2 # [ 53.724721] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:480464801worker1 # [ 53.739121] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:569364802worker2 # [ 53.728473] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=1764803worker1 # [ 53.743492] nix-grpc-daemon[780]: ts=2026-09-23T05:33:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_out=1284804client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...4805client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...4806client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.41 seconds)4807(finished: subtest: outputs read back through any worker carry the cache signature, in 6.84 seconds)4808subtest: two clients wanting the same drv build it once4809client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4810client # [ 53.861634] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4811client # Running as unit: dedup1.service; invocation ID: dc4e3751560f485d9ab65d464c10fde64812client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)4813client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4814client # [ 53.989710] nix[950]: don't know how to build these paths:4815client # [ 53.990669] nix[950]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4816worker1 # [ 53.988384] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569364817worker2 # [ 53.987987] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04818worker1 # [ 54.002186] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569364819worker2 # [ 53.992990] nix-grpc-daemon[783]: ts=2026-09-23T05:33:46Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=14820worker1 # [ 54.013754] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569364821client # [ 54.023691] nix[950]: copying 1 paths to 'lb:50051'...4822client # [ 54.024705] nix[950]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4823worker1 # [ 54.024934] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569364824worker1 # [ 54.036102] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4825worker1 # [ 54.142756] nix-daemon[847]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4826worker1 # [ 54.156704] nix-daemon[847]: reaped child process 1014, status = succeeded4827lb # [ 54.163773] niks3-server[969]: 2026/09/23 05:33:46 INFO Received uploads request method=POST path=/api/pending_closures4828worker1 # [ 54.173529] nix-grpc-daemon[860]: time=2026-09-23T05:33:46.380Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4829worker1 # [ 54.174966] nix-grpc-daemon[860]: time=2026-09-23T05:33:46.382Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4830lb # [ 54.195617] niks3-server[969]: 2026/09/23 05:33:46 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4831lb # [ 54.202246] niks3-server[969]: 2026/09/23 05:33:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4832worker1 # [ 54.211271] nix-grpc-daemon[860]: time=2026-09-23T05:33:46.418Z level=INFO msg="Uploading 1 narinfos"4833lb # [ 54.210881] niks3-server[969]: 2026/09/23 05:33:46 INFO Signed narinfos id=15 count=14834lb # [ 54.215703] niks3-server[969]: 2026/09/23 05:33:46 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4835lb # [ 54.228126] niks3-server[969]: 2026/09/23 05:33:46 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4836lb # [ 54.234757] niks3-server[969]: 2026/09/23 05:33:46 INFO Completed upload id=154837worker1 # [ 54.235282] nix-grpc-daemon[860]: time=2026-09-23T05:33:46.442Z level=INFO msg="Upload complete. (187ms)"4838lb # [ 54.237374] niks3-server[969]: 2026/09/23 05:33:46 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4839worker1 # [ 54.239674] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=4324840worker1 # [ 54.245254] nix-grpc-daemon[780]: ts=2026-09-23T05:33:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569364841worker1 # [ 54.253873] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4842worker1 # [ 54.262255] nix-daemon[847]: reaped child process 1009, status = succeeded4843worker1 # [ 54.267415] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4844client # [ 54.266352] nix[950]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4845client # [ 54.916175] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4846client # Running as unit: dedup2.service; invocation ID: ed332e3a3f944750b206fd51a2aa0bd34847client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.06 seconds)4848client # [ 55.081331] nix[966]: don't know how to build these paths:4849worker2 # [ 55.068808] nix-grpc-daemon[783]: ts=2026-09-23T05:33:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480464850client # [ 55.082154] nix[966]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4851worker2 # [ 55.080667] nix-grpc-daemon[783]: ts=2026-09-23T05:33:47Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04852worker1 # [ 55.094846] nix-grpc-daemon[780]: ts=2026-09-23T05:33:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569364853client # [ 55.099719] nix[966]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4854client: must succeed: sleep 24855lb # [ 57.978616] envoy[929]: conn peer=[2001:db8:1::1]:59628 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9507 tx=4354 ms=57254856lb # [ 57.982493] envoy[929]: conn peer=[2001:db8:1::1]:59888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2928 ms=1534857lb # [ 57.986259] envoy[929]: conn peer=[2001:db8:1::1]:59902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=784858lb # [ 57.990294] envoy[929]: conn peer=[2001:db8:1::1]:59896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=377 ms=864859lb # [ 57.993762] envoy[929]: conn peer=[2001:db8:1::1]:59892 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=934860client: (finished: must succeed: sleep 2, in 2.04 seconds)4861client: waiting for success: ! systemctl is-active dedup1 dedup24862worker1 # [ 58.285735] nix-daemon[847]: reaped child process 1018, status = succeeded4863worker1 # [ 58.362878] nix-daemon[847]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)4864worker1 # [ 58.373963] nix-daemon[847]: reaped child process 1052, status = succeeded4865lb # [ 58.383446] niks3-server[969]: 2026/09/23 05:33:50 INFO Received uploads request method=POST path=/api/pending_closures4866worker1 # [ 58.391342] nix-grpc-daemon[860]: time=2026-09-23T05:33:50.598Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4867worker1 # [ 58.392933] nix-grpc-daemon[860]: time=2026-09-23T05:33:50.600Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4868lb # [ 58.436554] niks3-server[969]: 2026/09/23 05:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4869worker1 # [ 58.441096] nix-grpc-daemon[860]: time=2026-09-23T05:33:50.648Z level=INFO msg="Uploading 1 narinfos"4870lb # [ 58.441888] niks3-server[969]: 2026/09/23 05:33:50 INFO Signed narinfos id=16 count=14871lb # [ 58.443935] niks3-server[969]: 2026/09/23 05:33:50 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4872lb # [ 58.452259] niks3-server[969]: 2026/09/23 05:33:50 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4873lb # [ 58.457687] niks3-server[969]: 2026/09/23 05:33:50 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4874lb # [ 58.463488] niks3-server[969]: 2026/09/23 05:33:50 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4875lb # [ 58.467905] niks3-server[969]: 2026/09/23 05:33:50 INFO Completed upload id=164876worker1 # [ 58.468422] nix-grpc-daemon[860]: time=2026-09-23T05:33:50.675Z level=INFO msg="Upload complete. (189ms)"4877lb # [ 58.473837] niks3-server[969]: 2026/09/23 05:33:50 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4878worker1 # [ 58.474818] nix-grpc-daemon[780]: ts=2026-09-23T05:33:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14879worker1 # [ 58.482614] nix-grpc-daemon[780]: ts=2026-09-23T05:33:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14880worker1 # [ 58.494179] nix-daemon[847]: reaped child process 1016, status = succeeded4881client # [ 58.505774] systemd[1]: dedup1.service: Deactivated successfully.4882client # [ 58.510279] systemd[1]: dedup1.service: Consumed 113ms CPU time over 4.646s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.4883client # [ 58.546112] systemd[1]: dedup2.service: Deactivated successfully.4884client # [ 58.548926] systemd[1]: dedup2.service: Consumed 135ms CPU time over 3.632s wall clock time, 33.4M memory peak, 6.1K incoming IP traffic, 5.8K outgoing IP traffic.4885client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4886client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4887client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4888worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4889worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4890worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4891worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4892worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4893worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4894worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4895worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4896(finished: subtest: two clients wanting the same drv build it once, in 5.74 seconds)4897subtest: --timeout and --max-silent-time reach the builder4898worker2 # [ 59.711918] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:480464899worker2 # [ 59.724231] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=04900worker2 # [ 59.728153] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=14901worker2 # [ 59.735755] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464902worker2 # [ 59.748059] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480464903worker2 # [ 59.759803] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480464904worker2 # [ 59.770809] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4905worker2 # [ 59.874434] nix-daemon[870]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)4906worker2 # [ 59.889778] nix-daemon[870]: reaped child process 1059, status = succeeded4907lb # [ 59.908437] niks3-server[969]: 2026/09/23 05:33:52 INFO Received uploads request method=POST path=/api/pending_closures4908worker2 # [ 59.906614] nix-grpc-daemon[922]: time=2026-09-23T05:33:52.256Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4909worker2 # [ 59.908626] nix-grpc-daemon[922]: time=2026-09-23T05:33:52.258Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"4910lb # [ 59.937888] niks3-server[969]: 2026/09/23 05:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4911worker2 # [ 59.935545] nix-grpc-daemon[922]: time=2026-09-23T05:33:52.285Z level=INFO msg="Uploading 1 narinfos"4912lb # [ 59.947713] niks3-server[969]: 2026/09/23 05:33:52 INFO Signed narinfos id=17 count=14913lb # [ 59.953094] niks3-server[969]: 2026/09/23 05:33:52 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst4914lb # [ 59.959147] niks3-server[969]: 2026/09/23 05:33:52 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls4915lb # [ 59.973637] niks3-server[969]: 2026/09/23 05:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4916worker2 # [ 59.969535] nix-grpc-daemon[922]: time=2026-09-23T05:33:52.319Z level=INFO msg="Upload complete. (187ms)"4917lb # [ 59.979957] niks3-server[969]: 2026/09/23 05:33:52 INFO Completed upload id=174918lb # [ 59.983135] niks3-server[969]: 2026/09/23 05:33:52 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo4919worker2 # [ 59.972731] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=4244920worker2 # [ 59.977701] nix-grpc-daemon[783]: ts=2026-09-23T05:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464921worker2 # [ 59.986521] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4922worker2 # [ 59.993471] nix-daemon[870]: reaped child process 1054, status = succeeded4923worker2 # [ 59.998616] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4924worker2 # [ 65.810172] nix-grpc-daemon[783]: ts=2026-09-23T05:33:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=5 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=04925worker2 # [ 65.821623] nix-daemon[870]: reaped child process 1061, status = succeeded4926worker2 # [ 65.822692] nix-daemon[870]: reaped child process 1063, status = succeeded4927worker1 # [ 66.025577] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569364928worker2 # [ 66.024222] nix-grpc-daemon[783]: ts=2026-09-23T05:33:58Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=04929worker2 # [ 66.028589] nix-grpc-daemon[783]: ts=2026-09-23T05:33:58Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=14930worker1 # [ 66.047059] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569364931worker1 # [ 66.059680] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569364932worker1 # [ 66.070922] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569364933worker1 # [ 66.081450] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4934worker1 # [ 66.186748] nix-daemon[847]: accepted connection from pid 1065, user nix-grpc-daemon (trusted)4935worker1 # [ 66.200587] nix-daemon[847]: reaped child process 1068, status = succeeded4936lb # [ 66.208224] niks3-server[969]: 2026/09/23 05:33:58 INFO Received uploads request method=POST path=/api/pending_closures4937worker1 # [ 66.217731] nix-grpc-daemon[860]: time=2026-09-23T05:33:58.424Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4938worker1 # [ 66.219854] nix-grpc-daemon[860]: time=2026-09-23T05:33:58.426Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"4939lb # [ 66.240303] niks3-server[969]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst4940lb # [ 66.253254] niks3-server[969]: 2026/09/23 05:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4941worker1 # [ 66.261300] nix-grpc-daemon[860]: time=2026-09-23T05:33:58.468Z level=INFO msg="Uploading 1 narinfos"4942lb # [ 66.260968] niks3-server[969]: 2026/09/23 05:33:58 INFO Signed narinfos id=18 count=14943lb # [ 66.267242] niks3-server[969]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls4944lb # [ 66.278446] niks3-server[969]: 2026/09/23 05:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4945lb # [ 66.285748] niks3-server[969]: 2026/09/23 05:33:58 INFO Completed upload id=184946lb # [ 66.288694] niks3-server[969]: 2026/09/23 05:33:58 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo4947worker1 # [ 66.288385] nix-grpc-daemon[860]: time=2026-09-23T05:33:58.495Z level=INFO msg="Upload complete. (195ms)"4948worker1 # [ 66.294514] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=4244949worker1 # [ 66.300495] nix-grpc-daemon[780]: ts=2026-09-23T05:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569364950worker1 # [ 66.312109] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4951worker1 # [ 66.320189] nix-daemon[847]: reaped child process 1063, status = succeeded4952worker1 # [ 66.327453] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4953lb # [ 67.979166] envoy[929]: conn peer=[2001:db8:1::1]:59912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1543 ms=44994954lb # [ 67.982880] envoy[929]: conn peer=[2001:db8:1::1]:59924 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1197 ms=34614955lb # [ 67.986429] envoy[929]: conn peer=[2001:db8:1::1]:59932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=935 ms=61594956worker1 # [ 72.341637] nix-grpc-daemon[780]: ts=2026-09-23T05:34:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=04957worker1 # [ 72.366355] nix-daemon[847]: reaped child process 1070, status = succeeded4958worker1 # [ 72.368727] nix-daemon[847]: reaped child process 1072, status = succeeded4959(finished: subtest: --timeout and --max-silent-time reach the builder, in 12.93 seconds)4960subtest: the first client leaving does not fail the second4961client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share4962client # [ 72.552436] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.4963client # Running as unit: share1.service; invocation ID: 35d75a25c14447419755cf8ccedbdda84964client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)4965client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share4966worker1 # [ 72.689267] nix-grpc-daemon[780]: ts=2026-09-23T05:34:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569364967client # [ 72.695192] nix[1037]: don't know how to build these paths:4968client # [ 72.696067] nix[1037]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv4969worker2 # [ 72.693812] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=04970worker2 # [ 72.699810] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=14971worker2 # [ 72.704312] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464972worker2 # [ 72.715632] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480464973client # [ 72.736033] nix[1037]: copying 1 paths to 'lb:50051'...4974client # [ 72.736872] nix[1037]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...4975worker2 # [ 72.726163] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480464976worker2 # [ 72.737087] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4977worker2 # [ 72.836082] nix-daemon[870]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4978worker2 # [ 72.848835] nix-daemon[870]: reaped child process 1093, status = succeeded4979lb # [ 72.868835] niks3-server[969]: 2026/09/23 05:34:05 INFO Received uploads request method=POST path=/api/pending_closures4980worker2 # [ 72.864878] nix-grpc-daemon[922]: time=2026-09-23T05:34:05.215Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4981worker2 # [ 72.866772] nix-grpc-daemon[922]: time=2026-09-23T05:34:05.216Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"4982lb # [ 72.897128] niks3-server[969]: 2026/09/23 05:34:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4983worker2 # [ 72.894993] nix-grpc-daemon[922]: time=2026-09-23T05:34:05.245Z level=INFO msg="Uploading 1 narinfos"4984lb # [ 72.906134] niks3-server[969]: 2026/09/23 05:34:05 INFO Signed narinfos id=19 count=14985lb # [ 72.919070] niks3-server[969]: 2026/09/23 05:34:05 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls4986lb # [ 72.923547] niks3-server[969]: 2026/09/23 05:34:05 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst4987lb # [ 72.934060] niks3-server[969]: 2026/09/23 05:34:05 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4988lb # [ 72.939116] niks3-server[969]: 2026/09/23 05:34:05 INFO Completed upload id=194989worker2 # [ 72.928496] nix-grpc-daemon[922]: time=2026-09-23T05:34:05.278Z level=INFO msg="Upload complete. (179ms)"4990lb # [ 72.942374] niks3-server[969]: 2026/09/23 05:34:05 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo4991worker2 # [ 72.933143] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=4324992worker2 # [ 72.940112] nix-grpc-daemon[783]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480464993worker2 # [ 72.948805] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4994worker2 # [ 72.959152] nix-daemon[870]: reaped child process 1088, status = succeeded4995worker2 # [ 72.963090] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4996client # [ 72.972792] nix[1037]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...4997client # [ 73.618861] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.4998client # Running as unit: share2.service; invocation ID: 8b3e7b28e3224dbe8549663f3e4e2d9e4999client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5000client # [ 73.765774] nix[1053]: don't know how to build these paths:5001client # [ 73.766663] nix[1053]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5002worker1 # [ 73.764873] nix-grpc-daemon[780]: ts=2026-09-23T05:34:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569365003worker2 # [ 73.760910] nix-grpc-daemon[783]: ts=2026-09-23T05:34:06Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05004client # [ 73.774619] nix[1053]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5005worker2 # [ 73.764267] nix-grpc-daemon[783]: ts=2026-09-23T05:34:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480465006client: must succeed: sleep 2; systemctl kill -s INT share15007client # [ 76.884480] systemd[1]: share1.service: Sent signal SIGINT to main process 1037 (nix) on client request.5008client # [ 76.892338] nix[1037]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5009client # [ 76.898071] nix[1037]: error: interrupted by the user5010client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5011client: waiting for success: ! systemctl is-active share15012client # [ 76.941261] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5013client # [ 76.942713] systemd[1]: share1.service: Failed with result 'exit-code'.5014client # [ 76.944617] systemd[1]: share1.service: Consumed 114ms CPU time over 4.392s wall clock time, 41.5M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.5015client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5016lb # [ 77.979954] envoy[929]: conn peer=[2001:db8:1::1]:41294 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=957 ms=63695017lb # [ 77.983801] envoy[929]: conn peer=[2001:db8:1::1]:41308 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=772 ms=42205018client: waiting for success: ! systemctl is-active share25019worker2 # [ 78.181606] nix-daemon[870]: reaped child process 1097, status = succeeded5020worker2 # [ 78.256556] nix-daemon[870]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)5021worker2 # [ 78.269206] nix-daemon[870]: reaped child process 1136, status = succeeded5022lb # [ 78.288467] niks3-server[969]: 2026/09/23 05:34:10 INFO Received uploads request method=POST path=/api/pending_closures5023worker2 # [ 78.287685] nix-grpc-daemon[922]: time=2026-09-23T05:34:10.637Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5024worker2 # [ 78.289422] nix-grpc-daemon[922]: time=2026-09-23T05:34:10.639Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5025lb # [ 78.345741] niks3-server[969]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5026lb # [ 78.350528] niks3-server[969]: 2026/09/23 05:34:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5027lb # [ 78.355553] niks3-server[969]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5028worker2 # [ 78.351515] nix-grpc-daemon[922]: time=2026-09-23T05:34:10.701Z level=INFO msg="Uploading 1 narinfos"5029lb # [ 78.361980] niks3-server[969]: 2026/09/23 05:34:10 INFO Signed narinfos id=20 count=15030lb # [ 78.368437] niks3-server[969]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5031lb # [ 78.374828] niks3-server[969]: 2026/09/23 05:34:10 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5032lb # [ 78.379277] niks3-server[969]: 2026/09/23 05:34:10 INFO Completed upload id=205033worker2 # [ 78.370104] nix-grpc-daemon[922]: time=2026-09-23T05:34:10.720Z level=INFO msg="Upload complete. (197ms)"5034lb # [ 78.381057] niks3-server[969]: 2026/09/23 05:34:10 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5035worker2 # [ 78.373627] nix-grpc-daemon[783]: ts=2026-09-23T05:34:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15036worker2 # [ 78.377176] nix-grpc-daemon[783]: ts=2026-09-23T05:34:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15037worker2 # [ 78.382713] nix-daemon[870]: reaped child process 1095, status = succeeded5038client # [ 78.457858] systemd[1]: share2.service: Deactivated successfully.5039client # [ 78.461880] systemd[1]: share2.service: Consumed 120ms CPU time over 4.842s wall clock time, 35.4M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.5040client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5041client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5042client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds)5043(finished: subtest: the first client leaving does not fail the second, in 6.76 seconds)5044subtest: build hook: nix-daemon with builders = grpc://lb5045client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5046client # [ 79.359828] systemd[1]: Started Nix Daemon.5047client # [ 79.449487] nix-daemon[1104]: accepted connection from pid 1101, user root (trusted)5048worker1 # [ 79.565278] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:569365049worker1 # [ 79.570654] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569365050worker2 # [ 79.568500] nix-grpc-daemon[783]: ts=2026-09-23T05:34:11Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05051worker2 # [ 79.570665] nix-grpc-daemon[783]: ts=2026-09-23T05:34:11Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15052worker1 # [ 79.583943] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569365053worker1 # [ 79.595683] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569365054worker1 # [ 79.606741] nix-grpc-daemon[780]: ts=2026-09-23T05:34:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569365055worker1 # [ 79.617925] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5056worker1 # [ 79.718186] nix-daemon[847]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5057worker1 # [ 79.730921] nix-daemon[847]: reaped child process 1114, status = succeeded5058lb # [ 79.738678] niks3-server[969]: 2026/09/23 05:34:11 INFO Received uploads request method=POST path=/api/pending_closures5059worker1 # [ 79.748067] nix-grpc-daemon[860]: time=2026-09-23T05:34:11.955Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5060worker1 # [ 79.749984] nix-grpc-daemon[860]: time=2026-09-23T05:34:11.956Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5061lb # [ 79.768842] niks3-server[969]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5062lb # [ 79.778058] niks3-server[969]: 2026/09/23 05:34:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5063lb # [ 79.783803] niks3-server[969]: 2026/09/23 05:34:12 INFO Signed narinfos id=21 count=15064worker1 # [ 79.787470] nix-grpc-daemon[860]: time=2026-09-23T05:34:11.994Z level=INFO msg="Uploading 1 narinfos"5065lb # [ 79.794136] niks3-server[969]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5066lb # [ 79.803751] niks3-server[969]: 2026/09/23 05:34:12 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5067lb # [ 79.810532] niks3-server[969]: 2026/09/23 05:34:12 INFO Completed upload id=215068worker1 # [ 79.810833] nix-grpc-daemon[860]: time=2026-09-23T05:34:12.018Z level=INFO msg="Upload complete. (181ms)"5069lb # [ 79.813457] niks3-server[969]: 2026/09/23 05:34:12 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5070worker1 # [ 79.815188] nix-grpc-daemon[780]: ts=2026-09-23T05:34:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=5125071worker1 # [ 79.821193] nix-grpc-daemon[780]: ts=2026-09-23T05:34:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569365072worker1 # [ 79.829706] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5073worker1 # [ 79.837039] nix-daemon[847]: reaped child process 1109, status = succeeded5074worker1 # [ 79.842732] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5075worker1 # [ 81.080554] nix-daemon[847]: reaped child process 1118, status = succeeded5076worker1 # [ 81.164955] nix-daemon[847]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)5077worker1 # [ 81.177598] nix-daemon[847]: reaped child process 1137, status = succeeded5078lb # [ 81.183467] niks3-server[969]: 2026/09/23 05:34:13 INFO Received uploads request method=POST path=/api/pending_closures5079worker1 # [ 81.193398] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.400Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5080worker1 # [ 81.195614] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.402Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5081lb # [ 81.230234] niks3-server[969]: 2026/09/23 05:34:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5082worker1 # [ 81.239488] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.446Z level=INFO msg="Uploading 1 narinfos"5083lb # [ 81.240654] niks3-server[969]: 2026/09/23 05:34:13 INFO Signed narinfos id=22 count=15084lb # [ 81.248690] niks3-server[969]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5085lb # [ 81.251761] niks3-server[969]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5086lb # [ 81.256797] niks3-server[969]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5087lb # [ 81.265054] niks3-server[969]: 2026/09/23 05:34:13 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5088lb # [ 81.268474] niks3-server[969]: 2026/09/23 05:34:13 INFO Completed upload id=225089worker1 # [ 81.269229] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.476Z level=INFO msg="Upload complete. (219ms)"5090lb # [ 81.271393] niks3-server[969]: 2026/09/23 05:34:13 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5091worker1 # [ 81.275864] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15092worker1 # [ 81.289814] nix-daemon[847]: reaped child process 1116, status = succeeded5093worker1 # [ 81.322809] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:569365094worker2 # [ 81.339517] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:480465095worker2 # [ 81.351771] nix-grpc-daemon[783]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5096worker2 # [ 81.366482] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=1285097worker2 # [ 81.571990] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:480465098worker2 # [ 81.578903] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480465099worker2 # [ 81.589377] nix-grpc-daemon[922]: time=2026-09-23T05:34:13.939Z level=INFO msg="All 1 paths already cached"5100worker2 # [ 81.603329] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05101worker2 # [ 81.607842] nix-grpc-daemon[783]: ts=2026-09-23T05:34:13Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15102worker1 # [ 81.621361] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569365103worker1 # [ 81.633229] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569365104worker1 # [ 81.645946] nix-grpc-daemon[860]: time=2026-09-23T05:34:13.853Z level=INFO msg="All 1 paths already cached"5105worker1 # [ 81.653376] nix-grpc-daemon[780]: ts=2026-09-23T05:34:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569365106worker1 # [ 81.661694] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5107worker1 # [ 81.768448] nix-daemon[847]: accepted connection from pid 1141, user nix-grpc-daemon (trusted)5108worker1 # [ 81.782938] nix-daemon[847]: reaped child process 1144, status = succeeded5109lb # [ 81.788394] niks3-server[969]: 2026/09/23 05:34:14 INFO Received uploads request method=POST path=/api/pending_closures5110worker1 # [ 81.796756] nix-grpc-daemon[860]: time=2026-09-23T05:34:14.004Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5111worker1 # [ 81.798299] nix-grpc-daemon[860]: time=2026-09-23T05:34:14.005Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5112lb # [ 81.822501] niks3-server[969]: 2026/09/23 05:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5113worker1 # [ 81.829837] nix-grpc-daemon[860]: time=2026-09-23T05:34:14.037Z level=INFO msg="Uploading 1 narinfos"5114lb # [ 81.830088] niks3-server[969]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5115lb # [ 81.833677] niks3-server[969]: 2026/09/23 05:34:14 INFO Signed narinfos id=23 count=15116lb # [ 81.840245] niks3-server[969]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5117lb # [ 81.852370] niks3-server[969]: 2026/09/23 05:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5118lb # [ 81.859415] niks3-server[969]: 2026/09/23 05:34:14 INFO Completed upload id=235119worker1 # [ 81.860105] nix-grpc-daemon[860]: time=2026-09-23T05:34:14.067Z level=INFO msg="Upload complete. (184ms)"5120lb # [ 81.865042] niks3-server[969]: 2026/09/23 05:34:14 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5121worker1 # [ 81.864435] nix-grpc-daemon[780]: ts=2026-09-23T05:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=6245122worker1 # [ 81.871078] nix-grpc-daemon[780]: ts=2026-09-23T05:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569365123worker1 # [ 81.881177] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5124worker1 # [ 81.884970] nix-daemon[847]: reaped child process 1139, status = succeeded5125worker1 # [ 81.889946] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5126worker1 # [ 82.915817] nix-daemon[847]: reaped child process 1148, status = succeeded5127worker1 # [ 82.992381] nix-daemon[847]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)5128worker1 # [ 83.006048] nix-daemon[847]: reaped child process 1167, status = succeeded5129lb # [ 83.011693] niks3-server[969]: 2026/09/23 05:34:15 INFO Received uploads request method=POST path=/api/pending_closures5130worker1 # [ 83.021148] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.228Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5131worker1 # [ 83.024930] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.229Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5132lb # [ 83.052170] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5133lb # [ 83.060260] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5134lb # [ 83.066788] niks3-server[969]: 2026/09/23 05:34:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5135lb # [ 83.073998] niks3-server[969]: 2026/09/23 05:34:15 INFO Signed narinfos id=24 count=15136worker1 # [ 83.074886] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.281Z level=INFO msg="Uploading 1 narinfos"5137lb # [ 83.080068] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5138lb # [ 83.090806] niks3-server[969]: 2026/09/23 05:34:15 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5139lb # [ 83.094634] niks3-server[969]: 2026/09/23 05:34:15 INFO Completed upload id=245140lb # [ 83.096217] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5141worker1 # [ 83.095295] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.302Z level=INFO msg="Upload complete. (198ms)"5142worker1 # [ 83.101757] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15143worker2 # [ 83.095810] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:480465144worker1 # [ 83.115878] nix-daemon[847]: reaped child process 1146, status = succeeded5145worker2 # [ 83.104387] nix-grpc-daemon[783]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5146worker1 # [ 83.143709] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:569365147worker1 # [ 83.148986] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_out=1765148worker2 # [ 83.302096] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:480465149worker2 # [ 83.310407] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480465150worker2 # [ 83.320250] nix-grpc-daemon[922]: time=2026-09-23T05:34:15.670Z level=INFO msg="All 2 paths already cached"5151worker2 # [ 83.334261] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05152worker2 # [ 83.338485] nix-grpc-daemon[783]: ts=2026-09-23T05:34:15Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15153worker1 # [ 83.357100] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569365154worker1 # [ 83.369861] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569365155worker1 # [ 83.382273] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.589Z level=INFO msg="All 2 paths already cached"5156worker1 # [ 83.390210] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569365157worker1 # [ 83.398537] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5158worker1 # [ 83.500727] nix-daemon[847]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5159worker1 # [ 83.513531] nix-daemon[847]: reaped child process 1174, status = succeeded5160lb # [ 83.520756] niks3-server[969]: 2026/09/23 05:34:15 INFO Received uploads request method=POST path=/api/pending_closures5161worker1 # [ 83.530385] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.737Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5162worker1 # [ 83.531816] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.739Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5163lb # [ 83.554108] niks3-server[969]: 2026/09/23 05:34:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5164worker1 # [ 83.561399] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.768Z level=INFO msg="Uploading 1 narinfos"5165lb # [ 83.562208] niks3-server[969]: 2026/09/23 05:34:15 INFO Signed narinfos id=25 count=15166lb # [ 83.567848] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5167lb # [ 83.573635] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5168lb # [ 83.587279] niks3-server[969]: 2026/09/23 05:34:15 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5169lb # [ 83.592372] niks3-server[969]: 2026/09/23 05:34:15 INFO Completed upload id=255170worker1 # [ 83.592599] nix-grpc-daemon[860]: time=2026-09-23T05:34:15.799Z level=INFO msg="Upload complete. (179ms)"5171worker1 # [ 83.596497] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_in=7525172lb # [ 83.599462] niks3-server[969]: 2026/09/23 05:34:15 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5173worker1 # [ 83.602388] nix-grpc-daemon[780]: ts=2026-09-23T05:34:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569365174worker1 # [ 83.612262] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5175worker1 # [ 83.620567] nix-daemon[847]: reaped child process 1169, status = succeeded5176worker1 # [ 83.625385] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5177worker1 # [ 84.439499] nix-daemon[847]: reaped child process 1178, status = succeeded5178worker1 # [ 84.515422] nix-daemon[847]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)5179worker1 # [ 84.528079] nix-daemon[847]: reaped child process 1197, status = succeeded5180lb # [ 84.537149] niks3-server[969]: 2026/09/23 05:34:16 INFO Received uploads request method=POST path=/api/pending_closures5181worker1 # [ 84.545592] nix-grpc-daemon[860]: time=2026-09-23T05:34:16.752Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5182worker1 # [ 84.547522] nix-grpc-daemon[860]: time=2026-09-23T05:34:16.754Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5183lb # [ 84.579936] niks3-server[969]: 2026/09/23 05:34:16 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5184lb # [ 84.586627] niks3-server[969]: 2026/09/23 05:34:16 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5185lb # [ 84.595472] niks3-server[969]: 2026/09/23 05:34:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5186worker1 # [ 84.601213] nix-grpc-daemon[860]: time=2026-09-23T05:34:16.808Z level=INFO msg="Uploading 1 narinfos"5187lb # [ 84.600955] niks3-server[969]: 2026/09/23 05:34:16 INFO Signed narinfos id=26 count=15188lb # [ 84.604552] niks3-server[969]: 2026/09/23 05:34:16 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5189lb # [ 84.614763] niks3-server[969]: 2026/09/23 05:34:16 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5190lb # [ 84.618169] niks3-server[969]: 2026/09/23 05:34:16 INFO Completed upload id=265191worker1 # [ 84.618860] nix-grpc-daemon[860]: time=2026-09-23T05:34:16.826Z level=INFO msg="Upload complete. (187ms)"5192lb # [ 84.620899] niks3-server[969]: 2026/09/23 05:34:16 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5193worker1 # [ 84.624870] nix-grpc-daemon[780]: ts=2026-09-23T05:34:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15194worker1 # [ 84.638263] nix-daemon[847]: reaped child process 1176, status = succeeded5195worker2 # [ 84.660720] nix-grpc-daemon[783]: ts=2026-09-23T05:34:17Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:480465196worker2 # [ 84.671877] nix-grpc-daemon[783]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5197worker1 # [ 84.711262] nix-grpc-daemon[780]: ts=2026-09-23T05:34:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:569365198worker1 # [ 84.716535] nix-grpc-daemon[780]: ts=2026-09-23T05:34:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=0 paths=1 nar_bytes_out=2325199client # [ 84.785416] nix-daemon[1104]: reaped child process 1106, status = succeeded5200client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.53 seconds)5201client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5202client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5203client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5204client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5205client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5206client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5207(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.58 seconds)5208subtest: an input only one worker has still reaches the builder5209client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5210client # this derivation will be built:5211client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5212client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5213client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.24 seconds)5214client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5215client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5216worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5217worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.12 seconds)5218worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5219worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5220client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25221client # [ 85.449275] nix-daemon[1104]: accepted connection from pid 1187, user root (trusted)5222client # this derivation will be built:5223client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5224client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5225worker2 # [ 85.557076] nix-grpc-daemon[783]: ts=2026-09-23T05:34:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:480465226worker2 # [ 85.562458] nix-grpc-daemon[783]: ts=2026-09-23T05:34:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480465227client # copying 1 paths...5228client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5229worker2 # [ 85.573785] nix-grpc-daemon[783]: ts=2026-09-23T05:34:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480465230worker2 # [ 85.585656] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5231worker2 # [ 85.691568] nix-daemon[870]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)5232worker2 # [ 85.705173] nix-daemon[870]: reaped child process 1155, status = succeeded5233lb # [ 85.724066] niks3-server[969]: 2026/09/23 05:34:17 INFO Received uploads request method=POST path=/api/pending_closures5234worker2 # [ 85.723811] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.073Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5235worker2 # [ 85.725305] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.073Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5236worker2 # [ 85.743549] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.093Z level=INFO msg="Uploading 1 narinfos"5237lb # [ 85.754375] niks3-server[969]: 2026/09/23 05:34:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5238lb # [ 85.757976] niks3-server[969]: 2026/09/23 05:34:18 INFO Signed narinfos id=27 count=15239lb # [ 85.766342] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5240lb # [ 85.778161] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5241lb # [ 85.784105] niks3-server[969]: 2026/09/23 05:34:18 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5242lb # [ 85.791253] niks3-server[969]: 2026/09/23 05:34:18 INFO Completed upload id=275243worker2 # [ 85.780801] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.131Z level=INFO msg="Upload complete. (184ms)"5244lb # [ 85.798581] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5245worker2 # [ 85.784964] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=1205246worker2 # [ 85.798609] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05247worker2 # [ 85.800638] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15248worker2 # [ 85.802740] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480465249worker2 # [ 85.806244] nix-daemon[870]: reaped child process 1150, status = succeeded5250worker2 # [ 85.809871] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480465251client # copying 2 paths to 'lb:50051'...5252worker2 # [ 85.819090] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480465253client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5254client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5255worker2 # [ 85.825678] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5256worker2 # [ 85.911111] nix-daemon[870]: accepted connection from pid 1161, user nix-grpc-daemon (trusted)5257worker2 # [ 85.924716] nix-daemon[870]: reaped child process 1164, status = succeeded5258lb # [ 85.942162] niks3-server[969]: 2026/09/23 05:34:18 INFO Received uploads request method=POST path=/api/pending_closures5259lb # [ 85.950851] niks3-server[969]: 2026/09/23 05:34:18 INFO Received uploads request method=POST path=/api/pending_closures5260worker2 # [ 85.949300] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.299Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5261worker2 # [ 85.950950] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.301Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5262worker2 # [ 85.954234] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.303Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5263lb # [ 85.997408] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5264lb # [ 86.000574] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5265lb # [ 86.003064] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5266lb # [ 86.006256] niks3-server[969]: 2026/09/23 05:34:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5267lb # [ 86.008321] niks3-server[969]: 2026/09/23 05:34:18 INFO Signed narinfos id=28 count=15268lb # [ 86.010548] niks3-server[969]: 2026/09/23 05:34:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5269worker2 # [ 86.000711] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.350Z level=INFO msg="Uploading 3 narinfos"5270lb # [ 86.012663] niks3-server[969]: 2026/09/23 05:34:18 INFO Signed narinfos id=29 count=25271lb # [ 86.015238] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5272lb # [ 86.027335] niks3-server[969]: 2026/09/23 05:34:18 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5273lb # [ 86.030988] niks3-server[969]: 2026/09/23 05:34:18 INFO Completed upload id=285274lb # [ 86.032293] niks3-server[969]: 2026/09/23 05:34:18 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5275lb # [ 86.035082] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5276lb # [ 86.037421] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5277lb # [ 86.040541] niks3-server[969]: 2026/09/23 05:34:18 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5278lb # [ 86.042388] niks3-server[969]: 2026/09/23 05:34:18 INFO Completed upload id=295279worker2 # [ 86.032093] nix-grpc-daemon[922]: time=2026-09-23T05:34:18.382Z level=INFO msg="Upload complete. (199ms)"5280worker2 # [ 86.037277] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=2 nar_bytes_in=9765281worker2 # [ 86.043549] nix-grpc-daemon[783]: ts=2026-09-23T05:34:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480465282worker2 # [ 86.054284] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5283worker2 # [ 86.064572] nix-daemon[870]: reaped child process 1159, status = succeeded5284worker2 # [ 86.069672] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5285client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5286client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5287client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5288worker2 # [ 86.908353] nix-daemon[870]: reaped child process 1168, status = succeeded5289worker2 # [ 86.989870] nix-daemon[870]: accepted connection from pid 1184, user nix-grpc-daemon (trusted)5290worker2 # [ 87.001885] nix-daemon[870]: reaped child process 1187, status = succeeded5291lb # [ 87.021791] niks3-server[969]: 2026/09/23 05:34:19 INFO Received uploads request method=POST path=/api/pending_closures5292worker2 # [ 87.019481] nix-grpc-daemon[922]: time=2026-09-23T05:34:19.369Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5293lb # [ 87.051611] niks3-server[969]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5294lb # [ 87.058638] niks3-server[969]: 2026/09/23 05:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5295worker2 # [ 87.054206] nix-grpc-daemon[922]: time=2026-09-23T05:34:19.404Z level=INFO msg="Uploading 1 narinfos"5296lb # [ 87.065348] niks3-server[969]: 2026/09/23 05:34:19 INFO Signed narinfos id=30 count=15297lb # [ 87.073866] niks3-server[969]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5298lb # [ 87.084108] niks3-server[969]: 2026/09/23 05:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5299lb # [ 87.091190] niks3-server[969]: 2026/09/23 05:34:19 INFO Completed upload id=305300worker2 # [ 87.080725] nix-grpc-daemon[922]: time=2026-09-23T05:34:19.431Z level=INFO msg="Upload complete. (204ms)"5301lb # [ 87.094883] niks3-server[969]: 2026/09/23 05:34:19 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5302worker2 # [ 87.086213] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15303worker2 # [ 87.100438] nix-daemon[870]: reaped child process 1166, status = succeeded5304client # copying 1 paths...5305worker1 # [ 87.147327] nix-grpc-daemon[780]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:569365306worker1 # [ 87.157784] nix-grpc-daemon[780]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5307client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5308worker2 # [ 87.177839] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:480465309worker2 # [ 87.183296] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=1205310client # [ 87.262958] nix-daemon[1104]: reaped child process 1190, status = succeeded5311client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 1.93 seconds)5312client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25313client # [ 87.365923] nix-daemon[1104]: accepted connection from pid 1209, user root (trusted)5314client # this derivation will be built:5315client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5316worker2 # [ 87.472954] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:480465317client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5318worker2 # [ 87.478807] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480465319worker2 # [ 87.484242] nix-grpc-daemon[922]: time=2026-09-23T05:34:19.833Z level=INFO msg="All 1 paths already cached"5320worker2 # [ 87.493668] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05321worker1 # [ 87.507778] nix-grpc-daemon[780]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569365322worker2 # [ 87.498540] nix-grpc-daemon[783]: ts=2026-09-23T05:34:19Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15323worker1 # [ 87.520141] nix-grpc-daemon[780]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569365324client # copying 1 paths to 'lb:50051'...5325client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5326worker1 # [ 87.537529] nix-grpc-daemon[780]: ts=2026-09-23T05:34:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569365327worker1 # [ 87.548698] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5328worker1 # [ 87.564780] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5)5329worker1 # [ 87.619432] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 132 ms (attempt 2/5)5330worker1 # [ 87.759434] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 3/5)5331worker1 # [ 87.798922] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 557 ms (attempt 4/5)5332lb # [ 87.980106] envoy[929]: conn peer=[2001:db8:1::1]:50254 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1197 ms=46715333lb # [ 87.983858] envoy[929]: conn peer=[2001:db8:1::1]:50258 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3408 tx=1921 ms=18645334lb # [ 87.987388] envoy[929]: conn peer=[2001:db8:1::1]:50266 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=334 ms=875335lb # [ 87.990858] envoy[929]: conn peer=[2001:db8:1::1]:40984 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=378 ms=165336lb # [ 87.994464] envoy[929]: conn peer=[2001:db8:1::1]:50276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2140 ms=15765337lb # [ 87.998540] envoy[929]: conn peer=[2001:db8:1::1]:40998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4297 tx=2371 ms=14145338lb # [ 88.002331] envoy[929]: conn peer=[2001:db8:1::1]:41010 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=185339lb # [ 88.005785] envoy[929]: conn peer=[2001:db8:1::1]:41014 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4218 tx=2055 ms=16375340lb # [ 88.009298] envoy[929]: conn peer=[2001:db8:1::1]:41022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=175341worker1 # [ 88.359800] nix-grpc-daemon[780]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5342worker1 # [ 88.363364] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5343worker1 # [ 88.368295] nix-grpc-daemon[780]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5344worker1 # [ 88.484764] nix-daemon[847]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)5345worker1 # [ 88.499486] nix-daemon[847]: reaped child process 1215, status = succeeded5346lb # [ 88.507574] niks3-server[969]: 2026/09/23 05:34:20 INFO Received uploads request method=POST path=/api/pending_closures5347worker1 # [ 88.516619] nix-grpc-daemon[860]: time=2026-09-23T05:34:20.723Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5348worker1 # [ 88.518244] nix-grpc-daemon[860]: time=2026-09-23T05:34:20.725Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5349lb # [ 88.540643] niks3-server[969]: 2026/09/23 05:34:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5350lb # [ 88.548720] niks3-server[969]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5351lb # [ 88.554429] niks3-server[969]: 2026/09/23 05:34:20 INFO Signed narinfos id=31 count=15352worker1 # [ 88.557972] nix-grpc-daemon[860]: time=2026-09-23T05:34:20.765Z level=INFO msg="Uploading 1 narinfos"5353lb # [ 88.562365] niks3-server[969]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5354lb # [ 88.572699] niks3-server[969]: 2026/09/23 05:34:20 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5355lb # [ 88.577484] niks3-server[969]: 2026/09/23 05:34:20 INFO Completed upload id=315356worker1 # [ 88.578119] nix-grpc-daemon[860]: time=2026-09-23T05:34:20.785Z level=INFO msg="Upload complete. (189ms)"5357worker1 # [ 88.581724] nix-grpc-daemon[780]: ts=2026-09-23T05:34:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 paths=1 nar_bytes_in=5605358lb # [ 88.584236] niks3-server[969]: 2026/09/23 05:34:20 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5359worker1 # [ 88.587443] nix-grpc-daemon[780]: ts=2026-09-23T05:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569365360worker1 # [ 88.597967] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5361worker1 # [ 88.607106] nix-daemon[847]: reaped child process 1205, status = succeeded5362worker1 # [ 88.609880] nix-daemon[847]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5363client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5364client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5365client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5366worker1 # [ 89.852507] nix-daemon[847]: reaped child process 1219, status = succeeded5367worker1 # [ 89.943678] nix-daemon[847]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5368worker1 # [ 89.960060] nix-daemon[847]: reaped child process 1238, status = succeeded5369lb # [ 89.966662] niks3-server[969]: 2026/09/23 05:34:22 INFO Received uploads request method=POST path=/api/pending_closures5370worker1 # [ 89.975421] nix-grpc-daemon[860]: time=2026-09-23T05:34:22.182Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5371lb # [ 89.997116] niks3-server[969]: 2026/09/23 05:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5372lb # [ 90.003261] niks3-server[969]: 2026/09/23 05:34:22 INFO Signed narinfos id=32 count=15373worker1 # [ 90.006812] nix-grpc-daemon[860]: time=2026-09-23T05:34:22.213Z level=INFO msg="Uploading 1 narinfos"5374lb # [ 90.011909] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5375lb # [ 90.023465] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5376lb # [ 90.036435] niks3-server[969]: 2026/09/23 05:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5377lb # [ 90.039827] niks3-server[969]: 2026/09/23 05:34:22 INFO Completed upload id=325378worker1 # [ 90.040340] nix-grpc-daemon[860]: time=2026-09-23T05:34:22.247Z level=INFO msg="Upload complete. (221ms)"5379lb # [ 90.043312] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5380worker1 # [ 90.043817] nix-grpc-daemon[780]: ts=2026-09-23T05:34:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56936 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15381worker1 # [ 90.057299] nix-daemon[847]: reaped child process 1217, status = succeeded5382client # copying 1 paths...5383worker2 # [ 90.079727] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:480465384worker2 # [ 90.091814] nix-grpc-daemon[783]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5385client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5386worker2 # [ 90.122722] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:480465387worker2 # [ 90.127757] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=1205388client # [ 90.247464] nix-daemon[1104]: reaped child process 1212, status = succeeded5389client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.99 seconds)5390client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25391client # [ 90.353479] nix-daemon[1104]: accepted connection from pid 1231, user root (trusted)5392client # this derivation will be built:5393client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5394worker1 # [ 90.472226] nix-grpc-daemon[780]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:569365395client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5396worker1 # [ 90.478949] nix-grpc-daemon[780]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569365397worker1 # [ 90.484149] nix-grpc-daemon[860]: time=2026-09-23T05:34:22.690Z level=INFO msg="All 1 paths already cached"5398worker2 # [ 90.483741] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05399worker2 # [ 90.489523] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15400worker2 # [ 90.495295] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480465401worker2 # [ 90.506829] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:480465402worker2 # [ 90.519905] nix-grpc-daemon[922]: time=2026-09-23T05:34:22.869Z level=INFO msg="All 1 paths already cached"5403client # copying 1 paths to 'lb:50051'...5404client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5405worker2 # [ 90.527846] nix-grpc-daemon[783]: ts=2026-09-23T05:34:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:480465406worker2 # [ 90.538449] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5407worker2 # [ 90.635703] nix-daemon[870]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)5408worker2 # [ 90.648528] nix-daemon[870]: reaped child process 1199, status = succeeded5409lb # [ 90.665974] niks3-server[969]: 2026/09/23 05:34:22 INFO Received uploads request method=POST path=/api/pending_closures5410worker2 # [ 90.665914] nix-grpc-daemon[922]: time=2026-09-23T05:34:23.016Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5411worker2 # [ 90.667901] nix-grpc-daemon[922]: time=2026-09-23T05:34:23.017Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5412lb # [ 90.698300] niks3-server[969]: 2026/09/23 05:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5413lb # [ 90.705464] niks3-server[969]: 2026/09/23 05:34:22 INFO Signed narinfos id=33 count=15414worker2 # [ 90.697560] nix-grpc-daemon[922]: time=2026-09-23T05:34:23.047Z level=INFO msg="Uploading 1 narinfos"5415lb # [ 90.710941] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5416lb # [ 90.714670] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5417lb # [ 90.725064] niks3-server[969]: 2026/09/23 05:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5418lb # [ 90.728219] niks3-server[969]: 2026/09/23 05:34:22 INFO Completed upload id=335419worker2 # [ 90.717899] nix-grpc-daemon[922]: time=2026-09-23T05:34:23.068Z level=INFO msg="Upload complete. (168ms)"5420lb # [ 90.730323] niks3-server[969]: 2026/09/23 05:34:22 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5421worker2 # [ 90.721688] nix-grpc-daemon[783]: ts=2026-09-23T05:34:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_in=5605422worker2 # [ 90.726763] nix-grpc-daemon[783]: ts=2026-09-23T05:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:480465423worker2 # [ 90.734277] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5424worker2 # [ 90.740345] nix-daemon[870]: reaped child process 1194, status = succeeded5425worker2 # [ 90.745069] nix-daemon[870]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5426client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5427client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5428client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5429worker2 # [ 91.775635] nix-daemon[870]: reaped child process 1203, status = succeeded5430worker2 # [ 91.858955] nix-daemon[870]: accepted connection from pid 1219, user nix-grpc-daemon (trusted)5431worker2 # [ 91.871788] nix-daemon[870]: reaped child process 1222, status = succeeded5432lb # [ 91.889741] niks3-server[969]: 2026/09/23 05:34:24 INFO Received uploads request method=POST path=/api/pending_closures5433worker2 # [ 91.888932] nix-grpc-daemon[922]: time=2026-09-23T05:34:24.239Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5434lb # [ 91.921352] niks3-server[969]: 2026/09/23 05:34:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5435lb # [ 91.930547] niks3-server[969]: 2026/09/23 05:34:24 INFO Signed narinfos id=34 count=15436worker2 # [ 91.922885] nix-grpc-daemon[922]: time=2026-09-23T05:34:24.273Z level=INFO msg="Uploading 1 narinfos"5437lb # [ 91.940731] niks3-server[969]: 2026/09/23 05:34:24 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5438lb # [ 91.947535] niks3-server[969]: 2026/09/23 05:34:24 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5439lb # [ 91.956471] niks3-server[969]: 2026/09/23 05:34:24 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5440lb # [ 91.964024] niks3-server[969]: 2026/09/23 05:34:24 INFO Completed upload id=345441worker2 # [ 91.953665] nix-grpc-daemon[922]: time=2026-09-23T05:34:24.304Z level=INFO msg="Upload complete. (199ms)"5442lb # [ 91.966745] niks3-server[969]: 2026/09/23 05:34:24 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5443worker2 # [ 91.959601] nix-grpc-daemon[783]: ts=2026-09-23T05:34:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15444client # copying 1 paths...5445worker2 # [ 91.969994] nix-grpc-daemon[783]: ts=2026-09-23T05:34:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:480465446client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5447worker2 # [ 91.978657] nix-daemon[870]: reaped child process 1201, status = succeeded5448worker2 # [ 91.988815] nix-grpc-daemon[783]: ts=2026-09-23T05:34:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:480465449worker2 # [ 91.992737] nix-grpc-daemon[783]: ts=2026-09-23T05:34:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48046 duration_s=0 paths=1 nar_bytes_out=1205450client # [ 92.112607] nix-daemon[1104]: reaped child process 1234, status = succeeded5451client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.86 seconds)5452(finished: subtest: an input only one worker has still reaches the builder, in 7.27 seconds)5453subtest: upload whose reference the worker lacks is completed from the cache5454client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5455client # this derivation will be built:5456client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5457client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5458client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5459client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5460client # this derivation will be built:5461client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5462client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5463client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.24 seconds)5464client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5465client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5466worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5467worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5468worker2 # [ 92.867109] systemd[1]: Stopping Nix worker-protocol over gRPC...5469worker2 # [ 93.229068] nix-grpc-daemon[783]: ts=2026-09-23T05:34:25Z level=info event=signal_exit5470lb # [ 93.739882] niks3-server[969]: 2026/09/23 05:34:25 INFO lead: released remote=192.168.1.4:329985471lb # [ 94.731466] niks3-server[969]: 2026/09/23 05:34:26 INFO lead: acquired remote=192.168.1.3:538865472worker1 # [ 94.736521] nix-grpc-daemon[780]: ts=2026-09-23T05:34:26Z level=info event=scheduler_take_over5473worker2 # [ 94.730159] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=scheduler_restarting5474lb # [ 94.739692] envoy[929]: [2026-09-23 05:34:27.000][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5475worker2 # [ 94.737307] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=scheduler_restarting addr=lb:500515476worker1 # [ 94.753082] nix-grpc-daemon[780]: ts=2026-09-23T05:34:26Z level=info event=scheduler_restarting addr=lb:500515477worker2 # [ 94.741565] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:480305478worker2 # [ 94.747759] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=worker_gone addr=192.168.1.4:500515479worker2 # [ 94.751184] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:480305480worker2 # [ 94.754971] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=worker_gone addr=192.168.1.3:500515481worker2 # [ 94.776806] nix-daemon[870]: reaped child process 872, status = succeeded5482worker1 # [ 94.954635] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_connected addr=lb:500515483worker1 # [ 94.958696] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5484worker2 # [ 94.952186] nix-grpc-daemon[783]: ts=2026-09-23T05:34:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5485worker1 # [ 95.160286] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_connected addr=lb:500515486worker1 # [ 95.164509] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5487worker1 # [ 95.365197] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=scheduler_connected addr=lb:500515488worker1 # [ 95.376874] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:530665489worker1 # [ 95.381451] nix-grpc-daemon[780]: ts=2026-09-23T05:34:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05490worker2 # [ 95.381310] nix-grpc-daemon[783]: E0000 00:00:1790141667.731645 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true5491worker2 # [ 95.595368] nix-grpc-daemon[783]: E0000 00:00:1790141667.945693 830 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5492worker2 # [ 95.633849] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5493worker2 # [ 95.636495] systemd[1]: Stopped Nix worker-protocol over gRPC.5494worker2 # [ 95.639530] systemd[1]: nix-grpc-daemon.service: Consumed 2.068s CPU time over 1min 17.575s wall clock time, 64.9M memory peak, 81.4K incoming IP traffic, 173.1K outgoing IP traffic.5495worker2 # [ 95.652396] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5496worker2 # [ 95.655737] systemd[1]: Closed Nix worker-protocol over gRPC.5497worker2 # [ 95.658482] systemd[1]: nix-grpc-daemon.socket: Consumed 294us CPU time over 1min 28.194s wall clock time, 256K memory peak, 343K incoming IP traffic, 97.1K outgoing IP traffic.5498lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux5499lb: must succeed: curl -sf localhost:9901/clusters5500lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5501lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds)5502client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25503worker1 # [ 95.872486] nix-grpc-daemon[780]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:569365504client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5505worker1 # [ 95.877901] nix-grpc-daemon[780]: ts=2026-09-23T05:34:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:569365506client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5507worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5508worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5509worker2 # [ 95.944676] systemd[1]: Listening on Nix worker-protocol over gRPC.5510worker1 # [ 96.005259] systemd[1]: Stopping Nix worker-protocol over gRPC...5511worker1 # [ 96.158722] nix-grpc-daemon[780]: ts=2026-09-23T05:34:28Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:530725512worker1 # [ 96.178181] nix-grpc-daemon[780]: ts=2026-09-23T05:34:28Z level=info event=signal_exit5513worker2 # [ 96.628819] systemd[1]: Starting scheduler-token.service...5514worker2 # [ 96.689106] systemd[1]: scheduler-token.service: Deactivated successfully.5515worker2 # [ 96.691505] systemd[1]: Finished scheduler-token.service.5516worker2 # [ 96.694566] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 59ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5517worker2 # [ 96.702398] systemd[1]: Starting Nix worker-protocol over gRPC...5518lb # [ 96.742126] niks3-server[969]: 2026/09/23 05:34:29 INFO lead: released remote=192.168.1.3:538865519worker2 # [ 96.782809] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5520worker2 # [ 96.784749] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5521worker2 # [ 96.787888] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15522worker2 # [ 96.792051] nix-grpc-daemon[1246]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5523worker2 # [ 96.794157] nix-grpc-daemon[1246]: E0000 00:00:1790141669.142501 1246 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5524worker2 # [ 96.799168] nix-grpc-daemon[1246]: E0000 00:00:1790141669.147884 1246 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5525worker2 # [ 96.803115] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5526worker2 # [ 96.804730] systemd[1]: Started Nix worker-protocol over gRPC.5527worker2 # [ 96.810069] nix-grpc-daemon[1246]: E0000 00:00:1790141669.159632 1289 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5528lb # [ 96.820751] niks3-server[969]: 2026/09/23 05:34:29 INFO lead: acquired remote=192.168.1.4:504605529worker2 # [ 96.814159] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_take_over5530worker2 # [ 96.824044] nix-grpc-daemon[1246]: E0000 00:00:1790141669.174530 1289 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5531worker2 # [ 96.828052] nix-grpc-daemon[1246]: E0000 00:00:1790141669.176841 1289 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5532worker2 # [ 96.838137] nix-grpc-daemon[1246]: E0000 00:00:1790141669.188605 1289 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5533worker2 # [ 96.847111] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_connected addr=lb:500515534worker2 # [ 96.848654] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5535worker2 # [ 97.050317] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_connected addr=lb:500515536worker2 # [ 97.054594] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5537worker2 # [ 97.255933] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_connected addr=lb:500515538worker2 # [ 97.260284] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5539worker2 # [ 97.461256] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_connected addr=lb:500515540worker2 # [ 97.465554] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5541worker2 # [ 97.666912] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=scheduler_connected addr=lb:500515542worker2 # [ 97.671225] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5543worker1 # [ 97.743601] nix-grpc-daemon[780]: ts=2026-09-23T05:34:29Z level=info event=scheduler_restarting5544lb # [ 97.741624] envoy[929]: [2026-09-23 05:34:30.002][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5545worker1 # [ 97.749169] nix-grpc-daemon[780]: ts=2026-09-23T05:34:29Z level=info event=scheduler_restarting addr=lb:500515546worker1 # [ 97.756099] nix-grpc-daemon[780]: ts=2026-09-23T05:34:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:530665547worker1 # [ 97.760132] nix-grpc-daemon[780]: ts=2026-09-23T05:34:29Z level=info event=worker_gone addr=192.168.1.3:500515548worker1 # [ 97.784185] nix-daemon[847]: reaped child process 849, status = succeeded5549worker2 # [ 97.873148] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=scheduler_connected addr=lb:500515550worker2 # [ 97.886149] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=oidc_jwks provider=mock keys=15551worker2 # [ 97.889946] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:329985552worker2 # [ 97.894584] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05553worker2 # [ 97.940927] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:30Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:330045554worker1 # [ 97.965703] nix-grpc-daemon[780]: ts=2026-09-23T05:34:30Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5555lb # [ 97.980444] envoy[929]: conn peer=[2001:db8:1::1]:41030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3742 tx=2124 ms=27075556lb # [ 97.984228] envoy[929]: conn peer=[2001:db8:1::1]:41042 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=595557lb # [ 97.987664] envoy[929]: conn peer=[2001:db8:1::1]:41058 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=555558lb # [ 97.991241] envoy[929]: conn peer=[2001:db8:1::1]:41052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3727 tx=2084 ms=15845559lb # [ 97.994741] envoy[929]: conn peer=[2001:db8:1::4]:41766 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2633 tx=869 ms=612025560lb # [ 97.998468] envoy[929]: conn peer=[2001:db8:1::1]:45018 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=175561lb # [ 98.002440] envoy[929]: conn peer=192.168.1.3:41782 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3191 tx=1318 ms=645835562worker1 # [ 98.354356] nix-grpc-daemon[780]: E0000 00:00:1790141670.561395 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5563worker1 # [ 99.600781] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5564worker1 # [ 99.604268] systemd[1]: Stopped Nix worker-protocol over gRPC.5565worker1 # [ 99.607227] systemd[1]: nix-grpc-daemon.service: Consumed 2.466s CPU time over 1min 21.619s wall clock time, 66M memory peak, 87.4K incoming IP traffic, 156.9K outgoing IP traffic.5566worker1 # [ 99.620767] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5567worker1 # [ 99.624952] systemd[1]: Closed Nix worker-protocol over gRPC.5568worker1 # [ 99.628909] systemd[1]: nix-grpc-daemon.socket: Consumed 303us CPU time over 1min 30.679s wall clock time, 256K memory peak, 217.7K incoming IP traffic, 89.3K outgoing IP traffic.5569lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux5570lb: must succeed: curl -sf localhost:9901/clusters5571lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5572lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.05 seconds)5573client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25574worker2 # [ 99.826363] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330345575worker2 # [ 99.837375] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5576worker2 # [ 99.854367] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)5577worker2 # [ 99.868492] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 2/5)5578worker2 # [ 99.987606] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 325 ms (attempt 3/5)5579worker2 # [ 100.314731] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 130 ms (attempt 4/5)5580worker2 # [ 100.447264] nix-grpc-daemon[1246]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5581worker2 # [ 100.450946] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5582client # copying 2 paths...5583client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5584worker2 # [ 100.472840] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330345585client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5586worker2 # [ 100.485139] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5587worker2 # [ 100.505255] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:32Z level=info event=niks3_push_started pid=13075588worker2 # [ 100.608981] nix-grpc-daemon[1307]: time=2026-09-23T05:34:32.959Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5589worker2 # [ 100.698099] nix-daemon[870]: accepted connection from pid 1320, user nix-grpc-daemon (trusted)5590worker2 # [ 100.710282] nix-daemon[870]: reaped child process 1323, status = succeeded5591lb # [ 100.730313] niks3-server[969]: 2026/09/23 05:34:32 INFO Received uploads request method=POST path=/api/pending_closures5592lb # [ 100.740638] niks3-server[969]: 2026/09/23 05:34:33 INFO Received uploads request method=POST path=/api/pending_closures5593worker2 # [ 100.737480] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.087Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5594worker2 # [ 100.739094] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.089Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5595worker2 # [ 100.740940] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.091Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5596lb # [ 100.799115] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5597lb # [ 100.801317] niks3-server[969]: 2026/09/23 05:34:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5598lb # [ 100.806926] niks3-server[969]: 2026/09/23 05:34:33 INFO Signed narinfos id=35 count=15599lb # [ 100.809218] niks3-server[969]: 2026/09/23 05:34:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5600lb # [ 100.812648] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5601lb # [ 100.815040] niks3-server[969]: 2026/09/23 05:34:33 INFO Signed narinfos id=36 count=25602worker2 # [ 100.805135] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.154Z level=INFO msg="Uploading 3 narinfos"5603lb # [ 100.824752] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5604lb # [ 100.833681] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5605lb # [ 100.841708] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5606lb # [ 100.845216] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5607lb # [ 100.848291] niks3-server[969]: 2026/09/23 05:34:33 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5608lb # [ 100.854108] niks3-server[969]: 2026/09/23 05:34:33 INFO Completed upload id=355609lb # [ 100.855556] niks3-server[969]: 2026/09/23 05:34:33 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5610lb # [ 100.858744] niks3-server[969]: 2026/09/23 05:34:33 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5611lb # [ 100.861307] niks3-server[969]: 2026/09/23 05:34:33 INFO Completed upload id=365612worker2 # [ 100.851120] nix-grpc-daemon[1307]: time=2026-09-23T05:34:33.201Z level=INFO msg="Upload complete. (238ms)"5613worker2 # [ 100.856777] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=0 paths=2 nar_bytes_in=3125614worker2 # [ 100.871297] nix-daemon[870]: reaped child process 1305, status = succeeded5615client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.21 seconds)5616worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5617worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5618worker1 # [ 100.970218] systemd[1]: Listening on Nix worker-protocol over gRPC.5619worker1 # [ 100.979604] systemd[1]: Starting Nix worker-protocol over gRPC...5620worker1 # [ 101.078440] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5621worker1 # [ 101.080672] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5622worker1 # [ 101.084571] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15623worker1 # [ 101.089056] nix-grpc-daemon[1261]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5624worker1 # [ 101.090396] nix-grpc-daemon[1261]: E0000 00:00:1790141673.295850 1261 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5625worker1 # [ 101.097070] nix-grpc-daemon[1261]: E0000 00:00:1790141673.301076 1261 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5626worker1 # [ 101.100725] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5627worker1 # [ 101.103511] systemd[1]: Started Nix worker-protocol over gRPC.5628worker1 # [ 101.106572] nix-grpc-daemon[1261]: E0000 00:00:1790141673.313924 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5629lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux5630lb: must succeed: curl -sf localhost:9901/clusters5631worker1 # [ 101.120159] nix-grpc-daemon[1261]: E0000 00:00:1790141673.327500 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5632worker1 # [ 101.124061] nix-grpc-daemon[1261]: E0000 00:00:1790141673.329848 1304 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5633worker1 # [ 101.134548] nix-grpc-daemon[1261]: E0000 00:00:1790141673.341842 1304 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5634worker1 # [ 101.146775] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=scheduler_connected addr=lb:500515635worker2 # [ 101.137918] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:33Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:329985636worker2 # [ 101.141717] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:33Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05637lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5638lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.06 seconds)5639(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.05 seconds)5640subtest: a niks3 restart does not cost the leader its role5641lb: must succeed: curl -sf localhost:9901/clusters5642lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5643lb: must succeed: curl -sf localhost:9901/clusters5644lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5645lb # [ 101.238347] systemd[1]: Stopping niks3 server...5646lb # [ 101.240180] niks3-server[969]: 2026/09/23 05:34:33 INFO Shutdown signal received, draining in-flight requests timeout=10s5647worker1 # [ 101.239431] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:33Z level=info event=scheduler_lock_lost status=2005648lb # [ 101.243873] niks3-server[969]: 2026/09/23 05:34:33 INFO lead: released remote=192.168.1.4:504605649worker2 # [ 101.233325] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:33Z level=info event=scheduler_lock_lost status=2005650lb # [ 106.059155] systemd[1]: niks3.service: Deactivated successfully.5651lb # [ 106.064573] systemd[1]: Stopped niks3 server.5652lb # [ 106.066350] systemd[1]: niks3.service: Consumed 762ms CPU time over 1min 26.540s wall clock time, 28.6M memory peak, 94.9K incoming IP traffic, 137.9K outgoing IP traffic.5653lb # [ 106.083552] systemd[1]: Starting niks3 server...5654lb # [ 106.143713] niks3-server[1135]: 2026/09/23 05:34:38 goose: no migrations to run. current version: 202609200000005655lb # [ 106.152299] niks3-server[1135]: 2026/09/23 05:34:38 OK 1_commit_pending_closure.sql (6.66ms)5656lb # [ 106.156183] niks3-server[1135]: 2026/09/23 05:34:38 OK 2_object_stats_trigger.sql (2.22ms)5657lb # [ 106.157256] niks3-server[1135]: 2026/09/23 05:34:38 goose: up to current file version: 25658lb # [ 106.158522] niks3-server[1135]: 2026/09/23 05:34:38 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5659lb # [ 106.169173] niks3-server[1135]: 2026/09/23 05:34:38 INFO Read proxy enabled — serving cache objects from S35660lb # [ 106.170555] niks3-server[1135]: 2026/09/23 05:34:38 INFO Using socket-activated listener address=0.0.0.0:57515661lb # [ 106.172699] systemd[1]: Started niks3 server.5662lb # [ 106.174364] niks3-server[1135]: 2026/09/23 05:34:38 INFO Starting HTTP server address=0.0.0.0:57515663lb # [ 106.179723] niks3-server[1135]: 2026/09/23 05:34:38 INFO systemd watchdog enabled interval=15s5664lb # [ 106.182057] niks3-server[1135]: 2026/09/23 05:34:38 INFO lead: acquired remote=192.168.1.4:505445665lb: waiting for unit niks3.service5666lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5667lb: must succeed: curl -sf localhost:9901/clusters5668lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5669worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5670worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5671lb: must succeed: curl -sf localhost:9901/clusters5672lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5673lb: must succeed: curl -sf localhost:9901/clusters5674lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5675worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5676worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5677client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25678worker2 # [ 106.490245] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330345679client # don't know how to build these paths:5680client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5681worker1 # [ 106.514897] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372165682worker1 # [ 106.527698] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5683worker1 # [ 106.544882] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5)5684worker1 # [ 106.585330] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 180 ms (attempt 2/5)5685worker1 # [ 106.767760] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 351 ms (attempt 3/5)5686worker1 # [ 107.121186] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 520 ms (attempt 4/5)5687worker1 # [ 107.645390] nix-grpc-daemon[1261]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5688worker1 # [ 107.648562] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5689worker2 # [ 107.675969] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05690worker2 # [ 107.682156] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15691worker2 # [ 107.722219] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330345692worker2 # [ 107.736497] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330345693client # copying 1 paths to 'lb:50051'...5694client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5695worker2 # [ 107.749414] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330345696worker2 # [ 107.760346] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5697worker2 # [ 107.864078] nix-daemon[870]: accepted connection from pid 1342, user nix-grpc-daemon (trusted)5698worker2 # [ 107.876873] nix-daemon[870]: reaped child process 1345, status = succeeded5699lb # [ 107.895692] niks3-server[1135]: 2026/09/23 05:34:40 INFO Received uploads request method=POST path=/api/pending_closures5700worker2 # [ 107.897322] nix-grpc-daemon[1307]: time=2026-09-23T05:34:40.247Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5701worker2 # [ 107.899742] nix-grpc-daemon[1307]: time=2026-09-23T05:34:40.249Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5702lb # [ 107.939822] niks3-server[1135]: 2026/09/23 05:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5703lb # [ 107.952233] niks3-server[1135]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5704lb # [ 107.967845] niks3-server[1135]: 2026/09/23 05:34:40 INFO Signed narinfos id=37 count=15705worker2 # [ 107.959529] nix-grpc-daemon[1307]: time=2026-09-23T05:34:40.309Z level=INFO msg="Uploading 1 narinfos"5706lb # [ 107.976362] niks3-server[1135]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5707lb # [ 107.979514] niks3-server[1135]: 2026/09/23 05:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5708lb # [ 107.982226] envoy[929]: conn peer=[2001:db8:1::1]:45022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=10475709lb # [ 107.988196] niks3-server[1135]: 2026/09/23 05:34:40 INFO Completed upload id=375710worker2 # [ 107.977832] nix-grpc-daemon[1307]: time=2026-09-23T05:34:40.328Z level=INFO msg="Upload complete. (205ms)"5711lb # [ 107.993060] niks3-server[1135]: 2026/09/23 05:34:40 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5712worker2 # [ 107.984117] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=0 paths=1 nar_bytes_in=5365713worker2 # [ 107.990449] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330345714worker2 # [ 108.001702] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5715worker2 # [ 108.010151] nix-daemon[870]: reaped child process 1340, status = succeeded5716client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5717worker2 # [ 108.018560] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5718client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5719client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5720client # farm-a> LOG-farm-a-after-niks3-restart5721worker2 # [ 109.451815] nix-daemon[870]: reaped child process 1349, status = succeeded5722worker2 # [ 109.537258] nix-daemon[870]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5723worker2 # [ 109.550046] nix-daemon[870]: reaped child process 1368, status = succeeded5724lb # [ 109.568494] niks3-server[1135]: 2026/09/23 05:34:41 INFO Received uploads request method=POST path=/api/pending_closures5725worker2 # [ 109.569436] nix-grpc-daemon[1307]: time=2026-09-23T05:34:41.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5726worker2 # [ 109.571241] nix-grpc-daemon[1307]: time=2026-09-23T05:34:41.921Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5727lb # [ 109.614752] niks3-server[1135]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5728lb # [ 109.619487] niks3-server[1135]: 2026/09/23 05:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5729lb # [ 109.625674] niks3-server[1135]: 2026/09/23 05:34:41 INFO Signed narinfos id=38 count=15730worker2 # [ 109.617712] nix-grpc-daemon[1307]: time=2026-09-23T05:34:41.968Z level=INFO msg="Uploading 1 narinfos"5731lb # [ 109.635743] niks3-server[1135]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5732lb # [ 109.640668] niks3-server[1135]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5733lb # [ 109.645786] niks3-server[1135]: 2026/09/23 05:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5734lb # [ 109.650906] niks3-server[1135]: 2026/09/23 05:34:41 INFO Completed upload id=385735worker2 # [ 109.640616] nix-grpc-daemon[1307]: time=2026-09-23T05:34:41.991Z level=INFO msg="Upload complete. (214ms)"5736lb # [ 109.652683] niks3-server[1135]: 2026/09/23 05:34:41 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5737worker2 # [ 109.648305] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15738worker2 # [ 109.660683] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05739worker2 # [ 109.665186] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15740worker2 # [ 109.673622] nix-daemon[870]: reaped child process 1347, status = succeeded5741worker2 # [ 109.704231] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330345742worker2 # [ 109.716417] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330345743worker2 # [ 109.730157] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.080Z level=INFO msg="All 1 paths already cached"5744client # copying 1 paths to 'lb:50051'...5745client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5746worker2 # [ 109.738923] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330345747worker2 # [ 109.749164] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5748worker2 # [ 109.843983] nix-daemon[870]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)5749worker2 # [ 109.857115] nix-daemon[870]: reaped child process 1377, status = succeeded5750lb # [ 109.875454] niks3-server[1135]: 2026/09/23 05:34:42 INFO Received uploads request method=POST path=/api/pending_closures5751worker2 # [ 109.875145] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.225Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5752worker2 # [ 109.876569] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.227Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5753lb # [ 109.910615] niks3-server[1135]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5754lb # [ 109.915960] niks3-server[1135]: 2026/09/23 05:34:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5755worker2 # [ 109.913158] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.263Z level=INFO msg="Uploading 1 narinfos"5756lb # [ 109.923794] niks3-server[1135]: 2026/09/23 05:34:42 INFO Signed narinfos id=39 count=15757lb # [ 109.930666] niks3-server[1135]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5758lb # [ 109.942401] niks3-server[1135]: 2026/09/23 05:34:42 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5759lb # [ 109.950510] niks3-server[1135]: 2026/09/23 05:34:42 INFO Completed upload id=395760lb # [ 109.953567] niks3-server[1135]: 2026/09/23 05:34:42 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5761worker2 # [ 109.940233] nix-grpc-daemon[1307]: time=2026-09-23T05:34:42.290Z level=INFO msg="Upload complete. (180ms)"5762worker2 # [ 109.945244] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=0 paths=1 nar_bytes_in=6565763worker2 # [ 109.951309] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330345764worker2 # [ 109.961769] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5765worker2 # [ 109.970437] nix-daemon[870]: reaped child process 1372, status = succeeded5766worker2 # [ 109.976121] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5767client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5768client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5769client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5770client # farm-b> LOG-farm-b-after-niks3-restart5771worker2 # [ 111.412644] nix-daemon[870]: reaped child process 1381, status = succeeded5772worker2 # [ 111.494839] nix-daemon[870]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)5773worker2 # [ 111.506309] nix-daemon[870]: reaped child process 1400, status = succeeded5774lb # [ 111.524896] niks3-server[1135]: 2026/09/23 05:34:43 INFO Received uploads request method=POST path=/api/pending_closures5775worker2 # [ 111.523254] nix-grpc-daemon[1307]: time=2026-09-23T05:34:43.873Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5776worker2 # [ 111.524903] nix-grpc-daemon[1307]: time=2026-09-23T05:34:43.875Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5777lb # [ 111.579299] niks3-server[1135]: 2026/09/23 05:34:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5778lb # [ 111.583940] niks3-server[1135]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5779lb # [ 111.591655] niks3-server[1135]: 2026/09/23 05:34:43 INFO Signed narinfos id=40 count=15780worker2 # [ 111.585454] nix-grpc-daemon[1307]: time=2026-09-23T05:34:43.935Z level=INFO msg="Uploading 1 narinfos"5781lb # [ 111.597286] niks3-server[1135]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5782lb # [ 111.600859] niks3-server[1135]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5783lb # [ 111.609121] niks3-server[1135]: 2026/09/23 05:34:43 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5784lb # [ 111.613524] niks3-server[1135]: 2026/09/23 05:34:43 INFO Completed upload id=405785worker2 # [ 111.602717] nix-grpc-daemon[1307]: time=2026-09-23T05:34:43.953Z level=INFO msg="Upload complete. (219ms)"5786lb # [ 111.615328] niks3-server[1135]: 2026/09/23 05:34:43 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5787worker2 # [ 111.608777] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15788worker2 # [ 111.622568] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:43Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05789worker2 # [ 111.627096] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:43Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15790worker2 # [ 111.634943] nix-daemon[870]: reaped child process 1379, status = succeeded5791worker2 # [ 111.665939] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330345792worker2 # [ 111.676689] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330345793worker2 # [ 111.688464] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.038Z level=INFO msg="All 2 paths already cached"5794client # copying 1 paths to 'lb:50051'...5795client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5796worker2 # [ 111.697359] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330345797worker2 # [ 111.705584] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5798worker2 # [ 111.799539] nix-daemon[870]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)5799worker2 # [ 111.812791] nix-daemon[870]: reaped child process 1409, status = succeeded5800lb # [ 111.830672] niks3-server[1135]: 2026/09/23 05:34:44 INFO Received uploads request method=POST path=/api/pending_closures5801worker2 # [ 111.828603] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.179Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5802worker2 # [ 111.830120] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.180Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5803lb # [ 111.866286] niks3-server[1135]: 2026/09/23 05:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5804worker2 # [ 111.864886] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.215Z level=INFO msg="Uploading 1 narinfos"5805lb # [ 111.875525] niks3-server[1135]: 2026/09/23 05:34:44 INFO Signed narinfos id=41 count=15806lb # [ 111.879935] niks3-server[1135]: 2026/09/23 05:34:44 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5807lb # [ 111.886125] niks3-server[1135]: 2026/09/23 05:34:44 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5808lb # [ 111.897545] niks3-server[1135]: 2026/09/23 05:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5809lb # [ 111.903564] niks3-server[1135]: 2026/09/23 05:34:44 INFO Completed upload id=415810worker2 # [ 111.892472] nix-grpc-daemon[1307]: time=2026-09-23T05:34:44.242Z level=INFO msg="Upload complete. (175ms)"5811lb # [ 111.906345] niks3-server[1135]: 2026/09/23 05:34:44 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5812worker2 # [ 111.896995] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=0 paths=1 nar_bytes_in=7845813worker2 # [ 111.902865] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330345814worker2 # [ 111.913117] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5815worker2 # [ 111.921150] nix-daemon[870]: reaped child process 1404, status = succeeded5816worker2 # [ 111.924715] nix-daemon[870]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5817client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5818client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5819client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5820client # farm-top> LOG-farm-top-after-niks3-restart5821worker2 # [ 113.161367] nix-daemon[870]: reaped child process 1413, status = succeeded5822worker2 # [ 113.244233] nix-daemon[870]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5823worker2 # [ 113.256193] nix-daemon[870]: reaped child process 1432, status = succeeded5824lb # [ 113.274431] niks3-server[1135]: 2026/09/23 05:34:45 INFO Received uploads request method=POST path=/api/pending_closures5825worker2 # [ 113.272746] nix-grpc-daemon[1307]: time=2026-09-23T05:34:45.623Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5826worker2 # [ 113.274573] nix-grpc-daemon[1307]: time=2026-09-23T05:34:45.624Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5827lb # [ 113.316169] niks3-server[1135]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5828lb # [ 113.322319] niks3-server[1135]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5829lb # [ 113.325904] niks3-server[1135]: 2026/09/23 05:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5830lb # [ 113.329826] niks3-server[1135]: 2026/09/23 05:34:45 INFO Signed narinfos id=42 count=15831worker2 # [ 113.321891] nix-grpc-daemon[1307]: time=2026-09-23T05:34:45.672Z level=INFO msg="Uploading 1 narinfos"5832lb # [ 113.336212] niks3-server[1135]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5833lb # [ 113.346944] niks3-server[1135]: 2026/09/23 05:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5834lb # [ 113.350444] niks3-server[1135]: 2026/09/23 05:34:45 INFO Completed upload id=425835worker2 # [ 113.341227] nix-grpc-daemon[1307]: time=2026-09-23T05:34:45.691Z level=INFO msg="Upload complete. (207ms)"5836lb # [ 113.354439] niks3-server[1135]: 2026/09/23 05:34:45 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5837worker2 # [ 113.349253] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33034 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15838worker2 # [ 113.362980] nix-daemon[870]: reaped child process 1411, status = succeeded5839client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 7.03 seconds)5840client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5841client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)5842(finished: subtest: a niks3 restart does not cost the leader its role, in 12.34 seconds)5843subtest: scheduler down: the other node takes the lock and keeps it5844lb: must succeed: curl -sf localhost:9901/clusters5845lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5846lb: must succeed: curl -sf localhost:9901/clusters5847lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5848lb: must succeed: curl -sf localhost:9901/clusters5849lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5850lb: must succeed: curl -sf localhost:9901/clusters5851lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5852worker2 # [ 113.651235] systemd[1]: Stopping Nix worker-protocol over gRPC...5853worker2 # [ 113.805632] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:46Z level=info event=signal_exit5854lb # [ 114.185165] niks3-server[1135]: 2026/09/23 05:34:46 INFO lead: released remote=192.168.1.4:505445855worker2 # [ 115.175878] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=scheduler_restarting5856lb # [ 115.185834] envoy[929]: [2026-09-23 05:34:47.446][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5857worker2 # [ 115.183950] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=scheduler_restarting addr=lb:500515858worker1 # [ 115.195295] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_restarting addr=lb:500515859worker2 # [ 115.189308] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:329985860worker2 # [ 115.193687] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=worker_gone addr=192.168.1.4:500515861worker2 # [ 115.197574] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:329985862worker2 # [ 115.200944] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=worker_gone addr=192.168.1.3:500515863worker2 # [ 115.222745] nix-daemon[870]: reaped child process 1299, status = succeeded5864worker1 # [ 115.395764] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_connected addr=lb:500515865worker1 # [ 115.400242] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5866worker2 # [ 115.396885] nix-grpc-daemon[1246]: ts=2026-09-23T05:34:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5867worker1 # [ 115.601199] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_connected addr=lb:500515868worker1 # [ 115.605438] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5869worker1 # [ 115.806619] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:500515870worker1 # [ 115.810859] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5871worker2 # [ 115.857496] nix-grpc-daemon[1246]: E0000 00:00:1790141688.207830 1292 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5872worker2 # [ 115.995102] nix-grpc-daemon[1246]: E0000 00:00:1790141688.345293 1292 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5873worker1 # [ 116.011687] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:500515874worker1 # [ 116.015879] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5875worker1 # [ 116.182925] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_take_over5876lb # [ 116.182637] niks3-server[1135]: 2026/09/23 05:34:48 INFO lead: acquired remote=192.168.1.3:410085877worker1 # [ 116.218254] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:500515878worker1 # [ 116.221380] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5879worker2 # [ 116.283253] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5880worker2 # [ 116.285903] systemd[1]: Stopped Nix worker-protocol over gRPC.5881worker2 # [ 116.289658] systemd[1]: nix-grpc-daemon.service: Consumed 959ms CPU time over 19.580s wall clock time, 38.3M memory peak, 46.8K incoming IP traffic, 85.2K outgoing IP traffic.5882worker2 # [ 116.301220] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5883worker2 # [ 116.305523] systemd[1]: Closed Nix worker-protocol over gRPC.5884worker2 # [ 116.309164] systemd[1]: nix-grpc-daemon.socket: Consumed 288us CPU time over 20.363s wall clock time, 256K memory peak, 218.6K incoming IP traffic, 34.9K outgoing IP traffic.5885worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5886worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)5887worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5888worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5889worker1 # [ 116.418625] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:500515890worker1 # [ 116.420471] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5891worker1 # [ 116.621605] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_connected addr=lb:500515892worker1 # [ 116.625760] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5893worker1 # [ 116.826795] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=scheduler_connected addr=lb:500515894worker1 # [ 116.834739] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:531725895worker1 # [ 116.838907] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05896worker1 # [ 117.159919] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:531885897worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5898worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5899client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25900worker1 # [ 117.598901] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372165901client # don't know how to build these paths:5902client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5903worker1 # [ 117.607691] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372165904worker1 # [ 117.631953] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05905worker1 # [ 117.637306] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15906worker1 # [ 117.675899] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372165907worker1 # [ 117.689276] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372165908client # copying 1 paths to 'lb:50051'...5909client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5910worker1 # [ 117.702289] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372165911worker1 # [ 117.714347] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5912worker1 # [ 117.734168] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:49Z level=info event=niks3_push_started pid=13385913worker1 # [ 117.843362] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.050Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5914worker1 # [ 117.938729] nix-daemon[847]: accepted connection from pid 1351, user nix-grpc-daemon (trusted)5915worker1 # [ 117.953832] nix-daemon[847]: reaped child process 1354, status = succeeded5916lb # [ 117.962809] niks3-server[1135]: 2026/09/23 05:34:50 INFO Received uploads request method=POST path=/api/pending_closures5917worker1 # [ 117.973328] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.180Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5918worker1 # [ 117.974873] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.182Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5919lb # [ 117.984472] envoy[929]: conn peer=[2001:db8:1::1]:38738 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9735 tx=4424 ms=68735920lb # [ 117.989235] envoy[929]: conn peer=[2001:db8:1::4]:55290 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5923 tx=665 ms=185445921lb # [ 118.015348] niks3-server[1135]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5922lb # [ 118.022537] niks3-server[1135]: 2026/09/23 05:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5923worker1 # [ 118.027547] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.234Z level=INFO msg="Uploading 1 narinfos"5924lb # [ 118.027184] niks3-server[1135]: 2026/09/23 05:34:50 INFO Signed narinfos id=43 count=15925lb # [ 118.037855] niks3-server[1135]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5926lb # [ 118.041807] niks3-server[1135]: 2026/09/23 05:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5927lb # [ 118.048410] niks3-server[1135]: 2026/09/23 05:34:50 INFO Completed upload id=435928worker1 # [ 118.048967] nix-grpc-daemon[1338]: time=2026-09-23T05:34:50.256Z level=INFO msg="Upload complete. (202ms)"5929lb # [ 118.050420] niks3-server[1135]: 2026/09/23 05:34:50 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5930worker1 # [ 118.053663] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=5205931worker1 # [ 118.058753] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372165932worker1 # [ 118.070225] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5933worker1 # [ 118.073810] nix-daemon[847]: reaped child process 1336, status = succeeded5934worker1 # [ 118.078926] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5935client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5936client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5937client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5938client # farm-a> LOG-farm-a-on-standby5939worker1 # [ 119.120651] nix-daemon[847]: reaped child process 1358, status = succeeded5940worker1 # [ 119.203784] nix-daemon[847]: accepted connection from pid 1374, user nix-grpc-daemon (trusted)5941worker1 # [ 119.219383] nix-daemon[847]: reaped child process 1377, status = succeeded5942lb # [ 119.227649] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received uploads request method=POST path=/api/pending_closures5943worker1 # [ 119.236494] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.443Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5944worker1 # [ 119.239242] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.445Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5945lb # [ 119.262705] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5946lb # [ 119.271689] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5947lb # [ 119.280695] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5948lb # [ 119.286532] niks3-server[1135]: 2026/09/23 05:34:51 INFO Signed narinfos id=44 count=15949worker1 # [ 119.289481] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.496Z level=INFO msg="Uploading 1 narinfos"5950lb # [ 119.293469] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5951lb # [ 119.302249] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5952lb # [ 119.306866] niks3-server[1135]: 2026/09/23 05:34:51 INFO Completed upload id=445953worker1 # [ 119.307373] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.514Z level=INFO msg="Upload complete. (219ms)"5954lb # [ 119.309560] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5955worker1 # [ 119.318100] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15956worker1 # [ 119.325803] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05957worker1 # [ 119.330500] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15958worker1 # [ 119.338502] nix-daemon[847]: reaped child process 1356, status = succeeded5959worker1 # [ 119.370715] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372165960worker1 # [ 119.381913] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372165961worker1 # [ 119.396196] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.603Z level=INFO msg="All 1 paths already cached"5962client # copying 1 paths to 'lb:50051'...5963client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5964worker1 # [ 119.404888] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372165965worker1 # [ 119.415163] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5966worker1 # [ 119.517989] nix-daemon[847]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)5967worker1 # [ 119.533305] nix-daemon[847]: reaped child process 1386, status = succeeded5968lb # [ 119.540996] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received uploads request method=POST path=/api/pending_closures5969worker1 # [ 119.550035] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.757Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5970worker1 # [ 119.551580] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.758Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5971lb # [ 119.575931] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5972lb # [ 119.583237] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5973lb # [ 119.590332] niks3-server[1135]: 2026/09/23 05:34:51 INFO Signed narinfos id=45 count=15974worker1 # [ 119.593386] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.800Z level=INFO msg="Uploading 1 narinfos"5975lb # [ 119.613540] niks3-server[1135]: 2026/09/23 05:34:51 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5976lb # [ 119.617622] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5977lb # [ 119.621405] niks3-server[1135]: 2026/09/23 05:34:51 INFO Completed upload id=455978worker1 # [ 119.622199] nix-grpc-daemon[1338]: time=2026-09-23T05:34:51.829Z level=INFO msg="Upload complete. (196ms)"5979lb # [ 119.623904] niks3-server[1135]: 2026/09/23 05:34:51 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5980worker1 # [ 119.626932] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=6405981worker1 # [ 119.633163] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372165982worker1 # [ 119.641894] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5983worker1 # [ 119.652253] nix-daemon[847]: reaped child process 1381, status = succeeded5984worker1 # [ 119.657215] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)5985client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5986client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5987client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5988client # farm-b> LOG-farm-b-on-standby5989worker1 # [ 120.485750] nix-daemon[847]: reaped child process 1390, status = succeeded5990worker1 # [ 120.563025] nix-daemon[847]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)5991worker1 # [ 120.577889] nix-daemon[847]: reaped child process 1409, status = succeeded5992lb # [ 120.586558] niks3-server[1135]: 2026/09/23 05:34:52 INFO Received uploads request method=POST path=/api/pending_closures5993worker1 # [ 120.595699] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.802Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5994worker1 # [ 120.597211] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.804Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5995lb # [ 120.630630] niks3-server[1135]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5996lb # [ 120.638147] niks3-server[1135]: 2026/09/23 05:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5997worker1 # [ 120.645105] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.852Z level=INFO msg="Uploading 1 narinfos"5998lb # [ 120.645197] niks3-server[1135]: 2026/09/23 05:34:52 INFO Signed narinfos id=46 count=15999lb # [ 120.650556] niks3-server[1135]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6000lb # [ 120.655460] niks3-server[1135]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6001lb # [ 120.664428] niks3-server[1135]: 2026/09/23 05:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6002lb # [ 120.667635] niks3-server[1135]: 2026/09/23 05:34:52 INFO Completed upload id=466003worker1 # [ 120.668166] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.875Z level=INFO msg="Upload complete. (204ms)"6004lb # [ 120.671521] niks3-server[1135]: 2026/09/23 05:34:52 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6005worker1 # [ 120.678984] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16006worker1 # [ 120.685636] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06007worker1 # [ 120.690197] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16008worker1 # [ 120.699361] nix-daemon[847]: reaped child process 1388, status = succeeded6009worker1 # [ 120.729130] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372166010worker1 # [ 120.742412] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372166011worker1 # [ 120.756108] nix-grpc-daemon[1338]: time=2026-09-23T05:34:52.963Z level=INFO msg="All 2 paths already cached"6012client # copying 1 paths to 'lb:50051'...6013client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6014worker1 # [ 120.765117] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372166015worker1 # [ 120.773732] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)6016worker1 # [ 120.877832] nix-daemon[847]: accepted connection from pid 1415, user nix-grpc-daemon (trusted)6017worker1 # [ 120.892432] nix-daemon[847]: reaped child process 1418, status = succeeded6018lb # [ 120.901467] niks3-server[1135]: 2026/09/23 05:34:53 INFO Received uploads request method=POST path=/api/pending_closures6019worker1 # [ 120.910507] nix-grpc-daemon[1338]: time=2026-09-23T05:34:53.117Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6020worker1 # [ 120.912034] nix-grpc-daemon[1338]: time=2026-09-23T05:34:53.119Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6021lb # [ 120.933209] niks3-server[1135]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6022lb # [ 120.939761] niks3-server[1135]: 2026/09/23 05:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6023worker1 # [ 120.945815] nix-grpc-daemon[1338]: time=2026-09-23T05:34:53.153Z level=INFO msg="Uploading 1 narinfos"6024lb # [ 120.945989] niks3-server[1135]: 2026/09/23 05:34:53 INFO Signed narinfos id=47 count=16025lb # [ 120.959528] niks3-server[1135]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6026lb # [ 120.966910] niks3-server[1135]: 2026/09/23 05:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6027lb # [ 120.972982] niks3-server[1135]: 2026/09/23 05:34:53 INFO Completed upload id=476028worker1 # [ 120.973680] nix-grpc-daemon[1338]: time=2026-09-23T05:34:53.180Z level=INFO msg="Upload complete. (187ms)"6029lb # [ 120.976331] niks3-server[1135]: 2026/09/23 05:34:53 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6030worker1 # [ 120.978165] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=7686031worker1 # [ 120.983267] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372166032worker1 # [ 120.994216] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)6033worker1 # [ 121.000163] nix-daemon[847]: reaped child process 1413, status = succeeded6034worker1 # [ 121.005335] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)6035client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6036client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6037client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6038client # farm-top> LOG-farm-top-on-standby6039worker1 # [ 122.043567] nix-daemon[847]: reaped child process 1422, status = succeeded6040worker1 # [ 122.136741] nix-daemon[847]: accepted connection from pid 1438, user nix-grpc-daemon (trusted)6041worker1 # [ 122.150796] nix-daemon[847]: reaped child process 1441, status = succeeded6042lb # [ 122.158902] niks3-server[1135]: 2026/09/23 05:34:54 INFO Received uploads request method=POST path=/api/pending_closures6043worker1 # [ 122.169105] nix-grpc-daemon[1338]: time=2026-09-23T05:34:54.376Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6044worker1 # [ 122.170600] nix-grpc-daemon[1338]: time=2026-09-23T05:34:54.377Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6045lb # [ 122.202641] niks3-server[1135]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6046lb # [ 122.211151] niks3-server[1135]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6047lb # [ 122.218981] niks3-server[1135]: 2026/09/23 05:34:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6048lb # [ 122.226281] niks3-server[1135]: 2026/09/23 05:34:54 INFO Signed narinfos id=48 count=16049worker1 # [ 122.227281] nix-grpc-daemon[1338]: time=2026-09-23T05:34:54.434Z level=INFO msg="Uploading 1 narinfos"6050lb # [ 122.234844] niks3-server[1135]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6051lb # [ 122.243130] niks3-server[1135]: 2026/09/23 05:34:54 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6052lb # [ 122.248347] niks3-server[1135]: 2026/09/23 05:34:54 INFO Completed upload id=486053lb # [ 122.250820] niks3-server[1135]: 2026/09/23 05:34:54 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6054worker1 # [ 122.250393] nix-grpc-daemon[1338]: time=2026-09-23T05:34:54.457Z level=INFO msg="Upload complete. (236ms)"6055worker1 # [ 122.258570] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16056worker1 # [ 122.273935] nix-daemon[847]: reaped child process 1420, status = succeeded6057client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 4.84 seconds)6058client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6059client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.11 seconds)6060worker2 # [ 122.441686] systemd[1]: Listening on Nix worker-protocol over gRPC.6061worker2 # [ 122.449829] systemd[1]: Starting scheduler-token.service...6062worker2 # [ 122.496729] systemd[1]: scheduler-token.service: Deactivated successfully.6063worker2 # [ 122.498467] systemd[1]: Finished scheduler-token.service.6064worker2 # [ 122.500198] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 49ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6065worker2 # [ 122.503759] systemd[1]: Starting Nix worker-protocol over gRPC...6066worker2 # [ 122.580969] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6067worker2 # [ 122.582921] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6068worker2 # [ 122.586115] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16069worker2 # [ 122.590488] nix-grpc-daemon[1453]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6070worker2 # [ 122.593183] nix-grpc-daemon[1453]: E0000 00:00:1790141694.940907 1453 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6071worker2 # [ 122.599565] nix-grpc-daemon[1453]: E0000 00:00:1790141694.946481 1453 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6072worker2 # [ 122.603467] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6073worker2 # [ 122.608122] systemd[1]: Started Nix worker-protocol over gRPC.6074worker2 # [ 122.610096] nix-grpc-daemon[1453]: E0000 00:00:1790141694.957067 1495 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6075worker2 # [ 122.620965] nix-grpc-daemon[1453]: E0000 00:00:1790141694.971467 1495 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6076worker2 # [ 122.625152] nix-grpc-daemon[1453]: E0000 00:00:1790141694.973707 1495 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6077lb: must succeed: curl -sf localhost:9901/clusters6078worker2 # [ 122.635625] nix-grpc-daemon[1453]: E0000 00:00:1790141694.986137 1495 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6079worker2 # [ 122.649178] nix-grpc-daemon[1453]: ts=2026-09-23T05:34:54Z level=info event=scheduler_connected addr=lb:500516080worker1 # [ 122.671190] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:54Z level=info event=oidc_jwks provider=mock keys=16081worker1 # [ 122.675846] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:531726082worker1 # [ 122.683098] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06083lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6084worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6085worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6086lb: must succeed: curl -sf localhost:9901/clusters6087lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6088lb: must succeed: curl -sf localhost:9901/clusters6089lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6090worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6091worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6092(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 9.34 seconds)6093subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6094client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6095client # [ 122.897730] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6096client # Running as unit: rs.service; invocation ID: 604ee7ef781c4a29bdd8a4f2016574ec6097client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6098worker1 # [ 123.034274] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:372166099client # [ 123.040918] nix[1358]: don't know how to build these paths:6100client # [ 123.041881] nix[1358]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6101worker1 # [ 123.048752] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06102worker1 # [ 123.054092] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16103worker1 # [ 123.093368] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372166104worker1 # [ 123.107743] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:372166105client # [ 123.118155] nix[1358]: copying 1 paths to 'lb:50051'...6106worker1 # [ 123.119290] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:372166107client # [ 123.120058] nix[1358]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6108worker1 # [ 123.132593] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)6109worker1 # [ 123.242351] nix-daemon[847]: accepted connection from pid 1454, user nix-grpc-daemon (trusted)6110worker1 # [ 123.257135] nix-daemon[847]: reaped child process 1457, status = succeeded6111lb # [ 123.265279] niks3-server[1135]: 2026/09/23 05:34:55 INFO Received uploads request method=POST path=/api/pending_closures6112worker1 # [ 123.276431] nix-grpc-daemon[1338]: time=2026-09-23T05:34:55.483Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6113worker1 # [ 123.277877] nix-grpc-daemon[1338]: time=2026-09-23T05:34:55.483Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6114lb # [ 123.296615] niks3-server[1135]: 2026/09/23 05:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6115worker1 # [ 123.302210] nix-grpc-daemon[1338]: time=2026-09-23T05:34:55.509Z level=INFO msg="Uploading 1 narinfos"6116lb # [ 123.303096] niks3-server[1135]: 2026/09/23 05:34:55 INFO Signed narinfos id=49 count=16117lb # [ 123.314660] niks3-server[1135]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6118lb # [ 123.324538] niks3-server[1135]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6119lb # [ 123.328644] niks3-server[1135]: 2026/09/23 05:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6120lb # [ 123.336367] niks3-server[1135]: 2026/09/23 05:34:55 INFO Completed upload id=496121worker1 # [ 123.337047] nix-grpc-daemon[1338]: time=2026-09-23T05:34:55.544Z level=INFO msg="Upload complete. (192ms)"6122lb # [ 123.339923] niks3-server[1135]: 2026/09/23 05:34:55 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6123worker1 # [ 123.341596] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=0 paths=1 nar_bytes_in=4166124worker1 # [ 123.348311] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:372166125worker1 # [ 123.359111] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)6126worker1 # [ 123.364707] nix-daemon[847]: reaped child process 1452, status = succeeded6127worker1 # [ 123.369897] nix-daemon[847]: accepted connection from pid 1261, user nix-grpc-daemon (trusted)6128client # [ 123.369422] nix[1358]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6129lb: must succeed: curl -sf localhost:9901/clusters6130lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6131lb: must succeed: curl -sf localhost:9901/clusters6132lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6133lb: must succeed: curl -sf localhost:9901/clusters6134lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6135lb: must succeed: curl -sf localhost:9901/clusters6136lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6137worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6138worker2 # [ 125.568077] systemd-journald[285]: Received client request to rotate journal, rotating.6139worker2 # [ 125.588401] systemd-journald[285]: Vacuuming done, freed 0B of archived journals from /var/log/journal/4fa082a50d854427bd7f5936381953f0.6140worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6141worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6142worker1 # [ 125.511707] systemd[1]: Stopping Nix worker-protocol over gRPC...6143worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)6144client: must succeed: sleep 5.06145worker1 # [ 126.107515] nix-grpc-daemon[1261]: ts=2026-09-23T05:34:58Z level=info event=draining6146lb # [ 127.984464] envoy[929]: conn peer=[2001:db8:1::1]:45890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9586 tx=4347 ms=46706147client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6148worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6149worker1 # [ 130.810718] nix-daemon[847]: reaped child process 1461, status = succeeded6150worker1 # [ 130.896957] nix-daemon[847]: accepted connection from pid 1502, user nix-grpc-daemon (trusted)6151worker1 # [ 130.911083] nix-daemon[847]: reaped child process 1505, status = succeeded6152lb # [ 130.921503] niks3-server[1135]: 2026/09/23 05:35:03 INFO Received uploads request method=POST path=/api/pending_closures6153worker1 # [ 130.932363] nix-grpc-daemon[1338]: time=2026-09-23T05:35:03.139Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6154worker1 # [ 130.933868] nix-grpc-daemon[1338]: time=2026-09-23T05:35:03.141Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6155lb # [ 130.970226] niks3-server[1135]: 2026/09/23 05:35:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6156worker1 # [ 130.974989] nix-grpc-daemon[1338]: time=2026-09-23T05:35:03.182Z level=INFO msg="Uploading 1 narinfos"6157lb # [ 130.975362] niks3-server[1135]: 2026/09/23 05:35:03 INFO Signed narinfos id=50 count=16158lb # [ 130.993664] niks3-server[1135]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6159lb # [ 130.996451] niks3-server[1135]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6160lb # [ 131.000654] niks3-server[1135]: 2026/09/23 05:35:03 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6161lb # [ 131.005077] niks3-server[1135]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6162lb # [ 131.007987] niks3-server[1135]: 2026/09/23 05:35:03 INFO Completed upload id=506163worker1 # [ 131.008646] nix-grpc-daemon[1338]: time=2026-09-23T05:35:03.215Z level=INFO msg="Upload complete. (223ms)"6164lb # [ 131.012173] niks3-server[1135]: 2026/09/23 05:35:03 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6165worker1 # [ 131.017725] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37216 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16166worker1 # [ 131.032241] nix-daemon[847]: reaped child process 1459, status = succeeded6167client # [ 131.045452] systemd[1]: rs.service: Deactivated successfully.6168client # [ 131.049140] systemd[1]: rs.service: Consumed 125ms CPU time over 8.150s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.2K outgoing IP traffic.6169worker1 # [ 131.109339] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:03Z level=info event=signal_exit6170lb # [ 131.181434] niks3-server[1135]: 2026/09/23 05:35:03 INFO lead: released remote=192.168.1.3:410086171worker1 # [ 132.185882] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=scheduler_restarting6172lb # [ 132.184698] envoy[929]: [2026-09-23 05:35:04.445][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6173worker1 # [ 132.192816] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=scheduler_restarting addr=lb:500516174worker2 # [ 132.182746] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_restarting addr=lb:500516175worker1 # [ 132.199394] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:531726176worker1 # [ 132.203261] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=worker_gone addr=192.168.1.3:500516177worker1 # [ 132.206885] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:531726178worker1 # [ 132.210969] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=worker_gone addr=192.168.1.4:500516179worker1 # [ 132.231561] nix-daemon[847]: reaped child process 1316, status = succeeded6180worker1 # [ 132.235611] nix-grpc-daemon[1261]: E0000 00:00:1790141704.442785 1307 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true6181worker2 # [ 132.383404] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_connected addr=lb:500516182worker2 # [ 132.387689] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6183worker1 # [ 132.407817] nix-grpc-daemon[1261]: ts=2026-09-23T05:35:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6184worker2 # [ 132.588880] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_connected addr=lb:500516185worker2 # [ 132.593958] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6186lb # [ 132.616845] niks3-server[1135]: 2026/09/23 05:35:04 INFO lead: acquired remote=192.168.1.4:331206187worker2 # [ 132.609486] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:04Z level=info event=scheduler_take_over6188worker1 # [ 132.678375] nix-grpc-daemon[1261]: E0000 00:00:1790141704.885360 1307 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6189worker1 # [ 132.779565] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6190worker1 # [ 132.783081] systemd[1]: Stopped Nix worker-protocol over gRPC.6191worker1 # [ 132.785620] systemd[1]: nix-grpc-daemon.service: Consumed 1.206s CPU time over 31.801s wall clock time, 38.2M memory peak, 47.5K incoming IP traffic, 98.3K outgoing IP traffic.6192worker1 # [ 132.801558] systemd[1]: Starting Nix worker-protocol over gRPC...6193worker2 # [ 132.794746] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:500516194worker2 # [ 132.799459] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6195worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.12 seconds)6196client: waiting for success: ! systemctl is-active rs6197client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6198client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6199worker1 # [ 132.899263] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6200worker1 # [ 132.903951] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6201worker1 # [ 132.908238] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16202worker1 # [ 132.912515] nix-grpc-daemon[1512]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6203worker1 # [ 132.916481] nix-grpc-daemon[1512]: E0000 00:00:1790141705.114317 1512 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6204worker1 # [ 132.919557] nix-grpc-daemon[1512]: E0000 00:00:1790141705.118271 1512 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6205worker1 # [ 132.922493] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6206worker1 # [ 132.924321] systemd[1]: Started Nix worker-protocol over gRPC.6207client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6208worker1 # [ 132.926421] nix-grpc-daemon[1512]: E0000 00:00:1790141705.132761 1554 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6209client: must fail: journalctl -u rs | grep -q 'reconnecting'6210worker1 # [ 132.937572] nix-grpc-daemon[1512]: E0000 00:00:1790141705.144862 1554 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6211worker1 # [ 132.942056] nix-grpc-daemon[1512]: E0000 00:00:1790141705.147184 1554 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6212client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6213lb: must succeed: curl -sf localhost:9901/clusters6214worker1 # [ 132.951809] nix-grpc-daemon[1512]: E0000 00:00:1790141705.159069 1554 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6215worker1 # [ 132.964248] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:500516216worker1 # [ 132.966414] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6217lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6218worker2 # [ 133.000123] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:500516219worker2 # [ 133.004080] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6220worker1 # [ 133.167966] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:500516221worker1 # [ 133.172254] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6222worker2 # [ 133.206183] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:500516223worker2 # [ 133.210141] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6224worker1 # [ 133.373562] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:500516225worker2 # [ 133.368279] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:384686226worker2 # [ 133.372220] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06227worker2 # [ 133.411136] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=scheduler_connected addr=lb:500516228worker2 # [ 133.421143] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=oidc_jwks provider=mock keys=16229worker2 # [ 133.424920] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:384686230worker2 # [ 133.428847] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06231lb: must succeed: curl -sf localhost:9901/clusters6232lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6233worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6234worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6235(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.24 seconds)6236subtest: scheduler restart mid-build: build finishes, no double build6237client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6238client # [ 134.135852] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6239client # Running as unit: mid.service; invocation ID: 7e078066fc7843f0bf57dce8d9225dfd6240client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6241worker2 # [ 134.271319] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:384806242worker2 # [ 134.282142] nix-daemon[870]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)6243client # [ 134.305653] nix[1394]: don't know how to build these paths:6244client # [ 134.306642] nix[1394]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6245worker2 # [ 134.304101] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:06Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06246worker2 # [ 134.308757] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:06Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16247worker1 # [ 134.374901] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404406248worker1 # [ 134.386515] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)6249worker1 # [ 134.403801] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5)6250worker1 # [ 134.493950] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 2/5)6251worker1 # [ 134.515636] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 271 ms (attempt 3/5)6252worker1 # [ 134.789838] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 509 ms (attempt 4/5)6253worker1 # [ 135.306513] nix-grpc-daemon[1512]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6254worker1 # [ 135.309624] nix-grpc-daemon[1512]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6255worker1 # [ 135.326885] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404406256client # [ 135.337799] nix[1394]: copying 1 paths to 'lb:50051'...6257client # [ 135.342189] nix[1394]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6258worker1 # [ 135.341815] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:404406259worker1 # [ 135.350690] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)6260worker1 # [ 135.372121] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=info event=niks3_push_started pid=15816261worker1 # [ 135.477848] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.684Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6262worker1 # [ 135.574096] nix-daemon[847]: accepted connection from pid 1594, user nix-grpc-daemon (trusted)6263worker1 # [ 135.588213] nix-daemon[847]: reaped child process 1597, status = succeeded6264lb # [ 135.596583] niks3-server[1135]: 2026/09/23 05:35:07 INFO Received uploads request method=POST path=/api/pending_closures6265worker2 # [ 135.586982] nix-grpc-daemon[1453]: ts=2026-09-23T05:35:07Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:384866266worker1 # [ 135.613140] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.820Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6267worker1 # [ 135.614679] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.822Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6268lb # [ 135.642087] niks3-server[1135]: 2026/09/23 05:35:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6269lb # [ 135.647537] niks3-server[1135]: 2026/09/23 05:35:07 INFO Signed narinfos id=51 count=16270worker1 # [ 135.651967] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.859Z level=INFO msg="Uploading 1 narinfos"6271lb # [ 135.652955] niks3-server[1135]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6272lb # [ 135.658705] niks3-server[1135]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6273lb # [ 135.669634] niks3-server[1135]: 2026/09/23 05:35:07 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6274lb # [ 135.672938] niks3-server[1135]: 2026/09/23 05:35:07 INFO Completed upload id=516275worker1 # [ 135.674614] nix-grpc-daemon[1581]: time=2026-09-23T05:35:07.881Z level=INFO msg="Upload complete. (193ms)"6276lb # [ 135.674912] niks3-server[1135]: 2026/09/23 05:35:07 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6277worker1 # [ 135.678648] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40440 duration_s=0 paths=1 nar_bytes_in=4246278worker1 # [ 135.684262] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404406279worker1 # [ 135.693741] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)6280worker1 # [ 135.698195] nix-daemon[847]: reaped child process 1579, status = succeeded6281worker1 # [ 135.702513] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)6282client # [ 135.701970] nix[1394]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6283lb: must succeed: curl -sf localhost:9901/clusters6284lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6285lb: must succeed: curl -sf localhost:9901/clusters6286lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6287worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6288worker2 # [ 136.636491] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1453 (nix-grpc-daemon) on client request.6289worker2 # [ 136.646633] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6290lb # [ 136.664327] niks3-server[1135]: 2026/09/23 05:35:08 INFO lead: released remote=192.168.1.4:331206291worker1 # [ 136.671854] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:08Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6292client # [ 136.672482] nix[1394]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6293lb # [ 136.671677] envoy[929]: [2026-09-23 05:35:08.928][929][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}6294worker2 # [ 136.666670] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6295worker2 # [ 136.668724] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6296worker2 # [ 136.669965] systemd[1]: nix-grpc-daemon.service: Consumed 229ms CPU time over 14.158s wall clock time, 14.1M memory peak, 10.6K incoming IP traffic, 21.3K outgoing IP traffic.6297worker2 # [ 136.680060] systemd[1]: Starting scheduler-token.service...6298worker2 # [ 136.684214] nix-daemon[870]: reaped child process 1563, status = succeeded6299worker2 # [ 136.716441] systemd[1]: scheduler-token.service: Deactivated successfully.6300worker2 # [ 136.718072] systemd[1]: Finished scheduler-token.service.6301worker2 # [ 136.719836] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 38ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6302worker2 # [ 136.721984] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6303worker2 # [ 136.725412] systemd[1]: Starting scheduler-token.service...6304worker2 # [ 136.759378] systemd[1]: scheduler-token.service: Deactivated successfully.6305worker2 # [ 136.760967] systemd[1]: Finished scheduler-token.service.6306worker2 # [ 136.762140] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 35ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6307worker2 # [ 136.767132] systemd[1]: Starting Nix worker-protocol over gRPC...6308worker2 # [ 136.843717] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6309worker2 # [ 136.848049] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6310worker2 # [ 136.851401] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16311worker2 # [ 136.858287] nix-grpc-daemon[1593]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6312worker1 # [ 136.873073] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:09Z level=info event=scheduler_connected addr=lb:500516313worker2 # [ 136.861693] nix-grpc-daemon[1593]: E0000 00:00:1790141709.201359 1593 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6314worker2 # [ 136.865102] nix-grpc-daemon[1593]: E0000 00:00:1790141709.205335 1593 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6315worker2 # [ 136.870088] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6316worker2 # [ 136.872350] systemd[1]: Started Nix worker-protocol over gRPC.6317lb # [ 136.882381] niks3-server[1135]: 2026/09/23 05:35:09 INFO lead: acquired remote=192.168.1.4:537706318worker2 # [ 136.874161] nix-grpc-daemon[1593]: E0000 00:00:1790141709.224679 1634 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6319worker2 # [ 136.878122] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=scheduler_take_over6320worker2 # [ 136.882662] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:385126321worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.28 seconds)6322client: must succeed: sleep 5.06323worker2 # [ 136.888630] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:385286324worker2 # [ 136.893101] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=16325worker2 # [ 136.903062] nix-grpc-daemon[1593]: E0000 00:00:1790141709.252718 1634 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6326worker2 # [ 136.905189] nix-grpc-daemon[1593]: E0000 00:00:1790141709.252810 1634 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6327worker2 # [ 136.916271] nix-grpc-daemon[1593]: E0000 00:00:1790141709.266737 1634 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6328worker2 # [ 136.925230] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=scheduler_connected addr=lb:500516329worker2 # [ 136.930066] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=oidc_jwks provider=mock keys=16330worker2 # [ 136.931675] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:385286331worker2 # [ 136.935612] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06332lb # [ 137.984757] envoy[929]: conn peer=[2001:db8:1::1]:45730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2943 tx=1521 ms=79926333lb # [ 137.988833] envoy[929]: conn peer=[2001:db8:1::3]:46462 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3049 tx=870 ms=312576334lb # [ 137.992416] envoy[929]: conn peer=[2001:db8:1::4]:55140 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6171 tx=512 ms=140206335client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6336client: waiting for success: ! systemctl is-active mid6337worker1 # [ 142.122745] nix-daemon[847]: reaped child process 1601, status = succeeded6338worker1 # [ 142.198899] nix-daemon[847]: accepted connection from pid 1634, user nix-grpc-daemon (trusted)6339worker1 # [ 142.210046] nix-daemon[847]: reaped child process 1637, status = succeeded6340lb # [ 142.218421] niks3-server[1135]: 2026/09/23 05:35:14 INFO Received uploads request method=POST path=/api/pending_closures6341worker1 # [ 142.228383] nix-grpc-daemon[1581]: time=2026-09-23T05:35:14.435Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6342worker1 # [ 142.231061] nix-grpc-daemon[1581]: time=2026-09-23T05:35:14.437Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6343lb # [ 142.266707] niks3-server[1135]: 2026/09/23 05:35:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6344lb # [ 142.273908] niks3-server[1135]: 2026/09/23 05:35:14 INFO Signed narinfos id=52 count=16345worker1 # [ 142.278494] nix-grpc-daemon[1581]: time=2026-09-23T05:35:14.485Z level=INFO msg="Uploading 1 narinfos"6346lb # [ 142.288664] niks3-server[1135]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6347lb # [ 142.303888] niks3-server[1135]: 2026/09/23 05:35:14 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6348lb # [ 142.308573] niks3-server[1135]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6349lb # [ 142.318032] niks3-server[1135]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6350lb # [ 142.322370] niks3-server[1135]: 2026/09/23 05:35:14 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6351lb # [ 142.326855] niks3-server[1135]: 2026/09/23 05:35:14 INFO Completed upload id=526352worker1 # [ 142.327482] nix-grpc-daemon[1581]: time=2026-09-23T05:35:14.534Z level=INFO msg="Upload complete. (213ms)"6353worker1 # [ 142.334678] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40440 duration_s=6 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16354worker1 # [ 142.349582] nix-daemon[847]: reaped child process 1599, status = succeeded6355client # [ 142.365363] systemd[1]: mid.service: Deactivated successfully.6356client # [ 142.369101] systemd[1]: mid.service: Consumed 132ms CPU time over 8.232s wall clock time, 43.4M memory peak, 7.6K incoming IP traffic, 8.6K outgoing IP traffic.6357client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6358client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6359client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6360(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.14 seconds)6361subtest: interrupting the client stops the build on the worker6362client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6363client # [ 143.247500] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6364client # Running as unit: intr.service; invocation ID: 03b278f3d08d4d8890724ec39aa85e826365client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6366worker2 # [ 143.357923] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:559846367worker2 # [ 143.367870] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6368client # [ 143.389987] nix[1430]: don't know how to build these paths:6369client # [ 143.391023] nix[1430]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6370worker2 # [ 143.390141] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:15Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06371worker2 # [ 143.394121] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:15Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16372worker2 # [ 143.434353] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846373worker2 # [ 143.446625] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5)6374worker2 # [ 143.477299] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 2/5)6375worker2 # [ 143.530190] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 290 ms (attempt 3/5)6376worker2 # [ 143.827366] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 260 ms (attempt 4/5)6377worker2 # [ 144.090363] nix-grpc-daemon[1593]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6378worker2 # [ 144.093351] nix-grpc-daemon[1593]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6379worker2 # [ 144.108452] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559846380client # [ 144.127971] nix[1430]: copying 1 paths to 'lb:50051'...6381client # [ 144.132148] nix[1430]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6382worker2 # [ 144.121411] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559846383worker2 # [ 144.131164] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6384worker2 # [ 144.152311] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=info event=niks3_push_started pid=16686385worker2 # [ 144.254450] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.604Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6386worker2 # [ 144.339087] nix-daemon[870]: accepted connection from pid 1681, user nix-grpc-daemon (trusted)6387worker2 # [ 144.350281] nix-daemon[870]: reaped child process 1684, status = succeeded6388lb # [ 144.369345] niks3-server[1135]: 2026/09/23 05:35:16 INFO Received uploads request method=POST path=/api/pending_closures6389worker2 # [ 144.364529] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.715Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6390worker2 # [ 144.366207] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.716Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6391lb # [ 144.393533] niks3-server[1135]: 2026/09/23 05:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6392lb # [ 144.396691] niks3-server[1135]: 2026/09/23 05:35:16 INFO Signed narinfos id=53 count=16393worker2 # [ 144.386317] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.736Z level=INFO msg="Uploading 1 narinfos"6394lb # [ 144.398154] niks3-server[1135]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6395lb # [ 144.403072] niks3-server[1135]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6396lb # [ 144.406985] niks3-server[1135]: 2026/09/23 05:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6397lb # [ 144.411079] niks3-server[1135]: 2026/09/23 05:35:16 INFO Completed upload id=536398worker2 # [ 144.401137] nix-grpc-daemon[1668]: time=2026-09-23T05:35:16.751Z level=INFO msg="Upload complete. (142ms)"6399lb # [ 144.413246] niks3-server[1135]: 2026/09/23 05:35:16 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6400worker2 # [ 144.403361] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=4246401worker2 # [ 144.407549] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846402worker2 # [ 144.416767] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6403worker2 # [ 144.420209] nix-daemon[870]: reaped child process 1666, status = succeeded6404worker2 # [ 144.427669] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6405client # [ 144.436975] nix[1430]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6406client: must succeed: systemctl kill -s INT intr6407client # [ 146.657167] systemd[1]: intr.service: Sent signal SIGINT to main process 1430 (nix) on client request.6408worker2 # [ 146.648434] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:18Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6409client # [ 146.663176] nix[1430]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6410client # [ 146.667952] nix[1430]: error: interrupted by the user6411client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6412client # [ 146.697677] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6413client # [ 146.700574] systemd[1]: intr.service: Failed with result 'exit-code'.6414client # [ 146.703946] systemd[1]: intr.service: Consumed 118ms CPU time over 3.453s wall clock time, 43.4M memory peak, 6.4K incoming IP traffic, 8.4K outgoing IP traffic.6415worker2 # [ 146.848356] nix-daemon[870]: reaped child process 1690, status = succeeded6416worker2 # [ 146.851267] nix-daemon[1692]: unexpected Nix daemon error: error: interrupted by the user6417worker2 # [ 146.855399] nix-daemon[870]: reaped child process 1692, status = failed with exit code 16418(finished: subtest: interrupting the client stops the build on the worker, in 4.60 seconds)6419subtest: a deploy mid-build drains: the build finishes, the new generation takes over6420client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6421client # [ 147.869555] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6422client # Running as unit: sw.service; invocation ID: bbf402ed07a34245875e21c1895d9ae16423client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6424lb # [ 147.984358] envoy[929]: conn peer=[2001:db8:1::1]:49868 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1590 ms=80726425lb # [ 147.988151] envoy[929]: conn peer=[2001:db8:1::1]:53634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=828 ms=33186426worker2 # [ 147.994805] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:559846427client # [ 148.011628] nix[1450]: don't know how to build these paths:6428client # [ 148.012582] nix[1450]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6429worker2 # [ 148.008142] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:20Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06430worker2 # [ 148.012558] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:20Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16431worker1 # [ 148.072492] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418486432worker1 # [ 148.081033] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418486433client # [ 148.087162] nix[1450]: copying 1 paths to 'lb:50051'...6434worker1 # [ 148.090801] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:418486435client # [ 148.089875] nix[1450]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6436worker1 # [ 148.096988] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)6437worker1 # [ 148.180129] nix-daemon[847]: accepted connection from pid 1669, user nix-grpc-daemon (trusted)6438worker1 # [ 148.192629] nix-daemon[847]: reaped child process 1672, status = succeeded6439lb # [ 148.199456] niks3-server[1135]: 2026/09/23 05:35:20 INFO Received uploads request method=POST path=/api/pending_closures6440worker1 # [ 148.207780] nix-grpc-daemon[1581]: time=2026-09-23T05:35:20.414Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6441worker1 # [ 148.209853] nix-grpc-daemon[1581]: time=2026-09-23T05:35:20.416Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6442lb # [ 148.229270] niks3-server[1135]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6443lb # [ 148.235275] niks3-server[1135]: 2026/09/23 05:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6444lb # [ 148.241070] niks3-server[1135]: 2026/09/23 05:35:20 INFO Signed narinfos id=54 count=16445worker1 # [ 148.244506] nix-grpc-daemon[1581]: time=2026-09-23T05:35:20.451Z level=INFO msg="Uploading 1 narinfos"6446lb # [ 148.250525] niks3-server[1135]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6447lb # [ 148.259488] niks3-server[1135]: 2026/09/23 05:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6448lb # [ 148.267688] niks3-server[1135]: 2026/09/23 05:35:20 INFO Completed upload id=546449worker1 # [ 148.270408] nix-grpc-daemon[1581]: time=2026-09-23T05:35:20.477Z level=INFO msg="Upload complete. (167ms)"6450lb # [ 148.271043] niks3-server[1135]: 2026/09/23 05:35:20 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6451worker1 # [ 148.272793] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41848 duration_s=0 paths=1 nar_bytes_in=4166452worker1 # [ 148.279202] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:418486453worker1 # [ 148.282986] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)6454worker1 # [ 148.285231] nix-daemon[847]: reaped child process 1667, status = succeeded6455worker1 # [ 148.290414] nix-daemon[847]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)6456client # [ 148.289931] nix[1450]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6457worker1: must succeed: systemctl reload nix-grpc-daemon.service6458worker1 # [ 149.161446] systemd[1]: Reloading Nix worker-protocol over gRPC...6459worker1 # [ 149.188584] systemd[1]: Reloaded Nix worker-protocol over gRPC.6460worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6461worker1: must succeed: pgrep -f 'read -t [9]0 x'6462worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6463client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26464worker2 # [ 149.358328] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:559846465client # don't know how to build these paths:6466client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6467worker1 # [ 149.378492] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:418486468worker2 # [ 149.388529] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06469worker2 # [ 149.393692] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16470worker2 # [ 149.445114] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846471worker2 # [ 149.458483] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559846472client # copying 1 paths to 'lb:50051'...6473worker2 # [ 149.471692] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559846474client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6475worker2 # [ 149.492150] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6476worker2 # [ 149.591263] nix-daemon[870]: accepted connection from pid 1743, user nix-grpc-daemon (trusted)6477worker2 # [ 149.604315] nix-daemon[870]: reaped child process 1746, status = succeeded6478lb # [ 149.621284] niks3-server[1135]: 2026/09/23 05:35:21 INFO Received uploads request method=POST path=/api/pending_closures6479worker2 # [ 149.619977] nix-grpc-daemon[1668]: time=2026-09-23T05:35:21.970Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6480worker2 # [ 149.622250] nix-grpc-daemon[1668]: time=2026-09-23T05:35:21.972Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6481lb # [ 149.651706] niks3-server[1135]: 2026/09/23 05:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6482lb # [ 149.657394] niks3-server[1135]: 2026/09/23 05:35:21 INFO Signed narinfos id=55 count=16483worker2 # [ 149.652771] nix-grpc-daemon[1668]: time=2026-09-23T05:35:22.002Z level=INFO msg="Uploading 1 narinfos"6484lb # [ 149.668312] niks3-server[1135]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6485lb # [ 149.672820] niks3-server[1135]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6486lb # [ 149.683862] niks3-server[1135]: 2026/09/23 05:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6487lb # [ 149.686922] niks3-server[1135]: 2026/09/23 05:35:21 INFO Completed upload id=556488worker2 # [ 149.676427] nix-grpc-daemon[1668]: time=2026-09-23T05:35:22.026Z level=INFO msg="Upload complete. (172ms)"6489lb # [ 149.689932] niks3-server[1135]: 2026/09/23 05:35:21 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6490worker2 # [ 149.680679] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=5286491worker2 # [ 149.685616] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846492worker2 # [ 149.694458] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6493worker2 # [ 149.700482] nix-daemon[870]: reaped child process 1741, status = succeeded6494worker2 # [ 149.704846] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6495client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6496client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6497worker1 # [ 149.919576] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:22Z level=info event=draining6498client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6499client # farm-a> LOG-farm-a-during-drain6500worker2 # [ 150.741347] nix-daemon[870]: reaped child process 1750, status = succeeded6501worker2 # [ 150.808594] nix-daemon[870]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6502worker2 # [ 150.820584] nix-daemon[870]: reaped child process 1769, status = succeeded6503lb # [ 150.839863] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received uploads request method=POST path=/api/pending_closures6504worker2 # [ 150.837911] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.188Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6505worker2 # [ 150.839630] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.189Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6506lb # [ 150.880856] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6507lb # [ 150.887347] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6508lb # [ 150.900410] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6509lb # [ 150.904472] niks3-server[1135]: 2026/09/23 05:35:23 INFO Signed narinfos id=56 count=16510worker2 # [ 150.896583] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.246Z level=INFO msg="Uploading 1 narinfos"6511lb # [ 150.911263] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6512lb # [ 150.921416] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6513lb # [ 150.925239] niks3-server[1135]: 2026/09/23 05:35:23 INFO Completed upload id=566514lb # [ 150.927249] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6515worker2 # [ 150.916221] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.266Z level=INFO msg="Upload complete. (203ms)"6516worker2 # [ 150.924065] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=3 outputs=16517worker2 # [ 150.934816] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06518worker2 # [ 150.939309] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16519worker2 # [ 150.948718] nix-daemon[870]: reaped child process 1748, status = succeeded6520worker2 # [ 150.979620] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846521worker2 # [ 150.994714] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559846522worker2 # [ 151.007839] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.358Z level=INFO msg="All 1 paths already cached"6523client # copying 1 paths to 'lb:50051'...6524worker2 # [ 151.016570] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559846525client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6526worker2 # [ 151.036140] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6527worker2 # [ 151.132877] nix-daemon[870]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6528worker2 # [ 151.145784] nix-daemon[870]: reaped child process 1778, status = succeeded6529lb # [ 151.163458] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received uploads request method=POST path=/api/pending_closures6530worker2 # [ 151.160377] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.510Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6531worker2 # [ 151.161925] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.512Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6532lb # [ 151.195064] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6533worker2 # [ 151.191976] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.542Z level=INFO msg="Uploading 1 narinfos"6534lb # [ 151.203725] niks3-server[1135]: 2026/09/23 05:35:23 INFO Signed narinfos id=57 count=16535lb # [ 151.209772] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6536lb # [ 151.216103] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6537lb # [ 151.228817] niks3-server[1135]: 2026/09/23 05:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6538lb # [ 151.235245] niks3-server[1135]: 2026/09/23 05:35:23 INFO Completed upload id=576539worker2 # [ 151.224782] nix-grpc-daemon[1668]: time=2026-09-23T05:35:23.575Z level=INFO msg="Upload complete. (176ms)"6540lb # [ 151.241186] niks3-server[1135]: 2026/09/23 05:35:23 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6541worker2 # [ 151.228871] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=6406542worker2 # [ 151.236239] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846543worker2 # [ 151.244912] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6544worker2 # [ 151.254823] nix-daemon[870]: reaped child process 1773, status = succeeded6545worker2 # [ 151.259083] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6546client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6547client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6548client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6549client # farm-b> LOG-farm-b-during-drain6550worker2 # [ 152.494829] nix-daemon[870]: reaped child process 1782, status = succeeded6551worker2 # [ 152.579087] nix-daemon[870]: accepted connection from pid 1798, user nix-grpc-daemon (trusted)6552worker2 # [ 152.592775] nix-daemon[870]: reaped child process 1801, status = succeeded6553lb # [ 152.608769] niks3-server[1135]: 2026/09/23 05:35:24 INFO Received uploads request method=POST path=/api/pending_closures6554worker2 # [ 152.609277] nix-grpc-daemon[1668]: time=2026-09-23T05:35:24.959Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6555worker2 # [ 152.610894] nix-grpc-daemon[1668]: time=2026-09-23T05:35:24.961Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6556lb # [ 152.652165] niks3-server[1135]: 2026/09/23 05:35:24 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6557lb # [ 152.664827] niks3-server[1135]: 2026/09/23 05:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6558lb # [ 152.670055] niks3-server[1135]: 2026/09/23 05:35:24 INFO Signed narinfos id=58 count=16559worker2 # [ 152.662331] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.012Z level=INFO msg="Uploading 1 narinfos"6560lb # [ 152.676843] niks3-server[1135]: 2026/09/23 05:35:24 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6561lb # [ 152.681447] niks3-server[1135]: 2026/09/23 05:35:24 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6562lb # [ 152.690243] niks3-server[1135]: 2026/09/23 05:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6563lb # [ 152.694188] niks3-server[1135]: 2026/09/23 05:35:24 INFO Completed upload id=586564worker2 # [ 152.683875] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.034Z level=INFO msg="Upload complete. (217ms)"6565lb # [ 152.696995] niks3-server[1135]: 2026/09/23 05:35:24 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6566worker2 # [ 152.691892] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=4 outputs=16567worker2 # [ 152.703989] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06568worker2 # [ 152.708178] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16569worker2 # [ 152.717331] nix-daemon[870]: reaped child process 1780, status = succeeded6570worker2 # [ 152.748237] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846571worker2 # [ 152.760849] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559846572worker2 # [ 152.774376] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.124Z level=INFO msg="All 2 paths already cached"6573client # copying 1 paths to 'lb:50051'...6574client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6575worker2 # [ 152.783553] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559846576worker2 # [ 152.794377] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6577worker2 # [ 152.889648] nix-daemon[870]: accepted connection from pid 1807, user nix-grpc-daemon (trusted)6578worker2 # [ 152.902091] nix-daemon[870]: reaped child process 1810, status = succeeded6579lb # [ 152.919936] niks3-server[1135]: 2026/09/23 05:35:25 INFO Received uploads request method=POST path=/api/pending_closures6580worker2 # [ 152.920448] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.270Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6581worker2 # [ 152.922087] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.272Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6582lb # [ 152.955960] niks3-server[1135]: 2026/09/23 05:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6583lb # [ 152.961487] niks3-server[1135]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6584worker2 # [ 152.958197] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.308Z level=INFO msg="Uploading 1 narinfos"6585lb # [ 152.969314] niks3-server[1135]: 2026/09/23 05:35:25 INFO Signed narinfos id=59 count=16586lb # [ 152.978640] niks3-server[1135]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6587lb # [ 152.990709] niks3-server[1135]: 2026/09/23 05:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6588lb # [ 152.998530] niks3-server[1135]: 2026/09/23 05:35:25 INFO Completed upload id=596589worker2 # [ 152.988155] nix-grpc-daemon[1668]: time=2026-09-23T05:35:25.338Z level=INFO msg="Upload complete. (180ms)"6590lb # [ 153.000611] niks3-server[1135]: 2026/09/23 05:35:25 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6591worker2 # [ 152.992394] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=7686592worker2 # [ 152.998423] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846593worker2 # [ 153.008873] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6594worker2 # [ 153.020808] nix-daemon[870]: reaped child process 1805, status = succeeded6595worker2 # [ 153.023962] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6596client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6597client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6598client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6599client # farm-top> LOG-farm-top-during-drain6600worker2 # [ 154.256672] nix-daemon[870]: reaped child process 1814, status = succeeded6601worker2 # [ 154.341384] nix-daemon[870]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6602worker2 # [ 154.354336] nix-daemon[870]: reaped child process 1833, status = succeeded6603lb # [ 154.371853] niks3-server[1135]: 2026/09/23 05:35:26 INFO Received uploads request method=POST path=/api/pending_closures6604worker2 # [ 154.372938] nix-grpc-daemon[1668]: time=2026-09-23T05:35:26.723Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6605worker2 # [ 154.375417] nix-grpc-daemon[1668]: time=2026-09-23T05:35:26.725Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6606lb # [ 154.413101] niks3-server[1135]: 2026/09/23 05:35:26 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6607lb # [ 154.425462] niks3-server[1135]: 2026/09/23 05:35:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6608lb # [ 154.430324] niks3-server[1135]: 2026/09/23 05:35:26 INFO Signed narinfos id=60 count=16609worker2 # [ 154.422536] nix-grpc-daemon[1668]: time=2026-09-23T05:35:26.772Z level=INFO msg="Uploading 1 narinfos"6610lb # [ 154.442378] niks3-server[1135]: 2026/09/23 05:35:26 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6611lb # [ 154.449955] niks3-server[1135]: 2026/09/23 05:35:26 INFO Completed upload id=606612worker2 # [ 154.439595] nix-grpc-daemon[1668]: time=2026-09-23T05:35:26.790Z level=INFO msg="Upload complete. (206ms)"6613lb # [ 154.452337] niks3-server[1135]: 2026/09/23 05:35:26 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6614lb # [ 154.455436] niks3-server[1135]: 2026/09/23 05:35:26 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6615lb # [ 154.458381] niks3-server[1135]: 2026/09/23 05:35:26 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6616worker2 # [ 154.445564] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=5 outputs=16617worker2 # [ 154.460740] nix-daemon[870]: reaped child process 1812, status = succeeded6618client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.31 seconds)6619client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6620client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)6621worker1: must succeed: pkill -f 'read -t [9]0 x'6622worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6623client: waiting for success: ! systemctl is-active sw6624worker1 # [ 154.710128] nix-daemon[847]: reaped child process 1676, status = succeeded6625worker1 # [ 154.781282] nix-daemon[847]: accepted connection from pid 1714, user nix-grpc-daemon (trusted)6626worker1 # [ 154.794397] nix-daemon[847]: reaped child process 1717, status = succeeded6627lb # [ 154.799853] niks3-server[1135]: 2026/09/23 05:35:27 INFO Received uploads request method=POST path=/api/pending_closures6628worker1 # [ 154.809058] nix-grpc-daemon[1581]: time=2026-09-23T05:35:27.016Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6629worker1 # [ 154.811235] nix-grpc-daemon[1581]: time=2026-09-23T05:35:27.018Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6630lb # [ 154.840283] niks3-server[1135]: 2026/09/23 05:35:27 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6631lb # [ 154.849980] niks3-server[1135]: 2026/09/23 05:35:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6632lb # [ 154.856660] niks3-server[1135]: 2026/09/23 05:35:27 INFO Signed narinfos id=61 count=16633worker1 # [ 154.861103] nix-grpc-daemon[1581]: time=2026-09-23T05:35:27.068Z level=INFO msg="Uploading 1 narinfos"6634lb # [ 154.860737] niks3-server[1135]: 2026/09/23 05:35:27 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6635lb # [ 154.868299] niks3-server[1135]: 2026/09/23 05:35:27 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6636lb # [ 154.878696] niks3-server[1135]: 2026/09/23 05:35:27 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6637lb # [ 154.884749] niks3-server[1135]: 2026/09/23 05:35:27 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6638lb # [ 154.887905] niks3-server[1135]: 2026/09/23 05:35:27 INFO Completed upload id=616639worker1 # [ 154.888589] nix-grpc-daemon[1581]: time=2026-09-23T05:35:27.095Z level=INFO msg="Upload complete. (187ms)"6640worker1 # [ 154.895848] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41848 duration_s=6 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=2 outputs=16641worker1 # [ 154.910121] nix-daemon[847]: reaped child process 1674, status = succeeded6642worker1 # [ 154.920310] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:27Z level=info event=signal_exit6643client # [ 154.924668] systemd[1]: sw.service: Deactivated successfully.6644client # [ 154.929168] systemd[1]: sw.service: Consumed 120ms CPU time over 7.058s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.5K outgoing IP traffic.6645client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6646client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6647client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6648worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6649worker2 # [ 155.924308] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:385286650worker2 # [ 155.930879] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:28Z level=info event=worker_gone addr=192.168.1.3:500516651worker1 # [ 155.934932] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:28Z level=info event=scheduler_restarting6652worker1 # [ 155.950185] nix-grpc-daemon[1512]: ts=2026-09-23T05:35:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6653worker1 # [ 155.972176] nix-daemon[847]: reaped child process 1569, status = succeeded6654worker1 # [ 156.825414] nix-grpc-daemon[1512]: E0000 00:00:1790141729.032545 1558 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6655worker1 # [ 156.952477] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6656worker1 # [ 156.957368] systemd[1]: nix-grpc-daemon.service: Consumed 644ms CPU time over 24.153s wall clock time, 35.8M memory peak, 25.1K incoming IP traffic, 35K outgoing IP traffic.6657worker1 # [ 156.972659] systemd[1]: Starting Nix worker-protocol over gRPC...6658worker1 # [ 157.056684] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6659worker1 # [ 157.058695] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6660worker1 # [ 157.061942] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16661worker1 # [ 157.066327] nix-grpc-daemon[1731]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6662worker1 # [ 157.067660] nix-grpc-daemon[1731]: E0000 00:00:1790141729.273547 1731 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6663worker1 # [ 157.073492] nix-grpc-daemon[1731]: E0000 00:00:1790141729.278836 1731 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6664worker1 # [ 157.077989] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6665worker1 # [ 157.081149] systemd[1]: Started Nix worker-protocol over gRPC.6666worker1 # [ 157.084090] nix-grpc-daemon[1731]: E0000 00:00:1790141729.290382 1773 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6667worker1 # [ 157.096832] nix-grpc-daemon[1731]: E0000 00:00:1790141729.304169 1773 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6668worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.32 seconds)6669lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux6670lb: must succeed: curl -sf localhost:9901/clusters6671worker1 # [ 157.100295] nix-grpc-daemon[1731]: E0000 00:00:1790141729.306451 1773 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6672worker1 # [ 157.111206] nix-grpc-daemon[1731]: E0000 00:00:1790141729.318517 1773 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6673worker1 # [ 157.124211] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=info event=scheduler_connected addr=lb:500516674worker2 # [ 157.116454] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:385286675worker2 # [ 157.120977] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06676lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6677lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.06 seconds)6678lb: must succeed: curl -sf localhost:9901/clusters6679lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6680worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6681worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6682(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 9.40 seconds)6683subtest: queue gauges show a burst while every build blocks6684lb: must succeed: curl -sf localhost:9901/clusters6685lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6686lb: must succeed: curl -sf localhost:9901/clusters6687lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6688worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6689worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6690client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6691client # [ 157.366832] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6692client # Running as unit: burst.service; invocation ID: 96dd86909bdd4f4584775c7a7c62fc146693client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6694lb: must succeed: curl -sf localhost:9901/clusters6695lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6696lb: must succeed: curl -sf localhost:9901/clusters6697lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6698worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6699worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6700worker1 # [ 157.504149] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514566701worker1 # [ 157.515814] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6702client # [ 157.528281] nix[1506]: don't know how to build these paths:6703client # [ 157.529213] nix[1506]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6704client # [ 157.530120] nix[1506]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6705client # [ 157.531166] nix[1506]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6706client # [ 157.532407] nix[1506]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6707worker2 # [ 157.524409] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=06708worker2 # [ 157.530575] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=16709worker2 # [ 157.534985] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=06710worker2 # [ 157.538939] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=16711worker2 # [ 157.543306] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=06712worker2 # [ 157.547251] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=06713worker1 # [ 157.585100] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514566714worker2 # [ 157.574967] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846715worker1 # [ 157.596476] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)6716worker2 # [ 157.587550] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559846717client # [ 157.608189] nix[1506]: copying 1 paths to 'lb:50051'...6718worker2 # [ 157.598688] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559846719client # [ 157.611142] nix[1506]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...6720worker1 # [ 157.612558] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 155 ms (attempt 2/5)6721worker2 # [ 157.611549] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6722worker2 # [ 157.705929] nix-daemon[870]: accepted connection from pid 1848, user nix-grpc-daemon (trusted)6723worker2 # [ 157.718615] nix-daemon[870]: reaped child process 1851, status = succeeded6724lb # [ 157.736911] niks3-server[1135]: 2026/09/23 05:35:29 INFO Received uploads request method=POST path=/api/pending_closures6725worker2 # [ 157.735364] nix-grpc-daemon[1668]: time=2026-09-23T05:35:30.085Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6726worker2 # [ 157.737508] nix-grpc-daemon[1668]: time=2026-09-23T05:35:30.087Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"6727lb # [ 157.769111] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6728lb # [ 157.774250] niks3-server[1135]: 2026/09/23 05:35:30 INFO Signed narinfos id=62 count=16729worker2 # [ 157.766577] nix-grpc-daemon[1668]: time=2026-09-23T05:35:30.116Z level=INFO msg="Uploading 1 narinfos"6730worker1 # [ 157.775518] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 3/5)6731lb # [ 157.783194] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst6732worker1 # [ 157.786599] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 387 ms (attempt 4/5)6733lb # [ 157.795412] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6734lb # [ 157.803147] niks3-server[1135]: 2026/09/23 05:35:30 INFO Completed upload id=626735worker2 # [ 157.792344] nix-grpc-daemon[1668]: time=2026-09-23T05:35:30.142Z level=INFO msg="Upload complete. (170ms)"6736lb # [ 157.806392] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls6737worker2 # [ 157.796718] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=4166738lb # [ 157.813780] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo6739worker2 # [ 157.801923] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846740worker2 # [ 157.813602] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6741worker2 # [ 157.815928] nix-daemon[870]: reaped child process 1846, status = succeeded6742worker2 # [ 157.820675] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6743client # [ 157.831196] nix[1506]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...6744lb # [ 157.985348] envoy[929]: conn peer=[2001:db8:1::1]:53650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9624 tx=4405 ms=51506745lb # [ 157.989859] envoy[929]: conn peer=[2001:db8:1::1]:53648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2945 tx=1519 ms=69006746lb # [ 157.993930] envoy[929]: conn peer=[2001:db8:1::3]:44178 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1597 tx=532 ms=229846747worker1 # [ 158.182898] nix-grpc-daemon[1731]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6748worker1 # [ 158.186779] nix-grpc-daemon[1731]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6749worker1 # [ 158.202423] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514566750client # [ 158.212573] nix[1506]: copying 1 paths to 'lb:50051'...6751client # [ 158.215760] nix[1506]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...6752worker1 # [ 158.217070] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514566753worker1 # [ 158.225549] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6754worker1 # [ 158.248946] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=info event=niks3_push_started pid=17926755worker1 # [ 158.348891] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.555Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6756worker1 # [ 158.432717] nix-daemon[847]: accepted connection from pid 1805, user nix-grpc-daemon (trusted)6757worker1 # [ 158.444400] nix-daemon[847]: reaped child process 1808, status = succeeded6758lb # [ 158.451985] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received uploads request method=POST path=/api/pending_closures6759worker1 # [ 158.460472] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.667Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6760worker1 # [ 158.461971] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.669Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"6761lb: must succeed: curl -sf localhost:9901/clusters6762lb # [ 158.492911] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6763lb # [ 158.498776] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls6764worker1 # [ 158.509787] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.716Z level=INFO msg="Uploading 1 narinfos"6765lb # [ 158.508808] niks3-server[1135]: 2026/09/23 05:35:30 INFO Signed narinfos id=63 count=16766lb # [ 158.513229] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst6767lb # [ 158.532124] niks3-server[1135]: 2026/09/23 05:35:30 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6768lb # [ 158.536565] niks3-server[1135]: 2026/09/23 05:35:30 INFO Completed upload id=636769worker1 # [ 158.537109] nix-grpc-daemon[1792]: time=2026-09-23T05:35:30.744Z level=INFO msg="Upload complete. (184ms)"6770lb # [ 158.539336] niks3-server[1135]: 2026/09/23 05:35:30 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo6771worker1 # [ 158.540106] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=4166772worker1 # [ 158.546129] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514566773worker1 # [ 158.555242] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6775lb: must succeed: curl -sf localhost:9901/clusters6776worker1 # [ 158.560600] nix-daemon[847]: reaped child process 1790, status = succeeded6777worker1 # [ 158.565455] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6778client # [ 158.565657] nix[1506]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...6779lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6780worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6781worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6782worker2 # [ 159.835614] nix-daemon[870]: reaped child process 1855, status = succeeded6783worker2 # [ 159.908616] nix-daemon[870]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6784worker2 # [ 159.921488] nix-daemon[870]: reaped child process 1886, status = succeeded6785lb # [ 159.938809] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures6786worker2 # [ 159.938373] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.288Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6787worker2 # [ 159.940069] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.290Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"6788lb # [ 159.981635] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6789lb # [ 159.990359] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls6790lb # [ 159.995766] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6791lb # [ 160.001255] niks3-server[1135]: 2026/09/23 05:35:32 INFO Signed narinfos id=64 count=16792worker2 # [ 159.993197] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.343Z level=INFO msg="Uploading 1 narinfos"6793lb # [ 160.005534] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6794worker1 # [ 160.006250] nix-daemon[847]: reaped child process 1812, status = succeeded6795lb # [ 160.010955] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6796lb # [ 160.015342] niks3-server[1135]: 2026/09/23 05:35:32 INFO Completed upload id=646797worker2 # [ 160.006261] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.356Z level=INFO msg="Upload complete. (178ms)"6798lb # [ 160.018725] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo6799worker2 # [ 160.011532] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=6 outputs=16800worker2 # [ 160.017949] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=16801worker2 # [ 160.023395] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846802worker2 # [ 160.031913] nix-daemon[870]: reaped child process 1853, status = succeeded6803worker2 # [ 160.033377] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559846804worker2 # [ 160.038843] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559846805client # [ 160.049866] nix[1506]: copying 1 paths to 'lb:50051'...6806client # [ 160.052515] nix[1506]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...6807worker2 # [ 160.047570] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6808worker1 # [ 160.086687] nix-daemon[847]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6809worker1 # [ 160.100218] nix-daemon[847]: reaped child process 1840, status = succeeded6810lb # [ 160.104691] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures6811worker1 # [ 160.114439] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6812worker1 # [ 160.116238] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.323Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"6813lb # [ 160.137767] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6814worker2 # [ 160.129947] nix-daemon[870]: accepted connection from pid 1890, user nix-grpc-daemon (trusted)6815lb # [ 160.143159] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6816lb # [ 160.145953] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6817lb # [ 160.149530] niks3-server[1135]: 2026/09/23 05:35:32 INFO Signed narinfos id=65 count=16818worker1 # [ 160.150316] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.357Z level=INFO msg="Uploading 1 narinfos"6819worker2 # [ 160.142404] nix-daemon[870]: reaped child process 1893, status = succeeded6820lb # [ 160.153267] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls6821lb # [ 160.157060] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6822lb # [ 160.159849] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures6823worker1 # [ 160.162907] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.370Z level=INFO msg="Upload complete. (189ms)"6824lb # [ 160.162929] niks3-server[1135]: 2026/09/23 05:35:32 INFO Completed upload id=656825worker1 # [ 160.165421] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=7 outputs=16826worker2 # [ 160.155226] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=16827lb # [ 160.166434] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo6828worker2 # [ 160.158923] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.509Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6829worker1 # [ 160.170989] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514566830worker2 # [ 160.160504] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.509Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"6831worker1 # [ 160.175513] nix-daemon[847]: reaped child process 1810, status = succeeded6832lb # [ 160.178986] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls6833worker1 # [ 160.181544] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514566834lb # [ 160.182075] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6835lb # [ 160.183846] niks3-server[1135]: 2026/09/23 05:35:32 INFO Signed narinfos id=66 count=16836worker2 # [ 160.174508] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.524Z level=INFO msg="Uploading 1 narinfos"6837worker1 # [ 160.192426] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514566838client # [ 160.192171] nix[1506]: copying 1 paths to 'lb:50051'...6839lb # [ 160.192621] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6840lb # [ 160.194707] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst6841client # [ 160.194348] nix[1506]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...6842lb # [ 160.198707] niks3-server[1135]: 2026/09/23 05:35:32 INFO Completed upload id=666843worker1 # [ 160.200462] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6844worker2 # [ 160.189226] nix-grpc-daemon[1668]: time=2026-09-23T05:35:32.539Z level=INFO msg="Upload complete. (136ms)"6845lb # [ 160.200709] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo6846worker2 # [ 160.191538] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=4166847worker2 # [ 160.194235] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559846848worker2 # [ 160.202710] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6849worker2 # [ 160.204079] nix-daemon[870]: reaped child process 1888, status = succeeded6850worker2 # [ 160.209250] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)6851client # [ 160.219183] nix[1506]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...6852worker1 # [ 160.282443] nix-daemon[847]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6853worker1 # [ 160.295145] nix-daemon[847]: reaped child process 1847, status = succeeded6854lb # [ 160.301083] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received uploads request method=POST path=/api/pending_closures6855worker1 # [ 160.309320] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.516Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6856worker1 # [ 160.311213] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.518Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"6857lb # [ 160.329352] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst6858lb # [ 160.336159] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6859lb # [ 160.339489] niks3-server[1135]: 2026/09/23 05:35:32 INFO Signed narinfos id=67 count=16860worker1 # [ 160.342967] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.550Z level=INFO msg="Uploading 1 narinfos"6861lb # [ 160.345887] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls6862lb # [ 160.349959] niks3-server[1135]: 2026/09/23 05:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6863lb # [ 160.353833] niks3-server[1135]: 2026/09/23 05:35:32 INFO Completed upload id=676864worker1 # [ 160.354140] nix-grpc-daemon[1792]: time=2026-09-23T05:35:32.561Z level=INFO msg="Upload complete. (148ms)"6865lb # [ 160.355422] niks3-server[1135]: 2026/09/23 05:35:32 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo6866worker1 # [ 160.356344] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=4166867worker1 # [ 160.360215] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514566868worker1 # [ 160.366165] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6869worker1 # [ 160.368400] nix-daemon[847]: reaped child process 1842, status = succeeded6870worker1 # [ 160.373471] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6871client # [ 160.372870] nix[1506]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...6872worker1 # [ 161.006412] nix-daemon[847]: reaped child process 1851, status = succeeded6873worker1 # [ 161.071651] nix-daemon[847]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6874worker1 # [ 161.085219] nix-daemon[847]: reaped child process 1875, status = succeeded6875lb # [ 161.090779] niks3-server[1135]: 2026/09/23 05:35:33 INFO Received uploads request method=POST path=/api/pending_closures6876worker1 # [ 161.100276] nix-grpc-daemon[1792]: time=2026-09-23T05:35:33.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6877worker1 # [ 161.101750] nix-grpc-daemon[1792]: time=2026-09-23T05:35:33.309Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"6878lb # [ 161.135494] niks3-server[1135]: 2026/09/23 05:35:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6879worker1 # [ 161.140574] nix-grpc-daemon[1792]: time=2026-09-23T05:35:33.347Z level=INFO msg="Uploading 1 narinfos"6880lb # [ 161.139840] niks3-server[1135]: 2026/09/23 05:35:33 INFO Signed narinfos id=68 count=16881lb # [ 161.151135] niks3-server[1135]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6882lb # [ 161.154666] niks3-server[1135]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls6883lb # [ 161.163466] niks3-server[1135]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6884lb # [ 161.167062] niks3-server[1135]: 2026/09/23 05:35:33 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6885lb # [ 161.170431] niks3-server[1135]: 2026/09/23 05:35:33 INFO Completed upload id=686886worker1 # [ 161.171079] nix-grpc-daemon[1792]: time=2026-09-23T05:35:33.378Z level=INFO msg="Upload complete. (191ms)"6887lb # [ 161.173256] niks3-server[1135]: 2026/09/23 05:35:33 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo6888worker1 # [ 161.176896] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=9 outputs=16889worker1 # [ 161.188840] nix-daemon[847]: reaped child process 1849, status = succeeded6890worker2 # [ 162.225692] nix-daemon[870]: reaped child process 1897, status = succeeded6891worker2 # [ 162.298875] nix-daemon[870]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6892worker2 # [ 162.312178] nix-daemon[870]: reaped child process 1925, status = succeeded6893lb # [ 162.329523] niks3-server[1135]: 2026/09/23 05:35:34 INFO Received uploads request method=POST path=/api/pending_closures6894worker2 # [ 162.327982] nix-grpc-daemon[1668]: time=2026-09-23T05:35:34.678Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6895worker2 # [ 162.329689] nix-grpc-daemon[1668]: time=2026-09-23T05:35:34.679Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"6896lb # [ 162.373738] niks3-server[1135]: 2026/09/23 05:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6897worker2 # [ 162.370172] nix-grpc-daemon[1668]: time=2026-09-23T05:35:34.720Z level=INFO msg="Uploading 1 narinfos"6898lb # [ 162.380413] niks3-server[1135]: 2026/09/23 05:35:34 INFO Signed narinfos id=69 count=16899lb # [ 162.387969] niks3-server[1135]: 2026/09/23 05:35:34 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6900lb # [ 162.393462] niks3-server[1135]: 2026/09/23 05:35:34 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6901lb # [ 162.401301] niks3-server[1135]: 2026/09/23 05:35:34 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls6902lb # [ 162.411507] niks3-server[1135]: 2026/09/23 05:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6903lb # [ 162.414547] niks3-server[1135]: 2026/09/23 05:35:34 INFO Completed upload id=696904worker2 # [ 162.405430] nix-grpc-daemon[1668]: time=2026-09-23T05:35:34.755Z level=INFO msg="Upload complete. (189ms)"6905lb # [ 162.418513] niks3-server[1135]: 2026/09/23 05:35:34 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo6906worker2 # [ 162.408075] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=8 outputs=16907worker2 # [ 162.415837] nix-daemon[870]: reaped child process 1895, status = succeeded6908client # [ 162.453692] systemd[1]: burst.service: Deactivated successfully.6909client # [ 162.457956] systemd[1]: burst.service: Consumed 156ms CPU time over 5.091s wall clock time, 53.7M memory peak, 13.8K incoming IP traffic, 18.5K outgoing IP traffic.6910client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6911client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)6912lb: must succeed: curl -sf localhost:9901/clusters6913lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6914lb: must succeed: curl -sf localhost:9901/clusters6915lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6916worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6917worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6918(finished: subtest: queue gauges show a burst while every build blocks, in 6.11 seconds)6919subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6920client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16921worker1 # [ 163.474585] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514566922worker1 # [ 163.480662] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514566923worker2 # [ 163.506171] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:35Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06924worker2 # [ 163.508271] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:35Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16925worker2 # [ 163.510110] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:35Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06926worker2 # [ 163.511665] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:35Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06927worker1 # [ 163.561714] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514566928worker1 # [ 163.574549] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514566929worker1 # [ 163.586401] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514566930worker1 # [ 163.594994] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6931worker1 # [ 163.696607] nix-daemon[847]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6932worker1 # [ 163.708632] nix-daemon[847]: reaped child process 1890, status = succeeded6933lb # [ 163.716424] niks3-server[1135]: 2026/09/23 05:35:35 INFO Received uploads request method=POST path=/api/pending_closures6934worker1 # [ 163.725682] nix-grpc-daemon[1792]: time=2026-09-23T05:35:35.932Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6935worker1 # [ 163.727792] nix-grpc-daemon[1792]: time=2026-09-23T05:35:35.934Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6936lb # [ 163.748813] niks3-server[1135]: 2026/09/23 05:35:36 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6937lb # [ 163.756483] niks3-server[1135]: 2026/09/23 05:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6938lb # [ 163.764650] niks3-server[1135]: 2026/09/23 05:35:36 INFO Signed narinfos id=70 count=16939worker1 # [ 163.768407] nix-grpc-daemon[1792]: time=2026-09-23T05:35:35.975Z level=INFO msg="Uploading 1 narinfos"6940lb # [ 163.771409] niks3-server[1135]: 2026/09/23 05:35:36 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6941lb # [ 163.783336] niks3-server[1135]: 2026/09/23 05:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6942lb # [ 163.789575] niks3-server[1135]: 2026/09/23 05:35:36 INFO Completed upload id=706943worker1 # [ 163.789636] nix-grpc-daemon[1792]: time=2026-09-23T05:35:35.996Z level=INFO msg="Upload complete. (181ms)"6944lb # [ 163.792973] niks3-server[1135]: 2026/09/23 05:35:36 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6945worker1 # [ 163.794463] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=5046946worker1 # [ 163.800505] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514566947worker1 # [ 163.809081] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6948worker1 # [ 163.819645] nix-daemon[847]: reaped child process 1885, status = succeeded6949worker1 # [ 163.824587] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6950worker1 # [ 165.060731] nix-daemon[847]: reaped child process 1894, status = succeeded6951worker1 # [ 165.135484] nix-daemon[847]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6952worker1 # [ 165.147109] nix-daemon[847]: reaped child process 1913, status = succeeded6953lb # [ 165.153772] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received uploads request method=POST path=/api/pending_closures6954worker1 # [ 165.163551] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.370Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6955worker1 # [ 165.165131] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.372Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6956lb # [ 165.199369] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6957lb # [ 165.206270] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6958worker1 # [ 165.216847] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.423Z level=INFO msg="Uploading 1 narinfos"6959lb # [ 165.217413] niks3-server[1135]: 2026/09/23 05:35:37 INFO Signed narinfos id=71 count=16960lb # [ 165.220681] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6961lb # [ 165.226427] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6962lb # [ 165.236864] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6963lb # [ 165.240835] niks3-server[1135]: 2026/09/23 05:35:37 INFO Completed upload id=716964worker1 # [ 165.241498] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.448Z level=INFO msg="Upload complete. (209ms)"6965lb # [ 165.244284] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6966worker2 # [ 165.237472] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:37Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16967worker2 # [ 165.242357] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:37Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06968worker1 # [ 165.247984] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=10 outputs=16969worker1 # [ 165.257748] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514566970worker1 # [ 165.268571] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514566971worker1 # [ 165.276809] nix-daemon[847]: reaped child process 1892, status = succeeded6972worker1 # [ 165.279598] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514566973worker1 # [ 165.285515] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6974worker1 # [ 165.367751] nix-daemon[847]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6975worker1 # [ 165.380918] nix-daemon[847]: reaped child process 1920, status = succeeded6976lb # [ 165.386152] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received uploads request method=POST path=/api/pending_closures6977worker1 # [ 165.394303] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.601Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6978worker1 # [ 165.396299] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.603Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6979lb # [ 165.417954] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6980worker1 # [ 165.424182] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.631Z level=INFO msg="Uploading 1 narinfos"6981lb # [ 165.423980] niks3-server[1135]: 2026/09/23 05:35:37 INFO Signed narinfos id=72 count=16982lb # [ 165.431381] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6983lb # [ 165.437723] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6984lb # [ 165.450587] niks3-server[1135]: 2026/09/23 05:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6985lb # [ 165.454430] niks3-server[1135]: 2026/09/23 05:35:37 INFO Completed upload id=726986worker1 # [ 165.456343] nix-grpc-daemon[1792]: time=2026-09-23T05:35:37.663Z level=INFO msg="Upload complete. (165ms)"6987lb # [ 165.457850] niks3-server[1135]: 2026/09/23 05:35:37 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6988worker1 # [ 165.458887] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=5046989worker1 # [ 165.464124] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514566990worker1 # [ 165.472691] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6991worker1 # [ 165.480785] nix-daemon[847]: reaped child process 1915, status = succeeded6992worker1 # [ 165.485386] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6993worker1 # [ 166.322893] nix-daemon[847]: reaped child process 1924, status = succeeded6994worker1 # [ 166.399930] nix-daemon[847]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6995worker1 # [ 166.411759] nix-daemon[847]: reaped child process 1943, status = succeeded6996lb # [ 166.417834] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received uploads request method=POST path=/api/pending_closures6997worker1 # [ 166.426641] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.633Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6998worker1 # [ 166.428137] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.635Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6999lb # [ 166.458462] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7000lb # [ 166.470927] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7001lb # [ 166.473921] niks3-server[1135]: 2026/09/23 05:35:38 INFO Signed narinfos id=73 count=17002worker1 # [ 166.476995] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.684Z level=INFO msg="Uploading 1 narinfos"7003lb # [ 166.483234] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7004lb # [ 166.488312] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7005lb # [ 166.498777] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7006lb # [ 166.501897] niks3-server[1135]: 2026/09/23 05:35:38 INFO Completed upload id=737007worker1 # [ 166.502320] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.709Z level=INFO msg="Upload complete. (209ms)"7008lb # [ 166.504528] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7009worker2 # [ 166.497115] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:38Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17010worker1 # [ 166.508089] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=11 outputs=17011worker2 # [ 166.502349] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:38Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07012worker1 # [ 166.518186] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567013worker1 # [ 166.529525] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514567014worker1 # [ 166.536530] nix-daemon[847]: reaped child process 1922, status = succeeded7015worker1 # [ 166.539702] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514567016worker1 # [ 166.545250] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7017worker1 # [ 166.628460] nix-daemon[847]: accepted connection from pid 1947, user nix-grpc-daemon (trusted)7018worker1 # [ 166.640453] nix-daemon[847]: reaped child process 1950, status = succeeded7019lb # [ 166.645421] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received uploads request method=POST path=/api/pending_closures7020worker1 # [ 166.654102] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.861Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7021worker1 # [ 166.655599] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.862Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7022lb # [ 166.678912] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7023lb # [ 166.685293] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7024lb # [ 166.690845] niks3-server[1135]: 2026/09/23 05:35:38 INFO Signed narinfos id=74 count=17025worker1 # [ 166.694039] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.901Z level=INFO msg="Uploading 1 narinfos"7026lb # [ 166.699277] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7027lb # [ 166.712319] niks3-server[1135]: 2026/09/23 05:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7028lb # [ 166.718770] niks3-server[1135]: 2026/09/23 05:35:38 INFO Completed upload id=747029worker1 # [ 166.719231] nix-grpc-daemon[1792]: time=2026-09-23T05:35:38.926Z level=INFO msg="Upload complete. (168ms)"7030lb # [ 166.720869] niks3-server[1135]: 2026/09/23 05:35:38 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7031worker1 # [ 166.721960] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=5047032worker1 # [ 166.726875] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567033worker1 # [ 166.736106] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7034worker1 # [ 166.743787] nix-daemon[847]: reaped child process 1945, status = succeeded7035worker1 # [ 166.748698] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7036worker1 # [ 167.980550] nix-daemon[847]: reaped child process 1954, status = succeeded7037lb # [ 167.985266] envoy[929]: conn peer=[2001:db8:1::1]:56168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9578 tx=5167 ms=49357038worker1 # [ 168.056341] nix-daemon[847]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)7039worker1 # [ 168.067888] nix-daemon[847]: reaped child process 1973, status = succeeded7040lb # [ 168.077083] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received uploads request method=POST path=/api/pending_closures7041worker1 # [ 168.086078] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.293Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7042worker1 # [ 168.087632] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.294Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7043lb # [ 168.120382] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7044worker1 # [ 168.129679] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.336Z level=INFO msg="Uploading 1 narinfos"7045lb # [ 168.129671] niks3-server[1135]: 2026/09/23 05:35:40 INFO Signed narinfos id=75 count=17046lb # [ 168.135634] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7047lb # [ 168.142714] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7048lb # [ 168.147445] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7049lb # [ 168.158885] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7050lb # [ 168.163117] niks3-server[1135]: 2026/09/23 05:35:40 INFO Completed upload id=757051worker1 # [ 168.165395] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.372Z level=INFO msg="Upload complete. (209ms)"7052lb # [ 168.167696] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7053worker2 # [ 168.160619] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:40Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17054worker2 # [ 168.165706] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:40Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07055worker1 # [ 168.174171] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=12 outputs=17056worker1 # [ 168.181689] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567057worker1 # [ 168.192519] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514567058worker1 # [ 168.200488] nix-daemon[847]: reaped child process 1952, status = succeeded7059worker1 # [ 168.203416] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.410Z level=INFO msg="All 1 paths already cached"7060worker1 # [ 168.206220] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514567061worker1 # [ 168.210840] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7062worker1 # [ 168.292476] nix-daemon[847]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)7063worker1 # [ 168.305441] nix-daemon[847]: reaped child process 1980, status = succeeded7064lb # [ 168.311531] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received uploads request method=POST path=/api/pending_closures7065worker1 # [ 168.319953] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.527Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7066worker1 # [ 168.321975] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.528Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7067lb # [ 168.341918] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7068lb # [ 168.350631] niks3-server[1135]: 2026/09/23 05:35:40 INFO Signed narinfos id=76 count=17069worker1 # [ 168.355086] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.562Z level=INFO msg="Uploading 1 narinfos"7070lb # [ 168.354637] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7071lb # [ 168.363989] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7072lb # [ 168.375245] niks3-server[1135]: 2026/09/23 05:35:40 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7073lb # [ 168.380931] niks3-server[1135]: 2026/09/23 05:35:40 INFO Completed upload id=767074worker1 # [ 168.380958] nix-grpc-daemon[1792]: time=2026-09-23T05:35:40.588Z level=INFO msg="Upload complete. (165ms)"7075lb # [ 168.384712] niks3-server[1135]: 2026/09/23 05:35:40 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7076worker1 # [ 168.384246] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=6247077worker1 # [ 168.389662] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567078worker1 # [ 168.400108] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7079worker1 # [ 168.404623] nix-daemon[847]: reaped child process 1975, status = succeeded7080worker1 # [ 168.409581] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7081worker1 # [ 170.035277] nix-daemon[847]: reaped child process 1984, status = succeeded7082worker1 # [ 170.107285] nix-daemon[847]: accepted connection from pid 2000, user nix-grpc-daemon (trusted)7083worker1 # [ 170.120159] nix-daemon[847]: reaped child process 2003, status = succeeded7084lb # [ 170.126921] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received uploads request method=POST path=/api/pending_closures7085worker1 # [ 170.136532] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.343Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7086worker1 # [ 170.138218] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.345Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7087lb # [ 170.167725] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7088lb # [ 170.176546] niks3-server[1135]: 2026/09/23 05:35:42 INFO Signed narinfos id=77 count=17089worker1 # [ 170.180387] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.387Z level=INFO msg="Uploading 1 narinfos"7090lb # [ 170.186747] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7091lb # [ 170.191096] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7092lb # [ 170.195677] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7093lb # [ 170.205418] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7094lb # [ 170.209879] niks3-server[1135]: 2026/09/23 05:35:42 INFO Completed upload id=777095lb # [ 170.211569] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7096worker1 # [ 170.210629] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.417Z level=INFO msg="Upload complete. (191ms)"7097worker2 # [ 170.206614] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:42Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17098worker2 # [ 170.211745] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:42Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07099worker1 # [ 170.219254] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=13 outputs=17100worker1 # [ 170.227463] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567101worker1 # [ 170.240519] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514567102worker1 # [ 170.247579] nix-daemon[847]: reaped child process 1982, status = succeeded7103worker1 # [ 170.250153] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.457Z level=INFO msg="All 1 paths already cached"7104worker1 # [ 170.255887] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514567105worker1 # [ 170.262845] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7106worker1 # [ 170.345361] nix-daemon[847]: accepted connection from pid 2007, user nix-grpc-daemon (trusted)7107worker1 # [ 170.358300] nix-daemon[847]: reaped child process 2010, status = succeeded7108lb # [ 170.364333] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received uploads request method=POST path=/api/pending_closures7109worker1 # [ 170.373463] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.580Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7110worker1 # [ 170.375455] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.582Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7111lb # [ 170.395929] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7112worker1 # [ 170.404635] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.611Z level=INFO msg="Uploading 1 narinfos"7113lb # [ 170.405122] niks3-server[1135]: 2026/09/23 05:35:42 INFO Signed narinfos id=78 count=17114lb # [ 170.407625] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7115lb # [ 170.416209] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7116lb # [ 170.427475] niks3-server[1135]: 2026/09/23 05:35:42 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7117lb # [ 170.434751] niks3-server[1135]: 2026/09/23 05:35:42 INFO Completed upload id=787118worker1 # [ 170.435289] nix-grpc-daemon[1792]: time=2026-09-23T05:35:42.642Z level=INFO msg="Upload complete. (166ms)"7119lb # [ 170.437922] niks3-server[1135]: 2026/09/23 05:35:42 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7120worker1 # [ 170.438976] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=6247121worker1 # [ 170.444720] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567122worker1 # [ 170.453592] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7123worker1 # [ 170.461165] nix-daemon[847]: reaped child process 2005, status = succeeded7124worker1 # [ 170.465940] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7125worker1 # [ 171.297395] nix-daemon[847]: reaped child process 2014, status = succeeded7126worker1 # [ 171.376181] nix-daemon[847]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)7127worker1 # [ 171.388970] nix-daemon[847]: reaped child process 2033, status = succeeded7128lb # [ 171.395934] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received uploads request method=POST path=/api/pending_closures7129worker1 # [ 171.404994] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.612Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7130worker1 # [ 171.406634] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.613Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7131lb # [ 171.433402] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7132lb # [ 171.446115] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7133lb # [ 171.454352] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7134lb # [ 171.459908] niks3-server[1135]: 2026/09/23 05:35:43 INFO Signed narinfos id=79 count=17135worker1 # [ 171.462618] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.669Z level=INFO msg="Uploading 1 narinfos"7136lb # [ 171.472103] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7137lb # [ 171.477624] niks3-server[1135]: 2026/09/23 05:35:43 INFO Completed upload id=797138worker1 # [ 171.477912] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.685Z level=INFO msg="Upload complete. (202ms)"7139lb # [ 171.479347] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7140lb # [ 171.482213] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7141worker2 # [ 171.474907] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:43Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17142worker1 # [ 171.484921] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=14 outputs=17143worker2 # [ 171.479934] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:43Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07144worker1 # [ 171.494327] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567145worker1 # [ 171.503213] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514567146worker1 # [ 171.512573] nix-daemon[847]: reaped child process 2012, status = succeeded7147worker1 # [ 171.514700] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.721Z level=INFO msg="All 1 paths already cached"7148worker1 # [ 171.517037] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514567149worker1 # [ 171.521811] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7150worker1 # [ 171.603538] nix-daemon[847]: accepted connection from pid 2037, user nix-grpc-daemon (trusted)7151worker1 # [ 171.617075] nix-daemon[847]: reaped child process 2040, status = succeeded7152lb # [ 171.621990] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received uploads request method=POST path=/api/pending_closures7153worker1 # [ 171.631339] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.838Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7154worker1 # [ 171.633294] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.840Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7155lb # [ 171.654585] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7156lb # [ 171.659477] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7157lb # [ 171.665524] niks3-server[1135]: 2026/09/23 05:35:43 INFO Signed narinfos id=80 count=17158worker1 # [ 171.668547] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.875Z level=INFO msg="Uploading 1 narinfos"7159lb # [ 171.672051] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7160lb # [ 171.680678] niks3-server[1135]: 2026/09/23 05:35:43 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7161lb # [ 171.686526] niks3-server[1135]: 2026/09/23 05:35:43 INFO Completed upload id=807162worker1 # [ 171.687183] nix-grpc-daemon[1792]: time=2026-09-23T05:35:43.894Z level=INFO msg="Upload complete. (160ms)"7163lb # [ 171.688696] niks3-server[1135]: 2026/09/23 05:35:43 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7164worker1 # [ 171.689957] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=6247165worker1 # [ 171.694646] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567166worker1 # [ 171.703865] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7167worker1 # [ 171.708188] nix-daemon[847]: reaped child process 2035, status = succeeded7168worker1 # [ 171.712992] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7169worker1 # [ 172.351279] nix-daemon[847]: reaped child process 2044, status = succeeded7170worker1 # [ 172.432641] nix-daemon[847]: accepted connection from pid 2060, user nix-grpc-daemon (trusted)7171worker1 # [ 172.445374] nix-daemon[847]: reaped child process 2063, status = succeeded7172lb # [ 172.452674] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received uploads request method=POST path=/api/pending_closures7173worker1 # [ 172.461866] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.668Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7174worker1 # [ 172.463401] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.670Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7175lb # [ 172.495352] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7176lb # [ 172.502132] niks3-server[1135]: 2026/09/23 05:35:44 INFO Signed narinfos id=81 count=17177worker1 # [ 172.506297] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.713Z level=INFO msg="Uploading 1 narinfos"7178lb # [ 172.509860] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7179lb # [ 172.518181] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7180lb # [ 172.522772] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7181lb # [ 172.531390] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7182lb # [ 172.534723] niks3-server[1135]: 2026/09/23 05:35:44 INFO Completed upload id=817183worker1 # [ 172.535312] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.742Z level=INFO msg="Upload complete. (214ms)"7184lb # [ 172.538365] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7185worker2 # [ 172.530851] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:44Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17186worker1 # [ 172.541520] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=15 outputs=17187worker2 # [ 172.536051] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:44Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07188worker1 # [ 172.550397] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567189worker1 # [ 172.560893] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514567190worker1 # [ 172.569882] nix-daemon[847]: reaped child process 2042, status = succeeded7191worker1 # [ 172.573631] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.779Z level=INFO msg="All 2 paths already cached"7192worker1 # [ 172.577844] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514567193worker1 # [ 172.581714] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7194worker1 # [ 172.663077] nix-daemon[847]: accepted connection from pid 2067, user nix-grpc-daemon (trusted)7195worker1 # [ 172.676313] nix-daemon[847]: reaped child process 2070, status = succeeded7196lb # [ 172.682122] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received uploads request method=POST path=/api/pending_closures7197worker1 # [ 172.691038] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.898Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7198worker1 # [ 172.692934] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.899Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7199lb # [ 172.715932] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7200lb # [ 172.723955] niks3-server[1135]: 2026/09/23 05:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7201lb # [ 172.729253] niks3-server[1135]: 2026/09/23 05:35:44 INFO Signed narinfos id=82 count=17202worker1 # [ 172.733100] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.940Z level=INFO msg="Uploading 1 narinfos"7203lb # [ 172.737173] niks3-server[1135]: 2026/09/23 05:35:44 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7204lb # [ 172.749181] niks3-server[1135]: 2026/09/23 05:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7205lb # [ 172.754094] niks3-server[1135]: 2026/09/23 05:35:45 INFO Completed upload id=827206worker1 # [ 172.754525] nix-grpc-daemon[1792]: time=2026-09-23T05:35:44.961Z level=INFO msg="Upload complete. (168ms)"7207lb # [ 172.757225] niks3-server[1135]: 2026/09/23 05:35:45 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7208worker1 # [ 172.758837] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=7527209worker1 # [ 172.764685] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567210worker1 # [ 172.775111] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7211worker1 # [ 172.780262] nix-daemon[847]: reaped child process 2065, status = succeeded7212worker1 # [ 172.785430] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7213worker1 # [ 174.023229] nix-daemon[847]: reaped child process 2074, status = succeeded7214worker1 # [ 174.099367] nix-daemon[847]: accepted connection from pid 2090, user nix-grpc-daemon (trusted)7215worker1 # [ 174.111412] nix-daemon[847]: reaped child process 2093, status = succeeded7216lb # [ 174.121246] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received uploads request method=POST path=/api/pending_closures7217worker1 # [ 174.131396] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.338Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7218worker1 # [ 174.133165] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.340Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7219lb # [ 174.165856] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7220lb # [ 174.171262] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7221lb # [ 174.177104] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7222lb # [ 174.183630] niks3-server[1135]: 2026/09/23 05:35:46 INFO Signed narinfos id=83 count=17223worker1 # [ 174.184377] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.391Z level=INFO msg="Uploading 1 narinfos"7224lb # [ 174.189980] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7225lb # [ 174.195255] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7226lb # [ 174.198575] niks3-server[1135]: 2026/09/23 05:35:46 INFO Completed upload id=837227worker1 # [ 174.199244] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.406Z level=INFO msg="Upload complete. (206ms)"7228lb # [ 174.201274] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7229worker2 # [ 174.194898] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:46Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17230worker1 # [ 174.205695] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=16 outputs=17231worker1 # [ 174.219840] nix-daemon[847]: reaped child process 2072, status = succeeded7232worker1 # [ 174.250585] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567233worker1 # [ 174.263147] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514567234worker1 # [ 174.276815] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.483Z level=INFO msg="All 2 paths already cached"7235worker1 # [ 174.285758] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514567236worker1 # [ 174.295132] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7237worker1 # [ 174.388472] nix-daemon[847]: accepted connection from pid 2097, user nix-grpc-daemon (trusted)7238worker1 # [ 174.401144] nix-daemon[847]: reaped child process 2100, status = succeeded7239lb # [ 174.407971] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received uploads request method=POST path=/api/pending_closures7240worker1 # [ 174.417195] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.624Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7241worker1 # [ 174.418677] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.626Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7242lb # [ 174.444423] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7243lb # [ 174.450953] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7244lb # [ 174.457127] niks3-server[1135]: 2026/09/23 05:35:46 INFO Signed narinfos id=84 count=17245worker1 # [ 174.460414] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.667Z level=INFO msg="Uploading 1 narinfos"7246lb # [ 174.464492] niks3-server[1135]: 2026/09/23 05:35:46 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7247lb # [ 174.473736] niks3-server[1135]: 2026/09/23 05:35:46 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7248lb # [ 174.479555] niks3-server[1135]: 2026/09/23 05:35:46 INFO Completed upload id=847249worker1 # [ 174.482450] nix-grpc-daemon[1792]: time=2026-09-23T05:35:46.689Z level=INFO msg="Upload complete. (176ms)"7250worker1 # [ 174.486606] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=7527251worker1 # [ 174.491677] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567252worker1 # [ 174.501129] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7253worker1 # [ 174.508926] nix-daemon[847]: reaped child process 2095, status = succeeded7254worker1 # [ 174.513844] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7255worker1 # [ 175.552558] nix-daemon[847]: reaped child process 2104, status = succeeded7256worker1 # [ 175.630716] nix-daemon[847]: accepted connection from pid 2120, user nix-grpc-daemon (trusted)7257worker1 # [ 175.643079] nix-daemon[847]: reaped child process 2123, status = succeeded7258lb # [ 175.651457] niks3-server[1135]: 2026/09/23 05:35:47 INFO Received uploads request method=POST path=/api/pending_closures7259worker1 # [ 175.660557] nix-grpc-daemon[1792]: time=2026-09-23T05:35:47.867Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7260worker1 # [ 175.662138] nix-grpc-daemon[1792]: time=2026-09-23T05:35:47.869Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7261lb # [ 175.693232] niks3-server[1135]: 2026/09/23 05:35:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7262lb # [ 175.699713] niks3-server[1135]: 2026/09/23 05:35:47 INFO Signed narinfos id=85 count=17263worker1 # [ 175.703108] nix-grpc-daemon[1792]: time=2026-09-23T05:35:47.910Z level=INFO msg="Uploading 1 narinfos"7264lb # [ 175.709855] niks3-server[1135]: 2026/09/23 05:35:47 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7265lb # [ 175.714977] niks3-server[1135]: 2026/09/23 05:35:47 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7266lb # [ 175.720103] niks3-server[1135]: 2026/09/23 05:35:47 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7267lb # [ 175.727886] niks3-server[1135]: 2026/09/23 05:35:47 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7268lb # [ 175.735383] niks3-server[1135]: 2026/09/23 05:35:47 INFO Completed upload id=857269lb # [ 175.736643] niks3-server[1135]: 2026/09/23 05:35:47 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7270worker1 # [ 175.735983] nix-grpc-daemon[1792]: time=2026-09-23T05:35:47.943Z level=INFO msg="Upload complete. (213ms)"7271worker2 # [ 175.731745] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:48Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17272worker1 # [ 175.744468] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=17 outputs=17273worker1 # [ 175.758240] nix-daemon[847]: reaped child process 2102, status = succeeded7274worker1 # [ 175.787582] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567275worker1 # [ 175.800273] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:514567276worker1 # [ 175.813414] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.020Z level=INFO msg="All 2 paths already cached"7277worker1 # [ 175.821856] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:514567278worker1 # [ 175.831166] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7279worker1 # [ 175.925742] nix-daemon[847]: accepted connection from pid 2127, user nix-grpc-daemon (trusted)7280worker1 # [ 175.938538] nix-daemon[847]: reaped child process 2130, status = succeeded7281lb # [ 175.945242] niks3-server[1135]: 2026/09/23 05:35:48 INFO Received uploads request method=POST path=/api/pending_closures7282worker1 # [ 175.955046] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.162Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7283worker1 # [ 175.957139] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.163Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7284lb # [ 175.980519] niks3-server[1135]: 2026/09/23 05:35:48 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7285lb # [ 175.990060] niks3-server[1135]: 2026/09/23 05:35:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7286lb # [ 175.994958] niks3-server[1135]: 2026/09/23 05:35:48 INFO Signed narinfos id=86 count=17287worker1 # [ 175.998908] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.206Z level=INFO msg="Uploading 1 narinfos"7288lb # [ 176.003253] niks3-server[1135]: 2026/09/23 05:35:48 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7289lb # [ 176.013360] niks3-server[1135]: 2026/09/23 05:35:48 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7290lb # [ 176.019761] niks3-server[1135]: 2026/09/23 05:35:48 INFO Completed upload id=867291worker1 # [ 176.020272] nix-grpc-daemon[1792]: time=2026-09-23T05:35:48.227Z level=INFO msg="Upload complete. (177ms)"7292lb # [ 176.024330] niks3-server[1135]: 2026/09/23 05:35:48 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7293worker1 # [ 176.024076] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 paths=1 nar_bytes_in=7527294worker1 # [ 176.031271] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:514567295worker1 # [ 176.040150] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7296worker1 # [ 176.048436] nix-daemon[847]: reaped child process 2125, status = succeeded7297worker1 # [ 176.053316] nix-daemon[847]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)7298worker1 # [ 176.689563] nix-daemon[847]: reaped child process 2134, status = succeeded7299worker1 # [ 176.771719] nix-daemon[847]: accepted connection from pid 2150, user nix-grpc-daemon (trusted)7300worker1 # [ 176.784605] nix-daemon[847]: reaped child process 2153, status = succeeded7301lb # [ 176.791933] niks3-server[1135]: 2026/09/23 05:35:49 INFO Received uploads request method=POST path=/api/pending_closures7302worker1 # [ 176.801404] nix-grpc-daemon[1792]: time=2026-09-23T05:35:49.008Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7303worker1 # [ 176.803131] nix-grpc-daemon[1792]: time=2026-09-23T05:35:49.010Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7304lb # [ 176.837145] niks3-server[1135]: 2026/09/23 05:35:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7305worker1 # [ 176.845243] nix-grpc-daemon[1792]: time=2026-09-23T05:35:49.052Z level=INFO msg="Uploading 1 narinfos"7306lb # [ 176.845139] niks3-server[1135]: 2026/09/23 05:35:49 INFO Signed narinfos id=87 count=17307lb # [ 176.852512] niks3-server[1135]: 2026/09/23 05:35:49 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7308lb # [ 176.858794] niks3-server[1135]: 2026/09/23 05:35:49 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7309lb # [ 176.867112] niks3-server[1135]: 2026/09/23 05:35:49 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7310lb # [ 176.874112] niks3-server[1135]: 2026/09/23 05:35:49 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7311lb # [ 176.878520] niks3-server[1135]: 2026/09/23 05:35:49 INFO Completed upload id=877312worker1 # [ 176.879322] nix-grpc-daemon[1792]: time=2026-09-23T05:35:49.086Z level=INFO msg="Upload complete. (219ms)"7313lb # [ 176.883061] niks3-server[1135]: 2026/09/23 05:35:49 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7314worker1 # [ 176.885608] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51456 duration_s=0 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=18 outputs=17315worker1 # [ 176.900200] nix-daemon[847]: reaped child process 2132, status = succeeded7316client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 13.60 seconds)7317worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7318worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7319client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7320client # [ 177.022765] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7321client # Running as unit: gpu.service; invocation ID: 5376ae82352a44b7996bfcf05090471b7322client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7323lb: must succeed: curl -sf localhost:9901/clusters7324lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7325lb: must succeed: curl -sf localhost:9901/clusters7326lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7327worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7328worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7329worker1 # [ 177.151168] nix-grpc-daemon[1731]: ts=2026-09-23T05:35:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:514567330client # [ 177.157144] nix[1576]: don't know how to build these paths:7331client # [ 177.158061] nix[1576]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7332worker2 # [ 177.148162] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559847333worker2 # [ 177.164277] nix-grpc-daemon[1593]: ts=2026-09-23T05:35:49Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07334client # [ 177.175737] nix[1576]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7335lb # [ 177.985634] envoy[929]: conn peer=[2001:db8:1::1]:42090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27124 tx=12268 ms=134287336lb: must succeed: curl -sf localhost:9901/clusters7337lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7338lb: must succeed: curl -sf localhost:9901/clusters7339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7340worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7341worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7342client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7343client # [ 207.178845] nix[1576]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7344client # [ 207.186823] nix[1576]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7345client # [ 207.220227] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7346client # [ 207.224168] systemd[1]: gpu.service: Failed with result 'exit-code'.7347client # [ 207.229152] systemd[1]: gpu.service: Consumed 136ms CPU time over 30.205s wall clock time, 37.6M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic.7348client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.41 seconds)7349client: must succeed: journalctl -u gpu -o cat7350client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7351lb: must succeed: curl -sf localhost:9901/clusters7352lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7353lb: must succeed: curl -sf localhost:9901/clusters7354lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7355worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7356worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7357(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 44.46 seconds)7358subtest: low disk drains a worker and builds go to the other7359worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7360lb # [ 207.986263] envoy[929]: conn peer=[2001:db8:1::1]:35838 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300497361worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.00 seconds)7362worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7363worker1 # [ 209.096252] nix-grpc-daemon[1731]: ts=2026-09-23T05:36:21Z level=info event=unhealthy reason=min_free7364worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds)7365client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27366client # don't know how to build these paths:7367client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7368worker2 # [ 210.015840] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:559847369worker2 # [ 210.021170] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559847370worker2 # [ 210.041444] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07371worker2 # [ 210.046382] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17372worker2 # [ 210.090483] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559847373worker2 # [ 210.103514] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559847374client # copying 1 paths to 'lb:50051'...7375client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7376worker2 # [ 210.116140] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559847377worker2 # [ 210.127165] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)7378worker2 # [ 210.231468] nix-daemon[870]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)7379worker2 # [ 210.245977] nix-daemon[870]: reaped child process 1966, status = succeeded7380lb # [ 210.263933] niks3-server[1135]: 2026/09/23 05:36:22 INFO Received uploads request method=POST path=/api/pending_closures7381worker2 # [ 210.262190] nix-grpc-daemon[1668]: time=2026-09-23T05:36:22.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7382worker2 # [ 210.264235] nix-grpc-daemon[1668]: time=2026-09-23T05:36:22.614Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7383lb # [ 210.304219] niks3-server[1135]: 2026/09/23 05:36:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7384lb # [ 210.309583] niks3-server[1135]: 2026/09/23 05:36:22 INFO Signed narinfos id=88 count=17385worker2 # [ 210.302311] nix-grpc-daemon[1668]: time=2026-09-23T05:36:22.652Z level=INFO msg="Uploading 1 narinfos"7386lb # [ 210.317952] niks3-server[1135]: 2026/09/23 05:36:22 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7387lb # [ 210.324450] niks3-server[1135]: 2026/09/23 05:36:22 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7388lb # [ 210.340329] niks3-server[1135]: 2026/09/23 05:36:22 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7389lb # [ 210.343043] niks3-server[1135]: 2026/09/23 05:36:22 INFO Completed upload id=887390worker2 # [ 210.332630] nix-grpc-daemon[1668]: time=2026-09-23T05:36:22.683Z level=INFO msg="Upload complete. (194ms)"7391lb # [ 210.345714] niks3-server[1135]: 2026/09/23 05:36:22 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7392worker2 # [ 210.337933] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=5127393worker2 # [ 210.344341] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559847394worker2 # [ 210.355200] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)7395worker2 # [ 210.363651] nix-daemon[870]: reaped child process 1961, status = succeeded7396worker2 # [ 210.371871] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)7397client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7398client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7399client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7400client # farm-a> LOG-farm-a-drain7401worker2 # [ 211.807232] nix-daemon[870]: reaped child process 1970, status = succeeded7402worker2 # [ 211.893272] nix-daemon[870]: accepted connection from pid 1986, user nix-grpc-daemon (trusted)7403worker2 # [ 211.904416] nix-daemon[870]: reaped child process 1989, status = succeeded7404lb # [ 211.923883] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received uploads request method=POST path=/api/pending_closures7405worker2 # [ 211.924561] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.274Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7406worker2 # [ 211.926193] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.276Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7407lb # [ 211.967935] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7408lb # [ 211.970354] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7409lb # [ 211.973425] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7410lb # [ 211.975757] niks3-server[1135]: 2026/09/23 05:36:24 INFO Signed narinfos id=89 count=17411worker2 # [ 211.966601] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.316Z level=INFO msg="Uploading 1 narinfos"7412lb # [ 211.980107] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7413lb # [ 211.983673] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7414lb # [ 211.988784] niks3-server[1135]: 2026/09/23 05:36:24 INFO Completed upload id=897415worker2 # [ 211.978338] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.328Z level=INFO msg="Upload complete. (198ms)"7416lb # [ 211.991678] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7417worker2 # [ 211.983647] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=19 outputs=17418worker2 # [ 211.993663] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07419worker2 # [ 211.995968] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17420worker2 # [ 212.001134] nix-daemon[870]: reaped child process 1968, status = succeeded7421worker2 # [ 212.038192] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559847422worker2 # [ 212.049183] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559847423worker2 # [ 212.067957] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.418Z level=INFO msg="All 1 paths already cached"7424client # copying 1 paths to 'lb:50051'...7425client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7426worker2 # [ 212.075944] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559847427worker2 # [ 212.086265] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)7428worker2 # [ 212.181514] nix-daemon[870]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)7429worker2 # [ 212.193804] nix-daemon[870]: reaped child process 1998, status = succeeded7430lb # [ 212.212205] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received uploads request method=POST path=/api/pending_closures7431worker2 # [ 212.214329] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.564Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7432worker2 # [ 212.215876] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.566Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7433lb # [ 212.249323] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7434lb # [ 212.252681] niks3-server[1135]: 2026/09/23 05:36:24 INFO Signed narinfos id=90 count=17435worker2 # [ 212.242150] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.592Z level=INFO msg="Uploading 1 narinfos"7436lb # [ 212.257584] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7437lb # [ 212.260254] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7438lb # [ 212.267846] niks3-server[1135]: 2026/09/23 05:36:24 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7439lb # [ 212.271633] niks3-server[1135]: 2026/09/23 05:36:24 INFO Completed upload id=907440worker2 # [ 212.261126] nix-grpc-daemon[1668]: time=2026-09-23T05:36:24.611Z level=INFO msg="Upload complete. (163ms)"7441lb # [ 212.273728] niks3-server[1135]: 2026/09/23 05:36:24 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7442worker2 # [ 212.263374] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=6247443worker2 # [ 212.267521] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559847444worker2 # [ 212.273761] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)7445worker2 # [ 212.275982] nix-daemon[870]: reaped child process 1993, status = succeeded7446worker2 # [ 212.281202] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)7447client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7448client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7449client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7450client # farm-b> LOG-farm-b-drain7451worker2 # [ 213.318143] nix-daemon[870]: reaped child process 2002, status = succeeded7452worker2 # [ 213.402635] nix-daemon[870]: accepted connection from pid 2018, user nix-grpc-daemon (trusted)7453worker2 # [ 213.413728] nix-daemon[870]: reaped child process 2021, status = succeeded7454lb # [ 213.433787] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received uploads request method=POST path=/api/pending_closures7455worker2 # [ 213.432719] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.783Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7456worker2 # [ 213.434272] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.784Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7457lb # [ 213.480214] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7458lb # [ 213.484994] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7459lb # [ 213.491510] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7460worker2 # [ 213.485168] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.835Z level=INFO msg="Uploading 1 narinfos"7461lb # [ 213.495483] niks3-server[1135]: 2026/09/23 05:36:25 INFO Signed narinfos id=91 count=17462lb # [ 213.505989] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7463lb # [ 213.511216] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7464lb # [ 213.515080] niks3-server[1135]: 2026/09/23 05:36:25 INFO Completed upload id=917465worker2 # [ 213.504633] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.855Z level=INFO msg="Upload complete. (216ms)"7466lb # [ 213.517974] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7467worker2 # [ 213.510984] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=20 outputs=17468worker2 # [ 213.522565] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07469worker2 # [ 213.526989] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17470worker2 # [ 213.532081] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559847471worker2 # [ 213.540307] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:559847472worker2 # [ 213.542937] nix-daemon[870]: reaped child process 2000, status = succeeded7473worker2 # [ 213.547764] nix-grpc-daemon[1668]: time=2026-09-23T05:36:25.898Z level=INFO msg="All 2 paths already cached"7474client # copying 1 paths to 'lb:50051'...7475worker2 # [ 213.553340] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:559847476client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7477worker2 # [ 213.559344] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)7478worker2 # [ 213.644050] nix-daemon[870]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)7479worker2 # [ 213.657964] nix-daemon[870]: reaped child process 2030, status = succeeded7480lb # [ 213.673991] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received uploads request method=POST path=/api/pending_closures7481worker2 # [ 213.672219] nix-grpc-daemon[1668]: time=2026-09-23T05:36:26.022Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7482worker2 # [ 213.674268] nix-grpc-daemon[1668]: time=2026-09-23T05:36:26.024Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7483lb # [ 213.707241] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7484worker2 # [ 213.702962] nix-grpc-daemon[1668]: time=2026-09-23T05:36:26.053Z level=INFO msg="Uploading 1 narinfos"7485lb # [ 213.714549] niks3-server[1135]: 2026/09/23 05:36:25 INFO Signed narinfos id=92 count=17486lb # [ 213.720183] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7487lb # [ 213.726143] niks3-server[1135]: 2026/09/23 05:36:25 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7488lb # [ 213.738147] niks3-server[1135]: 2026/09/23 05:36:25 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7489lb # [ 213.742796] niks3-server[1135]: 2026/09/23 05:36:26 INFO Completed upload id=927490worker2 # [ 213.732223] nix-grpc-daemon[1668]: time=2026-09-23T05:36:26.082Z level=INFO msg="Upload complete. (168ms)"7491lb # [ 213.745114] niks3-server[1135]: 2026/09/23 05:36:26 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7492worker2 # [ 213.735773] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=0 paths=1 nar_bytes_in=7527493worker2 # [ 213.740793] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:559847494worker2 # [ 213.752105] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)7495worker2 # [ 213.755622] nix-daemon[870]: reaped child process 2025, status = succeeded7496worker2 # [ 213.760724] nix-daemon[870]: accepted connection from pid 1593, user nix-grpc-daemon (trusted)7497client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7498client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7499client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7500client # farm-top> LOG-farm-top-drain7501worker2 # [ 214.575209] nix-daemon[870]: reaped child process 2034, status = succeeded7502worker2 # [ 214.650121] nix-daemon[870]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)7503worker2 # [ 214.662406] nix-daemon[870]: reaped child process 2053, status = succeeded7504lb # [ 214.683421] niks3-server[1135]: 2026/09/23 05:36:26 INFO Received uploads request method=POST path=/api/pending_closures7505worker2 # [ 214.680725] nix-grpc-daemon[1668]: time=2026-09-23T05:36:27.031Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7506worker2 # [ 214.682383] nix-grpc-daemon[1668]: time=2026-09-23T05:36:27.032Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7507lb # [ 214.722222] niks3-server[1135]: 2026/09/23 05:36:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7508worker2 # [ 214.718112] nix-grpc-daemon[1668]: time=2026-09-23T05:36:27.068Z level=INFO msg="Uploading 1 narinfos"7509lb # [ 214.728324] niks3-server[1135]: 2026/09/23 05:36:26 INFO Signed narinfos id=93 count=17510lb # [ 214.739618] niks3-server[1135]: 2026/09/23 05:36:27 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7511lb # [ 214.744430] niks3-server[1135]: 2026/09/23 05:36:27 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7512lb # [ 214.756348] niks3-server[1135]: 2026/09/23 05:36:27 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7513lb # [ 214.762769] niks3-server[1135]: 2026/09/23 05:36:27 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7514lb # [ 214.764723] niks3-server[1135]: 2026/09/23 05:36:27 INFO Completed upload id=937515worker2 # [ 214.754362] nix-grpc-daemon[1668]: time=2026-09-23T05:36:27.104Z level=INFO msg="Upload complete. (187ms)"7516lb # [ 214.767470] niks3-server[1135]: 2026/09/23 05:36:27 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7517worker2 # [ 214.760809] nix-grpc-daemon[1593]: ts=2026-09-23T05:36:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55984 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=21 outputs=17518worker2 # [ 214.775678] nix-daemon[870]: reaped child process 2032, status = succeeded7519client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.93 seconds)7520client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7521client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7522worker1: must succeed: rm /nix/.rw-store/fill7523worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7524worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7525worker1 # [ 216.099157] nix-grpc-daemon[1731]: ts=2026-09-23T05:36:28Z level=info event=healthy reason=min_free7526worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)7527(finished: subtest: low disk drains a worker and builds go to the other, in 8.38 seconds)7528(finished: run the VM test script, in 217.07 seconds)7529test script finished in 217.25s7530cleanup7531kill QemuMachine (pid 50)7532client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7533client # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down7534client # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down7535client # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down7536kill QemuMachine (pid 49)7537lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7538lb # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down7539lb # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down7540lb # [2026-09-23T05:36:29Z INFO virtiofsd] Client disconnected, shutting down7541kill QemuMachine (pid 51)7542worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7543worker1 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down7544worker1 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down7545worker1 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down7546kill QemuMachine (pid 52)7547worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7548worker2 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down7549worker2 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down7550worker2 # [2026-09-23T05:36:30Z INFO virtiofsd] Client disconnected, shutting down7551(finished: cleanup, in 1.10 seconds)7552additionally exposed symbols:7553 client, lb, worker1, worker2,7554 vlan1,7555 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh