vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #507
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Formatting '/build/vm-state-worker1/tmp.KFxaCdrqfs', fmt=raw size=107374182414worker1: QEMU running (pid 50)15client # Disk image does not exist, creating the virtualisation disk image...16client: QEMU running (pid 49)17client # Formatting '/build/vm-state-client/tmp.ftzp170xTa', fmt=raw size=107374182418worker1 # mke2fs 1.47.4 (6-Mar-2025)19client # mke2fs 1.47.4 (6-Mar-2025)20worker1 # Discarding device blocks: 0/262144 done21client # Discarding device blocks: 0/262144 done22worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes23client # Creating filesystem with 262144 4k blocks and 65536 inodes24worker1 # Filesystem UUID: 61cbd5eb-9e12-4441-b02d-3175ebe2edab25client # Filesystem UUID: 47e36270-68ac-45a2-9cc9-af0e4995ac3d26worker1 # Superblock backups stored on blocks:27lb # Disk image does not exist, creating the virtualisation disk image...28client # Superblock backups stored on blocks:29lb: QEMU running (pid 51)30worker2: QEMU running (pid 52)31worker2 # Disk image does not exist, creating the virtualisation disk image...32worker1 # 32768, 98304, 163840, 22937633client # 32768, 98304, 163840, 22937634lb # Formatting '/build/vm-state-lb/tmp.6tm8pRXrOn', fmt=raw size=107374182435worker1 # 36client # 37lb # mke2fs 1.47.4 (6-Mar-2025)38worker1 # Allocating group tables: 0/8 done39client # Allocating group tables: 0/8 done40lb # Discarding device blocks: 0/262144 done41worker1 # Writing inode tables: 0/8 done42client # Writing inode tables: 0/8 done43lb # Creating filesystem with 262144 4k blocks and 65536 inodes44worker1 # Creating journal (8192 blocks): done45client # Creating journal (8192 blocks): done46lb # Filesystem UUID: c36a0cd3-53b5-493e-acf1-7fa6cde1972c47worker1 # Writing superblocks and filesystem accounting information: 0/8 done48client # Writing superblocks and filesystem accounting information: 0/8 done49lb # Superblock backups stored on blocks:50worker1 # 51client # 52lb # 32768, 98304, 163840, 22937653worker1 # Virtualisation disk image created.54client # Virtualisation disk image created.55lb # 56worker1 # Starting virtiofs daemons...57client # Starting virtiofs daemons...58lb # Allocating group tables: 0/8 done59worker1 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)60client # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)61lb # Writing inode tables: 0/8 done62worker1 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether63client # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether64lb # Creating journal (8192 blocks): done65worker1 # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...66client # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...67lb # Writing superblocks and filesystem accounting information: 0/8 done68worker1 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69client # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)70lb # 71worker1 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72client # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73lb # Virtualisation disk image created.74worker1 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75client # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...76lb # Starting virtiofs daemons...77worker1 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78client # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79lb # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80worker1 # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...81client # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82lb # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83worker1 # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...84client # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...85lb # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...86worker1 # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests87client # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests88lb # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)89worker1 # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests90client # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests91lb # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker1 # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests93client # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests94lb # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...95worker2 # Formatting '/build/vm-state-worker2/tmp.FV547AVCxP', fmt=raw size=107374182496lb # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)97worker2 # mke2fs 1.47.4 (6-Mar-2025)98lb # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether99worker2 # Discarding device blocks: 0/262144 done100lb # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...101worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes102lb # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests103worker2 # Filesystem UUID: 729abcc3-a3de-4225-8a96-535984ba4cc0104lb # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests105worker2 # Superblock backups stored on blocks:106lb # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests107worker2 # 32768, 98304, 163840, 229376108(finished: start all VMs, in 0.24 seconds)109worker2 # 110client: waiting for unit network-addresses-eth1.service111worker2 # Allocating group tables: 0/8 done112client: waiting for the VM to finish booting113worker2 # Writing inode tables: 0/8 done114worker2 # Creating journal (8192 blocks): done115worker2 # Writing superblocks and filesystem accounting information: 0/8 done116worker2 # 117worker2 # Virtualisation disk image created.118worker2 # Starting virtiofs daemons...119worker2 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker2 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker2 # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker2 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker2 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker2 # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker2 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker2 # [2026-09-22T16:53:42Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker2 # [2026-09-22T16:53:42Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker2 # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests129worker2 # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests130worker2 # [2026-09-22T16:53:42Z INFO virtiofsd] Client connected, servicing requests131client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135client # 136client # 137client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139client # 140client # 141client # 142client # 143client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00144client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...145client # 146client # 147worker2 # 148worker2 # 149lb # 150lb # 151worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00152lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00153worker1 # 154worker1 # 155worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...156worker2 # 157worker2 # 158worker2 # 159worker2 # 160lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...161lb # 162lb # 163lb # 164lb # 165worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00166worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00167lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00168worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...169worker1 # 170worker1 # 171worker1 # 172worker1 # 173worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00174worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...175worker2 # 176worker2 # 177lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...178lb # 179lb # 180worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker1 # 182worker1 # 183client # Booting from ROM...184worker2 # Booting from ROM...185lb # Booting from ROM...186worker1 # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nwrxkcyq2rabn5jfqyww8chp4fbw12l7-nixos-system-client-test/init regInfo=/nix/store/9mbchw87wm67kspwy4r01ypax3r3mych-closure-info/registration console=ttyS0,115200n8 console=tty0189client # [ 0.000000] BIOS-provided physical RAM map:190client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200client # [ 0.000000] NX (Execute Disable) protection: active201client # [ 0.000000] APIC: Static calls initialized202client # [ 0.000000] SMBIOS 2.8 present.203client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014204client # [ 0.000000] DMI: Memory slots populated: 1/1205client # [ 0.000000] Hypervisor detected: KVM206client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000207client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00208client # [ 0.000000] kvm-clock: using sched offset of 475967401 cycles209client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns210client # [ 0.000005] tsc: Detected 2400.012 MHz processor211client # [ 0.000809] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000212client # [ 0.000835] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs213client # [ 0.000838] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT214lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026215worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026216client # [ 0.002722] found SMP MP-table at [mem 0x000f5450-0x000f545f]217client # [ 0.002733] Using GB pages for direct mapping218client # [ 0.002853] RAMDISK: [mem 0x5e370000-0x5ffcffff]219client # [ 0.002862] ACPI: Early table checksum verification disabled220client # [ 0.002865] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )221lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w2dy91fandddgvvjqa8m7gkv6yi4sgxg-nixos-system-lb-test/init regInfo=/nix/store/xn52n2bv8v838xy21kw2w4b6pvfd5i39-closure-info/registration console=ttyS0,115200n8 console=tty0222client # [ 0.002869] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)223worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qir84w600srafs16f9kikb4gknc3p104-nixos-system-worker2-test/init regInfo=/nix/store/5cdi1aq72z624lzk19yyg9mqb18j8azk-closure-info/registration console=ttyS0,115200n8 console=tty0224lb # [ 0.000000] BIOS-provided physical RAM map:225worker2 # [ 0.000000] BIOS-provided physical RAM map:226client # [ 0.002873] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)227lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable228worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable229worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026230lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved231client # [ 0.002880] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)232worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved233client # [ 0.002882] ACPI: FACS 0x000000005FFE0000 000040234lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved235worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved236client # [ 0.002883] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)237lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable238worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable239lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved240client # [ 0.002885] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)241worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved242worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mdpl287kr399rvszyfvd3fa6h47a12dq-nixos-system-worker1-test/init regInfo=/nix/store/76664a5z5p7jsx3wsqacr084ngmgfx5j-closure-info/registration console=ttyS0,115200n8 console=tty0243lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved244worker1 # [ 0.000000] BIOS-provided physical RAM map:245worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved246client # [ 0.002886] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)247lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved248worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved249worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable250client # [ 0.002888] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)251lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved252worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved253worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved254client # [ 0.002889] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]255lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved256worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved257worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved258client # [ 0.002890] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]259lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved260worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable261worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved262lb # [ 0.000000] NX (Execute Disable) protection: active263client # [ 0.002890] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]264worker2 # [ 0.000000] NX (Execute Disable) protection: active265worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved266lb # [ 0.000000] APIC: Static calls initialized267worker2 # [ 0.000000] APIC: Static calls initialized268client # [ 0.002891] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]269lb # [ 0.000000] SMBIOS 2.8 present.270worker2 # [ 0.000000] SMBIOS 2.8 present.271worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved272client # [ 0.002891] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]273worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved274lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014275worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014276client # [ 0.002892] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]277lb # [ 0.000000] DMI: Memory slots populated: 1/1278worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved279worker2 # [ 0.000000] DMI: Memory slots populated: 1/1280lb # [ 0.000000] Hypervisor detected: KVM281worker2 # [ 0.000000] Hypervisor detected: KVM282client # [ 0.002892] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]283lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000284worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved285worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000286client # [ 0.003121] No NUMA configuration found287lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00288worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved289worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00290client # [ 0.003121] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]291lb # [ 0.000001] kvm-clock: using sched offset of 481633311 cycles292worker1 # [ 0.000000] NX (Execute Disable) protection: active293worker2 # [ 0.000000] kvm-clock: using sched offset of 485115325 cycles294client # [ 0.003124] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]295worker1 # [ 0.000000] APIC: Static calls initialized296client # [ 0.005394] Zone ranges:297worker1 # [ 0.000000] SMBIOS 2.8 present.298lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns299worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns300client # [ 0.005395] DMA [mem 0x0000000000001000-0x0000000000ffffff]301lb # [ 0.000005] tsc: Detected 2400.012 MHz processor302worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor303client # [ 0.005396] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]304worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014305client # [ 0.005397] Normal empty306lb # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000307worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000308worker1 # [ 0.000000] DMI: Memory slots populated: 1/1309client # [ 0.005398] Device empty310worker1 # [ 0.000000] Hypervisor detected: KVM311client # [ 0.005399] Movable zone start for each node312lb # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs313worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs314client # [ 0.005400] Early memory node ranges315worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000316lb # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT317worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT318worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00319client # [ 0.005400] node 0: [mem 0x0000000000001000-0x000000000009efff]320lb # [ 0.002728] found SMP MP-table at [mem 0x000f5450-0x000f545f]321worker1 # [ 0.000001] kvm-clock: using sched offset of 505198204 cycles322worker2 # [ 0.002727] found SMP MP-table at [mem 0x000f5450-0x000f545f]323client # [ 0.005401] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]324lb # [ 0.002740] Using GB pages for direct mapping325worker2 # [ 0.002738] Using GB pages for direct mapping326client # [ 0.005402] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]327lb # [ 0.002818] RAMDISK: [mem 0x5e36e000-0x5ffcffff]328worker2 # [ 0.002870] RAMDISK: [mem 0x5e36f000-0x5ffcffff]329worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns330client # [ 0.005421] On node 0, zone DMA: 1 pages in unavailable ranges331lb # [ 0.002827] ACPI: Early table checksum verification disabled332worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor333worker2 # [ 0.002879] ACPI: Early table checksum verification disabled334lb # [ 0.002830] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )335client # [ 0.005679] On node 0, zone DMA: 97 pages in unavailable ranges336worker1 # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000337worker2 # [ 0.002882] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )338client # [ 0.031780] On node 0, zone DMA32: 40 pages in unavailable ranges339lb # [ 0.002833] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)340worker1 # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs341worker2 # [ 0.002886] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)342client # [ 0.032244] ACPI: PM-Timer IO Port: 0x608343client # [ 0.032256] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])344worker1 # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT345lb # [ 0.002837] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)346worker2 # [ 0.002890] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)347worker1 # [ 0.002749] found SMP MP-table at [mem 0x000f5450-0x000f545f]348client # [ 0.032285] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23349lb # [ 0.002844] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)350worker2 # [ 0.002897] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)351worker1 # [ 0.002760] Using GB pages for direct mapping352client # [ 0.032288] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)353worker1 # [ 0.002875] RAMDISK: [mem 0x5e370000-0x5ffcffff]354worker2 # [ 0.002899] ACPI: FACS 0x000000005FFE0000 000040355lb # [ 0.002846] ACPI: FACS 0x000000005FFE0000 000040356client # [ 0.032289] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)357worker1 # [ 0.002884] ACPI: Early table checksum verification disabled358lb # [ 0.002848] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)359worker2 # [ 0.002901] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)360worker1 # [ 0.002887] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )361client # [ 0.032291] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)362worker2 # [ 0.002902] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)363lb # [ 0.002849] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)364client # [ 0.032292] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)365worker1 # [ 0.002891] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)366lb # [ 0.002851] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)367worker2 # [ 0.002904] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)368client # [ 0.032292] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)369worker1 # [ 0.002895] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)370client # [ 0.032295] ACPI: Using ACPI (MADT) for SMP configuration information371lb # [ 0.002852] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)372worker2 # [ 0.002905] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)373client # [ 0.032296] ACPI: HPET id: 0x8086a201 base: 0xfed00000374worker1 # [ 0.002902] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)375lb # [ 0.002854] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]376worker2 # [ 0.002907] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]377worker1 # [ 0.002904] ACPI: FACS 0x000000005FFE0000 000040378client # [ 0.032299] TSC deadline timer available379client # [ 0.032303] CPU topo: Max. logical packages: 1380worker2 # [ 0.002908] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]381lb # [ 0.002855] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]382client # [ 0.032304] CPU topo: Max. logical dies: 1383worker1 # [ 0.002905] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)384worker2 # [ 0.002908] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]385client # [ 0.032304] CPU topo: Max. dies per package: 1386lb # [ 0.002855] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]387worker1 # [ 0.002907] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)388client # [ 0.032308] CPU topo: Max. threads per core: 1389lb # [ 0.002856] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]390worker2 # [ 0.002909] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]391client # [ 0.032308] CPU topo: Num. cores per package: 1392worker1 # [ 0.002908] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)393worker2 # [ 0.002909] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]394lb # [ 0.002856] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]395client # [ 0.032309] CPU topo: Num. threads per package: 1396client # [ 0.032309] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs397worker2 # [ 0.002910] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]398lb # [ 0.002857] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]399worker1 # [ 0.002910] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)400client # [ 0.032329] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()401worker2 # [ 0.002910] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]402lb # [ 0.002857] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]403worker1 # [ 0.002911] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]404worker2 # [ 0.003142] No NUMA configuration found405lb # [ 0.003088] No NUMA configuration found406worker1 # [ 0.002912] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]407client # [ 0.032359] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]408worker2 # [ 0.003143] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]409lb # [ 0.003089] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]410worker1 # [ 0.002913] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]411client # [ 0.032360] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]412worker2 # [ 0.003145] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]413lb # [ 0.003092] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]414worker2 # [ 0.005638] Zone ranges:415lb # [ 0.005561] Zone ranges:416client # [ 0.032362] [mem 0x60000000-0xafffffff] available for PCI devices417worker1 # [ 0.002913] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]418worker2 # [ 0.005638] DMA [mem 0x0000000000001000-0x0000000000ffffff]419client # [ 0.032363] Booting paravirtualized kernel on KVM420lb # [ 0.005562] DMA [mem 0x0000000000001000-0x0000000000ffffff]421worker1 # [ 0.002914] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]422lb # [ 0.005563] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]423worker2 # [ 0.005640] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]424lb # [ 0.005564] Normal empty425worker2 # [ 0.005641] Normal empty426worker1 # [ 0.002914] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]427client # [ 0.032366] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns428worker2 # [ 0.005642] Device empty429lb # [ 0.005565] Device empty430worker2 # [ 0.005643] Movable zone start for each node431lb # [ 0.005566] Movable zone start for each node432worker1 # [ 0.002915] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]433client # [ 0.036828] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1434worker2 # [ 0.005643] Early memory node ranges435lb # [ 0.005566] Early memory node ranges436worker1 # [ 0.003143] No NUMA configuration found437client # [ 0.038972] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152438worker2 # [ 0.005644] node 0: [mem 0x0000000000001000-0x000000000009efff]439lb # [ 0.005567] node 0: [mem 0x0000000000001000-0x000000000009efff]440worker1 # [ 0.003144] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]441client # [ 0.039017] kvm-guest: PV spinlocks disabled, single CPU442worker2 # [ 0.005645] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]443lb # [ 0.005568] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]444worker1 # [ 0.003146] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]445worker1 # [ 0.005602] Zone ranges:446lb # [ 0.005569] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]447worker2 # [ 0.005646] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]448worker1 # [ 0.005603] DMA [mem 0x0000000000001000-0x0000000000ffffff]449worker2 # [ 0.005671] On node 0, zone DMA: 1 pages in unavailable ranges450lb # [ 0.005587] On node 0, zone DMA: 1 pages in unavailable ranges451worker1 # [ 0.005604] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]452worker1 # [ 0.005606] Normal empty453worker2 # [ 0.005928] On node 0, zone DMA: 97 pages in unavailable ranges454lb # [ 0.005844] On node 0, zone DMA: 97 pages in unavailable ranges455worker1 # [ 0.005607] Device empty456worker2 # [ 0.032054] On node 0, zone DMA32: 40 pages in unavailable ranges457lb # [ 0.032061] On node 0, zone DMA32: 40 pages in unavailable ranges458worker1 # [ 0.005607] Movable zone start for each node459worker2 # [ 0.032513] ACPI: PM-Timer IO Port: 0x608460lb # [ 0.032525] ACPI: PM-Timer IO Port: 0x608461client # [ 0.039018] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nwrxkcyq2rabn5jfqyww8chp4fbw12l7-nixos-system-client-test/init regInfo=/nix/store/9mbchw87wm67kspwy4r01ypax3r3mych-closure-info/registration console=ttyS0,115200n8 console=tty0462worker1 # [ 0.005608] Early memory node ranges463lb # [ 0.032536] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])464worker2 # [ 0.032524] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])465worker1 # [ 0.005608] node 0: [mem 0x0000000000001000-0x000000000009efff]466worker2 # [ 0.032554] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23467lb # [ 0.032565] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23468worker1 # [ 0.005609] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]469client # [ 0.039111] Unknown kernel command line parameters "regInfo=/nix/store/9mbchw87wm67kspwy4r01ypax3r3mych-closure-info/registration", will be passed to user space.470client # [ 0.039123] random: crng init done471lb # [ 0.032568] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)472worker2 # [ 0.032557] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)473worker1 # [ 0.005611] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]474lb # [ 0.032570] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)475worker1 # [ 0.005630] On node 0, zone DMA: 1 pages in unavailable ranges476client # [ 0.039124] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes477worker2 # [ 0.032558] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)478worker1 # [ 0.005898] On node 0, zone DMA: 97 pages in unavailable ranges479lb # [ 0.032571] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)480client # [ 0.041260] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)481worker2 # [ 0.032559] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)482worker1 # [ 0.032164] On node 0, zone DMA32: 40 pages in unavailable ranges483lb # [ 0.032572] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)484worker2 # [ 0.032560] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)485client # [ 0.042487] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)486worker1 # [ 0.032621] ACPI: PM-Timer IO Port: 0x608487client # [ 0.042517] Fallback order for Node 0: 0488worker2 # [ 0.032561] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)489lb # [ 0.032573] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)490worker1 # [ 0.032633] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])491client # [ 0.042520] Built 1 zonelists, mobility grouping on. Total pages: 393078492worker2 # [ 0.032563] ACPI: Using ACPI (MADT) for SMP configuration information493lb # [ 0.032575] ACPI: Using ACPI (MADT) for SMP configuration information494client # [ 0.042521] Policy zone: DMA32495worker1 # [ 0.032662] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23496worker2 # [ 0.032564] ACPI: HPET id: 0x8086a201 base: 0xfed00000497lb # [ 0.032576] ACPI: HPET id: 0x8086a201 base: 0xfed00000498worker1 # [ 0.032671] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)499worker2 # [ 0.032568] TSC deadline timer available500client # [ 0.045012] mem auto-init: stack:all(zero), heap alloc:on, heap free:off501lb # [ 0.032580] TSC deadline timer available502worker2 # [ 0.032572] CPU topo: Max. logical packages: 1503lb # [ 0.032584] CPU topo: Max. logical packages: 1504worker1 # [ 0.032672] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)505client # [ 0.048104] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1506worker2 # [ 0.032572] CPU topo: Max. logical dies: 1507lb # [ 0.032585] CPU topo: Max. logical dies: 1508client # [ 0.051386] allocated 3145728 bytes of page_ext509worker1 # [ 0.032673] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)510worker2 # [ 0.032573] CPU topo: Max. dies per package: 1511lb # [ 0.032585] CPU topo: Max. dies per package: 1512client # [ 0.061429] ftrace: allocating 48787 entries in 192 pages513worker2 # [ 0.032576] CPU topo: Max. threads per core: 1514lb # [ 0.032588] CPU topo: Max. threads per core: 1515worker1 # [ 0.032674] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)516client # [ 0.061431] ftrace: allocated 192 pages with 2 groups517worker2 # [ 0.032577] CPU topo: Num. cores per package: 1518lb # [ 0.032589] CPU topo: Num. cores per package: 1519client # [ 0.062272] Dynamic Preempt: lazy520worker1 # [ 0.032675] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)521worker2 # [ 0.032577] CPU topo: Num. threads per package: 1522lb # [ 0.032589] CPU topo: Num. threads per package: 1523client # [ 0.062434] rcu: Preemptible hierarchical RCU implementation.524worker1 # [ 0.032677] ACPI: Using ACPI (MADT) for SMP configuration information525worker2 # [ 0.032577] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs526lb # [ 0.032589] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs527client # [ 0.062435] rcu: RCU event tracing is enabled.528worker1 # [ 0.032678] ACPI: HPET id: 0x8086a201 base: 0xfed00000529worker2 # [ 0.032596] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()530client # [ 0.062436] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.531worker1 # [ 0.032682] TSC deadline timer available532lb # [ 0.032609] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()533worker1 # [ 0.032686] CPU topo: Max. logical packages: 1534client # [ 0.062437] Trampoline variant of Tasks RCU enabled.535worker2 # [ 0.032629] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]536lb # [ 0.032640] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]537worker1 # [ 0.032686] CPU topo: Max. logical dies: 1538client # [ 0.062438] Rude variant of Tasks RCU enabled.539worker2 # [ 0.032631] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]540worker1 # [ 0.032687] CPU topo: Max. dies per package: 1541client # [ 0.062438] Tracing variant of Tasks RCU enabled.542lb # [ 0.032642] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]543worker1 # [ 0.032690] CPU topo: Max. threads per core: 1544lb # [ 0.032643] [mem 0x60000000-0xafffffff] available for PCI devices545worker2 # [ 0.032632] [mem 0x60000000-0xafffffff] available for PCI devices546client # [ 0.062439] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.547worker1 # [ 0.032691] CPU topo: Num. cores per package: 1548worker2 # [ 0.032633] Booting paravirtualized kernel on KVM549lb # [ 0.032645] Booting paravirtualized kernel on KVM550worker1 # [ 0.032691] CPU topo: Num. threads per package: 1551client # [ 0.062439] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1552worker1 # [ 0.032691] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs553worker2 # [ 0.032636] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns554lb # [ 0.032648] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns555client # [ 0.062488] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.556worker1 # [ 0.032710] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()557worker2 # [ 0.037115] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1558lb # [ 0.037111] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1559client # [ 0.062490] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.560worker1 # [ 0.032739] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]561worker2 # [ 0.039245] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152562lb # [ 0.039247] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152563worker2 # [ 0.039291] kvm-guest: PV spinlocks disabled, single CPU564client # [ 0.062491] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.565lb # [ 0.039292] kvm-guest: PV spinlocks disabled, single CPU566worker1 # [ 0.032741] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]567client # [ 0.067063] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16568worker1 # [ 0.032742] [mem 0x60000000-0xafffffff] available for PCI devices569worker1 # [ 0.032744] Booting paravirtualized kernel on KVM570client # [ 0.067336] rcu: srcu_init: Setting srcu_struct sizes based on contention.571client # [ 0.067343] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns572worker1 # [ 0.032746] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns573worker1 # [ 0.037213] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1574worker2 # [ 0.039292] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qir84w600srafs16f9kikb4gknc3p104-nixos-system-worker2-test/init regInfo=/nix/store/5cdi1aq72z624lzk19yyg9mqb18j8azk-closure-info/registration console=ttyS0,115200n8 console=tty0575lb # [ 0.039293] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w2dy91fandddgvvjqa8m7gkv6yi4sgxg-nixos-system-lb-test/init regInfo=/nix/store/xn52n2bv8v838xy21kw2w4b6pvfd5i39-closure-info/registration console=ttyS0,115200n8 console=tty0576client # [ 0.067446] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)577client # [ 0.070965] Console: colour VGA+ 80x25578worker1 # [ 0.039350] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152579client # [ 0.070968] printk: legacy console [tty0] enabled580worker1 # [ 0.039395] kvm-guest: PV spinlocks disabled, single CPU581worker2 # [ 0.039387] Unknown kernel command line parameters "regInfo=/nix/store/5cdi1aq72z624lzk19yyg9mqb18j8azk-closure-info/registration", will be passed to user space.582lb # [ 0.039386] Unknown kernel command line parameters "regInfo=/nix/store/xn52n2bv8v838xy21kw2w4b6pvfd5i39-closure-info/registration", will be passed to user space.583client # [ 0.100491] printk: legacy console [ttyS0] enabled584worker2 # [ 0.039400] random: crng init done585lb # [ 0.039398] random: crng init done586client # [ 0.198582] ACPI: Core revision 20250807587worker2 # [ 0.039400] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes588lb # [ 0.039398] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes589worker2 # [ 0.041509] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)590lb # [ 0.041516] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)591client # [ 0.199517] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns592worker2 # [ 0.042567] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)593lb # [ 0.042572] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)594worker1 # [ 0.039397] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mdpl287kr399rvszyfvd3fa6h47a12dq-nixos-system-worker1-test/init regInfo=/nix/store/76664a5z5p7jsx3wsqacr084ngmgfx5j-closure-info/registration console=ttyS0,115200n8 console=tty0595client # [ 0.200995] APIC: Switch to symmetric I/O mode setup596worker2 # [ 0.042598] Fallback order for Node 0: 0597lb # [ 0.042603] Fallback order for Node 0: 0598client # [ 0.201929] x2apic enabled599worker2 # [ 0.042601] Built 1 zonelists, mobility grouping on. Total pages: 393078600lb # [ 0.042606] Built 1 zonelists, mobility grouping on. Total pages: 393078601worker2 # [ 0.042602] Policy zone: DMA32602lb # [ 0.042607] Policy zone: DMA32603worker1 # [ 0.039491] Unknown kernel command line parameters "regInfo=/nix/store/76664a5z5p7jsx3wsqacr084ngmgfx5j-closure-info/registration", will be passed to user space.604client # [ 0.202667] APIC: Switched APIC routing to: physical x2apic605worker2 # [ 0.045275] mem auto-init: stack:all(zero), heap alloc:on, heap free:off606worker1 # [ 0.039503] random: crng init done607lb # [ 0.045246] mem auto-init: stack:all(zero), heap alloc:on, heap free:off608worker2 # [ 0.048343] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1609lb # [ 0.048334] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1610worker1 # [ 0.039504] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes611worker2 # [ 0.051680] allocated 3145728 bytes of page_ext612lb # [ 0.051664] allocated 3145728 bytes of page_ext613client # [ 0.204440] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1614worker1 # [ 0.041609] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)615worker2 # [ 0.061606] ftrace: allocating 48787 entries in 192 pages616lb # [ 0.061500] ftrace: allocating 48787 entries in 192 pages617worker2 # [ 0.061609] ftrace: allocated 192 pages with 2 groups618lb # [ 0.061502] ftrace: allocated 192 pages with 2 groups619worker1 # [ 0.042659] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)620worker2 # [ 0.062451] Dynamic Preempt: lazy621lb # [ 0.062328] Dynamic Preempt: lazy622client # [ 0.205383] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns623worker1 # [ 0.042696] Fallback order for Node 0: 0624lb # [ 0.062441] rcu: Preemptible hierarchical RCU implementation.625worker2 # [ 0.062577] rcu: Preemptible hierarchical RCU implementation.626worker1 # [ 0.042699] Built 1 zonelists, mobility grouping on. Total pages: 393078627worker2 # [ 0.062578] rcu: RCU event tracing is enabled.628lb # [ 0.062442] rcu: RCU event tracing is enabled.629worker1 # [ 0.042700] Policy zone: DMA32630client # [ 0.206960] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)631worker2 # [ 0.062579] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.632lb # [ 0.062442] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.633worker1 # [ 0.045322] mem auto-init: stack:all(zero), heap alloc:on, heap free:off634client # [ 0.209045] x86/cpu: User Mode Instruction Prevention (UMIP) activated635lb # [ 0.062444] Trampoline variant of Tasks RCU enabled.636worker2 # [ 0.062580] Trampoline variant of Tasks RCU enabled.637worker1 # [ 0.048459] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1638worker2 # [ 0.062581] Rude variant of Tasks RCU enabled.639lb # [ 0.062444] Rude variant of Tasks RCU enabled.640worker1 # [ 0.051733] allocated 3145728 bytes of page_ext641client # [ 0.210089] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127642worker2 # [ 0.062581] Tracing variant of Tasks RCU enabled.643lb # [ 0.062445] Tracing variant of Tasks RCU enabled.644worker1 # [ 0.061560] ftrace: allocating 48787 entries in 192 pages645worker2 # [ 0.062582] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.646client # [ 0.210957] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0647lb # [ 0.062445] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.648worker1 # [ 0.061562] ftrace: allocated 192 pages with 2 groups649worker1 # [ 0.062407] Dynamic Preempt: lazy650lb # [ 0.062446] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1651worker2 # [ 0.062582] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1652worker1 # [ 0.062587] rcu: Preemptible hierarchical RCU implementation.653client # [ 0.211961] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto654worker1 # [ 0.062587] rcu: RCU event tracing is enabled.655worker2 # [ 0.062601] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.656lb # [ 0.062511] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.657worker1 # [ 0.062588] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.658worker2 # [ 0.062602] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.659lb # [ 0.062512] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.660client # [ 0.212957] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl661worker1 # [ 0.062590] Trampoline variant of Tasks RCU enabled.662worker1 # [ 0.062590] Rude variant of Tasks RCU enabled.663client # [ 0.214957] Transient Scheduler Attacks: Vulnerable: No microcode664worker2 # [ 0.062603] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.665lb # [ 0.062513] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.666worker1 # [ 0.062590] Tracing variant of Tasks RCU enabled.667worker2 # [ 0.067009] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16668lb # [ 0.066878] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16669client # [ 0.215957] Spectre V2 : Mitigation: Enhanced / Automatic IBRS670worker1 # [ 0.062591] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.671lb # [ 0.067144] rcu: srcu_init: Setting srcu_struct sizes based on contention.672worker2 # [ 0.067315] rcu: srcu_init: Setting srcu_struct sizes based on contention.673client # [ 0.216957] Speculative Return Stack Overflow: Mitigation: Safe RET674worker1 # [ 0.062592] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1675lb # [ 0.067151] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns676worker2 # [ 0.067323] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns677worker1 # [ 0.062601] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.678worker2 # [ 0.067428] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)679lb # [ 0.067254] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)680worker1 # [ 0.062602] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.681lb # [ 0.070762] Console: colour VGA+ 80x25682worker2 # [ 0.070974] Console: colour VGA+ 80x25683worker2 # [ 0.070977] printk: legacy console [tty0] enabled684lb # [ 0.070765] printk: legacy console [tty0] enabled685worker1 # [ 0.062603] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.686lb # [ 0.100188] printk: legacy console [ttyS0] enabled687worker2 # [ 0.100508] printk: legacy console [ttyS0] enabled688client # [ 0.217957] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization689worker1 # [ 0.066946] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16690worker2 # [ 0.196265] ACPI: Core revision 20250807691lb # [ 0.196121] ACPI: Core revision 20250807692client # [ 0.219965] Spectre V2 : Enabling IBPB for BPF693worker1 # [ 0.067230] rcu: srcu_init: Setting srcu_struct sizes based on contention.694client # [ 0.221471] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier695worker2 # [ 0.197151] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns696lb # [ 0.197007] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns697worker1 # [ 0.067238] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns698client # [ 0.222914] active return thunk: srso_alias_return_thunk699worker2 # [ 0.198634] APIC: Switch to symmetric I/O mode setup700lb # [ 0.198492] APIC: Switch to symmetric I/O mode setup701worker1 # [ 0.067344] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)702worker2 # [ 0.199570] x2apic enabled703worker1 # [ 0.070922] Console: colour VGA+ 80x25704lb # [ 0.199499] x2apic enabled705client # [ 0.223591] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'706worker1 # [ 0.070925] printk: legacy console [tty0] enabled707worker2 # [ 0.200310] APIC: Switched APIC routing to: physical x2apic708lb # [ 0.200269] APIC: Switched APIC routing to: physical x2apic709worker1 # [ 0.100364] printk: legacy console [ttyS0] enabled710client # [ 0.224957] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'711worker1 # [ 0.196435] ACPI: Core revision 20250807712client # [ 0.225956] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'713worker2 # [ 0.202108] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1714lb # [ 0.202082] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1715client # [ 0.226957] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'716worker1 # [ 0.197338] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns717client # [ 0.227957] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'718worker1 # [ 0.198838] APIC: Switch to symmetric I/O mode setup719worker2 # [ 0.203082] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns720lb # [ 0.203063] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns721worker1 # [ 0.199759] x2apic enabled722client # [ 0.228898] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'723worker2 # [ 0.204710] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)724lb # [ 0.204728] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)725worker1 # [ 0.200498] APIC: Switched APIC routing to: physical x2apic726client # [ 0.229957] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'727worker2 # [ 0.205795] x86/cpu: User Mode Instruction Prevention (UMIP) activated728lb # [ 0.205813] x86/cpu: User Mode Instruction Prevention (UMIP) activated729client # [ 0.230957] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256730worker2 # [ 0.206853] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127731worker1 # [ 0.202319] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1732lb # [ 0.206854] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127733client # [ 0.231957] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64734worker2 # [ 0.207707] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0735lb # [ 0.208388] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0736client # [ 0.232957] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512737worker1 # [ 0.203285] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns738client # [ 0.233957] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024739worker2 # [ 0.208710] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto740lb # [ 0.208729] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto741worker1 # [ 0.204935] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)742client # [ 0.234957] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8743worker2 # [ 0.209707] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl744worker1 # [ 0.206020] x86/cpu: User Mode Instruction Prevention (UMIP) activated745lb # [ 0.210725] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl746client # [ 0.235883] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.747worker2 # [ 0.210707] Transient Scheduler Attacks: Vulnerable: No microcode748lb # [ 0.211725] Transient Scheduler Attacks: Vulnerable: No microcode749worker1 # [ 0.207066] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127750worker2 # [ 0.211706] Spectre V2 : Mitigation: Enhanced / Automatic IBRS751lb # [ 0.212725] Spectre V2 : Mitigation: Enhanced / Automatic IBRS752worker1 # [ 0.207932] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0753worker2 # [ 0.212707] Speculative Return Stack Overflow: Mitigation: Safe RET754lb # [ 0.213725] Speculative Return Stack Overflow: Mitigation: Safe RET755worker1 # [ 0.208936] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto756worker1 # [ 0.209932] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl757worker1 # [ 0.210932] Transient Scheduler Attacks: Vulnerable: No microcode758worker2 # [ 0.213707] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization759lb # [ 0.214724] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization760worker1 # [ 0.212675] Spectre V2 : Mitigation: Enhanced / Automatic IBRS761worker2 # [ 0.215714] Spectre V2 : Enabling IBPB for BPF762lb # [ 0.216732] Spectre V2 : Enabling IBPB for BPF763worker1 # [ 0.213630] Speculative Return Stack Overflow: Mitigation: Safe RET764worker2 # [ 0.216708] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier765lb # [ 0.217726] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier766worker2 # [ 0.217708] active return thunk: srso_alias_return_thunk767lb # [ 0.218725] active return thunk: srso_alias_return_thunk768worker2 # [ 0.218728] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'769lb # [ 0.220378] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'770worker2 # [ 0.219707] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'771lb # [ 0.221620] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'772worker1 # [ 0.213932] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization773worker2 # [ 0.220707] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'774worker1 # [ 0.215939] Spectre V2 : Enabling IBPB for BPF775lb # [ 0.222464] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'776worker2 # [ 0.221707] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'777lb # [ 0.223463] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'778worker1 # [ 0.216933] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier779worker2 # [ 0.222706] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'780lb # [ 0.224476] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'781worker1 # [ 0.217932] active return thunk: srso_alias_return_thunk782worker2 # [ 0.223707] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'783lb # [ 0.225462] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'784worker1 # [ 0.219587] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'785worker2 # [ 0.224706] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'786lb # [ 0.226497] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'787worker1 # [ 0.220932] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'788worker2 # [ 0.225707] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256789lb # [ 0.227654] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256790worker1 # [ 0.221932] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'791worker2 # [ 0.227424] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64792lb # [ 0.228725] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64793worker1 # [ 0.222932] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'794worker2 # [ 0.228444] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512795lb # [ 0.229724] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512796worker1 # [ 0.223932] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'797worker2 # [ 0.229430] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024798lb # [ 0.230725] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024799worker1 # [ 0.224931] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'800worker2 # [ 0.230454] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8801lb # [ 0.231724] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8802worker1 # [ 0.225931] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'803worker2 # [ 0.230706] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.804lb # [ 0.232657] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.805worker1 # [ 0.226932] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256806worker1 # [ 0.227932] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64807worker1 # [ 0.228932] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808worker1 # [ 0.229931] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809worker1 # [ 0.230932] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810worker1 # [ 0.231932] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811client # [ 0.270443] Freeing SMP alternatives memory: 44K812client # [ 0.270960] pid_max: default: 32768 minimum: 301813client # [ 0.272060] LSM: initializing lsm=capability,landlock,yama,bpf,ima814client # [ 0.273059] landlock: Up and running.815client # [ 0.273703] Yama: becoming mindful.816client # [ 0.274626] LSM support for eBPF active817client # [ 0.275624] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818client # [ 0.276970] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819client # [ 0.279251] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820client # [ 0.280536] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821client # [ 0.280966] ... version: 2822client # [ 0.281708] ... bit width: 48823client # [ 0.281959] ... generic counters: 6824client # [ 0.282702] ... generic bitmap: 000000000000003f825client # [ 0.282959] ... fixed-purpose counters: 0826client # [ 0.283697] ... fixed-purpose bitmap: 0000000000000000827client # [ 0.283959] ... value mask: 0000ffffffffffff828client # [ 0.284892] ... max period: 00007fffffffffff829client # [ 0.285696] ... global_ctrl mask: 000000000000003f830client # [ 0.286057] signal: max sigframe size: 3376831client # [ 0.286875] rcu: Hierarchical SRCU implementation.832client # [ 0.287554] rcu: Max phase no-delay instances is 400.833worker2 # [ 0.265929] Freeing SMP alternatives memory: 44K834lb # [ 0.267112] Freeing SMP alternatives memory: 44K835worker2 # [ 0.266710] pid_max: default: 32768 minimum: 301836lb # [ 0.267728] pid_max: default: 32768 minimum: 301837worker2 # [ 0.267542] LSM: initializing lsm=capability,landlock,yama,bpf,ima838lb # [ 0.268802] LSM: initializing lsm=capability,landlock,yama,bpf,ima839worker2 # [ 0.268552] landlock: Up and running.840lb # [ 0.269822] landlock: Up and running.841worker2 # [ 0.269342] Yama: becoming mindful.842lb # [ 0.270439] Yama: becoming mindful.843client # [ 0.292523] smp: Bringing up secondary CPUs ...844worker2 # [ 0.269919] LSM support for eBPF active845lb # [ 0.270936] LSM support for eBPF active846client # [ 0.292973] smp: Brought up 1 node, 1 CPU847worker2 # [ 0.270836] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)848lb # [ 0.271841] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)849client # [ 0.293647] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)850worker1 # [ 0.267176] Freeing SMP alternatives memory: 44K851worker2 # [ 0.271746] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)852lb # [ 0.272769] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)853worker1 # [ 0.267931] pid_max: default: 32768 minimum: 301854client # [ 0.294133] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)855worker1 # [ 0.268560] LSM: initializing lsm=capability,landlock,yama,bpf,ima856client # [ 0.295219] devtmpfs: initialized857worker1 # [ 0.269790] landlock: Up and running.858worker1 # [ 0.270354] Yama: becoming mindful.859worker2 # [ 0.274779] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)860lb # [ 0.275697] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)861client # [ 0.295978] x86/mm: Memory block size: 128MB862worker1 # [ 0.271144] LSM support for eBPF active863lb # [ 0.276311] Performance Events: Fam17h+ core perfctr, AMD PMU driver.864worker2 # [ 0.276282] Performance Events: Fam17h+ core perfctr, AMD PMU driver.865worker1 # [ 0.272051] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)866lb # [ 0.276729] ... version: 2867worker2 # [ 0.276715] ... version: 2868client # [ 0.297716] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)869lb # [ 0.277397] ... bit width: 48870worker2 # [ 0.277370] ... bit width: 48871worker1 # [ 0.272970] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)872lb # [ 0.277728] ... generic counters: 6873worker2 # [ 0.277709] ... generic counters: 6874client # [ 0.298818] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).875lb # [ 0.278403] ... generic bitmap: 000000000000003f876worker2 # [ 0.278406] ... generic bitmap: 000000000000003f877client # [ 0.300030] pinctrl core: initialized pinctrl subsystem878lb # [ 0.278736] ... fixed-purpose counters: 0879worker2 # [ 0.278710] ... fixed-purpose counters: 0880client # [ 0.301128] PM: RTC time: 16:53:42, date: 2026-09-22881worker1 # [ 0.275708] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)882lb # [ 0.279426] ... fixed-purpose bitmap: 0000000000000000883worker2 # [ 0.279381] ... fixed-purpose bitmap: 0000000000000000884lb # [ 0.279728] ... value mask: 0000ffffffffffff885worker2 # [ 0.279709] ... value mask: 0000ffffffffffff886worker1 # [ 0.276445] Performance Events: Fam17h+ core perfctr, AMD PMU driver.887lb # [ 0.280598] ... max period: 00007fffffffffff888worker2 # [ 0.280630] ... max period: 00007fffffffffff889worker1 # [ 0.276937] ... version: 2890lb # [ 0.281384] ... global_ctrl mask: 000000000000003f891worker2 # [ 0.281388] ... global_ctrl mask: 000000000000003f892worker1 # [ 0.277678] ... bit width: 48893lb # [ 0.281856] signal: max sigframe size: 3376894client # [ 0.304668] NET: Registered PF_NETLINK/PF_ROUTE protocol family895worker2 # [ 0.281805] signal: max sigframe size: 3376896worker1 # [ 0.277934] ... generic counters: 6897lb # [ 0.282596] rcu: Hierarchical SRCU implementation.898worker2 # [ 0.282609] rcu: Hierarchical SRCU implementation.899worker1 # [ 0.278615] ... generic bitmap: 000000000000003f900client # [ 0.305452] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations901lb # [ 0.283288] rcu: Max phase no-delay instances is 400.902worker2 # [ 0.283269] rcu: Max phase no-delay instances is 400.903worker1 # [ 0.278980] ... fixed-purpose counters: 0904client # [ 0.305984] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations905worker1 # [ 0.279656] ... fixed-purpose bitmap: 0000000000000000906worker1 # [ 0.279935] ... value mask: 0000ffffffffffff907client # [ 0.307242] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations908worker1 # [ 0.280875] ... max period: 00007fffffffffff909client # [ 0.307970] audit: initializing netlink subsys (disabled)910worker1 # [ 0.281665] ... global_ctrl mask: 000000000000003f911worker1 # [ 0.282045] signal: max sigframe size: 3376912client # [ 0.309166] thermal_sys: Registered thermal governor 'fair_share'913lb # [ 0.288531] smp: Bringing up secondary CPUs ...914worker2 # [ 0.288327] smp: Bringing up secondary CPUs ...915worker1 # [ 0.282879] rcu: Hierarchical SRCU implementation.916client # [ 0.309168] thermal_sys: Registered thermal governor 'bang_bang'917lb # [ 0.289273] smp: Brought up 1 node, 1 CPU918worker2 # [ 0.288723] smp: Brought up 1 node, 1 CPU919worker1 # [ 0.283537] rcu: Max phase no-delay instances is 400.920client # [ 0.309961] thermal_sys: Registered thermal governor 'step_wise'921lb # [ 0.289728] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)922worker2 # [ 0.289359] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)923client # [ 0.310869] thermal_sys: Registered thermal governor 'user_space'924client # [ 0.311666] audit: type=2000 audit(1790096022.966:1): state=initialized audit_enabled=0 res=1925lb # [ 0.290897] Memory: 1454640K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)926worker2 # [ 0.289872] Memory: 1454604K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)927client # [ 0.312962] thermal_sys: Registered thermal governor 'power_allocator'928lb # [ 0.291962] devtmpfs: initialized929worker2 # [ 0.290957] devtmpfs: initialized930client # [ 0.312985] cpuidle: using governor menu931lb # [ 0.292705] x86/mm: Memory block size: 128MB932worker2 # [ 0.291729] x86/mm: Memory block size: 128MB933worker1 # [ 0.288522] smp: Bringing up secondary CPUs ...934client # [ 0.315164] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5935worker1 # [ 0.288948] smp: Brought up 1 node, 1 CPU936lb # [ 0.294309] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)937worker2 # [ 0.293434] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)938worker1 # [ 0.289645] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)939client # [ 0.316197] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]940lb # [ 0.294751] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).941worker2 # [ 0.294585] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).942client # [ 0.316963] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry943lb # [ 0.295808] pinctrl core: initialized pinctrl subsystem944worker2 # [ 0.295807] pinctrl core: initialized pinctrl subsystem945client # [ 0.317971] PCI: Using configuration type 1 for base access946worker1 # [ 0.290147] Memory: 1454592K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)947lb # [ 0.296901] PM: RTC time: 16:53:42, date: 2026-09-22948worker2 # [ 0.296902] PM: RTC time: 16:53:42, date: 2026-09-22949worker1 # [ 0.291191] devtmpfs: initialized950client # [ 0.319044] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.951worker1 # [ 0.292133] x86/mm: Memory block size: 128MB952lb # [ 0.300389] NET: Registered PF_NETLINK/PF_ROUTE protocol family953worker1 # [ 0.293909] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)954worker2 # [ 0.300470] NET: Registered PF_NETLINK/PF_ROUTE protocol family955lb # [ 0.301238] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations956worker1 # [ 0.294816] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).957worker2 # [ 0.301183] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations958worker1 # [ 0.296008] pinctrl core: initialized pinctrl subsystem959client # [ 0.324204] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages960lb # [ 0.301750] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations961worker2 # [ 0.301733] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations962worker1 # [ 0.297062] PM: RTC time: 16:53:42, date: 2026-09-22963client # [ 0.324960] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page964lb # [ 0.302998] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations965worker2 # [ 0.302989] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations966lb # [ 0.303737] audit: initializing netlink subsys (disabled)967worker2 # [ 0.303719] audit: initializing netlink subsys (disabled)968lb # [ 0.304843] thermal_sys: Registered thermal governor 'fair_share'969worker2 # [ 0.304916] thermal_sys: Registered thermal governor 'fair_share'970worker1 # [ 0.300643] NET: Registered PF_NETLINK/PF_ROUTE protocol family971lb # [ 0.304846] thermal_sys: Registered thermal governor 'bang_bang'972worker2 # [ 0.304918] thermal_sys: Registered thermal governor 'bang_bang'973lb # [ 0.305728] thermal_sys: Registered thermal governor 'step_wise'974client # [ 0.329960] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages975worker2 # [ 0.305710] thermal_sys: Registered thermal governor 'step_wise'976worker1 # [ 0.301412] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations977lb # [ 0.306642] thermal_sys: Registered thermal governor 'user_space'978worker2 # [ 0.306671] thermal_sys: Registered thermal governor 'user_space'979client # [ 0.330959] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page980worker1 # [ 0.301959] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations981lb # [ 0.307429] audit: type=2000 audit(1790096022.989:1): state=initialized audit_enabled=0 res=1982worker2 # [ 0.307414] audit: type=2000 audit(1790096022.989:1): state=initialized audit_enabled=0 res=1983lb # [ 0.308730] thermal_sys: Registered thermal governor 'power_allocator'984worker1 # [ 0.303213] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations985worker2 # [ 0.308712] thermal_sys: Registered thermal governor 'power_allocator'986lb # [ 0.308754] cpuidle: using governor menu987worker2 # [ 0.308727] cpuidle: using governor menu988worker1 # [ 0.303944] audit: initializing netlink subsys (disabled)989lb # [ 0.310910] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5990worker1 # [ 0.305115] thermal_sys: Registered thermal governor 'fair_share'991worker2 # [ 0.310927] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5992worker1 # [ 0.305117] thermal_sys: Registered thermal governor 'bang_bang'993lb # [ 0.311939] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]994worker2 # [ 0.311987] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]995lb # [ 0.312730] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry996worker1 # [ 0.305938] audit: type=2000 audit(1790096022.994:1): state=initialized audit_enabled=0 res=1997worker2 # [ 0.312711] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry998lb # [ 0.313741] PCI: Using configuration type 1 for base access999worker1 # [ 0.307935] thermal_sys: Registered thermal governor 'step_wise'1000worker2 # [ 0.313721] PCI: Using configuration type 1 for base access1001worker1 # [ 0.307936] thermal_sys: Registered thermal governor 'user_space'1002lb # [ 0.314823] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1003worker2 # [ 0.314806] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1004worker1 # [ 0.308898] thermal_sys: Registered thermal governor 'power_allocator'1005client # [ 0.341357] ACPI: Added _OSI(Module Device)1006worker1 # [ 0.309732] cpuidle: using governor menu1007client # [ 0.341962] ACPI: Added _OSI(Processor Device)1008client # [ 0.342728] ACPI: Added _OSI(Processor Aggregator Device)1009worker1 # [ 0.311517] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51010worker1 # [ 0.312167] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1011worker1 # [ 0.312937] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1012worker2 # [ 0.319956] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1013worker1 # [ 0.313946] PCI: Using configuration type 1 for base access1014worker2 # [ 0.320710] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1015lb # [ 0.322002] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1016lb # [ 0.322728] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1017worker1 # [ 0.315075] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1018client # [ 0.348873] ACPI: 1 ACPI AML tables successfully acquired and loaded1019worker2 # [ 0.325710] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1020worker2 # [ 0.326710] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1021lb # [ 0.327729] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1022lb # [ 0.328727] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1023worker1 # [ 0.321957] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1024client # [ 0.355380] ACPI: Interpreter enabled1025client # [ 0.355975] ACPI: PM: (supports S0 S3 S4 S5)1026worker1 # [ 0.322935] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1027client # [ 0.356688] ACPI: Using IOAPIC for interrupt routing1028client # [ 0.357005] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1029client # [ 0.357960] PCI: Using E820 reservations for host bridge windows1030worker1 # [ 0.327936] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1031lb # [ 0.337308] ACPI: Added _OSI(Module Device)1032worker1 # [ 0.328935] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1033client # [ 0.361048] ACPI: Enabled 2 GPEs in block 00 to 3F1034worker2 # [ 0.337062] ACPI: Added _OSI(Module Device)1035lb # [ 0.337729] ACPI: Added _OSI(Processor Device)1036worker2 # [ 0.337711] ACPI: Added _OSI(Processor Device)1037lb # [ 0.338489] ACPI: Added _OSI(Processor Aggregator Device)1038worker2 # [ 0.338450] ACPI: Added _OSI(Processor Aggregator Device)1039lb # [ 0.344280] ACPI: 1 ACPI AML tables successfully acquired and loaded1040client # [ 0.369680] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1041client # [ 0.369965] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1042worker1 # [ 0.340181] ACPI: Added _OSI(Module Device)1043worker2 # [ 0.347259] ACPI: 1 ACPI AML tables successfully acquired and loaded1044lb # [ 0.347691] ACPI: Interpreter enabled1045worker1 # [ 0.340928] ACPI: Added _OSI(Processor Device)1046lb # [ 0.348177] ACPI: PM: (supports S0 S3 S4 S5)1047client # [ 0.371680] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1048worker1 # [ 0.341690] ACPI: Added _OSI(Processor Aggregator Device)1049client # [ 0.373031] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1050client # [ 0.374299] PCI host bridge to bus 0000:001051lb # [ 0.350728] ACPI: Using IOAPIC for interrupt routing1052worker2 # [ 0.350958] ACPI: Interpreter enabled1053client # [ 0.374964] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1054worker2 # [ 0.351631] ACPI: PM: (supports S0 S3 S4 S5)1055lb # [ 0.351596] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1056client # [ 0.375960] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1057worker2 # [ 0.352196] ACPI: Using IOAPIC for interrupt routing1058client # [ 0.376959] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1059worker2 # [ 0.352777] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1060worker1 # [ 0.347315] ACPI: 1 ACPI AML tables successfully acquired and loaded1061client # [ 0.377959] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1062worker2 # [ 0.353710] PCI: Using E820 reservations for host bridge windows1063lb # [ 0.354727] PCI: Using E820 reservations for host bridge windows1064lb # [ 0.355778] ACPI: Enabled 2 GPEs in block 00 to 3F1065client # [ 0.378960] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1066client # [ 0.379960] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1067worker2 # [ 0.356773] ACPI: Enabled 2 GPEs in block 00 to 3F1068client # [ 0.380961] pci_bus 0000:00: root bus resource [bus 00-ff]1069worker1 # [ 0.351347] ACPI: Interpreter enabled1070worker1 # [ 0.351950] ACPI: PM: (supports S0 S3 S4 S5)1071client # [ 0.382003] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1072worker1 # [ 0.352698] ACPI: Using IOAPIC for interrupt routing1073client # [ 0.383431] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1074worker1 # [ 0.355008] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1075lb # [ 0.364381] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1076worker1 # [ 0.355935] PCI: Using E820 reservations for host bridge windows1077worker2 # [ 0.365445] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1078lb # [ 0.366071] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1079worker1 # [ 0.359066] ACPI: Enabled 2 GPEs in block 00 to 3F1080worker2 # [ 0.365717] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1081lb # [ 0.366805] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1082client # [ 0.386024] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1083worker2 # [ 0.367286] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1084lb # [ 0.367849] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1085client # [ 0.387096] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1086lb # [ 0.369085] PCI host bridge to bus 0000:001087worker2 # [ 0.367832] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1088client # [ 0.388022] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1089lb # [ 0.369731] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1090worker2 # [ 0.369046] PCI host bridge to bus 0000:001091lb # [ 0.370727] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1092worker2 # [ 0.369714] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1093client # [ 0.389289] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1094lb # [ 0.371727] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1095worker2 # [ 0.370710] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1096worker1 # [ 0.366388] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1097lb # [ 0.372727] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1098worker2 # [ 0.371709] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1099client # [ 0.390700] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1100lb # [ 0.373727] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1101worker2 # [ 0.372710] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1102worker1 # [ 0.366941] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1103lb # [ 0.374727] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1104worker2 # [ 0.373709] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1105worker1 # [ 0.368027] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1106lb # [ 0.375729] pci_bus 0000:00: root bus resource [bus 00-ff]1107client # [ 0.392977] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1108worker2 # [ 0.374710] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1109worker1 # [ 0.369058] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1110client # [ 0.393789] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1111worker2 # [ 0.375724] pci_bus 0000:00: root bus resource [bus 00-ff]1112lb # [ 0.376657] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1113worker1 # [ 0.370270] PCI host bridge to bus 0000:001114client # [ 0.394685] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1115worker2 # [ 0.376681] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1116worker1 # [ 0.370940] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1117client # [ 0.394965] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1118lb # [ 0.378126] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1119worker1 # [ 0.371934] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1120worker2 # [ 0.378177] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1121worker1 # [ 0.372934] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1122client # [ 0.396454] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1123worker1 # [ 0.373934] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1124worker1 # [ 0.374934] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1125client # [ 0.397976] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1126worker1 # [ 0.375934] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1127client # [ 0.398844] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1128worker1 # [ 0.376935] pci_bus 0000:00: root bus resource [bus 00-ff]1129client # [ 0.399705] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1130lb # [ 0.380794] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1131worker1 # [ 0.377953] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1132lb # [ 0.381740] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1133client # [ 0.400507] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1134worker2 # [ 0.380805] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1135lb # [ 0.382679] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1136worker1 # [ 0.379425] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1137worker2 # [ 0.381722] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1138worker2 # [ 0.382678] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1139lb # [ 0.383621] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1140client # [ 0.402975] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1141worker2 # [ 0.383636] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1142lb # [ 0.385534] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1143client # [ 0.404604] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1144client # [ 0.404981] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1145worker2 # [ 0.385468] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1146lb # [ 0.387736] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1147client # [ 0.406499] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1148lb # [ 0.388569] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1149worker1 # [ 0.382030] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1150worker2 # [ 0.388750] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1151lb # [ 0.389453] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1152worker1 # [ 0.382947] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1153client # [ 0.407968] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1154lb # [ 0.389733] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1155worker2 # [ 0.389608] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1156worker1 # [ 0.383923] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1157client # [ 0.408781] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1158worker2 # [ 0.390449] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1159worker2 # [ 0.390715] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1160client # [ 0.409723] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1161worker1 # [ 0.384854] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1162lb # [ 0.391379] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1163client # [ 0.409965] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1164worker2 # [ 0.392259] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1165worker1 # [ 0.386642] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1166lb # [ 0.392735] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1167client # [ 0.411509] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1168lb # [ 0.393588] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1169lb # [ 0.394457] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1170worker2 # [ 0.395748] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1171worker1 # [ 0.388971] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1172client # [ 0.412972] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1173worker2 # [ 0.396563] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1174worker1 # [ 0.389798] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1175client # [ 0.413944] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1176lb # [ 0.395286] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1177worker2 # [ 0.397403] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1178worker1 # [ 0.390653] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1179worker1 # [ 0.390940] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1180client # [ 0.415395] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1181worker2 # [ 0.398286] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1182worker1 # [ 0.392613] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1183client # [ 0.416973] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1184lb # [ 0.397735] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1185lb # [ 0.399351] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1186client # [ 0.417981] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1187worker1 # [ 0.394973] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1188lb # [ 0.399749] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1189worker2 # [ 0.400983] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1190worker1 # [ 0.395841] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1191worker2 # [ 0.402373] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1192client # [ 0.419536] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1193lb # [ 0.401272] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1194worker1 # [ 0.396700] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1195worker2 # [ 0.402731] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1196client # [ 0.420972] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1197worker1 # [ 0.398363] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1198lb # [ 0.402735] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1199worker2 # [ 0.404307] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1200client # [ 0.421981] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1201lb # [ 0.403565] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1202lb # [ 0.404451] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1203client # [ 0.423564] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1204lb # [ 0.404733] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1205worker2 # [ 0.406615] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1206worker1 # [ 0.401541] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1207worker2 # [ 0.407361] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1208client # [ 0.424972] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1209worker1 # [ 0.401941] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1210worker2 # [ 0.407731] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1211lb # [ 0.406816] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1212worker1 # [ 0.402866] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1213client # [ 0.425945] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1214worker2 # [ 0.408716] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1215lb # [ 0.408739] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1216worker1 # [ 0.404352] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1217client # [ 0.427540] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1218lb # [ 0.409678] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1219worker2 # [ 0.410245] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1220worker1 # [ 0.405971] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1221lb # [ 0.411139] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1222client # [ 0.429004] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1223worker1 # [ 0.406866] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1224worker2 # [ 0.411753] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1225client # [ 0.429854] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1226worker2 # [ 0.412701] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1227worker1 # [ 0.407700] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1228lb # [ 0.412740] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1229client # [ 0.430728] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1230lb # [ 0.413673] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1231worker1 # [ 0.407940] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1232worker2 # [ 0.414277] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1233client # [ 0.432401] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1234worker1 # [ 0.409586] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1235lb # [ 0.415102] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1236worker2 # [ 0.415756] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1237worker1 # [ 0.410823] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1238lb # [ 0.417740] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1239client # [ 0.434998] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1240worker2 # [ 0.416725] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1241worker1 # [ 0.411724] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1242client # [ 0.435841] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1243lb # [ 0.418671] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1244worker2 # [ 0.418279] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1245client # [ 0.436710] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1246worker1 # [ 0.413428] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1247lb # [ 0.420072] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1248client # [ 0.437550] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1249worker2 # [ 0.419755] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1250worker1 # [ 0.414948] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1251lb # [ 0.421740] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1252worker2 # [ 0.420720] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1253client # [ 0.438649] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1254lb # [ 0.422673] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1255worker1 # [ 0.415875] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1256client # [ 0.439194] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1257worker2 # [ 0.422299] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1258worker1 # [ 0.417492] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1259lb # [ 0.424058] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1260client # [ 0.440598] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1261worker2 # [ 0.423722] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1262worker1 # [ 0.418778] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1263lb # [ 0.425763] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1264client # [ 0.441165] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1265worker2 # [ 0.424732] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1266worker1 # [ 0.419732] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1267lb # [ 0.426583] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1268client # [ 0.442599] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1269lb # [ 0.427449] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1270worker2 # [ 0.426238] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1271worker1 # [ 0.421295] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1272lb # [ 0.428271] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1273worker1 # [ 0.422935] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1274worker2 # [ 0.428540] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1275worker1 # [ 0.423945] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1276worker2 # [ 0.429328] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1277lb # [ 0.429735] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1278worker2 # [ 0.429731] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1279worker1 # [ 0.425946] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1280lb # [ 0.430577] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1281worker2 # [ 0.431285] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1282lb # [ 0.431749] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1283worker1 # [ 0.427952] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1284worker1 # [ 0.428825] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1285lb # [ 0.433295] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1286worker1 # [ 0.429678] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1287worker2 # [ 0.433747] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1288lb # [ 0.434448] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1289worker2 # [ 0.434632] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1290worker1 # [ 0.430515] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1291worker2 # [ 0.435469] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1292client # [ 0.443173] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1293lb # [ 0.434926] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1294lb # [ 0.436369] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1295client # [ 0.445586] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1296worker2 # [ 0.436264] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1297worker1 # [ 0.431973] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1298lb # [ 0.436925] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1299client # [ 0.446240] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1300worker2 # [ 0.437493] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1301worker1 # [ 0.432941] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1302worker1 # [ 0.433876] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1303worker2 # [ 0.437914] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1304client # [ 0.447250] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1305lb # [ 0.438351] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1306worker1 # [ 0.435343] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1307client # [ 0.448228] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1308worker2 # [ 0.439424] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1309worker1 # [ 0.436500] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1310client # [ 0.450006] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1311worker2 # [ 0.439912] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1312client # [ 0.450860] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1313worker1 # [ 0.437132] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1314worker2 # [ 0.441511] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1315worker1 # [ 0.438510] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1316client # [ 0.452019] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1317client # [ 0.453617] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1318worker1 # [ 0.439130] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1319worker1 # [ 0.440463] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1320client # [ 0.457435] ACPI: PCI: Interrupt link LNKA configured for IRQ 101321lb # [ 0.439669] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1322client # [ 0.458097] ACPI: PCI: Interrupt link LNKB configured for IRQ 101323client # [ 0.459064] ACPI: PCI: Interrupt link LNKC configured for IRQ 111324lb # [ 0.442184] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1325client # [ 0.460056] ACPI: PCI: Interrupt link LNKD configured for IRQ 111326lb # [ 0.442981] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1327client # [ 0.461056] ACPI: PCI: Interrupt link LNKE configured for IRQ 101328client # [ 0.462056] ACPI: PCI: Interrupt link LNKF configured for IRQ 101329worker2 # [ 0.442574] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1330lb # [ 0.444012] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1331client # [ 0.463061] ACPI: PCI: Interrupt link LNKG configured for IRQ 111332worker2 # [ 0.444417] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1333lb # [ 0.444970] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1334client # [ 0.464060] ACPI: PCI: Interrupt link LNKH configured for IRQ 111335worker2 # [ 0.444991] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1336client # [ 0.464997] ACPI: PCI: Interrupt link GSIA configured for IRQ 161337worker1 # [ 0.441196] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1338client # [ 0.465972] ACPI: PCI: Interrupt link GSIB configured for IRQ 171339lb # [ 0.446756] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1340worker2 # [ 0.446036] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1341worker1 # [ 0.443479] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1342client # [ 0.466977] ACPI: PCI: Interrupt link GSIC configured for IRQ 181343lb # [ 0.447583] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1344worker2 # [ 0.446965] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1345client # [ 0.467970] ACPI: PCI: Interrupt link GSID configured for IRQ 191346worker1 # [ 0.444181] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1347lb # [ 0.449041] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1348client # [ 0.468971] ACPI: PCI: Interrupt link GSIE configured for IRQ 201349worker2 # [ 0.448772] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1350client # [ 0.469948] ACPI: PCI: Interrupt link GSIF configured for IRQ 211351worker1 # [ 0.445221] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1352lb # [ 0.450452] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1353worker2 # [ 0.449616] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1354client # [ 0.470703] ACPI: PCI: Interrupt link GSIG configured for IRQ 221355worker1 # [ 0.446219] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1356client # [ 0.470973] ACPI: PCI: Interrupt link GSIH configured for IRQ 231357worker2 # [ 0.450730] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1358worker1 # [ 0.447871] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1359client # [ 0.473009] iommu: Default domain type: Translated1360worker2 # [ 0.452438] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1361lb # [ 0.453689] ACPI: PCI: Interrupt link LNKA configured for IRQ 101362worker1 # [ 0.448588] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1363client # [ 0.473793] iommu: DMA domain TLB invalidation policy: lazy mode1364lb # [ 0.454536] ACPI: PCI: Interrupt link LNKB configured for IRQ 101365client # [ 0.474937] ACPI: bus type USB registered1366worker1 # [ 0.449250] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1367lb # [ 0.455539] ACPI: PCI: Interrupt link LNKC configured for IRQ 111368client # [ 0.475489] usbcore: registered new interface driver usbfs1369client # [ 0.475976] usbcore: registered new interface driver hub1370lb # [ 0.456539] ACPI: PCI: Interrupt link LNKD configured for IRQ 111371worker1 # [ 0.450934] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1372client # [ 0.476866] usbcore: registered new device driver usb1373worker2 # [ 0.456089] ACPI: PCI: Interrupt link LNKA configured for IRQ 101374lb # [ 0.457543] ACPI: PCI: Interrupt link LNKE configured for IRQ 101375worker2 # [ 0.456817] ACPI: PCI: Interrupt link LNKB configured for IRQ 101376lb # [ 0.458540] ACPI: PCI: Interrupt link LNKF configured for IRQ 101377client # [ 0.478466] NetLabel: Initializing1378client # [ 0.478961] NetLabel: domain hash size = 1281379worker2 # [ 0.457808] ACPI: PCI: Interrupt link LNKC configured for IRQ 111380lb # [ 0.459535] ACPI: PCI: Interrupt link LNKG configured for IRQ 111381client # [ 0.479696] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1382worker2 # [ 0.458804] ACPI: PCI: Interrupt link LNKD configured for IRQ 111383lb # [ 0.460536] ACPI: PCI: Interrupt link LNKH configured for IRQ 111384worker1 # [ 0.454813] ACPI: PCI: Interrupt link LNKA configured for IRQ 101385client # [ 0.479997] NetLabel: unlabeled traffic allowed by default1386worker2 # [ 0.459803] ACPI: PCI: Interrupt link LNKE configured for IRQ 101387lb # [ 0.461452] ACPI: PCI: Interrupt link GSIA configured for IRQ 161388client # [ 0.480903] PCI: Using ACPI for IRQ routing1389worker1 # [ 0.456044] ACPI: PCI: Interrupt link LNKB configured for IRQ 101390lb # [ 0.461742] ACPI: PCI: Interrupt link GSIB configured for IRQ 171391worker2 # [ 0.460803] ACPI: PCI: Interrupt link LNKF configured for IRQ 101392worker1 # [ 0.457036] ACPI: PCI: Interrupt link LNKC configured for IRQ 111393lb # [ 0.462693] ACPI: PCI: Interrupt link GSIC configured for IRQ 181394worker2 # [ 0.461809] ACPI: PCI: Interrupt link LNKG configured for IRQ 111395worker1 # [ 0.458035] ACPI: PCI: Interrupt link LNKD configured for IRQ 111396lb # [ 0.463447] ACPI: PCI: Interrupt link GSID configured for IRQ 191397worker2 # [ 0.462807] ACPI: PCI: Interrupt link LNKH configured for IRQ 111398lb # [ 0.463739] ACPI: PCI: Interrupt link GSIE configured for IRQ 201399worker1 # [ 0.459030] ACPI: PCI: Interrupt link LNKE configured for IRQ 101400worker2 # [ 0.463744] ACPI: PCI: Interrupt link GSIA configured for IRQ 161401lb # [ 0.464666] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402worker1 # [ 0.460030] ACPI: PCI: Interrupt link LNKF configured for IRQ 101403worker2 # [ 0.464730] ACPI: PCI: Interrupt link GSIB configured for IRQ 171404lb # [ 0.465451] ACPI: PCI: Interrupt link GSIG configured for IRQ 221405worker1 # [ 0.461037] ACPI: PCI: Interrupt link LNKG configured for IRQ 111406worker2 # [ 0.465723] ACPI: PCI: Interrupt link GSIC configured for IRQ 181407lb # [ 0.465740] ACPI: PCI: Interrupt link GSIH configured for IRQ 231408worker1 # [ 0.462031] ACPI: PCI: Interrupt link LNKH configured for IRQ 111409worker2 # [ 0.466709] ACPI: PCI: Interrupt link GSID configured for IRQ 191410worker1 # [ 0.462968] ACPI: PCI: Interrupt link GSIA configured for IRQ 161411worker2 # [ 0.467634] ACPI: PCI: Interrupt link GSIE configured for IRQ 201412lb # [ 0.467671] iommu: Default domain type: Translated1413worker1 # [ 0.463922] ACPI: PCI: Interrupt link GSIB configured for IRQ 171414worker2 # [ 0.468409] ACPI: PCI: Interrupt link GSIF configured for IRQ 211415lb # [ 0.468312] iommu: DMA domain TLB invalidation policy: lazy mode1416worker2 # [ 0.468721] ACPI: PCI: Interrupt link GSIG configured for IRQ 221417worker1 # [ 0.464670] ACPI: PCI: Interrupt link GSIC configured for IRQ 181418lb # [ 0.468993] ACPI: bus type USB registered1419worker2 # [ 0.469642] ACPI: PCI: Interrupt link GSIH configured for IRQ 231420worker1 # [ 0.464950] ACPI: PCI: Interrupt link GSID configured for IRQ 191421lb # [ 0.469735] usbcore: registered new interface driver usbfs1422lb # [ 0.470607] usbcore: registered new interface driver hub1423worker1 # [ 0.465950] ACPI: PCI: Interrupt link GSIE configured for IRQ 201424worker2 # [ 0.471530] iommu: Default domain type: Translated1425lb # [ 0.471377] usbcore: registered new device driver usb1426worker1 # [ 0.466916] ACPI: PCI: Interrupt link GSIF configured for IRQ 211427worker2 # [ 0.472274] iommu: DMA domain TLB invalidation policy: lazy mode1428worker1 # [ 0.467654] ACPI: PCI: Interrupt link GSIG configured for IRQ 221429lb # [ 0.472582] NetLabel: Initializing1430worker2 # [ 0.472985] ACPI: bus type USB registered1431worker1 # [ 0.467946] ACPI: PCI: Interrupt link GSIH configured for IRQ 231432lb # [ 0.473152] NetLabel: domain hash size = 1281433worker2 # [ 0.473712] usbcore: registered new interface driver usbfs1434lb # [ 0.473727] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1435worker2 # [ 0.474622] usbcore: registered new interface driver hub1436worker1 # [ 0.469993] iommu: Default domain type: Translated1437lb # [ 0.474767] NetLabel: unlabeled traffic allowed by default1438worker2 # [ 0.475381] usbcore: registered new device driver usb1439worker1 # [ 0.470748] iommu: DMA domain TLB invalidation policy: lazy mode1440lb # [ 0.475738] PCI: Using ACPI for IRQ routing1441worker1 # [ 0.471962] ACPI: bus type USB registered1442worker2 # [ 0.476634] NetLabel: Initializing1443worker2 # [ 0.477114] NetLabel: domain hash size = 1281444worker1 # [ 0.472674] usbcore: registered new interface driver usbfs1445worker2 # [ 0.477709] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1446worker1 # [ 0.472947] usbcore: registered new interface driver hub1447worker2 # [ 0.478659] NetLabel: unlabeled traffic allowed by default1448worker1 # [ 0.473824] usbcore: registered new device driver usb1449worker2 # [ 0.479397] PCI: Using ACPI for IRQ routing1450worker1 # [ 0.475389] NetLabel: Initializing1451worker1 # [ 0.475936] NetLabel: domain hash size = 1281452worker1 # [ 0.476663] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453worker1 # [ 0.476974] NetLabel: unlabeled traffic allowed by default1454worker1 # [ 0.477874] PCI: Using ACPI for IRQ routing1455client # [ 0.525638] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456client # [ 0.525956] pci 0000:00:01.0: vgaarb: bridge control possible1457client # [ 0.525956] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458client # [ 0.525965] vgaarb: loaded1459client # [ 0.526747] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460client # [ 0.526961] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461client # [ 0.532047] clocksource: Switched to clocksource kvm-clock1462client # [ 0.535642] VFS: Disk quotas dquot_6.6.01463client # [ 0.536357] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464client # [ 0.537711] pnp: PnP ACPI init1465lb # [ 0.518405] pci 0000:00:01.0: vgaarb: setting as boot VGA device1466client # [ 0.538448] ACPI: IRQ 4 override to edge(!), high(!)1467lb # [ 0.518723] pci 0000:00:01.0: vgaarb: bridge control possible1468client # [ 0.539417] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1469lb # [ 0.518723] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1470lb # [ 0.518731] vgaarb: loaded1471client # [ 0.541012] pnp: PnP ACPI: found 5 devices1472lb # [ 0.519463] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01473lb # [ 0.519729] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1474worker2 # [ 0.522985] pci 0000:00:01.0: vgaarb: setting as boot VGA device1475worker2 # [ 0.523705] pci 0000:00:01.0: vgaarb: bridge control possible1476worker2 # [ 0.523705] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1477worker2 # [ 0.523714] vgaarb: loaded1478lb # [ 0.523807] clocksource: Switched to clocksource kvm-clock1479worker1 # [ 0.522484] pci 0000:00:01.0: vgaarb: setting as boot VGA device1480worker2 # [ 0.524500] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01481worker1 # [ 0.522931] pci 0000:00:01.0: vgaarb: bridge control possible1482lb # [ 0.525505] VFS: Disk quotas dquot_6.6.01483worker2 # [ 0.525325] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1484client # [ 0.548672] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1485worker1 # [ 0.522931] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1486client # [ 0.550079] clocksource: Switched to clocksource acpi_pm1487worker1 # [ 0.522939] vgaarb: loaded1488client # [ 0.551088] NET: Registered PF_INET protocol family1489worker1 # [ 0.523681] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01490lb # [ 0.527981] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1491worker1 # [ 0.523935] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1492lb # [ 0.529299] pnp: PnP ACPI init1493client # [ 0.552319] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1494worker2 # [ 0.528798] clocksource: Switched to clocksource kvm-clock1495lb # [ 0.530088] ACPI: IRQ 4 override to edge(!), high(!)1496lb # [ 0.531015] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1497lb # [ 0.532528] pnp: PnP ACPI: found 5 devices1498worker2 # [ 0.532319] VFS: Disk quotas dquot_6.6.01499worker1 # [ 0.527023] clocksource: Switched to clocksource kvm-clock1500worker2 # [ 0.533060] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1501worker2 # [ 0.534400] pnp: PnP ACPI init1502worker2 # [ 0.535140] ACPI: IRQ 4 override to edge(!), high(!)1503worker1 # [ 0.530643] VFS: Disk quotas dquot_6.6.01504worker2 # [ 0.536081] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1505worker1 # [ 0.531404] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1506worker2 # [ 0.537608] pnp: PnP ACPI: found 5 devices1507worker1 # [ 0.532787] pnp: PnP ACPI init1508worker1 # [ 0.533605] ACPI: IRQ 4 override to edge(!), high(!)1509lb # [ 0.540096] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1510worker1 # [ 0.534630] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1511lb # [ 0.541496] clocksource: Switched to clocksource acpi_pm1512worker1 # [ 0.536218] pnp: PnP ACPI: found 5 devices1513lb # [ 0.542436] NET: Registered PF_INET protocol family1514lb # [ 0.543618] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1515client # [ 0.568855] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1516worker2 # [ 0.545202] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1517worker2 # [ 0.546610] clocksource: Switched to clocksource acpi_pm1518client # [ 0.570433] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1519worker2 # [ 0.547623] NET: Registered PF_INET protocol family1520client # [ 0.572455] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1521worker2 # [ 0.548868] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1522worker1 # [ 0.543811] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1523client # [ 0.574230] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1524worker1 # [ 0.545303] clocksource: Switched to clocksource acpi_pm1525client # [ 0.575419] TCP: Hash tables configured (established 16384 bind 16384)1526worker1 # [ 0.546344] NET: Registered PF_INET protocol family1527client # [ 0.576692] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1528worker1 # [ 0.547536] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1529client # [ 0.578064] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1530client # [ 0.579174] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1531client # [ 0.580394] NET: Registered PF_UNIX/PF_LOCAL protocol family1532client # [ 0.581304] NET: Registered PF_XDP protocol family1533client # [ 0.582161] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1534client # [ 0.583149] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1535client # [ 0.584161] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1536lb # [ 0.560297] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1537client # [ 0.585232] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1538lb # [ 0.561812] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1539client # [ 0.586299] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1540lb # [ 0.563080] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1541client # [ 0.587350] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1542lb # [ 0.564378] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1543client # [ 0.589102] ACPI: \_SB_.GSIA: Enabled at IRQ 161544lb # [ 0.565602] TCP: Hash tables configured (established 16384 bind 16384)1545worker2 # [ 0.565548] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1546lb # [ 0.566679] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1547worker2 # [ 0.566949] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1548lb # [ 0.567896] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1549client # [ 0.592024] ACPI: \_SB_.GSIB: Enabled at IRQ 171550lb # [ 0.568946] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1551worker2 # [ 0.568204] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1552lb # [ 0.570112] NET: Registered PF_UNIX/PF_LOCAL protocol family1553client # [ 0.594007] ACPI: \_SB_.GSIC: Enabled at IRQ 181554worker2 # [ 0.569552] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1555worker1 # [ 0.564018] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1556lb # [ 0.571024] NET: Registered PF_XDP protocol family1557worker2 # [ 0.570751] TCP: Hash tables configured (established 16384 bind 16384)1558lb # [ 0.571833] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1559worker1 # [ 0.565539] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1560client # [ 0.596037] ACPI: \_SB_.GSID: Enabled at IRQ 191561worker2 # [ 0.571837] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1562lb # [ 0.572794] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1563worker1 # [ 0.566851] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1564worker2 # [ 0.573058] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1565lb # [ 0.573747] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1566client # [ 0.597814] PCI: CLS 0 bytes, default 641567worker1 # [ 0.568190] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1568worker2 # [ 0.574110] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1569lb # [ 0.574796] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1570client # [ 0.598773] Trying to unpack rootfs image as initramfs...1571worker1 # [ 0.569408] TCP: Hash tables configured (established 16384 bind 16384)1572worker2 # [ 0.575267] NET: Registered PF_UNIX/PF_LOCAL protocol family1573lb # [ 0.575834] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1574worker2 # [ 0.576161] NET: Registered PF_XDP protocol family1575worker1 # [ 0.570504] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1576lb # [ 0.576898] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1577worker2 # [ 0.576991] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1578worker1 # [ 0.571771] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1579lb # [ 0.578532] ACPI: \_SB_.GSIA: Enabled at IRQ 161580worker2 # [ 0.578013] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1581worker1 # [ 0.572880] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1582worker2 # [ 0.578995] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1583client # [ 0.602869] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1584worker1 # [ 0.574189] NET: Registered PF_UNIX/PF_LOCAL protocol family1585lb # [ 0.580779] ACPI: \_SB_.GSIB: Enabled at IRQ 171586worker2 # [ 0.580068] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1587worker1 # [ 0.575145] NET: Registered PF_XDP protocol family1588worker2 # [ 0.581129] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1589worker1 # [ 0.576081] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1590lb # [ 0.582967] ACPI: \_SB_.GSIC: Enabled at IRQ 181591worker2 # [ 0.582192] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1592worker1 # [ 0.577080] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1593worker1 # [ 0.578068] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1594worker2 # [ 0.583868] ACPI: \_SB_.GSIA: Enabled at IRQ 161595lb # [ 0.584996] ACPI: \_SB_.GSID: Enabled at IRQ 191596worker1 # [ 0.579206] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1597lb # [ 0.586704] PCI: CLS 0 bytes, default 641598worker1 # [ 0.580366] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1599worker2 # [ 0.586643] ACPI: \_SB_.GSIB: Enabled at IRQ 171600lb # [ 0.587699] Trying to unpack rootfs image as initramfs...1601worker1 # [ 0.581512] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1602worker1 # [ 0.583271] ACPI: \_SB_.GSIA: Enabled at IRQ 161603worker2 # [ 0.588925] ACPI: \_SB_.GSIC: Enabled at IRQ 181604worker2 # [ 0.590932] ACPI: \_SB_.GSID: Enabled at IRQ 191605worker1 # [ 0.585503] ACPI: \_SB_.GSIB: Enabled at IRQ 171606lb # [ 0.592019] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1607worker2 # [ 0.592701] PCI: CLS 0 bytes, default 641608worker1 # [ 0.587851] ACPI: \_SB_.GSIC: Enabled at IRQ 181609worker2 # [ 0.593624] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1610worker1 # [ 0.589839] ACPI: \_SB_.GSID: Enabled at IRQ 191611worker2 # [ 0.595270] Trying to unpack rootfs image as initramfs...1612worker1 # [ 0.591504] PCI: CLS 0 bytes, default 641613worker1 # [ 0.592455] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614worker1 # [ 0.594267] Trying to unpack rootfs image as initramfs...1615client # [ 0.644347] Initialise system trusted keyrings1616client # [ 0.646703] workingset: timestamp_bits=40 max_order=19 bucket_order=01617lb # [ 0.633474] Initialise system trusted keyrings1618lb # [ 0.636658] workingset: timestamp_bits=40 max_order=19 bucket_order=01619worker2 # [ 0.639161] Initialise system trusted keyrings1620worker2 # [ 0.642458] workingset: timestamp_bits=40 max_order=19 bucket_order=01621worker1 # [ 0.638747] Initialise system trusted keyrings1622client # [ 0.668771] Key type asymmetric registered1623client # [ 0.669479] Asymmetric key parser 'x509' registered1624client # [ 0.670324] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1625worker1 # [ 0.641776] workingset: timestamp_bits=40 max_order=19 bucket_order=01626client # [ 0.675805] io scheduler mq-deadline registered1627client # [ 0.676542] io scheduler kyber registered1628client # [ 0.679297] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1629client # [ 0.680534] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630lb # [ 0.657809] Key type asymmetric registered1631lb # [ 0.658512] Asymmetric key parser 'x509' registered1632lb # [ 0.659358] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1633client # [ 0.686073] Linux agpgart interface v0.1031634client # [ 0.686841] ACPI: bus type drm_connector registered1635worker2 # [ 0.663652] Key type asymmetric registered1636lb # [ 0.664706] io scheduler mq-deadline registered1637worker2 # [ 0.664342] Asymmetric key parser 'x509' registered1638lb # [ 0.665452] io scheduler kyber registered1639worker2 # [ 0.665180] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1640client # [ 0.690132] usbcore: registered new interface driver usbserial_generic1641client # [ 0.691203] usbserial: USB Serial support registered for generic1642lb # [ 0.668167] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1643worker2 # [ 0.668554] io scheduler mq-deadline registered1644worker1 # [ 0.663791] Key type asymmetric registered1645worker2 # [ 0.669277] io scheduler kyber registered1646lb # [ 0.669424] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1647worker1 # [ 0.664547] Asymmetric key parser 'x509' registered1648client # [ 0.693661] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1649client # [ 0.693661] Please enable it if your BIOS has the CPPC option.1650worker1 # [ 0.665455] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1651client # [ 0.695796] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1652worker2 # [ 0.673008] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1653lb # [ 0.674991] Linux agpgart interface v0.1031654worker2 # [ 0.674256] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1655client # [ 0.698792] drop_monitor: Initializing network drop monitor service1656lb # [ 0.675742] ACPI: bus type drm_connector registered1657client # [ 0.699939] NET: Registered PF_INET6 protocol family1658worker1 # [ 0.670861] io scheduler mq-deadline registered1659worker1 # [ 0.671678] io scheduler kyber registered1660worker2 # [ 0.678365] Linux agpgart interface v0.1031661lb # [ 0.678961] usbcore: registered new interface driver usbserial_generic1662client # [ 0.703187] Segment Routing with IPv61663worker2 # [ 0.679107] ACPI: bus type drm_connector registered1664lb # [ 0.680074] usbserial: USB Serial support registered for generic1665worker1 # [ 0.675277] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1666client # [ 0.705659] In-situ OAM (IOAM) with IPv61667client # [ 0.706717] IPI shorthand broadcast: enabled1668worker1 # [ 0.676597] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1669lb # [ 0.682612] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1670lb # [ 0.682612] Please enable it if your BIOS has the CPPC option.1671worker2 # [ 0.683854] usbcore: registered new interface driver usbserial_generic1672lb # [ 0.684749] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1673worker2 # [ 0.684905] usbserial: USB Serial support registered for generic1674worker1 # [ 0.680510] Linux agpgart interface v0.1031675worker2 # [ 0.685857] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1676worker1 # [ 0.681319] ACPI: bus type drm_connector registered1677worker2 # [ 0.685857] Please enable it if your BIOS has the CPPC option.1678lb # [ 0.688741] drop_monitor: Initializing network drop monitor service1679worker2 # [ 0.687970] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1680lb # [ 0.689863] NET: Registered PF_INET6 protocol family1681client # [ 0.715026] sched_clock: Marking stable (579014565, 135613002)->(788144809, -73517242)1682worker1 # [ 0.686158] usbcore: registered new interface driver usbserial_generic1683lb # [ 0.693049] Segment Routing with IPv61684worker1 # [ 0.687299] usbserial: USB Serial support registered for generic1685lb # [ 0.693718] In-situ OAM (IOAM) with IPv61686worker2 # [ 0.693549] drop_monitor: Initializing network drop monitor service1687worker2 # [ 0.694702] NET: Registered PF_INET6 protocol family1688lb # [ 0.695880] IPI shorthand broadcast: enabled1689client # [ 0.719929] registered taskstats version 11690worker1 # [ 0.689735] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1691worker1 # [ 0.689735] Please enable it if your BIOS has the CPPC option.1692client # [ 0.720899] Loading compiled-in X.509 certificates1693worker2 # [ 0.697866] Segment Routing with IPv61694worker1 # [ 0.691983] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1695worker2 # [ 0.698557] In-situ OAM (IOAM) with IPv61696worker1 # [ 0.694855] drop_monitor: Initializing network drop monitor service1697worker1 # [ 0.696098] NET: Registered PF_INET6 protocol family1698worker2 # [ 0.701712] IPI shorthand broadcast: enabled1699lb # [ 0.704183] sched_clock: Marking stable (570014972, 133572943)->(776067940, -72480025)1700worker1 # [ 0.701155] Segment Routing with IPv61701worker1 # [ 0.701870] In-situ OAM (IOAM) with IPv61702lb # [ 0.709725] registered taskstats version 11703worker1 # [ 0.704020] IPI shorthand broadcast: enabled1704lb # [ 0.710676] Loading compiled-in X.509 certificates1705worker2 # [ 0.710021] sched_clock: Marking stable (576020556, 133383453)->(782672597, -73268588)1706worker2 # [ 0.713532] registered taskstats version 11707client # [ 0.738662] Demotion targets for Node 0: null1708worker2 # [ 0.714467] Loading compiled-in X.509 certificates1709client # [ 0.740683] Key type .fscrypt registered1710client # [ 0.741325] Key type fscrypt-provisioning registered1711worker1 # [ 0.712120] sched_clock: Marking stable (578016824, 133692651)->(785376653, -73667178)1712client # [ 0.744786] ima: No TPM chip found, activating TPM-bypass!1713client # [ 0.745693] ima: Allocated hash algorithm: sha11714client # [ 0.746433] ima: No architecture policies found1715worker1 # [ 0.716931] registered taskstats version 11716worker1 # [ 0.717934] Loading compiled-in X.509 certificates1717client # [ 0.748860] PM: Magic number: 10:708:8941718client # [ 0.750515] RAS: Correctable Errors collector initialized.1719lb # [ 0.727129] Demotion targets for Node 0: null1720lb # [ 0.730634] Key type .fscrypt registered1721lb # [ 0.731299] Key type fscrypt-provisioning registered1722lb # [ 0.732224] ima: No TPM chip found, activating TPM-bypass!1723worker2 # [ 0.732411] Demotion targets for Node 0: null1724worker2 # [ 0.733266] Key type .fscrypt registered1725lb # [ 0.734599] ima: Allocated hash algorithm: sha11726lb # [ 0.735365] ima: No architecture policies found1727client # [ 0.760303] clk: Disabling unused clocks1728worker2 # [ 0.736402] Key type fscrypt-provisioning registered1729lb # [ 0.737600] PM: Magic number: 10:708:8941730worker2 # [ 0.737291] ima: No TPM chip found, activating TPM-bypass!1731worker2 # [ 0.738161] ima: Allocated hash algorithm: sha11732client # [ 0.762653] PM: genpd: Disabling unused power domains1733lb # [ 0.739204] RAS: Correctable Errors collector initialized.1734worker2 # [ 0.738962] ima: No architecture policies found1735worker1 # [ 0.735723] Demotion targets for Node 0: null1736worker2 # [ 0.741411] PM: Magic number: 10:708:8941737worker2 # [ 0.742978] RAS: Correctable Errors collector initialized.1738worker1 # [ 0.738755] Key type .fscrypt registered1739worker1 # [ 0.739476] Key type fscrypt-provisioning registered1740worker1 # [ 0.740470] ima: No TPM chip found, activating TPM-bypass!1741lb # [ 0.748273] clk: Disabling unused clocks1742worker1 # [ 0.742726] ima: Allocated hash algorithm: sha11743worker1 # [ 0.743563] ima: No architecture policies found1744lb # [ 0.750621] PM: genpd: Disabling unused power domains1745worker2 # [ 0.752028] clk: Disabling unused clocks1746worker1 # [ 0.747727] PM: Magic number: 10:708:8941747worker1 # [ 0.749418] RAS: Correctable Errors collector initialized.1748worker2 # [ 0.756419] PM: genpd: Disabling unused power domains1749worker1 # [ 0.757407] clk: Disabling unused clocks1750worker1 # [ 0.760723] PM: genpd: Disabling unused power domains1751lb # [ 0.914260] Freeing initrd memory: 29064K1752client # [ 0.940473] Freeing initrd memory: 29056K1753lb # [ 0.917429] Freeing unused decrypted memory: 2028K1754lb # [ 0.920038] Freeing unused kernel image (initmem) memory: 3652K1755client # [ 0.943828] Freeing unused decrypted memory: 2028K1756lb # [ 0.921094] Write protecting the kernel read-only data: 32768k1757worker2 # [ 0.921310] Freeing initrd memory: 29060K1758lb # [ 0.922912] Freeing unused kernel image (text/rodata gap) memory: 1184K1759client # [ 0.946692] Freeing unused kernel image (initmem) memory: 3652K1760client # [ 0.947776] Write protecting the kernel read-only data: 32768k1761lb # [ 0.924375] Freeing unused kernel image (rodata/data gap) memory: 720K1762worker2 # [ 0.924536] Freeing unused decrypted memory: 2028K1763client # [ 0.949722] Freeing unused kernel image (text/rodata gap) memory: 1184K1764worker2 # [ 0.927086] Freeing unused kernel image (initmem) memory: 3652K1765client # [ 0.951295] Freeing unused kernel image (rodata/data gap) memory: 720K1766worker1 # [ 0.922178] Freeing initrd memory: 29056K1767worker2 # [ 0.928153] Write protecting the kernel read-only data: 32768k1768worker2 # [ 0.929980] Freeing unused kernel image (text/rodata gap) memory: 1184K1769worker1 # [ 0.925323] Freeing unused decrypted memory: 2028K1770worker2 # [ 0.931542] Freeing unused kernel image (rodata/data gap) memory: 720K1771worker1 # [ 0.927929] Freeing unused kernel image (initmem) memory: 3652K1772worker1 # [ 0.929056] Write protecting the kernel read-only data: 32768k1773worker1 # [ 0.930936] Freeing unused kernel image (text/rodata gap) memory: 1184K1774worker1 # [ 0.932444] Freeing unused kernel image (rodata/data gap) memory: 720K1775lb # [ 0.976170] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776lb # [ 0.977217] Run /init as init process1777client # [ 1.002137] x86/mm: Checked W+X mappings: passed, no W+X pages found.1778client # [ 1.003354] Run /init as init process1779worker2 # [ 0.982402] x86/mm: Checked W+X mappings: passed, no W+X pages found.1780worker2 # [ 0.983535] Run /init as init process1781lb # [ 0.987925] systemd[1]: Inserted module 'autofs4'1782worker1 # [ 0.983533] x86/mm: Checked W+X mappings: passed, no W+X pages found.1783worker1 # [ 0.984677] Run /init as init process1784client # [ 1.015089] systemd[1]: Inserted module 'autofs4'1785worker2 # [ 0.994461] systemd[1]: Inserted module 'autofs4'1786worker1 # [ 0.995746] systemd[1]: Inserted module 'autofs4'1787lb # [ 1.007783] fuse: init (API version 7.45)1788client # [ 1.035092] fuse: init (API version 7.45)1789worker2 # [ 1.014080] fuse: init (API version 7.45)1790lb # [ 1.015200] ACPI: \_SB_.GSIG: Enabled at IRQ 221791lb # [ 1.017532] ACPI: \_SB_.GSIH: Enabled at IRQ 231792client # [ 1.042719] ACPI: \_SB_.GSIG: Enabled at IRQ 221793lb # [ 1.020716] ACPI: \_SB_.GSIE: Enabled at IRQ 201794worker1 # [ 1.015521] fuse: init (API version 7.45)1795client # [ 1.045410] ACPI: \_SB_.GSIH: Enabled at IRQ 231796worker2 # [ 1.021563] ACPI: \_SB_.GSIG: Enabled at IRQ 221797lb # [ 1.022632] ACPI: \_SB_.GSIF: Enabled at IRQ 211798worker2 # [ 1.024022] ACPI: \_SB_.GSIH: Enabled at IRQ 231799client # [ 1.048863] ACPI: \_SB_.GSIE: Enabled at IRQ 201800client # [ 1.051048] ACPI: \_SB_.GSIF: Enabled at IRQ 211801worker2 # [ 1.027271] ACPI: \_SB_.GSIE: Enabled at IRQ 201802lb # [ 1.027918] virtiofs virtio5: discovered new tag: nix-store1803worker1 # [ 1.023073] ACPI: \_SB_.GSIG: Enabled at IRQ 221804lb # [ 1.029438] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1805worker2 # [ 1.029307] ACPI: \_SB_.GSIF: Enabled at IRQ 211806worker1 # [ 1.025602] ACPI: \_SB_.GSIH: Enabled at IRQ 231807client # [ 1.057279] virtiofs virtio5: discovered new tag: nix-store1808worker1 # [ 1.028671] ACPI: \_SB_.GSIE: Enabled at IRQ 201809worker2 # [ 1.034638] virtiofs virtio5: discovered new tag: nix-store1810client # [ 1.058929] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1811lb # [ 1.035944] virtiofs virtio6: discovered new tag: shared1812worker1 # [ 1.030732] ACPI: \_SB_.GSIF: Enabled at IRQ 211813worker2 # [ 1.036182] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1814lb # [ 1.037426] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1815lb # [ 1.040779] virtiofs virtio7: discovered new tag: xchg1816worker1 # [ 1.035998] virtiofs virtio5: discovered new tag: nix-store1817lb # [ 1.042182] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1818client # [ 1.066706] virtiofs virtio6: discovered new tag: shared1819worker2 # [ 1.042904] virtiofs virtio6: discovered new tag: shared1820worker1 # [ 1.037580] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1821client # [ 1.068172] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1822worker2 # [ 1.044608] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1823client # [ 1.071662] virtiofs virtio7: discovered new tag: xchg1824worker2 # [ 1.047982] virtiofs virtio7: discovered new tag: xchg1825client # [ 1.073103] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1826worker1 # [ 1.044004] virtiofs virtio6: discovered new tag: shared1827worker2 # [ 1.049544] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1828worker1 # [ 1.045553] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1829worker1 # [ 1.050016] virtiofs virtio7: discovered new tag: xchg1830worker1 # [ 1.051646] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1831lb # [ 1.065665] systemd[1]: Successfully made /usr/ read-only.1832client # [ 1.094480] systemd[1]: Successfully made /usr/ read-only.1833worker2 # [ 1.072261] systemd[1]: Successfully made /usr/ read-only.1834worker1 # [ 1.072509] systemd[1]: Successfully made /usr/ read-only.1835lb # [ 1.402454] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836lb # [ 1.414355] systemd[1]: Detected virtualization kvm.1837lb # [ 1.416495] systemd[1]: Detected architecture x86-64.1838client # [ 1.431059] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1839lb # [ 1.418622] systemd[1]: Running in initrd.1840client # [ 1.442938] systemd[1]: Detected virtualization kvm.1841worker2 # [ 1.408944] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842client # [ 1.445038] systemd[1]: Detected architecture x86-64.1843lb # [ 1.421102] systemd[1]: Initializing machine ID from random generator.1844worker2 # [ 1.421049] systemd[1]: Detected virtualization kvm.1845client # [ 1.447103] systemd[1]: Running in initrd.1846lb # [ 1.423928] systemd[1]: Hostname set to <lb>.1847worker2 # [ 1.423209] systemd[1]: Detected architecture x86-64.1848worker1 # [ 1.409127] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1849worker2 # [ 1.425352] systemd[1]: Running in initrd.1850client # [ 1.449504] systemd[1]: Initializing machine ID from random generator.1851worker1 # [ 1.421212] systemd[1]: Detected virtualization kvm.1852client # [ 1.452353] systemd[1]: Hostname set to <client>.1853worker2 # [ 1.427825] systemd[1]: Initializing machine ID from random generator.1854worker1 # [ 1.423368] systemd[1]: Detected architecture x86-64.1855worker2 # [ 1.430721] systemd[1]: Hostname set to <worker2>.1856worker1 # [ 1.425540] systemd[1]: Running in initrd.1857worker1 # [ 1.428045] systemd[1]: Initializing machine ID from random generator.1858worker1 # [ 1.430892] systemd[1]: Hostname set to <worker1>.1859lb # [ 1.650158] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860client # [ 1.675946] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861worker1 # [ 1.649791] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862worker2 # [ 1.657342] systemd[1]: bpf-restrict-fs: LSM BPF program attached1863lb # [ 1.700277] systemd[1]: Queued start job for default target Initrd Default Target.1864lb # [ 1.704918] systemd[1]: Created slice Slice /system/modprobe.1865lb # [ 1.706105] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1866client # [ 1.729759] systemd[1]: Queued start job for default target Initrd Default Target.1867lb # [ 1.707482] systemd[1]: Expecting device /dev/disk/by-label/nixos...1868lb # [ 1.708595] systemd[1]: Reached target Path Units.1869worker1 # [ 1.702318] systemd[1]: Queued start job for default target Initrd Default Target.1870lb # [ 1.709449] systemd[1]: Reached target Slice Units.1871client # [ 1.733039] systemd[1]: Created slice Slice /system/modprobe.1872lb # [ 1.710297] systemd[1]: Reached target Swaps.1873lb # [ 1.711095] systemd[1]: Reached target Timer Units.1874client # [ 1.734215] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1875worker1 # [ 1.705999] systemd[1]: Created slice Slice /system/modprobe.1876client # [ 1.735636] systemd[1]: Expecting device /dev/disk/by-label/nixos...1877lb # [ 1.712050] systemd[1]: Listening on D-Bus System Message Bus Socket.1878worker2 # [ 1.711219] systemd[1]: Queued start job for default target Initrd Default Target.1879client # [ 1.736670] systemd[1]: Reached target Path Units.1880lb # [ 1.713180] systemd[1]: Listening on Journal Socket (/dev/log).1881worker1 # [ 1.707105] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1882client # [ 1.737514] systemd[1]: Reached target Slice Units.1883lb # [ 1.714292] systemd[1]: Listening on Journal Sockets.1884client # [ 1.738373] systemd[1]: Reached target Swaps.1885worker1 # [ 1.708464] systemd[1]: Expecting device /dev/disk/by-label/nixos...1886lb # [ 1.715264] systemd[1]: Listening on udev Control Socket.1887client # [ 1.739141] systemd[1]: Reached target Timer Units.1888worker1 # [ 1.709557] systemd[1]: Reached target Path Units.1889worker1 # [ 1.710425] systemd[1]: Reached target Slice Units.1890lb # [ 1.716321] systemd[1]: Listening on udev Kernel Socket.1891worker2 # [ 1.715737] systemd[1]: Created slice Slice /system/modprobe.1892client # [ 1.740136] systemd[1]: Listening on D-Bus System Message Bus Socket.1893worker1 # [ 1.711276] systemd[1]: Reached target Swaps.1894lb # [ 1.717235] systemd[1]: Reached target Socket Units.1895client # [ 1.741282] systemd[1]: Listening on Journal Socket (/dev/log).1896worker2 # [ 1.716876] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1897worker1 # [ 1.712053] systemd[1]: Reached target Timer Units.1898client # [ 1.742389] systemd[1]: Listening on Journal Sockets.1899worker2 # [ 1.718231] systemd[1]: Expecting device /dev/disk/by-label/nixos...1900worker1 # [ 1.712954] systemd[1]: Listening on D-Bus System Message Bus Socket.1901lb # [ 1.719126] systemd[1]: Starting Create List of Static Device Nodes...1902client # [ 1.743357] systemd[1]: Listening on udev Control Socket.1903worker2 # [ 1.719282] systemd[1]: Reached target Path Units.1904worker1 # [ 1.714112] systemd[1]: Listening on Journal Socket (/dev/log).1905worker2 # [ 1.720120] systemd[1]: Reached target Slice Units.1906client # [ 1.744326] systemd[1]: Listening on udev Kernel Socket.1907worker1 # [ 1.715290] systemd[1]: Listening on Journal Sockets.1908worker2 # [ 1.721007] systemd[1]: Reached target Swaps.1909client # [ 1.745261] systemd[1]: Reached target Socket Units.1910worker1 # [ 1.716247] systemd[1]: Listening on udev Control Socket.1911worker2 # [ 1.721816] systemd[1]: Reached target Timer Units.1912lb # [ 1.722660] systemd[1]: Starting Load Kernel Module configfs...1913worker1 # [ 1.717240] systemd[1]: Listening on udev Kernel Socket.1914worker2 # [ 1.722798] systemd[1]: Listening on D-Bus System Message Bus Socket.1915client # [ 1.747049] systemd[1]: Starting Create List of Static Device Nodes...1916worker1 # [ 1.718163] systemd[1]: Reached target Socket Units.1917worker2 # [ 1.724025] systemd[1]: Listening on Journal Socket (/dev/log).1918worker2 # [ 1.725232] systemd[1]: Listening on Journal Sockets.1919worker1 # [ 1.719922] systemd[1]: Starting Create List of Static Device Nodes...1920worker2 # [ 1.726232] systemd[1]: Listening on udev Control Socket.1921client # [ 1.750703] systemd[1]: Starting Load Kernel Module configfs...1922worker2 # [ 1.727235] systemd[1]: Listening on udev Kernel Socket.1923worker2 # [ 1.728199] systemd[1]: Reached target Socket Units.1924worker1 # [ 1.723776] systemd[1]: Starting Load Kernel Module configfs...1925worker2 # [ 1.730067] systemd[1]: Starting Create List of Static Device Nodes...1926worker2 # [ 1.734469] systemd[1]: Starting Load Kernel Module configfs...1927lb # [ 1.731661] systemd[1]: Starting Journal Service...1928client # [ 1.760823] systemd[1]: Starting Journal Service...1929worker1 # [ 1.733154] systemd[1]: Starting Journal Service...1930worker2 # [ 1.744022] systemd[1]: Starting Journal Service...1931lb # [ 1.758781] systemd[1]: Starting Load Kernel Modules...1932client # [ 1.784248] systemd[1]: Starting Load Kernel Modules...1933worker1 # [ 1.754922] systemd[1]: Starting Load Kernel Modules...1934lb # [ 1.762692] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1935client # [ 1.788763] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1936worker1 # [ 1.761819] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1937lb # [ 1.769782] systemd[1]: Starting Coldplug All udev Devices...1938worker2 # [ 1.769604] systemd[1]: Starting Load Kernel Modules...1939client # [ 1.797883] systemd[1]: Starting Coldplug All udev Devices...1940worker2 # [ 1.773490] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1941worker1 # [ 1.768780] systemd[1]: Starting Coldplug All udev Devices...1942worker2 # [ 1.781579] systemd[1]: Starting Coldplug All udev Devices...1943lb # [ 1.782329] systemd-journald[66]: Collecting audit messages is disabled.1944worker1 # [ 1.778544] systemd-journald[66]: Collecting audit messages is disabled.1945lb # [ 1.784870] systemd[1]: Finished Create List of Static Device Nodes.1946client # [ 1.809185] systemd-journald[66]: Collecting audit messages is disabled.1947worker1 # [ 1.782774] systemd[1]: Finished Create List of Static Device Nodes.1948client # [ 1.812703] systemd[1]: Finished Create List of Static Device Nodes.1949lb # [ 1.790796] systemd[1]: modprobe@configfs.service: Deactivated successfully.1950worker1 # [ 1.785328] systemd[1]: modprobe@configfs.service: Deactivated successfully.1951client # [ 1.815284] systemd[1]: modprobe@configfs.service: Deactivated successfully.1952worker2 # [ 1.792902] systemd-journald[66]: Collecting audit messages is disabled.1953worker2 # [ 1.794465] systemd[1]: Finished Create List of Static Device Nodes.1954lb # [ 1.796009] systemd[1]: Finished Load Kernel Module configfs.1955worker1 # [ 1.794116] systemd[1]: Finished Load Kernel Module configfs.1956client # [ 1.824040] systemd[1]: Finished Load Kernel Module configfs.1957worker2 # [ 1.800817] systemd[1]: modprobe@configfs.service: Deactivated successfully.1958lb # [ 1.802010] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1959worker1 # [ 1.799148] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1960client # [ 1.829093] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1961worker2 # [ 1.807806] systemd[1]: Finished Load Kernel Module configfs.1962worker2 # [ 1.813813] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1963lb # [ 1.815669] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1964worker1 # [ 1.811773] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1965client # [ 1.842705] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1966lb # [ 1.823016] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1967worker1 # [ 1.817555] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1968client # [ 1.848067] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1969worker2 # [ 1.827472] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1970worker1 # [ 1.828851] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1971lb # [ 1.834967] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1972worker2 # [ 1.834130] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1973client # [ 1.862014] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1974worker2 # [ 1.842585] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1975lb # [ 1.857207] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1976worker1 # [ 1.851242] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1977client # [ 1.884233] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1978worker2 # [ 1.863951] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1979worker1 # [ 1.859890] systemd[1]: Starting Create Static Device Nodes in /dev...1980lb # [ 1.866905] systemd[1]: Starting Create Static Device Nodes in /dev...1981client # [ 1.892847] systemd[1]: Starting Create Static Device Nodes in /dev...1982worker2 # [ 1.872585] systemd[1]: Starting Create Static Device Nodes in /dev...1983worker1 # [ 1.871276] systemd[1]: Finished Load Kernel Modules.1984lb # [ 1.878661] systemd[1]: Finished Load Kernel Modules.1985client # [ 1.904180] systemd[1]: Finished Load Kernel Modules.1986worker2 # [ 1.884473] systemd[1]: Finished Load Kernel Modules.1987worker1 # [ 1.880383] systemd[1]: Starting Apply Kernel Variables...1988lb # [ 1.889676] systemd[1]: Starting Apply Kernel Variables...1989client # [ 1.915748] systemd[1]: Starting Apply Kernel Variables...1990worker2 # [ 1.893600] systemd[1]: Starting Apply Kernel Variables...1991lb # [ 1.771469] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.906697] systemd[1]: Started Journal Service.1992worker1 # [ 1.768430] systemd-modules-load[67]: Inserted module 'dm_mod'1993client # [ 1.796339] systemd-modules-load[67]: Inserted module 'dm_mod'1994worker1 # [ 1.903308] systemd[1]: Finished Create Static Device Nodes in /dev.1995client # [ 1.933156] systemd[1]: Started Journal Service.1996lb # 1997worker1 # [ 1.770609] systemd-modules-load[67]: Inserted module 'virtio_balloon'1998client # [ 1.798410] systemd-modules-load[67]: Inserted module 'virtio_balloon'1999lb # [ 1.777345] systemd-modules-load[68]: Inserted module 'virtio_balloon'2000worker1 # [ 1.771676] systemd-modules-load[67]: Inserted module 'virtio_gpu'
loading 2,622 hidden lines…
4623worker1 # [ 35.917718] nix-daemon[845]: reaped child process 876, status = succeeded4624worker2 # [ 35.922582] nix-grpc-daemon[781]: ts=2026-09-22T16:54:19Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14625worker1 # [ 35.959347] nix-grpc-daemon[780]: ts=2026-09-22T16:54:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442364626worker1 # [ 35.971710] nix-grpc-daemon[780]: ts=2026-09-22T16:54:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442364627worker1 # [ 35.985813] nix-grpc-daemon[857]: time=2026-09-22T16:54:18.300Z level=INFO msg="All 1 paths already cached"4628client # copying 1 paths to 'lb:50051'...4629worker1 # [ 35.994516] nix-grpc-daemon[780]: ts=2026-09-22T16:54:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442364630client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4631worker1 # [ 36.012569] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4632worker1 # [ 36.112964] nix-daemon[845]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4633worker1 # [ 36.126512] nix-daemon[845]: reaped child process 904, status = succeeded4634lb # [ 36.141402] niks3-server[969]: 2026/09/22 16:54:19 INFO Received uploads request method=POST path=/api/pending_closures4635worker1 # [ 36.144229] nix-grpc-daemon[857]: time=2026-09-22T16:54:18.458Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4636worker1 # [ 36.145790] nix-grpc-daemon[857]: time=2026-09-22T16:54:18.460Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4637lb # [ 36.173957] niks3-server[969]: 2026/09/22 16:54:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4638lb # [ 36.179913] niks3-server[969]: 2026/09/22 16:54:19 INFO Signed narinfos id=3 count=14639worker1 # [ 36.181052] nix-grpc-daemon[857]: time=2026-09-22T16:54:18.495Z level=INFO msg="Uploading 1 narinfos"4640lb # [ 36.187889] niks3-server[969]: 2026/09/22 16:54:19 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4641lb # [ 36.200213] niks3-server[969]: 2026/09/22 16:54:19 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4642lb # [ 36.205884] niks3-server[969]: 2026/09/22 16:54:19 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4643lb # [ 36.212798] niks3-server[969]: 2026/09/22 16:54:19 INFO Completed upload id=34644worker1 # [ 36.207053] nix-grpc-daemon[857]: time=2026-09-22T16:54:18.521Z level=INFO msg="Upload complete. (180ms)"4645lb # [ 36.215746] niks3-server[969]: 2026/09/22 16:54:19 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4646worker1 # [ 36.213369] nix-grpc-daemon[780]: ts=2026-09-22T16:54:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_in=6244647worker1 # [ 36.218872] nix-grpc-daemon[780]: ts=2026-09-22T16:54:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442364648worker1 # [ 36.230108] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4649worker1 # [ 36.237233] nix-daemon[845]: reaped child process 899, status = succeeded4650worker1 # [ 36.242167] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4651client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4652client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4653lb # [ 36.629909] envoy[915]: conn peer=[2001:db8:1::1]:40488 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=423 ms=26124654lb # [ 36.633531] envoy[915]: conn peer=[2001:db8:1::1]:44272 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=634655lb # [ 36.636708] envoy[915]: conn peer=[2001:db8:1::1]:44278 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=74656lb # [ 36.639834] envoy[915]: conn peer=[2001:db8:1::1]:44286 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=114657client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4658client # farm-b> LOG-farm-b-t14659worker1 # [ 37.055346] nix-daemon[845]: reaped child process 908, status = succeeded4660worker1 # [ 37.130716] nix-daemon[845]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4661worker1 # [ 37.141368] nix-daemon[845]: reaped child process 927, status = succeeded4662lb # [ 37.155883] niks3-server[969]: 2026/09/22 16:54:20 INFO Received uploads request method=POST path=/api/pending_closures4663worker1 # [ 37.158139] nix-grpc-daemon[857]: time=2026-09-22T16:54:19.472Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4664worker1 # [ 37.159768] nix-grpc-daemon[857]: time=2026-09-22T16:54:19.474Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4665lb # [ 37.195491] niks3-server[969]: 2026/09/22 16:54:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4666worker1 # [ 37.199209] nix-grpc-daemon[857]: time=2026-09-22T16:54:19.513Z level=INFO msg="Uploading 1 narinfos"4667lb # [ 37.205495] niks3-server[969]: 2026/09/22 16:54:20 INFO Signed narinfos id=4 count=14668lb # [ 37.215258] niks3-server[969]: 2026/09/22 16:54:20 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4669lb # [ 37.220549] niks3-server[969]: 2026/09/22 16:54:20 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4670lb # [ 37.224669] niks3-server[969]: 2026/09/22 16:54:20 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4671lb # [ 37.235743] niks3-server[969]: 2026/09/22 16:54:20 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4672lb # [ 37.241586] niks3-server[969]: 2026/09/22 16:54:20 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4673lb # [ 37.244859] niks3-server[969]: 2026/09/22 16:54:20 INFO Completed upload id=44674worker1 # [ 37.239189] nix-grpc-daemon[857]: time=2026-09-22T16:54:19.553Z level=INFO msg="Upload complete. (190ms)"4675worker2 # [ 37.255138] nix-grpc-daemon[781]: ts=2026-09-22T16:54:20Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04676worker1 # [ 37.246229] nix-grpc-daemon[780]: ts=2026-09-22T16:54:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14677worker2 # [ 37.256955] nix-grpc-daemon[781]: ts=2026-09-22T16:54:20Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14678worker1 # [ 37.260676] nix-daemon[845]: reaped child process 906, status = succeeded4679worker1 # [ 37.294594] nix-grpc-daemon[780]: ts=2026-09-22T16:54:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442364680worker1 # [ 37.306547] nix-grpc-daemon[780]: ts=2026-09-22T16:54:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442364681worker1 # [ 37.320369] nix-grpc-daemon[857]: time=2026-09-22T16:54:19.634Z level=INFO msg="All 2 paths already cached"4682client # copying 1 paths to 'lb:50051'...4683client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4684worker1 # [ 37.329502] nix-grpc-daemon[780]: ts=2026-09-22T16:54:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442364685worker1 # [ 37.342152] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4686worker1 # [ 37.441765] nix-daemon[845]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4687worker1 # [ 37.455480] nix-daemon[845]: reaped child process 934, status = succeeded4688lb # [ 37.466350] niks3-server[969]: 2026/09/22 16:54:20 INFO Received uploads request method=POST path=/api/pending_closures4689worker1 # [ 37.466731] nix-grpc-daemon[857]: time=2026-09-22T16:54:19.781Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4690worker1 # [ 37.468185] nix-grpc-daemon[857]: time=2026-09-22T16:54:19.782Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4691lb # [ 37.483835] niks3-server[969]: 2026/09/22 16:54:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4692lb # [ 37.485902] niks3-server[969]: 2026/09/22 16:54:20 INFO Signed narinfos id=5 count=14693worker1 # [ 37.481127] nix-grpc-daemon[857]: time=2026-09-22T16:54:19.795Z level=INFO msg="Uploading 1 narinfos"4694lb # [ 37.490264] niks3-server[969]: 2026/09/22 16:54:20 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4695lb # [ 37.492916] niks3-server[969]: 2026/09/22 16:54:20 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4696lb # [ 37.498884] niks3-server[969]: 2026/09/22 16:54:20 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4697lb # [ 37.503890] niks3-server[969]: 2026/09/22 16:54:20 INFO Completed upload id=54698worker1 # [ 37.497971] nix-grpc-daemon[857]: time=2026-09-22T16:54:19.812Z level=INFO msg="Upload complete. (141ms)"4699lb # [ 37.505156] niks3-server[969]: 2026/09/22 16:54:20 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4700worker1 # [ 37.500616] nix-grpc-daemon[780]: ts=2026-09-22T16:54:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_in=7524701worker1 # [ 37.503520] nix-grpc-daemon[780]: ts=2026-09-22T16:54:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442364702worker1 # [ 37.509287] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4703worker1 # [ 37.515036] nix-daemon[845]: reaped child process 929, status = succeeded4704worker1 # [ 37.519539] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4705client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4706client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4707client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4708client # farm-top> LOG-farm-top-t14709worker1 # [ 39.135882] nix-daemon[845]: reaped child process 938, status = succeeded4710worker1 # [ 39.218353] nix-daemon[845]: accepted connection from pid 955, user nix-grpc-daemon (trusted)4711worker1 # [ 39.232255] nix-daemon[845]: reaped child process 958, status = succeeded4712lb # [ 39.243903] niks3-server[969]: 2026/09/22 16:54:22 INFO Received uploads request method=POST path=/api/pending_closures4713worker1 # [ 39.247732] nix-grpc-daemon[857]: time=2026-09-22T16:54:21.562Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4714worker1 # [ 39.250064] nix-grpc-daemon[857]: time=2026-09-22T16:54:21.563Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4715lb # [ 39.284732] niks3-server[969]: 2026/09/22 16:54:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4716lb # [ 39.289193] niks3-server[969]: 2026/09/22 16:54:22 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4717lb # [ 39.296343] niks3-server[969]: 2026/09/22 16:54:22 INFO Signed narinfos id=6 count=14718worker1 # [ 39.292951] nix-grpc-daemon[857]: time=2026-09-22T16:54:21.607Z level=INFO msg="Uploading 1 narinfos"4719lb # [ 39.300277] niks3-server[969]: 2026/09/22 16:54:22 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4720lb # [ 39.306935] niks3-server[969]: 2026/09/22 16:54:22 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4721lb # [ 39.312199] niks3-server[969]: 2026/09/22 16:54:22 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4722lb # [ 39.315673] niks3-server[969]: 2026/09/22 16:54:22 INFO Completed upload id=64723worker1 # [ 39.309718] nix-grpc-daemon[857]: time=2026-09-22T16:54:21.624Z level=INFO msg="Upload complete. (181ms)"4724lb # [ 39.318364] niks3-server[969]: 2026/09/22 16:54:22 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4725worker1 # [ 39.314997] nix-grpc-daemon[780]: ts=2026-09-22T16:54:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14726worker1 # [ 39.328274] nix-daemon[845]: reaped child process 936, status = succeeded4727client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 4.92 seconds)4728client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4729client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.12 seconds)4730client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4731client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4732client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4733client # copying 3 paths...4734client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4735client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4736client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4737client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.18 seconds)4738(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.29 seconds)4739subtest: repeat is answered Cached by the scheduler without building4740worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4741worker1 # finding garbage collector roots...4742worker1 # deleting specified paths...4743worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4744worker1 # deleting unused links...4745worker1 # note: hard linking is currently saving 0.0 KiB4746worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4747worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4748worker2 # finding garbage collector roots...4749worker2 # deleting specified paths...4750worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4751worker2 # deleting unused links...4752worker2 # note: hard linking is currently saving 0.0 KiB4753worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.18 seconds)4754client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24755worker1 # [ 40.212289] nix-grpc-daemon[780]: ts=2026-09-22T16:54:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442364756client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4757client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4758worker1 # [ 40.227905] nix-grpc-daemon[780]: ts=2026-09-22T16:54:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442364759worker1 # [ 40.236490] nix-grpc-daemon[857]: time=2026-09-22T16:54:22.551Z level=INFO msg="All 2 paths already cached"4760worker2 # [ 40.267606] nix-grpc-daemon[781]: ts=2026-09-22T16:54:23Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14761client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.23 seconds)4762client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4763client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds)4764lb: must succeed: curl -sf localhost:9901/clusters4765lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4766lb: must succeed: curl -sf localhost:9901/clusters4767lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4768worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4769worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4770(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.82 seconds)4771subtest: inputs already in the cache are not sent to the scheduler4772lb: must succeed: curl -sf localhost:9901/clusters4773lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4774lb: must succeed: curl -sf localhost:9901/clusters4775lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4776worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4777worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.02 seconds)4778client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24779worker2 # [ 40.755346] nix-grpc-daemon[781]: ts=2026-09-22T16:54:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608024780client # don't know how to build these paths:4781client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4782worker1 # [ 40.757382] nix-grpc-daemon[780]: ts=2026-09-22T16:54:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442364783worker1 # [ 40.767414] nix-grpc-daemon[857]: time=2026-09-22T16:54:23.081Z level=INFO msg="All 2 paths already cached"4784worker2 # [ 40.784251] nix-grpc-daemon[781]: ts=2026-09-22T16:54:23Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04785worker2 # [ 40.786206] nix-grpc-daemon[781]: ts=2026-09-22T16:54:23Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14786worker1 # [ 40.782833] nix-grpc-daemon[780]: ts=2026-09-22T16:54:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442364787worker1 # [ 40.795581] nix-grpc-daemon[780]: ts=2026-09-22T16:54:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442364788worker1 # [ 40.813455] nix-grpc-daemon[857]: time=2026-09-22T16:54:23.128Z level=INFO msg="All 2 paths already cached"4789client # copying 1 paths to 'lb:50051'...4790client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4791worker1 # [ 40.822528] nix-grpc-daemon[780]: ts=2026-09-22T16:54:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442364792worker1 # [ 40.834843] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4793worker1 # [ 40.939606] nix-daemon[845]: accepted connection from pid 975, user nix-grpc-daemon (trusted)4794worker1 # [ 40.954287] nix-daemon[845]: reaped child process 978, status = succeeded4795lb # [ 40.966394] niks3-server[969]: 2026/09/22 16:54:24 INFO Received uploads request method=POST path=/api/pending_closures4796worker1 # [ 40.968588] nix-grpc-daemon[857]: time=2026-09-22T16:54:23.282Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4797worker1 # [ 40.969990] nix-grpc-daemon[857]: time=2026-09-22T16:54:23.282Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4798lb # [ 40.996413] niks3-server[969]: 2026/09/22 16:54:24 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4799lb # [ 41.003971] niks3-server[969]: 2026/09/22 16:54:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4800worker1 # [ 41.002915] nix-grpc-daemon[857]: time=2026-09-22T16:54:23.317Z level=INFO msg="Uploading 1 narinfos"4801lb # [ 41.008629] niks3-server[969]: 2026/09/22 16:54:24 INFO Signed narinfos id=7 count=14802lb # [ 41.017563] niks3-server[969]: 2026/09/22 16:54:24 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4803lb # [ 41.024249] niks3-server[969]: 2026/09/22 16:54:24 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4804lb # [ 41.028277] niks3-server[969]: 2026/09/22 16:54:24 INFO Completed upload id=74805worker1 # [ 41.022361] nix-grpc-daemon[857]: time=2026-09-22T16:54:23.336Z level=INFO msg="Upload complete. (174ms)"4806lb # [ 41.029841] niks3-server[969]: 2026/09/22 16:54:24 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4807worker1 # [ 41.026191] nix-grpc-daemon[780]: ts=2026-09-22T16:54:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_in=7524808worker1 # [ 41.031642] nix-grpc-daemon[780]: ts=2026-09-22T16:54:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442364809worker1 # [ 41.038521] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4810worker1 # [ 41.047483] nix-daemon[845]: reaped child process 973, status = succeeded4811worker1 # [ 41.050208] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)4812client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4813client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4814client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4815client # farm-top> LOG-farm-top-t1b4816worker1 # [ 42.302710] nix-daemon[845]: reaped child process 982, status = succeeded4817worker1 # [ 42.378645] nix-daemon[845]: accepted connection from pid 999, user nix-grpc-daemon (trusted)4818worker1 # [ 42.391711] nix-daemon[845]: reaped child process 1002, status = succeeded4819lb # [ 42.403832] niks3-server[969]: 2026/09/22 16:54:25 INFO Received uploads request method=POST path=/api/pending_closures4820worker1 # [ 42.405992] nix-grpc-daemon[857]: time=2026-09-22T16:54:24.720Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4821worker1 # [ 42.408340] nix-grpc-daemon[857]: time=2026-09-22T16:54:24.722Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4822lb # [ 42.446262] niks3-server[969]: 2026/09/22 16:54:25 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4823lb # [ 42.453687] niks3-server[969]: 2026/09/22 16:54:25 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4824lb # [ 42.462691] niks3-server[969]: 2026/09/22 16:54:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4825worker1 # [ 42.463758] nix-grpc-daemon[857]: time=2026-09-22T16:54:24.778Z level=INFO msg="Uploading 1 narinfos"4826lb # [ 42.469485] niks3-server[969]: 2026/09/22 16:54:25 INFO Signed narinfos id=8 count=14827lb # [ 42.475214] niks3-server[969]: 2026/09/22 16:54:25 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4828lb # [ 42.484923] niks3-server[969]: 2026/09/22 16:54:25 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4829lb # [ 42.488205] niks3-server[969]: 2026/09/22 16:54:25 INFO Completed upload id=84830worker1 # [ 42.482497] nix-grpc-daemon[857]: time=2026-09-22T16:54:24.797Z level=INFO msg="Upload complete. (223ms)"4831lb # [ 42.490186] niks3-server[969]: 2026/09/22 16:54:25 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4832worker1 # [ 42.488760] nix-grpc-daemon[780]: ts=2026-09-22T16:54:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14833worker1 # [ 42.504193] nix-daemon[845]: reaped child process 980, status = succeeded4834client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.91 seconds)4835client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4836client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.11 seconds)4837lb: must succeed: curl -sf localhost:9901/clusters4838lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4839lb: must succeed: curl -sf localhost:9901/clusters4840lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4841worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4842worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4843(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.21 seconds)4844subtest: outputs read back through any worker carry the cache signature4845client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24846client # don't know how to build these paths:4847client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4848worker1 # [ 42.863133] nix-grpc-daemon[780]: ts=2026-09-22T16:54:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442364849worker2 # [ 42.874156] nix-grpc-daemon[781]: ts=2026-09-22T16:54:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608024850worker2 # [ 42.895848] nix-grpc-daemon[781]: ts=2026-09-22T16:54:26Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04851worker2 # [ 42.901300] nix-grpc-daemon[781]: ts=2026-09-22T16:54:26Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.4:50051 clients=14852worker2 # [ 42.906560] nix-grpc-daemon[781]: ts=2026-09-22T16:54:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608024853worker2 # [ 42.918888] nix-grpc-daemon[781]: ts=2026-09-22T16:54:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608024854client # copying 1 paths to 'lb:50051'...4855client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4856worker2 # [ 42.930610] nix-grpc-daemon[781]: ts=2026-09-22T16:54:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608024857worker2 # [ 42.941680] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4858worker2 # [ 42.959400] nix-grpc-daemon[781]: ts=2026-09-22T16:54:26Z level=info event=niks3_push_started pid=9194859worker2 # [ 43.084524] nix-grpc-daemon[919]: time=2026-09-22T16:54:26.232Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4860worker2 # [ 43.175083] nix-daemon[865]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4861worker2 # [ 43.188968] nix-daemon[865]: reaped child process 935, status = succeeded4862lb # [ 43.194291] niks3-server[969]: 2026/09/22 16:54:26 INFO Received uploads request method=POST path=/api/pending_closures4863worker2 # [ 43.198549] nix-grpc-daemon[919]: time=2026-09-22T16:54:26.347Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4864worker2 # [ 43.199981] nix-grpc-daemon[919]: time=2026-09-22T16:54:26.349Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4865lb # [ 43.216483] niks3-server[969]: 2026/09/22 16:54:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4866lb # [ 43.219050] niks3-server[969]: 2026/09/22 16:54:26 INFO Signed narinfos id=9 count=14867worker2 # [ 43.220821] nix-grpc-daemon[919]: time=2026-09-22T16:54:26.369Z level=INFO msg="Uploading 1 narinfos"4868lb # [ 43.223076] niks3-server[969]: 2026/09/22 16:54:26 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4869lb # [ 43.225842] niks3-server[969]: 2026/09/22 16:54:26 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4870lb # [ 43.230457] niks3-server[969]: 2026/09/22 16:54:26 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4871lb # [ 43.234475] niks3-server[969]: 2026/09/22 16:54:26 INFO Completed upload id=94872worker2 # [ 43.235663] nix-grpc-daemon[919]: time=2026-09-22T16:54:26.384Z level=INFO msg="Upload complete. (147ms)"4873lb # [ 43.236494] niks3-server[969]: 2026/09/22 16:54:26 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4874worker2 # [ 43.237663] nix-grpc-daemon[781]: ts=2026-09-22T16:54:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_in=5044875worker2 # [ 43.243398] nix-grpc-daemon[781]: ts=2026-09-22T16:54:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608024876worker2 # [ 43.247558] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4877worker2 # [ 43.250124] nix-daemon[865]: reaped child process 917, status = succeeded4878worker2 # [ 43.255146] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4879client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4880client # farm-a> worker2: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4881client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4882client # farm-a> LOG-farm-a-sig4883worker2 # [ 44.089698] nix-daemon[865]: reaped child process 939, status = succeeded4884worker2 # [ 44.172500] nix-daemon[865]: accepted connection from pid 955, user nix-grpc-daemon (trusted)4885worker2 # [ 44.185075] nix-daemon[865]: reaped child process 958, status = succeeded4886lb # [ 44.192747] niks3-server[969]: 2026/09/22 16:54:27 INFO Received uploads request method=POST path=/api/pending_closures4887worker2 # [ 44.201441] nix-grpc-daemon[919]: time=2026-09-22T16:54:27.350Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4888worker2 # [ 44.203234] nix-grpc-daemon[919]: time=2026-09-22T16:54:27.352Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4889lb # [ 44.236134] niks3-server[969]: 2026/09/22 16:54:27 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4890lb # [ 44.244069] niks3-server[969]: 2026/09/22 16:54:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4891lb # [ 44.249496] niks3-server[969]: 2026/09/22 16:54:27 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4892worker2 # [ 44.257437] nix-grpc-daemon[919]: time=2026-09-22T16:54:27.406Z level=INFO msg="Uploading 1 narinfos"4893lb # [ 44.257671] niks3-server[969]: 2026/09/22 16:54:27 INFO Signed narinfos id=10 count=14894lb # [ 44.262637] niks3-server[969]: 2026/09/22 16:54:27 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4895lb # [ 44.273592] niks3-server[969]: 2026/09/22 16:54:27 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4896lb # [ 44.278388] niks3-server[969]: 2026/09/22 16:54:27 INFO Completed upload id=104897worker2 # [ 44.278448] nix-grpc-daemon[919]: time=2026-09-22T16:54:27.427Z level=INFO msg="Upload complete. (216ms)"4898worker2 # [ 44.283987] nix-grpc-daemon[781]: ts=2026-09-22T16:54:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14899worker2 # [ 44.296142] nix-grpc-daemon[781]: ts=2026-09-22T16:54:27Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04900worker2 # [ 44.301214] nix-grpc-daemon[781]: ts=2026-09-22T16:54:27Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=14901worker2 # [ 44.310742] nix-daemon[865]: reaped child process 937, status = succeeded4902worker2 # [ 44.340876] nix-grpc-daemon[781]: ts=2026-09-22T16:54:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608024903worker2 # [ 44.352800] nix-grpc-daemon[781]: ts=2026-09-22T16:54:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608024904worker2 # [ 44.365486] nix-grpc-daemon[919]: time=2026-09-22T16:54:27.514Z level=INFO msg="All 1 paths already cached"4905client # copying 1 paths to 'lb:50051'...4906client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4907worker2 # [ 44.374217] nix-grpc-daemon[781]: ts=2026-09-22T16:54:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608024908worker2 # [ 44.384135] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4909worker2 # [ 44.477264] nix-daemon[865]: accepted connection from pid 964, user nix-grpc-daemon (trusted)4910worker2 # [ 44.492045] nix-daemon[865]: reaped child process 967, status = succeeded4911lb # [ 44.496253] niks3-server[969]: 2026/09/22 16:54:27 INFO Received uploads request method=POST path=/api/pending_closures4912worker2 # [ 44.504617] nix-grpc-daemon[919]: time=2026-09-22T16:54:27.653Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4913worker2 # [ 44.506095] nix-grpc-daemon[919]: time=2026-09-22T16:54:27.655Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4914lb # [ 44.527980] niks3-server[969]: 2026/09/22 16:54:27 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4915lb # [ 44.537192] niks3-server[969]: 2026/09/22 16:54:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4916worker2 # [ 44.543607] nix-grpc-daemon[919]: time=2026-09-22T16:54:27.692Z level=INFO msg="Uploading 1 narinfos"4917lb # [ 44.543926] niks3-server[969]: 2026/09/22 16:54:27 INFO Signed narinfos id=11 count=14918lb # [ 44.548450] niks3-server[969]: 2026/09/22 16:54:27 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4919lb # [ 44.562124] niks3-server[969]: 2026/09/22 16:54:27 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4920lb # [ 44.567217] niks3-server[969]: 2026/09/22 16:54:27 INFO Completed upload id=114921worker2 # [ 44.567101] nix-grpc-daemon[919]: time=2026-09-22T16:54:27.716Z level=INFO msg="Upload complete. (172ms)"4922worker2 # [ 44.571237] nix-grpc-daemon[781]: ts=2026-09-22T16:54:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_in=6244923lb # [ 44.573388] niks3-server[969]: 2026/09/22 16:54:27 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4924worker2 # [ 44.578358] nix-grpc-daemon[781]: ts=2026-09-22T16:54:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608024925worker2 # [ 44.590189] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4926worker2 # [ 44.592678] nix-daemon[865]: reaped child process 962, status = succeeded4927worker2 # [ 44.597752] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4928client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4929client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4930client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4931client # farm-b> LOG-farm-b-sig4932worker2 # [ 45.412723] nix-daemon[865]: reaped child process 971, status = succeeded4933worker2 # [ 45.486089] nix-daemon[865]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4934worker2 # [ 45.497686] nix-daemon[865]: reaped child process 990, status = succeeded4935lb # [ 45.504449] niks3-server[969]: 2026/09/22 16:54:28 INFO Received uploads request method=POST path=/api/pending_closures4936worker2 # [ 45.508470] nix-grpc-daemon[919]: time=2026-09-22T16:54:28.657Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4937worker2 # [ 45.510327] nix-grpc-daemon[919]: time=2026-09-22T16:54:28.659Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4938lb # [ 45.525762] niks3-server[969]: 2026/09/22 16:54:28 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4939lb # [ 45.527764] niks3-server[969]: 2026/09/22 16:54:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4940worker2 # [ 45.531295] nix-grpc-daemon[919]: time=2026-09-22T16:54:28.680Z level=INFO msg="Uploading 1 narinfos"4941lb # [ 45.530987] niks3-server[969]: 2026/09/22 16:54:28 INFO Signed narinfos id=12 count=14942lb # [ 45.535043] niks3-server[969]: 2026/09/22 16:54:28 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4943lb # [ 45.541952] niks3-server[969]: 2026/09/22 16:54:28 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4944lb # [ 45.546535] niks3-server[969]: 2026/09/22 16:54:28 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4945lb # [ 45.549097] niks3-server[969]: 2026/09/22 16:54:28 INFO Completed upload id=124946worker2 # [ 45.550097] nix-grpc-daemon[919]: time=2026-09-22T16:54:28.699Z level=INFO msg="Upload complete. (146ms)"4947lb # [ 45.552030] niks3-server[969]: 2026/09/22 16:54:28 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4948worker2 # [ 45.552706] nix-grpc-daemon[781]: ts=2026-09-22T16:54:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14949worker2 # [ 45.560466] nix-grpc-daemon[781]: ts=2026-09-22T16:54:28Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04950worker2 # [ 45.562213] nix-grpc-daemon[781]: ts=2026-09-22T16:54:28Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=14951worker2 # [ 45.565431] nix-daemon[865]: reaped child process 969, status = succeeded4952worker2 # [ 45.605221] nix-grpc-daemon[781]: ts=2026-09-22T16:54:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608024953worker2 # [ 45.615526] nix-grpc-daemon[781]: ts=2026-09-22T16:54:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608024954worker2 # [ 45.626490] nix-grpc-daemon[919]: time=2026-09-22T16:54:28.775Z level=INFO msg="All 2 paths already cached"4955client # copying 1 paths to 'lb:50051'...4956client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4957worker2 # [ 45.634339] nix-grpc-daemon[781]: ts=2026-09-22T16:54:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608024958worker2 # [ 45.640876] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4959worker2 # [ 45.730604] nix-daemon[865]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4960worker2 # [ 45.742812] nix-daemon[865]: reaped child process 999, status = succeeded4961lb # [ 45.750975] niks3-server[969]: 2026/09/22 16:54:28 INFO Received uploads request method=POST path=/api/pending_closures4962worker2 # [ 45.758870] nix-grpc-daemon[919]: time=2026-09-22T16:54:28.908Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4963worker2 # [ 45.760809] nix-grpc-daemon[919]: time=2026-09-22T16:54:28.909Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4964lb # [ 45.779555] niks3-server[969]: 2026/09/22 16:54:28 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4965lb # [ 45.786374] niks3-server[969]: 2026/09/22 16:54:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4966lb # [ 45.790953] niks3-server[969]: 2026/09/22 16:54:28 INFO Signed narinfos id=13 count=14967worker2 # [ 45.794128] nix-grpc-daemon[919]: time=2026-09-22T16:54:28.943Z level=INFO msg="Uploading 1 narinfos"4968lb # [ 45.802517] niks3-server[969]: 2026/09/22 16:54:28 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4969lb # [ 45.813261] niks3-server[969]: 2026/09/22 16:54:29 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4970lb # [ 45.822112] niks3-server[969]: 2026/09/22 16:54:29 INFO Completed upload id=134971lb # [ 45.825299] niks3-server[969]: 2026/09/22 16:54:29 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4972worker2 # [ 45.824366] nix-grpc-daemon[919]: time=2026-09-22T16:54:28.973Z level=INFO msg="Upload complete. (172ms)"4973worker2 # [ 45.828394] nix-grpc-daemon[781]: ts=2026-09-22T16:54:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_in=7524974worker2 # [ 45.834214] nix-grpc-daemon[781]: ts=2026-09-22T16:54:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608024975worker2 # [ 45.843673] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4976worker2 # [ 45.852905] nix-daemon[865]: reaped child process 994, status = succeeded4977worker2 # [ 45.857635] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4978client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4979client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4980lb # [ 46.630403] envoy[915]: conn peer=[2001:db8:1::1]:44302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9451 tx=4341 ms=47874981lb # [ 46.634483] envoy[915]: conn peer=[2001:db8:1::1]:44312 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=624982lb # [ 46.637971] envoy[915]: conn peer=[2001:db8:1::1]:44326 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4178 tx=1877 ms=17524983client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4984client # farm-top> LOG-farm-top-sig4985worker2 # [ 47.073397] nix-daemon[865]: reaped child process 1003, status = succeeded4986worker2 # [ 47.146076] nix-daemon[865]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4987worker2 # [ 47.158903] nix-daemon[865]: reaped child process 1022, status = succeeded4988lb # [ 47.163602] niks3-server[969]: 2026/09/22 16:54:30 INFO Received uploads request method=POST path=/api/pending_closures4989worker2 # [ 47.168146] nix-grpc-daemon[919]: time=2026-09-22T16:54:30.317Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4990worker2 # [ 47.169866] nix-grpc-daemon[919]: time=2026-09-22T16:54:30.318Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4991lb # [ 47.183369] niks3-server[969]: 2026/09/22 16:54:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4992lb # [ 47.185276] niks3-server[969]: 2026/09/22 16:54:30 INFO Signed narinfos id=14 count=14993worker2 # [ 47.186345] nix-grpc-daemon[919]: time=2026-09-22T16:54:30.335Z level=INFO msg="Uploading 1 narinfos"4994lb # [ 47.191051] niks3-server[969]: 2026/09/22 16:54:30 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4995lb # [ 47.195183] niks3-server[969]: 2026/09/22 16:54:30 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4996lb # [ 47.197488] niks3-server[969]: 2026/09/22 16:54:30 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4997lb # [ 47.202059] niks3-server[969]: 2026/09/22 16:54:30 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4998lb # [ 47.206362] niks3-server[969]: 2026/09/22 16:54:30 INFO Completed upload id=144999worker2 # [ 47.207454] nix-grpc-daemon[919]: time=2026-09-22T16:54:30.356Z level=INFO msg="Upload complete. (143ms)"5000lb # [ 47.209716] niks3-server[969]: 2026/09/22 16:54:30 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5001worker2 # [ 47.209888] nix-grpc-daemon[781]: ts=2026-09-22T16:54:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15002worker2 # [ 47.217528] nix-daemon[865]: reaped child process 1001, status = succeeded5003client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.50 seconds)5004client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5005client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.13 seconds)5006client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5007worker1 # [ 47.484680] nix-grpc-daemon[780]: ts=2026-09-22T16:54:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:477185008client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5009client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5010client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5011client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5012worker1 # [ 47.504900] nix-grpc-daemon[780]: ts=2026-09-22T16:54:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:477185013worker1 # [ 47.511744] nix-grpc-daemon[780]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5014worker1 # [ 47.522936] nix-grpc-daemon[780]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5015worker1 # [ 47.532548] nix-grpc-daemon[780]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5016client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.19 seconds)5017client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5018worker2 # [ 47.678812] nix-grpc-daemon[781]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:576065019worker2 # [ 47.721604] nix-grpc-daemon[781]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:576065020client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.22 seconds)5021client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5022client # finding garbage collector roots...5023client # removing stale temporary roots file "/nix/var/nix/temproots/822"5024client # removing stale temporary roots file "/nix/var/nix/temproots/863"5025client # removing stale temporary roots file "/nix/var/nix/temproots/877"5026client # removing stale temporary roots file "/nix/var/nix/temproots/855"5027client # removing stale temporary roots file "/nix/var/nix/temproots/843"5028client # removing stale temporary roots file "/nix/var/nix/temproots/805"5029client # removing stale temporary roots file "/nix/var/nix/temproots/885"5030client # removing stale temporary roots file "/nix/var/nix/temproots/900"5031client # deleting specified paths...5032client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5033client # deleting unused links...5034client # note: hard linking is currently saving 0.0 KiB5035worker2 # [ 48.053270] nix-grpc-daemon[781]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608025036worker1 # [ 48.050575] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365037worker1 # [ 48.059930] nix-grpc-daemon[857]: time=2026-09-22T16:54:30.374Z level=INFO msg="All 1 paths already cached"5038worker1 # [ 48.066239] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442365039worker2 # [ 48.078828] nix-grpc-daemon[781]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:608025040client # copying 3 paths...5041worker1 # [ 48.083892] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442365042client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5043worker2 # [ 48.106594] nix-grpc-daemon[781]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:608025044worker1 # [ 48.106891] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:442365045worker1 # [ 48.109493] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_out=1765046worker2 # [ 48.112769] nix-grpc-daemon[781]: ts=2026-09-22T16:54:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_out=2325047worker2 # [ 48.119892] nix-grpc-daemon[781]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:608025048worker2 # [ 48.125188] nix-grpc-daemon[781]: ts=2026-09-22T16:54:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_out=1285049client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5050client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5051client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.41 seconds)5052(finished: subtest: outputs read back through any worker carry the cache signature, in 5.45 seconds)5053subtest: two clients wanting the same drv build it once5054client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5055client # [ 48.244692] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5056client # Running as unit: dedup1.service; invocation ID: d0d4055a8f514ac3bf1174328d85d8965057client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)5058client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5059client # [ 48.382472] nix[953]: don't know how to build these paths:5060client # [ 48.383398] nix[953]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5061worker1 # [ 48.353218] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442365062worker2 # [ 48.369273] nix-grpc-daemon[781]: ts=2026-09-22T16:54:31Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05063worker1 # [ 48.364648] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365064worker2 # [ 48.373844] nix-grpc-daemon[781]: ts=2026-09-22T16:54:31Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=15065worker1 # [ 48.376333] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365066client # [ 48.415075] nix[953]: copying 1 paths to 'lb:50051'...5067client # [ 48.415831] nix[953]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5068worker1 # [ 48.387125] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442365069worker1 # [ 48.399154] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5070worker1 # [ 48.505827] nix-daemon[845]: accepted connection from pid 1009, user nix-grpc-daemon (trusted)5071worker1 # [ 48.519884] nix-daemon[845]: reaped child process 1012, status = succeeded5072lb # [ 48.534833] niks3-server[969]: 2026/09/22 16:54:31 INFO Received uploads request method=POST path=/api/pending_closures5073worker1 # [ 48.534981] nix-grpc-daemon[857]: time=2026-09-22T16:54:30.849Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5074worker1 # [ 48.536710] nix-grpc-daemon[857]: time=2026-09-22T16:54:30.851Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5075lb # [ 48.564473] niks3-server[969]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5076lb # [ 48.571831] niks3-server[969]: 2026/09/22 16:54:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5077lb # [ 48.578528] niks3-server[969]: 2026/09/22 16:54:31 INFO Signed narinfos id=15 count=15078worker1 # [ 48.575990] nix-grpc-daemon[857]: time=2026-09-22T16:54:30.890Z level=INFO msg="Uploading 1 narinfos"5079lb # [ 48.592225] niks3-server[969]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5080lb # [ 48.598621] niks3-server[969]: 2026/09/22 16:54:31 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5081lb # [ 48.605209] niks3-server[969]: 2026/09/22 16:54:31 INFO Completed upload id=155082worker1 # [ 48.599384] nix-grpc-daemon[857]: time=2026-09-22T16:54:30.914Z level=INFO msg="Upload complete. (188ms)"5083lb # [ 48.609159] niks3-server[969]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5084worker1 # [ 48.604614] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_in=4325085worker1 # [ 48.609944] nix-grpc-daemon[780]: ts=2026-09-22T16:54:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365086worker1 # [ 48.621130] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5087worker1 # [ 48.625159] nix-daemon[845]: reaped child process 1007, status = succeeded5088worker1 # [ 48.629664] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5089client # [ 48.657301] nix[953]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5090client # [ 49.288747] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5091client # Running as unit: dedup2.service; invocation ID: c15dd471774b4631a0b16155ab24b1855092client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.06 seconds)5093worker1 # [ 49.426309] nix-grpc-daemon[780]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442365094client # [ 49.462910] nix[969]: this derivation will be built:5095worker2 # [ 49.443324] nix-grpc-daemon[781]: ts=2026-09-22T16:54:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:608025096worker2 # [ 49.454144] nix-grpc-daemon[781]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...5097client # [ 49.491620] nix[969]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5098worker2 # [ 49.475975] nix-grpc-daemon[781]: ts=2026-09-22T16:54:32Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05099worker1 # [ 49.479411] nix-grpc-daemon[780]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365100client # [ 49.509737] nix[969]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5101client: must succeed: sleep 25102client: (finished: must succeed: sleep 2, in 2.04 seconds)5103client: waiting for success: ! systemctl is-active dedup1 dedup25104worker1 # [ 52.653888] nix-daemon[845]: reaped child process 1016, status = succeeded5105worker1 # [ 52.721254] nix-daemon[845]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)5106worker1 # [ 52.736341] nix-daemon[845]: reaped child process 1049, status = succeeded5107lb # [ 52.745706] niks3-server[969]: 2026/09/22 16:54:35 INFO Received uploads request method=POST path=/api/pending_closures5108worker1 # [ 52.744498] nix-grpc-daemon[857]: time=2026-09-22T16:54:35.058Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5109worker1 # [ 52.745964] nix-grpc-daemon[857]: time=2026-09-22T16:54:35.060Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5110lb # [ 52.771192] niks3-server[969]: 2026/09/22 16:54:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5111lb # [ 52.773206] niks3-server[969]: 2026/09/22 16:54:35 INFO Signed narinfos id=16 count=15112worker1 # [ 52.768742] nix-grpc-daemon[857]: time=2026-09-22T16:54:35.083Z level=INFO msg="Uploading 1 narinfos"5113lb # [ 52.777555] niks3-server[969]: 2026/09/22 16:54:35 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5114lb # [ 52.780304] niks3-server[969]: 2026/09/22 16:54:35 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5115lb # [ 52.786027] niks3-server[969]: 2026/09/22 16:54:35 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5116lb # [ 52.791079] niks3-server[969]: 2026/09/22 16:54:35 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5117worker1 # [ 52.788835] nix-grpc-daemon[857]: time=2026-09-22T16:54:35.103Z level=INFO msg="Upload complete. (149ms)"5118lb # [ 52.795096] niks3-server[969]: 2026/09/22 16:54:35 INFO Completed upload id=165119worker1 # [ 52.791707] nix-grpc-daemon[780]: ts=2026-09-22T16:54:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15120lb # [ 52.799372] niks3-server[969]: 2026/09/22 16:54:35 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5121worker1 # [ 52.794853] nix-grpc-daemon[780]: ts=2026-09-22T16:54:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15122worker1 # [ 52.801560] nix-daemon[845]: reaped child process 1014, status = succeeded5123client # [ 52.852553] systemd[1]: dedup1.service: Deactivated successfully.5124client # [ 52.856193] systemd[1]: dedup1.service: Consumed 129ms CPU time over 4.607s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5125client # [ 52.892146] systemd[1]: dedup2.service: Deactivated successfully.5126client # [ 52.894306] systemd[1]: dedup2.service: Consumed 146ms CPU time over 3.603s wall clock time, 35.4M memory peak, 6.6K incoming IP traffic, 6.1K outgoing IP traffic.5127client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)5128client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5129client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5130worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5131worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5132worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5133worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5134worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5135worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5136worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5137worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5138(finished: subtest: two clients wanting the same drv build it once, in 5.72 seconds)5139subtest: --timeout and --max-silent-time reach the builder5140worker2 # [ 54.062415] nix-grpc-daemon[781]: ts=2026-09-22T16:54:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608025141worker2 # [ 54.074157] nix-grpc-daemon[781]: ts=2026-09-22T16:54:37Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05142worker2 # [ 54.079401] nix-grpc-daemon[781]: ts=2026-09-22T16:54:37Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=15143worker2 # [ 54.085268] nix-grpc-daemon[781]: ts=2026-09-22T16:54:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025144worker2 # [ 54.094454] nix-grpc-daemon[781]: ts=2026-09-22T16:54:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608025145worker2 # [ 54.104290] nix-grpc-daemon[781]: ts=2026-09-22T16:54:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608025146worker2 # [ 54.114376] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5147worker2 # [ 54.209310] nix-daemon[865]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)5148worker2 # [ 54.221873] nix-daemon[865]: reaped child process 1057, status = succeeded5149lb # [ 54.229671] niks3-server[969]: 2026/09/22 16:54:37 INFO Received uploads request method=POST path=/api/pending_closures5150worker2 # [ 54.233578] nix-grpc-daemon[919]: time=2026-09-22T16:54:37.382Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5151worker2 # [ 54.235513] nix-grpc-daemon[919]: time=2026-09-22T16:54:37.384Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5152lb # [ 54.244907] niks3-server[969]: 2026/09/22 16:54:37 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5153lb # [ 54.254310] niks3-server[969]: 2026/09/22 16:54:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5154lb # [ 54.259279] niks3-server[969]: 2026/09/22 16:54:37 INFO Signed narinfos id=17 count=15155worker2 # [ 54.262555] nix-grpc-daemon[919]: time=2026-09-22T16:54:37.411Z level=INFO msg="Uploading 1 narinfos"5156lb # [ 54.265321] niks3-server[969]: 2026/09/22 16:54:37 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5157lb # [ 54.276896] niks3-server[969]: 2026/09/22 16:54:37 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5158lb # [ 54.282371] niks3-server[969]: 2026/09/22 16:54:37 INFO Completed upload id=175159worker2 # [ 54.283436] nix-grpc-daemon[919]: time=2026-09-22T16:54:37.432Z level=INFO msg="Upload complete. (157ms)"5160lb # [ 54.285879] niks3-server[969]: 2026/09/22 16:54:37 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5161worker2 # [ 54.285619] nix-grpc-daemon[781]: ts=2026-09-22T16:54:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_in=4245162worker2 # [ 54.289725] nix-grpc-daemon[781]: ts=2026-09-22T16:54:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025163worker2 # [ 54.295774] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5164worker2 # [ 54.298036] nix-daemon[865]: reaped child process 1052, status = succeeded5165worker2 # [ 54.303599] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5166lb # [ 56.631112] envoy[915]: conn peer=[2001:db8:1::1]:59158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9494 tx=4364 ms=43555167lb # [ 56.635088] envoy[915]: conn peer=[2001:db8:1::1]:59176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=815168lb # [ 56.638509] envoy[915]: conn peer=[2001:db8:1::1]:59160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1365169lb # [ 56.641973] envoy[915]: conn peer=[2001:db8:1::1]:59202 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=675170lb # [ 56.645512] envoy[915]: conn peer=[2001:db8:1::1]:59190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=376 ms=745171lb # [ 56.649547] envoy[915]: conn peer=[2001:db8:1::1]:59212 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1519 ms=44585172lb # [ 56.653163] envoy[915]: conn peer=[2001:db8:1::1]:59224 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1576 tx=1598 ms=34315173worker2 # [ 59.915350] nix-grpc-daemon[781]: ts=2026-09-22T16:54:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=5 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05174worker2 # [ 59.926368] nix-daemon[865]: reaped child process 1061, status = succeeded5175worker2 # [ 59.927342] nix-daemon[865]: reaped child process 1059, status = succeeded5176worker2 # [ 60.126261] nix-grpc-daemon[781]: ts=2026-09-22T16:54:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608025177worker2 # [ 60.136974] nix-grpc-daemon[781]: ts=2026-09-22T16:54:43Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05178worker1 # [ 60.134094] nix-grpc-daemon[780]: ts=2026-09-22T16:54:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365179worker2 # [ 60.141742] nix-grpc-daemon[781]: ts=2026-09-22T16:54:43Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=15180worker1 # [ 60.142128] nix-grpc-daemon[780]: ts=2026-09-22T16:54:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365181worker1 # [ 60.149027] nix-grpc-daemon[780]: ts=2026-09-22T16:54:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442365182worker1 # [ 60.152849] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5183worker1 # [ 60.238080] nix-daemon[845]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)5184worker1 # [ 60.252174] nix-daemon[845]: reaped child process 1065, status = succeeded5185lb # [ 60.264759] niks3-server[969]: 2026/09/22 16:54:43 INFO Received uploads request method=POST path=/api/pending_closures5186worker1 # [ 60.267177] nix-grpc-daemon[857]: time=2026-09-22T16:54:42.581Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5187worker1 # [ 60.268579] nix-grpc-daemon[857]: time=2026-09-22T16:54:42.581Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5188lb # [ 60.292192] niks3-server[969]: 2026/09/22 16:54:43 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5189lb # [ 60.303709] niks3-server[969]: 2026/09/22 16:54:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5190lb # [ 60.309886] niks3-server[969]: 2026/09/22 16:54:43 INFO Signed narinfos id=18 count=15191worker1 # [ 60.307731] nix-grpc-daemon[857]: time=2026-09-22T16:54:42.622Z level=INFO msg="Uploading 1 narinfos"5192lb # [ 60.324414] niks3-server[969]: 2026/09/22 16:54:43 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5193lb # [ 60.331960] niks3-server[969]: 2026/09/22 16:54:43 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5194lb # [ 60.334615] niks3-server[969]: 2026/09/22 16:54:43 INFO Completed upload id=185195worker1 # [ 60.328720] nix-grpc-daemon[857]: time=2026-09-22T16:54:42.643Z level=INFO msg="Upload complete. (169ms)"5196worker1 # [ 60.330733] nix-grpc-daemon[780]: ts=2026-09-22T16:54:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_in=4245197lb # [ 60.339144] niks3-server[969]: 2026/09/22 16:54:43 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5198worker1 # [ 60.335408] nix-grpc-daemon[780]: ts=2026-09-22T16:54:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365199worker1 # [ 60.341373] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5200worker1 # [ 60.343615] nix-daemon[845]: reaped child process 1060, status = succeeded5201worker1 # [ 60.348703] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5202worker1 # [ 66.161593] nix-grpc-daemon[780]: ts=2026-09-22T16:54:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=5 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05203worker1 # [ 66.185956] nix-daemon[845]: reaped child process 1067, status = succeeded5204worker1 # [ 66.188153] nix-daemon[845]: reaped child process 1069, status = succeeded5205(finished: subtest: --timeout and --max-silent-time reach the builder, in 12.37 seconds)5206subtest: the first client leaving does not fail the second5207client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5208client # [ 66.348574] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5209client # Running as unit: share1.service; invocation ID: 780de06f94f5464d82f231ceaf52db925210client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5211client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5212worker1 # [ 66.467291] nix-grpc-daemon[780]: ts=2026-09-22T16:54:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442365213client # [ 66.501623] nix[1043]: don't know how to build these paths:5214client # [ 66.502748] nix[1043]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5215worker2 # [ 66.488788] nix-grpc-daemon[781]: ts=2026-09-22T16:54:49Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05216worker2 # [ 66.494488] nix-grpc-daemon[781]: ts=2026-09-22T16:54:49Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15217worker2 # [ 66.498948] nix-grpc-daemon[781]: ts=2026-09-22T16:54:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025218worker2 # [ 66.511078] nix-grpc-daemon[781]: ts=2026-09-22T16:54:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608025219client # [ 66.543078] nix[1043]: copying 1 paths to 'lb:50051'...5220client # [ 66.543898] nix[1043]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5221worker2 # [ 66.521801] nix-grpc-daemon[781]: ts=2026-09-22T16:54:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608025222worker2 # [ 66.532796] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5223worker2 # [ 66.629259] nix-daemon[865]: accepted connection from pid 1088, user nix-grpc-daemon (trusted)5224lb # [ 66.631488] envoy[915]: conn peer=[2001:db8:1::1]:56142 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=956 ms=59065225lb # [ 66.633129] envoy[915]: conn peer=[2001:db8:1::1]:56154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=956 ms=60545226worker2 # [ 66.641750] nix-daemon[865]: reaped child process 1091, status = succeeded5227lb # [ 66.648805] niks3-server[969]: 2026/09/22 16:54:49 INFO Received uploads request method=POST path=/api/pending_closures5228worker2 # [ 66.653431] nix-grpc-daemon[919]: time=2026-09-22T16:54:49.801Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5229worker2 # [ 66.654822] nix-grpc-daemon[919]: time=2026-09-22T16:54:49.802Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5230lb # [ 66.662711] niks3-server[969]: 2026/09/22 16:54:49 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5231lb # [ 66.668184] niks3-server[969]: 2026/09/22 16:54:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5232lb # [ 66.670267] niks3-server[969]: 2026/09/22 16:54:49 INFO Signed narinfos id=19 count=15233worker2 # [ 66.670271] nix-grpc-daemon[919]: time=2026-09-22T16:54:49.819Z level=INFO msg="Uploading 1 narinfos"5234lb # [ 66.676863] niks3-server[969]: 2026/09/22 16:54:49 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5235lb # [ 66.680247] niks3-server[969]: 2026/09/22 16:54:49 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5236lb # [ 66.683049] niks3-server[969]: 2026/09/22 16:54:49 INFO Completed upload id=195237worker2 # [ 66.682766] nix-grpc-daemon[919]: time=2026-09-22T16:54:49.831Z level=INFO msg="Upload complete. (138ms)"5238worker2 # [ 66.684817] nix-grpc-daemon[781]: ts=2026-09-22T16:54:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_in=4325239lb # [ 66.685920] niks3-server[969]: 2026/09/22 16:54:49 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5240worker2 # [ 66.688352] nix-grpc-daemon[781]: ts=2026-09-22T16:54:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025241worker2 # [ 66.693280] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5242worker2 # [ 66.697972] nix-daemon[865]: reaped child process 1086, status = succeeded5243worker2 # [ 66.702488] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5244client # [ 66.724148] nix[1043]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5245client # [ 67.411726] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5246client # Running as unit: share2.service; invocation ID: 0ec343d32acb4176ab17b065ec474a215247client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5248worker2 # [ 67.531877] nix-grpc-daemon[781]: ts=2026-09-22T16:54:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608025249client # [ 67.561448] nix[1059]: this derivation will be built:5250worker1 # [ 67.535168] nix-grpc-daemon[780]: ts=2026-09-22T16:54:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442365251worker1 # [ 67.542794] nix-grpc-daemon[780]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' from 'http://lb:5751'...5252client # [ 67.579951] nix[1059]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5253worker2 # [ 67.563072] nix-grpc-daemon[781]: ts=2026-09-22T16:54:50Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05254worker2 # [ 67.567702] nix-grpc-daemon[781]: ts=2026-09-22T16:54:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025255client # [ 67.592220] nix[1059]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5256client: must succeed: sleep 2; systemctl kill -s INT share15257client # [ 70.582842] systemd[1]: share1.service: Sent signal SIGINT to main process 1043 (nix) on client request.5258client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.03 seconds)5259client: waiting for success: ! systemctl is-active share15260client # [ 70.588680] nix[1043]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5261client # [ 70.596803] nix[1043]: error: interrupted by the user5262client # [ 70.636356] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5263client # [ 70.638369] systemd[1]: share1.service: Failed with result 'exit-code'.5264client # [ 70.640460] systemd[1]: share1.service: Consumed 122ms CPU time over 4.290s wall clock time, 43.5M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.5265client: (finished: waiting for success: ! systemctl is-active share1, in 1.06 seconds)5266client: waiting for success: ! systemctl is-active share25267worker2 # [ 71.949616] nix-daemon[865]: reaped child process 1095, status = succeeded5268worker2 # [ 72.026548] nix-daemon[865]: accepted connection from pid 1131, user nix-grpc-daemon (trusted)5269worker2 # [ 72.039435] nix-daemon[865]: reaped child process 1134, status = succeeded5270lb # [ 72.046783] niks3-server[969]: 2026/09/22 16:54:55 INFO Received uploads request method=POST path=/api/pending_closures5271worker2 # [ 72.052637] nix-grpc-daemon[919]: time=2026-09-22T16:54:55.201Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5272worker2 # [ 72.054529] nix-grpc-daemon[919]: time=2026-09-22T16:54:55.203Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5273lb # [ 72.087805] niks3-server[969]: 2026/09/22 16:54:55 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5274lb # [ 72.092656] niks3-server[969]: 2026/09/22 16:54:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5275worker2 # [ 72.100370] nix-grpc-daemon[919]: time=2026-09-22T16:54:55.249Z level=INFO msg="Uploading 1 narinfos"5276lb # [ 72.100478] niks3-server[969]: 2026/09/22 16:54:55 INFO Signed narinfos id=20 count=15277lb # [ 72.105669] niks3-server[969]: 2026/09/22 16:54:55 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5278lb # [ 72.110985] niks3-server[969]: 2026/09/22 16:54:55 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5279lb # [ 72.117956] niks3-server[969]: 2026/09/22 16:54:55 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5280lb # [ 72.121130] niks3-server[969]: 2026/09/22 16:54:55 INFO Completed upload id=205281worker2 # [ 72.122055] nix-grpc-daemon[919]: time=2026-09-22T16:54:55.271Z level=INFO msg="Upload complete. (207ms)"5282lb # [ 72.124277] niks3-server[969]: 2026/09/22 16:54:55 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5283worker2 # [ 72.125252] nix-grpc-daemon[781]: ts=2026-09-22T16:54:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15284worker2 # [ 72.130190] nix-grpc-daemon[781]: ts=2026-09-22T16:54:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15285worker2 # [ 72.136889] nix-daemon[865]: reaped child process 1093, status = succeeded5286client # [ 72.220821] systemd[1]: share2.service: Deactivated successfully.5287client # [ 72.225456] systemd[1]: share2.service: Consumed 124ms CPU time over 4.813s wall clock time, 35.4M memory peak, 6.6K incoming IP traffic, 6K outgoing IP traffic.5288client: (finished: waiting for success: ! systemctl is-active share2, in 1.06 seconds)5289client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5290client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds)5291(finished: subtest: the first client leaving does not fail the second, in 6.57 seconds)5292subtest: build hook: nix-daemon with builders = grpc://lb5293client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5294client # [ 72.969509] systemd[1]: Started Nix Daemon.5295client # [ 73.064506] nix-daemon[1110]: accepted connection from pid 1107, user root (trusted)5296worker2 # [ 73.167506] nix-grpc-daemon[781]: ts=2026-09-22T16:54:56Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:608025297worker1 # [ 73.164966] nix-grpc-daemon[780]: ts=2026-09-22T16:54:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365298worker2 # [ 73.179941] nix-grpc-daemon[781]: ts=2026-09-22T16:54:56Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05299worker1 # [ 73.176180] nix-grpc-daemon[780]: ts=2026-09-22T16:54:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365300worker2 # [ 73.184382] nix-grpc-daemon[781]: ts=2026-09-22T16:54:56Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15301worker1 # [ 73.185684] nix-grpc-daemon[780]: ts=2026-09-22T16:54:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365302worker1 # [ 73.194797] nix-grpc-daemon[780]: ts=2026-09-22T16:54:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442365303worker1 # [ 73.205551] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5304worker1 # [ 73.303676] nix-daemon[845]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5305worker1 # [ 73.316053] nix-daemon[845]: reaped child process 1112, status = succeeded5306lb # [ 73.329548] niks3-server[969]: 2026/09/22 16:54:56 INFO Received uploads request method=POST path=/api/pending_closures5307worker1 # [ 73.328246] nix-grpc-daemon[857]: time=2026-09-22T16:54:55.642Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5308worker1 # [ 73.329715] nix-grpc-daemon[857]: time=2026-09-22T16:54:55.642Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5309lb # [ 73.342298] niks3-server[969]: 2026/09/22 16:54:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5310lb # [ 73.346151] niks3-server[969]: 2026/09/22 16:54:56 INFO Signed narinfos id=21 count=15311worker1 # [ 73.340318] nix-grpc-daemon[857]: time=2026-09-22T16:54:55.654Z level=INFO msg="Uploading 1 narinfos"5312lb # [ 73.348957] niks3-server[969]: 2026/09/22 16:54:56 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5313lb # [ 73.351703] niks3-server[969]: 2026/09/22 16:54:56 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5314lb # [ 73.355737] niks3-server[969]: 2026/09/22 16:54:56 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5315lb # [ 73.358901] niks3-server[969]: 2026/09/22 16:54:56 INFO Completed upload id=215316worker1 # [ 73.352972] nix-grpc-daemon[857]: time=2026-09-22T16:54:55.667Z level=INFO msg="Upload complete. (136ms)"5317worker1 # [ 73.354989] nix-grpc-daemon[780]: ts=2026-09-22T16:54:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_in=5125318lb # [ 73.361850] niks3-server[969]: 2026/09/22 16:54:56 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5319worker1 # [ 73.358286] nix-grpc-daemon[780]: ts=2026-09-22T16:54:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365320worker1 # [ 73.364701] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5321worker1 # [ 73.366886] nix-daemon[845]: reaped child process 1107, status = succeeded5322worker1 # [ 73.371779] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5323worker1 # [ 74.388093] nix-daemon[845]: reaped child process 1116, status = succeeded5324worker1 # [ 74.456965] nix-daemon[845]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)5325worker1 # [ 74.471148] nix-daemon[845]: reaped child process 1135, status = succeeded5326lb # [ 74.483411] niks3-server[969]: 2026/09/22 16:54:57 INFO Received uploads request method=POST path=/api/pending_closures5327worker1 # [ 74.485967] nix-grpc-daemon[857]: time=2026-09-22T16:54:56.800Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5328worker1 # [ 74.488786] nix-grpc-daemon[857]: time=2026-09-22T16:54:56.802Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5329lb # [ 74.524894] niks3-server[969]: 2026/09/22 16:54:57 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5330lb # [ 74.530281] niks3-server[969]: 2026/09/22 16:54:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5331lb # [ 74.535970] niks3-server[969]: 2026/09/22 16:54:57 INFO Signed narinfos id=22 count=15332worker1 # [ 74.533208] nix-grpc-daemon[857]: time=2026-09-22T16:54:56.847Z level=INFO msg="Uploading 1 narinfos"5333lb # [ 74.542459] niks3-server[969]: 2026/09/22 16:54:57 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5334lb # [ 74.551889] niks3-server[969]: 2026/09/22 16:54:57 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5335lb # [ 74.558545] niks3-server[969]: 2026/09/22 16:54:57 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5336lb # [ 74.563937] niks3-server[969]: 2026/09/22 16:54:57 INFO Completed upload id=225337worker1 # [ 74.557867] nix-grpc-daemon[857]: time=2026-09-22T16:54:56.872Z level=INFO msg="Upload complete. (179ms)"5338lb # [ 74.569258] niks3-server[969]: 2026/09/22 16:54:57 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5339worker1 # [ 74.563941] nix-grpc-daemon[780]: ts=2026-09-22T16:54:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15340worker1 # [ 74.572717] nix-grpc-daemon[780]: ts=2026-09-22T16:54:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442365341worker1 # [ 74.584785] nix-daemon[845]: reaped child process 1114, status = succeeded5342worker2 # [ 74.598791] nix-grpc-daemon[781]: ts=2026-09-22T16:54:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:608025343worker2 # [ 74.609478] nix-grpc-daemon[781]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5344worker2 # [ 74.622955] nix-grpc-daemon[781]: ts=2026-09-22T16:54:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_out=1285345worker2 # [ 74.794408] nix-grpc-daemon[781]: ts=2026-09-22T16:54:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:608025346worker1 # [ 74.796932] nix-grpc-daemon[780]: ts=2026-09-22T16:54:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365347worker1 # [ 74.806130] nix-grpc-daemon[857]: time=2026-09-22T16:54:57.120Z level=INFO msg="All 1 paths already cached"5348worker2 # [ 74.826280] nix-grpc-daemon[781]: ts=2026-09-22T16:54:57Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05349worker2 # [ 74.830825] nix-grpc-daemon[781]: ts=2026-09-22T16:54:57Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15350worker1 # [ 74.827326] nix-grpc-daemon[780]: ts=2026-09-22T16:54:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365351worker1 # [ 74.840081] nix-grpc-daemon[780]: ts=2026-09-22T16:54:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365352worker1 # [ 74.850984] nix-grpc-daemon[857]: time=2026-09-22T16:54:57.165Z level=INFO msg="All 1 paths already cached"5353worker1 # [ 74.858839] nix-grpc-daemon[780]: ts=2026-09-22T16:54:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442365354worker1 # [ 74.869253] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5355worker1 # [ 74.967078] nix-daemon[845]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)5356worker1 # [ 74.980887] nix-daemon[845]: reaped child process 1142, status = succeeded5357lb # [ 74.991448] niks3-server[969]: 2026/09/22 16:54:58 INFO Received uploads request method=POST path=/api/pending_closures5358worker1 # [ 74.993495] nix-grpc-daemon[857]: time=2026-09-22T16:54:57.308Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5359worker1 # [ 74.995148] nix-grpc-daemon[857]: time=2026-09-22T16:54:57.309Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5360lb # [ 75.021882] niks3-server[969]: 2026/09/22 16:54:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5361lb # [ 75.031113] niks3-server[969]: 2026/09/22 16:54:58 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5362lb # [ 75.036918] niks3-server[969]: 2026/09/22 16:54:58 INFO Signed narinfos id=23 count=15363worker1 # [ 75.033602] nix-grpc-daemon[857]: time=2026-09-22T16:54:57.348Z level=INFO msg="Uploading 1 narinfos"5364lb # [ 75.043965] niks3-server[969]: 2026/09/22 16:54:58 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5365lb # [ 75.052198] niks3-server[969]: 2026/09/22 16:54:58 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5366lb # [ 75.059129] niks3-server[969]: 2026/09/22 16:54:58 INFO Completed upload id=235367worker1 # [ 75.053223] nix-grpc-daemon[857]: time=2026-09-22T16:54:57.367Z level=INFO msg="Upload complete. (171ms)"5368lb # [ 75.065092] niks3-server[969]: 2026/09/22 16:54:58 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5369worker1 # [ 75.058335] nix-grpc-daemon[780]: ts=2026-09-22T16:54:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_in=6245370worker1 # [ 75.064429] nix-grpc-daemon[780]: ts=2026-09-22T16:54:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365371worker1 # [ 75.074472] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5372worker1 # [ 75.080843] nix-daemon[845]: reaped child process 1137, status = succeeded5373worker1 # [ 75.085250] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5374worker1 # [ 76.503339] nix-daemon[845]: reaped child process 1146, status = succeeded5375worker1 # [ 76.571589] nix-daemon[845]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5376worker1 # [ 76.584165] nix-daemon[845]: reaped child process 1165, status = succeeded5377lb # [ 76.596621] niks3-server[969]: 2026/09/22 16:54:59 INFO Received uploads request method=POST path=/api/pending_closures5378worker1 # [ 76.599359] nix-grpc-daemon[857]: time=2026-09-22T16:54:58.913Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5379worker1 # [ 76.602058] nix-grpc-daemon[857]: time=2026-09-22T16:54:58.915Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5380lb # [ 76.632415] envoy[915]: conn peer=[2001:db8:1::1]:38552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=773 ms=41165381lb # [ 76.637937] envoy[915]: conn peer=[2001:db8:1::1]:38562 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1589 tx=1598 ms=46465382lb # [ 76.642885] envoy[915]: conn peer=[2001:db8:1::1]:58928 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=334 ms=425383lb # [ 76.648125] envoy[915]: conn peer=[2001:db8:1::1]:58918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3409 tx=1918 ms=14725384lb # [ 76.659481] niks3-server[969]: 2026/09/22 16:54:59 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5385lb # [ 76.662549] niks3-server[969]: 2026/09/22 16:54:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5386lb # [ 76.666305] niks3-server[969]: 2026/09/22 16:54:59 INFO Signed narinfos id=24 count=15387worker1 # [ 76.662823] nix-grpc-daemon[857]: time=2026-09-22T16:54:58.977Z level=INFO msg="Uploading 1 narinfos"5388lb # [ 76.671532] niks3-server[969]: 2026/09/22 16:54:59 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5389lb # [ 76.673893] niks3-server[969]: 2026/09/22 16:54:59 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5390lb # [ 76.679178] niks3-server[969]: 2026/09/22 16:54:59 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5391lb # [ 76.681719] niks3-server[969]: 2026/09/22 16:54:59 INFO Completed upload id=245392worker1 # [ 76.675971] nix-grpc-daemon[857]: time=2026-09-22T16:54:58.990Z level=INFO msg="Upload complete. (182ms)"5393lb # [ 76.684787] niks3-server[969]: 2026/09/22 16:54:59 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5394worker1 # [ 76.681919] nix-grpc-daemon[780]: ts=2026-09-22T16:54:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15395worker1 # [ 76.690504] nix-grpc-daemon[780]: ts=2026-09-22T16:54:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442365396worker1 # [ 76.702048] nix-daemon[845]: reaped child process 1144, status = succeeded5397worker2 # [ 76.708098] nix-grpc-daemon[781]: ts=2026-09-22T16:54:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:608025398worker2 # [ 76.713217] nix-grpc-daemon[781]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5399worker2 # [ 76.718829] nix-grpc-daemon[781]: ts=2026-09-22T16:54:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_out=1765400worker1 # [ 76.929935] nix-grpc-daemon[780]: ts=2026-09-22T16:54:59Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:442365401worker2 # [ 76.938860] nix-grpc-daemon[781]: ts=2026-09-22T16:55:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608025402worker2 # [ 76.949340] nix-grpc-daemon[919]: time=2026-09-22T16:55:00.097Z level=INFO msg="All 2 paths already cached"5403worker2 # [ 76.963892] nix-grpc-daemon[781]: ts=2026-09-22T16:55:00Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05404worker1 # [ 76.960370] nix-grpc-daemon[780]: ts=2026-09-22T16:54:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365405worker2 # [ 76.969626] nix-grpc-daemon[781]: ts=2026-09-22T16:55:00Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15406worker1 # [ 76.972867] nix-grpc-daemon[780]: ts=2026-09-22T16:54:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365407worker1 # [ 76.985335] nix-grpc-daemon[857]: time=2026-09-22T16:54:59.299Z level=INFO msg="All 2 paths already cached"5408worker1 # [ 76.992583] nix-grpc-daemon[780]: ts=2026-09-22T16:54:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442365409worker1 # [ 77.002162] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5410worker1 # [ 77.106195] nix-daemon[845]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5411worker1 # [ 77.120591] nix-daemon[845]: reaped child process 1172, status = succeeded5412lb # [ 77.132619] niks3-server[969]: 2026/09/22 16:55:00 INFO Received uploads request method=POST path=/api/pending_closures5413worker1 # [ 77.134951] nix-grpc-daemon[857]: time=2026-09-22T16:54:59.449Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5414worker1 # [ 77.136443] nix-grpc-daemon[857]: time=2026-09-22T16:54:59.451Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5415lb # [ 77.167193] niks3-server[969]: 2026/09/22 16:55:00 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5416lb # [ 77.176987] niks3-server[969]: 2026/09/22 16:55:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5417lb # [ 77.181158] niks3-server[969]: 2026/09/22 16:55:00 INFO Signed narinfos id=25 count=15418worker1 # [ 77.178238] nix-grpc-daemon[857]: time=2026-09-22T16:54:59.492Z level=INFO msg="Uploading 1 narinfos"5419lb # [ 77.187270] niks3-server[969]: 2026/09/22 16:55:00 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5420lb # [ 77.200602] niks3-server[969]: 2026/09/22 16:55:00 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5421lb # [ 77.205907] niks3-server[969]: 2026/09/22 16:55:00 INFO Completed upload id=255422worker1 # [ 77.202233] nix-grpc-daemon[857]: time=2026-09-22T16:54:59.516Z level=INFO msg="Upload complete. (187ms)"5423lb # [ 77.213378] niks3-server[969]: 2026/09/22 16:55:00 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5424worker1 # [ 77.206353] nix-grpc-daemon[780]: ts=2026-09-22T16:54:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_in=7525425worker1 # [ 77.213183] nix-grpc-daemon[780]: ts=2026-09-22T16:54:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365426worker1 # [ 77.222725] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5427worker1 # [ 77.233195] nix-daemon[845]: reaped child process 1167, status = succeeded5428worker1 # [ 77.236981] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5429worker1 # [ 78.468650] nix-daemon[845]: reaped child process 1176, status = succeeded5430worker1 # [ 78.544435] nix-daemon[845]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)5431worker1 # [ 78.556542] nix-daemon[845]: reaped child process 1195, status = succeeded5432lb # [ 78.568862] niks3-server[969]: 2026/09/22 16:55:01 INFO Received uploads request method=POST path=/api/pending_closures5433worker1 # [ 78.566849] nix-grpc-daemon[857]: time=2026-09-22T16:55:00.881Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5434worker1 # [ 78.568275] nix-grpc-daemon[857]: time=2026-09-22T16:55:00.882Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5435lb # [ 78.591859] niks3-server[969]: 2026/09/22 16:55:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5436lb # [ 78.594520] niks3-server[969]: 2026/09/22 16:55:01 INFO Signed narinfos id=26 count=15437worker1 # [ 78.589886] nix-grpc-daemon[857]: time=2026-09-22T16:55:00.904Z level=INFO msg="Uploading 1 narinfos"5438lb # [ 78.600360] niks3-server[969]: 2026/09/22 16:55:01 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5439lb # [ 78.602518] niks3-server[969]: 2026/09/22 16:55:01 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5440lb # [ 78.604957] niks3-server[969]: 2026/09/22 16:55:01 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5441lb # [ 78.608938] niks3-server[969]: 2026/09/22 16:55:01 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5442lb # [ 78.612915] niks3-server[969]: 2026/09/22 16:55:01 INFO Completed upload id=265443worker1 # [ 78.606907] nix-grpc-daemon[857]: time=2026-09-22T16:55:00.921Z level=INFO msg="Upload complete. (162ms)"5444lb # [ 78.614137] niks3-server[969]: 2026/09/22 16:55:01 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5445worker1 # [ 78.609485] nix-grpc-daemon[780]: ts=2026-09-22T16:55:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15446worker1 # [ 78.616108] nix-daemon[845]: reaped child process 1174, status = succeeded5447worker2 # [ 78.662927] nix-grpc-daemon[781]: ts=2026-09-22T16:55:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:608025448worker2 # [ 78.675378] nix-grpc-daemon[781]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5449worker1 # [ 78.698879] nix-grpc-daemon[780]: ts=2026-09-22T16:55:01Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:442365450worker1 # [ 78.704218] nix-grpc-daemon[780]: ts=2026-09-22T16:55:01Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_out=2325451client # [ 78.801685] nix-daemon[1110]: reaped child process 1112, status = succeeded5452client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.94 seconds)5453client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5454client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5455client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5456client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5457client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5458client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5459(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.99 seconds)5460subtest: an input only one worker has still reaches the builder5461client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5462client # this derivation will be built:5463client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5464client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5465client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.25 seconds)5466client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5467client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5468worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5469worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.12 seconds)5470worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5471worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.01 seconds)5472client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25473client # [ 79.464083] nix-daemon[1110]: accepted connection from pid 1193, user root (trusted)5474client # this derivation will be built:5475client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5476worker2 # [ 79.566706] nix-grpc-daemon[781]: ts=2026-09-22T16:55:02Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:608025477worker2 # [ 79.568985] nix-grpc-daemon[781]: ts=2026-09-22T16:55:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608025478client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5479client # copying 1 paths...5480worker2 # [ 79.574571] nix-grpc-daemon[781]: ts=2026-09-22T16:55:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608025481client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5482worker2 # [ 79.578406] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5483worker2 # [ 79.674626] nix-daemon[865]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)5484worker2 # [ 79.687777] nix-daemon[865]: reaped child process 1153, status = succeeded5485lb # [ 79.694934] niks3-server[969]: 2026/09/22 16:55:02 INFO Received uploads request method=POST path=/api/pending_closures5486worker2 # [ 79.698899] nix-grpc-daemon[919]: time=2026-09-22T16:55:02.847Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5487worker2 # [ 79.700847] nix-grpc-daemon[919]: time=2026-09-22T16:55:02.849Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5488lb # [ 79.710151] niks3-server[969]: 2026/09/22 16:55:02 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5489lb # [ 79.713668] niks3-server[969]: 2026/09/22 16:55:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5490lb # [ 79.716506] niks3-server[969]: 2026/09/22 16:55:02 INFO Signed narinfos id=27 count=15491worker2 # [ 79.716462] nix-grpc-daemon[919]: time=2026-09-22T16:55:02.865Z level=INFO msg="Uploading 1 narinfos"5492lb # [ 79.720080] niks3-server[969]: 2026/09/22 16:55:02 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5493lb # [ 79.723922] niks3-server[969]: 2026/09/22 16:55:02 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5494lb # [ 79.727460] niks3-server[969]: 2026/09/22 16:55:02 INFO Completed upload id=275495worker2 # [ 79.727059] nix-grpc-daemon[919]: time=2026-09-22T16:55:02.876Z level=INFO msg="Upload complete. (138ms)"5496worker2 # [ 79.729387] nix-grpc-daemon[781]: ts=2026-09-22T16:55:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_in=1205497lb # [ 79.733545] niks3-server[969]: 2026/09/22 16:55:02 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5498worker2 # [ 79.735297] nix-daemon[865]: reaped child process 1148, status = succeeded5499worker2 # [ 79.739915] nix-grpc-daemon[781]: ts=2026-09-22T16:55:02Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05500worker2 # [ 79.741901] nix-grpc-daemon[781]: ts=2026-09-22T16:55:02Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15501worker2 # [ 79.746220] nix-grpc-daemon[781]: ts=2026-09-22T16:55:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025502worker2 # [ 79.752237] nix-grpc-daemon[781]: ts=2026-09-22T16:55:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608025503client # copying 2 paths to 'lb:50051'...5504worker2 # [ 79.760690] nix-grpc-daemon[781]: ts=2026-09-22T16:55:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608025505client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5506client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5507worker2 # [ 79.776479] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5508worker2 # [ 79.857579] nix-daemon[865]: accepted connection from pid 1159, user nix-grpc-daemon (trusted)5509worker2 # [ 79.871599] nix-daemon[865]: reaped child process 1162, status = succeeded5510lb # [ 79.876702] niks3-server[969]: 2026/09/22 16:55:03 INFO Received uploads request method=POST path=/api/pending_closures5511lb # [ 79.881890] niks3-server[969]: 2026/09/22 16:55:03 INFO Received uploads request method=POST path=/api/pending_closures5512worker2 # [ 79.886232] nix-grpc-daemon[919]: time=2026-09-22T16:55:03.035Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5513worker2 # [ 79.888048] nix-grpc-daemon[919]: time=2026-09-22T16:55:03.036Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5514worker2 # [ 79.890457] nix-grpc-daemon[919]: time=2026-09-22T16:55:03.036Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5515lb # [ 79.907074] niks3-server[969]: 2026/09/22 16:55:03 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5516lb # [ 79.910305] niks3-server[969]: 2026/09/22 16:55:03 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5517lb # [ 79.914680] niks3-server[969]: 2026/09/22 16:55:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5518lb # [ 79.918415] niks3-server[969]: 2026/09/22 16:55:03 INFO Signed narinfos id=28 count=15519lb # [ 79.920214] niks3-server[969]: 2026/09/22 16:55:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5520lb # [ 79.923160] niks3-server[969]: 2026/09/22 16:55:03 INFO Signed narinfos id=29 count=25521worker2 # [ 79.923321] nix-grpc-daemon[919]: time=2026-09-22T16:55:03.072Z level=INFO msg="Uploading 3 narinfos"5522lb # [ 79.925376] niks3-server[969]: 2026/09/22 16:55:03 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5523lb # [ 79.930204] niks3-server[969]: 2026/09/22 16:55:03 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5524lb # [ 79.941889] niks3-server[969]: 2026/09/22 16:55:03 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5525lb # [ 79.947674] niks3-server[969]: 2026/09/22 16:55:03 INFO Completed upload id=285526lb # [ 79.949657] niks3-server[969]: 2026/09/22 16:55:03 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5527lb # [ 79.951914] niks3-server[969]: 2026/09/22 16:55:03 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5528lb # [ 79.953563] niks3-server[969]: 2026/09/22 16:55:03 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5529lb # [ 79.956481] niks3-server[969]: 2026/09/22 16:55:03 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5530lb # [ 79.958769] niks3-server[969]: 2026/09/22 16:55:03 INFO Completed upload id=295531worker2 # [ 79.958518] nix-grpc-daemon[919]: time=2026-09-22T16:55:03.107Z level=INFO msg="Upload complete. (175ms)"5532worker2 # [ 79.962496] nix-grpc-daemon[781]: ts=2026-09-22T16:55:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=2 nar_bytes_in=9765533worker2 # [ 79.966868] nix-grpc-daemon[781]: ts=2026-09-22T16:55:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025534worker2 # [ 79.979090] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5535worker2 # [ 79.985078] nix-daemon[865]: reaped child process 1157, status = succeeded5536client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5537worker2 # [ 79.987858] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5538client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5539client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5540worker2 # [ 81.428782] nix-daemon[865]: reaped child process 1166, status = succeeded5541worker2 # [ 81.508287] nix-daemon[865]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5542worker2 # [ 81.521138] nix-daemon[865]: reaped child process 1185, status = succeeded5543lb # [ 81.528567] niks3-server[969]: 2026/09/22 16:55:04 INFO Received uploads request method=POST path=/api/pending_closures5544worker2 # [ 81.536514] nix-grpc-daemon[919]: time=2026-09-22T16:55:04.685Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5545lb # [ 81.559642] niks3-server[969]: 2026/09/22 16:55:04 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5546lb # [ 81.565269] niks3-server[969]: 2026/09/22 16:55:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5547lb # [ 81.573316] niks3-server[969]: 2026/09/22 16:55:04 INFO Signed narinfos id=30 count=15548worker2 # [ 81.576027] nix-grpc-daemon[919]: time=2026-09-22T16:55:04.725Z level=INFO msg="Uploading 1 narinfos"5549lb # [ 81.581144] niks3-server[969]: 2026/09/22 16:55:04 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5550lb # [ 81.595175] niks3-server[969]: 2026/09/22 16:55:04 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5551lb # [ 81.602248] niks3-server[969]: 2026/09/22 16:55:04 INFO Completed upload id=305552worker2 # [ 81.602102] nix-grpc-daemon[919]: time=2026-09-22T16:55:04.751Z level=INFO msg="Upload complete. (204ms)"5553lb # [ 81.605528] niks3-server[969]: 2026/09/22 16:55:04 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5554worker2 # [ 81.607415] nix-grpc-daemon[781]: ts=2026-09-22T16:55:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15555client # copying 1 paths...5556worker2 # [ 81.617436] nix-grpc-daemon[781]: ts=2026-09-22T16:55:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:608025557client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5558worker2 # [ 81.628222] nix-daemon[865]: reaped child process 1164, status = succeeded5559worker1 # [ 81.627158] nix-grpc-daemon[780]: ts=2026-09-22T16:55:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:442365560worker1 # [ 81.632131] nix-grpc-daemon[780]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5561worker1 # [ 81.638890] nix-grpc-daemon[780]: ts=2026-09-22T16:55:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_out=1205562client # [ 81.759830] nix-daemon[1110]: reaped child process 1196, status = succeeded5563client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.41 seconds)5564client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25565client # [ 81.862122] nix-daemon[1110]: accepted connection from pid 1215, user root (trusted)5566client # this derivation will be built:5567client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5568worker2 # [ 81.966182] nix-grpc-daemon[781]: ts=2026-09-22T16:55:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:608025569client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5570worker1 # [ 81.963665] nix-grpc-daemon[780]: ts=2026-09-22T16:55:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365571worker1 # [ 81.969633] nix-grpc-daemon[857]: time=2026-09-22T16:55:04.284Z level=INFO msg="All 1 paths already cached"5572worker2 # [ 81.987172] nix-grpc-daemon[781]: ts=2026-09-22T16:55:05Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05573worker2 # [ 81.991528] nix-grpc-daemon[781]: ts=2026-09-22T16:55:05Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15574worker1 # [ 82.025166] nix-grpc-daemon[780]: ts=2026-09-22T16:55:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365575worker1 # [ 82.033528] nix-grpc-daemon[780]: ts=2026-09-22T16:55:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365576client # copying 1 paths to 'lb:50051'...5577client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5578worker1 # [ 82.047644] nix-grpc-daemon[780]: ts=2026-09-22T16:55:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442365579worker1 # [ 82.058803] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5580worker1 # [ 82.074169] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5)5581worker1 # [ 82.115190] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 1 ms (attempt 2/5)5582worker1 # [ 82.122109] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 337 ms (attempt 3/5)5583worker1 # [ 82.467533] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 314 ms (attempt 4/5)5584worker1 # [ 82.790883] nix-grpc-daemon[780]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5585worker1 # [ 82.794393] nix-grpc-daemon[780]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5586worker1 # [ 82.799350] nix-grpc-daemon[780]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5587worker1 # [ 82.915089] nix-daemon[845]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)5588worker1 # [ 82.929912] nix-daemon[845]: reaped child process 1213, status = succeeded5589lb # [ 82.944461] niks3-server[969]: 2026/09/22 16:55:06 INFO Received uploads request method=POST path=/api/pending_closures5590worker1 # [ 82.947095] nix-grpc-daemon[857]: time=2026-09-22T16:55:05.261Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5591worker1 # [ 82.948587] nix-grpc-daemon[857]: time=2026-09-22T16:55:05.263Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5592lb # [ 82.976146] niks3-server[969]: 2026/09/22 16:55:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5593worker1 # [ 82.977713] nix-grpc-daemon[857]: time=2026-09-22T16:55:05.292Z level=INFO msg="Uploading 1 narinfos"5594lb # [ 82.983873] niks3-server[969]: 2026/09/22 16:55:06 INFO Signed narinfos id=31 count=15595lb # [ 82.991239] niks3-server[969]: 2026/09/22 16:55:06 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5596lb # [ 82.999420] niks3-server[969]: 2026/09/22 16:55:06 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5597lb # [ 83.008668] niks3-server[969]: 2026/09/22 16:55:06 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5598lb # [ 83.014687] niks3-server[969]: 2026/09/22 16:55:06 INFO Completed upload id=315599worker1 # [ 83.008930] nix-grpc-daemon[857]: time=2026-09-22T16:55:05.323Z level=INFO msg="Upload complete. (190ms)"5600lb # [ 83.016716] niks3-server[969]: 2026/09/22 16:55:06 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5601worker1 # [ 83.012884] nix-grpc-daemon[780]: ts=2026-09-22T16:55:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=0 paths=1 nar_bytes_in=5605602worker1 # [ 83.018389] nix-grpc-daemon[780]: ts=2026-09-22T16:55:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442365603worker1 # [ 83.028123] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5604worker1 # [ 83.038775] nix-daemon[845]: reaped child process 1203, status = succeeded5605worker1 # [ 83.042251] nix-daemon[845]: accepted connection from pid 780, user nix-grpc-daemon (trusted)5606client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5607client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5608client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5609worker1 # [ 84.083378] nix-daemon[845]: reaped child process 1217, status = succeeded5610worker1 # [ 84.161952] nix-daemon[845]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)5611worker1 # [ 84.173326] nix-daemon[845]: reaped child process 1236, status = succeeded5612lb # [ 84.190179] niks3-server[969]: 2026/09/22 16:55:07 INFO Received uploads request method=POST path=/api/pending_closures5613worker1 # [ 84.190154] nix-grpc-daemon[857]: time=2026-09-22T16:55:06.504Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5614lb # [ 84.215852] niks3-server[969]: 2026/09/22 16:55:07 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5615lb # [ 84.228283] niks3-server[969]: 2026/09/22 16:55:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5616worker1 # [ 84.227777] nix-grpc-daemon[857]: time=2026-09-22T16:55:06.542Z level=INFO msg="Uploading 1 narinfos"5617lb # [ 84.234165] niks3-server[969]: 2026/09/22 16:55:07 INFO Signed narinfos id=32 count=15618lb # [ 84.240478] niks3-server[969]: 2026/09/22 16:55:07 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5619lb # [ 84.253863] niks3-server[969]: 2026/09/22 16:55:07 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5620worker1 # [ 84.254232] nix-grpc-daemon[857]: time=2026-09-22T16:55:06.568Z level=INFO msg="Upload complete. (204ms)"5621lb # [ 84.260247] niks3-server[969]: 2026/09/22 16:55:07 INFO Completed upload id=325622worker1 # [ 84.256834] nix-grpc-daemon[780]: ts=2026-09-22T16:55:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44236 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15623lb # [ 84.265871] niks3-server[969]: 2026/09/22 16:55:07 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5624client # copying 1 paths...5625worker1 # [ 84.263979] nix-daemon[845]: reaped child process 1215, status = succeeded5626worker2 # [ 84.272964] nix-grpc-daemon[781]: ts=2026-09-22T16:55:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:608025627worker2 # [ 84.281903] nix-grpc-daemon[781]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5628client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5629worker2 # [ 84.306923] nix-grpc-daemon[781]: ts=2026-09-22T16:55:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:608025630worker2 # [ 84.312123] nix-grpc-daemon[781]: ts=2026-09-22T16:55:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_out=1205631client # [ 84.442805] nix-daemon[1110]: reaped child process 1218, status = succeeded5632client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.69 seconds)5633client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25634client # [ 84.549966] nix-daemon[1110]: accepted connection from pid 1237, user root (trusted)5635client # this derivation will be built:5636client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5637worker2 # [ 84.648953] nix-grpc-daemon[781]: ts=2026-09-22T16:55:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:608025638worker1 # [ 84.645881] nix-grpc-daemon[780]: ts=2026-09-22T16:55:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442365639client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5640worker1 # [ 84.652294] nix-grpc-daemon[857]: time=2026-09-22T16:55:06.966Z level=INFO msg="All 1 paths already cached"5641worker2 # [ 84.670194] nix-grpc-daemon[781]: ts=2026-09-22T16:55:07Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05642worker2 # [ 84.676468] nix-grpc-daemon[781]: ts=2026-09-22T16:55:07Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15643worker2 # [ 84.681467] nix-grpc-daemon[781]: ts=2026-09-22T16:55:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025644worker2 # [ 84.692952] nix-grpc-daemon[781]: ts=2026-09-22T16:55:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608025645worker2 # [ 84.705658] nix-grpc-daemon[919]: time=2026-09-22T16:55:07.854Z level=INFO msg="All 1 paths already cached"5646client # copying 1 paths to 'lb:50051'...5647client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5648worker2 # [ 84.713440] nix-grpc-daemon[781]: ts=2026-09-22T16:55:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608025649worker2 # [ 84.724691] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5650worker2 # [ 84.818498] nix-daemon[865]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)5651worker2 # [ 84.831549] nix-daemon[865]: reaped child process 1197, status = succeeded5652lb # [ 84.837267] niks3-server[969]: 2026/09/22 16:55:08 INFO Received uploads request method=POST path=/api/pending_closures5653worker2 # [ 84.842156] nix-grpc-daemon[919]: time=2026-09-22T16:55:07.991Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5654worker2 # [ 84.844238] nix-grpc-daemon[919]: time=2026-09-22T16:55:07.992Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5655lb # [ 84.853271] niks3-server[969]: 2026/09/22 16:55:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5656lb # [ 84.855277] niks3-server[969]: 2026/09/22 16:55:08 INFO Signed narinfos id=33 count=15657worker2 # [ 84.855294] nix-grpc-daemon[919]: time=2026-09-22T16:55:08.004Z level=INFO msg="Uploading 1 narinfos"5658lb # [ 84.861885] niks3-server[969]: 2026/09/22 16:55:08 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5659lb # [ 84.865176] niks3-server[969]: 2026/09/22 16:55:08 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5660lb # [ 84.868326] niks3-server[969]: 2026/09/22 16:55:08 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5661lb # [ 84.872365] niks3-server[969]: 2026/09/22 16:55:08 INFO Completed upload id=335662worker2 # [ 84.872106] nix-grpc-daemon[919]: time=2026-09-22T16:55:08.021Z level=INFO msg="Upload complete. (136ms)"5663lb # [ 84.873862] niks3-server[969]: 2026/09/22 16:55:08 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5664worker2 # [ 84.874231] nix-grpc-daemon[781]: ts=2026-09-22T16:55:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_in=5605665worker2 # [ 84.878849] nix-grpc-daemon[781]: ts=2026-09-22T16:55:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608025666worker2 # [ 84.885191] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5667worker2 # [ 84.887607] nix-daemon[865]: reaped child process 1192, status = succeeded5668worker2 # [ 84.892904] nix-daemon[865]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5669client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5670client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5671client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5672worker2 # [ 85.909955] nix-daemon[865]: reaped child process 1201, status = succeeded5673worker2 # [ 85.992639] nix-daemon[865]: accepted connection from pid 1217, user nix-grpc-daemon (trusted)5674worker2 # [ 86.006883] nix-daemon[865]: reaped child process 1220, status = succeeded5675lb # [ 86.016575] niks3-server[969]: 2026/09/22 16:55:09 INFO Received uploads request method=POST path=/api/pending_closures5676worker2 # [ 86.025215] nix-grpc-daemon[919]: time=2026-09-22T16:55:09.174Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5677lb # [ 86.045950] niks3-server[969]: 2026/09/22 16:55:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5678worker2 # [ 86.053070] nix-grpc-daemon[919]: time=2026-09-22T16:55:09.201Z level=INFO msg="Uploading 1 narinfos"5679lb # [ 86.053214] niks3-server[969]: 2026/09/22 16:55:09 INFO Signed narinfos id=34 count=15680lb # [ 86.060939] niks3-server[969]: 2026/09/22 16:55:09 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5681lb # [ 86.068376] niks3-server[969]: 2026/09/22 16:55:09 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5682lb # [ 86.081743] niks3-server[969]: 2026/09/22 16:55:09 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5683lb # [ 86.086570] niks3-server[969]: 2026/09/22 16:55:09 INFO Completed upload id=345684worker2 # [ 86.089159] nix-grpc-daemon[919]: time=2026-09-22T16:55:09.238Z level=INFO msg="Upload complete. (189ms)"5685lb # [ 86.093410] niks3-server[969]: 2026/09/22 16:55:09 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5686worker2 # [ 86.094378] nix-grpc-daemon[781]: ts=2026-09-22T16:55:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15687client # copying 1 paths...5688worker1 # [ 86.100985] nix-grpc-daemon[780]: ts=2026-09-22T16:55:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442365689worker2 # [ 86.109153] nix-daemon[865]: reaped child process 1199, status = succeeded5690worker1 # [ 86.107917] nix-grpc-daemon[780]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5691client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5692worker2 # [ 86.136040] nix-grpc-daemon[781]: ts=2026-09-22T16:55:09Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:608025693worker2 # [ 86.138479] nix-grpc-daemon[781]: ts=2026-09-22T16:55:09Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60802 duration_s=0 paths=1 nar_bytes_out=1205694client # [ 86.269891] nix-daemon[1110]: reaped child process 1240, status = succeeded5695client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.83 seconds)5696(finished: subtest: an input only one worker has still reaches the builder, in 7.41 seconds)5697subtest: upload whose reference the worker lacks is completed from the cache5698client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5699client # this derivation will be built:5700client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5701client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5702client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.26 seconds)5703client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5704client # this derivation will be built:5705client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5706lb # [ 86.632772] envoy[915]: conn peer=[2001:db8:1::1]:58942 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=635707lb # [ 86.636472] envoy[915]: conn peer=[2001:db8:1::1]:58930 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3863 tx=2140 ms=19805708lb # [ 86.639952] envoy[915]: conn peer=[2001:db8:1::1]:58956 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4311 tx=2395 ms=17855709lb # [ 86.643477] envoy[915]: conn peer=[2001:db8:1::1]:58970 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=195710lb # [ 86.647064] envoy[915]: conn peer=[2001:db8:1::1]:58976 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4221 tx=2027 ms=21265711lb # [ 86.650561] envoy[915]: conn peer=[2001:db8:1::1]:58980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=595712lb # [ 86.653929] envoy[915]: conn peer=[2001:db8:1::1]:58984 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3744 tx=2123 ms=23985713lb # [ 86.657917] envoy[915]: conn peer=[2001:db8:1::1]:35756 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=575714lb # [ 86.661303] envoy[915]: conn peer=[2001:db8:1::1]:35770 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2082 ms=15415715lb # [ 86.664728] envoy[915]: conn peer=[2001:db8:1::1]:35780 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=555716client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5717client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.25 seconds)5718client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5719client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds)5720worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5721worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.10 seconds)5722worker2 # [ 87.018539] systemd[1]: Stopping Nix worker-protocol over gRPC...5723worker2 # [ 87.869183] nix-grpc-daemon[781]: ts=2026-09-22T16:55:11Z level=info event=signal_exit5724lb # [ 88.847940] niks3-server[969]: 2026/09/22 16:55:12 INFO lead: released remote=192.168.1.4:418545725lb # [ 89.838517] niks3-server[969]: 2026/09/22 16:55:13 INFO lead: acquired remote=192.168.1.3:337965726worker1 # [ 89.836875] nix-grpc-daemon[780]: ts=2026-09-22T16:55:12Z level=info event=scheduler_take_over5727worker2 # [ 89.847908] nix-grpc-daemon[781]: ts=2026-09-22T16:55:12Z level=info event=scheduler_restarting5728worker2 # [ 89.855150] nix-grpc-daemon[781]: ts=2026-09-22T16:55:13Z level=info event=scheduler_restarting addr=lb:500515729worker1 # [ 89.849772] nix-grpc-daemon[780]: ts=2026-09-22T16:55:12Z level=info event=scheduler_restarting addr=lb:500515730worker2 # [ 89.860931] nix-grpc-daemon[781]: ts=2026-09-22T16:55:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:607885731worker2 # [ 89.864556] nix-grpc-daemon[781]: ts=2026-09-22T16:55:13Z level=info event=worker_gone addr=192.168.1.4:500515732worker2 # [ 89.868184] nix-grpc-daemon[781]: ts=2026-09-22T16:55:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:607885733worker2 # [ 89.871622] nix-grpc-daemon[781]: ts=2026-09-22T16:55:13Z level=info event=worker_gone addr=192.168.1.3:500515734worker2 # [ 89.893864] nix-daemon[865]: reaped child process 867, status = succeeded5735worker2 # [ 89.997512] nix-grpc-daemon[781]: E0000 00:00:1790096113.146356 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5736worker2 # [ 90.002511] nix-grpc-daemon[781]: E0000 00:00:1790096113.146543 826 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5737worker1 # [ 90.050942] nix-grpc-daemon[780]: ts=2026-09-22T16:55:12Z level=info event=scheduler_connected addr=lb:500515738worker1 # [ 90.062659] nix-grpc-daemon[780]: ts=2026-09-22T16:55:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:377985739worker2 # [ 90.069867] nix-grpc-daemon[781]: ts=2026-09-22T16:55:13Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5740worker1 # [ 90.066759] nix-grpc-daemon[780]: ts=2026-09-22T16:55:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05741worker2 # [ 91.967092] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5742worker2 # [ 91.968726] systemd[1]: Stopped Nix worker-protocol over gRPC.5743worker2 # [ 91.970719] systemd[1]: nix-grpc-daemon.service: Consumed 1.886s CPU time over 1min 15.253s wall clock time, 63.8M memory peak, 91.1K incoming IP traffic, 236.3K outgoing IP traffic.5744worker2 # [ 91.975422] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5745worker2 # [ 91.977284] systemd[1]: Closed Nix worker-protocol over gRPC.5746worker2 # [ 91.978440] systemd[1]: nix-grpc-daemon.socket: Consumed 301us CPU time over 1min 23.359s wall clock time, 256K memory peak, 107.4K incoming IP traffic, 90.1K outgoing IP traffic.5747lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5748lb: must succeed: curl -sf localhost:9901/clusters5749lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5750lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.03 seconds)5751client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25752worker1 # [ 92.150368] nix-grpc-daemon[780]: ts=2026-09-22T16:55:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442365753client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5754worker1 # [ 92.154221] nix-grpc-daemon[780]: ts=2026-09-22T16:55:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442365755client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5756worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5757worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5758worker2 # [ 92.229871] systemd[1]: Listening on Nix worker-protocol over gRPC.5759worker1 # [ 92.247372] systemd[1]: Stopping Nix worker-protocol over gRPC...5760worker1 # [ 92.820832] nix-grpc-daemon[780]: ts=2026-09-22T16:55:15Z level=info event=signal_exit5761lb # [ 92.849454] niks3-server[969]: 2026/09/22 16:55:16 INFO lead: released remote=192.168.1.3:337965762worker2 # [ 92.969910] systemd[1]: Starting scheduler-token.service...5763worker2 # [ 93.029873] systemd[1]: scheduler-token.service: Deactivated successfully.5764worker2 # [ 93.033136] systemd[1]: Finished scheduler-token.service.5765worker2 # [ 93.035758] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 63ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5766worker2 # [ 93.044384] systemd[1]: Starting Nix worker-protocol over gRPC...5767worker2 # [ 93.130820] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5768worker2 # [ 93.135319] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5769worker2 # [ 93.140210] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15770worker2 # [ 93.142174] nix-grpc-daemon[1244]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5771worker2 # [ 93.143483] nix-grpc-daemon[1244]: E0000 00:00:1790096116.287741 1244 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5772worker2 # [ 93.145685] nix-grpc-daemon[1244]: E0000 00:00:1790096116.289160 1244 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5773worker2 # [ 93.154967] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5774worker2 # [ 93.157329] systemd[1]: Started Nix worker-protocol over gRPC.5775worker2 # [ 93.158969] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=scheduler_take_over5776lb # [ 93.159417] niks3-server[969]: 2026/09/22 16:55:16 INFO lead: acquired remote=192.168.1.4:501305777worker2 # [ 93.161149] nix-grpc-daemon[1244]: E0000 00:00:1790096116.309420 1284 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5778worker2 # [ 93.175140] nix-grpc-daemon[1244]: E0000 00:00:1790096116.324228 1284 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5779worker2 # [ 93.180057] nix-grpc-daemon[1244]: E0000 00:00:1790096116.326426 1284 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5780worker2 # [ 93.189057] nix-grpc-daemon[1244]: E0000 00:00:1790096116.338120 1284 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5781worker2 # [ 93.197629] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=scheduler_connected addr=lb:500515782worker2 # [ 93.199652] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5783worker2 # [ 93.401079] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=scheduler_connected addr=lb:500515784worker2 # [ 93.405207] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5785worker2 # [ 93.606553] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=scheduler_connected addr=lb:500515786worker2 # [ 93.610617] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5787worker2 # [ 93.812304] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=scheduler_connected addr=lb:500515788worker2 # [ 93.816144] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5789worker1 # [ 93.847491] nix-grpc-daemon[780]: ts=2026-09-22T16:55:16Z level=info event=scheduler_restarting5790worker1 # [ 93.850866] nix-grpc-daemon[780]: ts=2026-09-22T16:55:16Z level=info event=scheduler_restarting addr=lb:500515791worker1 # [ 93.854232] nix-grpc-daemon[780]: ts=2026-09-22T16:55:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:377985792worker1 # [ 93.857811] nix-grpc-daemon[780]: ts=2026-09-22T16:55:16Z level=info event=worker_gone addr=192.168.1.3:500515793worker1 # [ 93.878112] nix-daemon[845]: reaped child process 847, status = succeeded5794worker1 # [ 93.949407] nix-grpc-daemon[780]: E0000 00:00:1790096116.263874 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5795worker1 # [ 93.954789] nix-grpc-daemon[780]: E0000 00:00:1790096116.269310 825 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5796worker2 # [ 94.017402] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:17Z level=info event=scheduler_connected addr=lb:500515797worker2 # [ 94.021534] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5798worker1 # [ 94.057814] nix-grpc-daemon[780]: ts=2026-09-22T16:55:16Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5799worker2 # [ 94.231157] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:17Z level=info event=scheduler_connected addr=lb:500515800worker2 # [ 94.235824] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:17Z level=info event=oidc_jwks provider=mock keys=15801worker2 # [ 94.238714] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:529245802worker2 # [ 94.242648] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05803worker1 # [ 95.926868] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5804worker1 # [ 95.928547] systemd[1]: Stopped Nix worker-protocol over gRPC.5805worker1 # [ 95.930227] systemd[1]: nix-grpc-daemon.service: Consumed 2.268s CPU time over 1min 19.302s wall clock time, 66M memory peak, 100.6K incoming IP traffic, 187.3K outgoing IP traffic.5806worker1 # [ 95.934820] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5807worker1 # [ 95.936589] systemd[1]: Closed Nix worker-protocol over gRPC.5808worker1 # [ 95.937898] systemd[1]: nix-grpc-daemon.socket: Consumed 303us CPU time over 1min 27.475s wall clock time, 256K memory peak, 105K incoming IP traffic, 92.1K outgoing IP traffic.5809lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5810lb: must succeed: curl -sf localhost:9901/clusters5811lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5812lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.04 seconds)5813client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25814worker2 # [ 96.121466] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529385815worker2 # [ 96.133317] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5816worker2 # [ 96.150540] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 1/5)5817worker2 # [ 96.229285] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 2/5)5818worker2 # [ 96.406346] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 3/5)5819worker2 # [ 96.577710] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 503 ms (attempt 4/5)5820lb # [ 96.633469] envoy[915]: conn peer=[2001:db8:1::4]:52728 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=55356 tx=2969 ms=731645821lb # [ 96.637647] envoy[915]: conn peer=[2001:db8:1::1]:35790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=125822lb # [ 96.641267] envoy[915]: conn peer=[2001:db8:1::3]:36718 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15478 tx=3279 ms=772065823worker2 # [ 97.083273] nix-grpc-daemon[1244]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5824worker2 # [ 97.086687] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5825client # copying 2 paths...5826client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5827worker2 # [ 97.108944] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529385828client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5829worker2 # [ 97.122278] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5830worker2 # [ 97.143235] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:20Z level=info event=niks3_push_started pid=13045831worker2 # [ 97.257357] nix-grpc-daemon[1304]: time=2026-09-22T16:55:20.405Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5832worker2 # [ 97.352157] nix-daemon[865]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5833worker2 # [ 97.366521] nix-daemon[865]: reaped child process 1319, status = succeeded5834lb # [ 97.373677] niks3-server[969]: 2026/09/22 16:55:20 INFO Received uploads request method=POST path=/api/pending_closures5835lb # [ 97.386084] niks3-server[969]: 2026/09/22 16:55:20 INFO Received uploads request method=POST path=/api/pending_closures5836worker2 # [ 97.393469] nix-grpc-daemon[1304]: time=2026-09-22T16:55:20.542Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5837worker2 # [ 97.395192] nix-grpc-daemon[1304]: time=2026-09-22T16:55:20.544Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5838worker2 # [ 97.397230] nix-grpc-daemon[1304]: time=2026-09-22T16:55:20.546Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5839lb # [ 97.441073] niks3-server[969]: 2026/09/22 16:55:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5840lb # [ 97.450301] niks3-server[969]: 2026/09/22 16:55:20 INFO Signed narinfos id=35 count=15841lb # [ 97.452776] niks3-server[969]: 2026/09/22 16:55:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5842lb # [ 97.461476] niks3-server[969]: 2026/09/22 16:55:20 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5843lb # [ 97.465703] niks3-server[969]: 2026/09/22 16:55:20 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5844lb # [ 97.468262] niks3-server[969]: 2026/09/22 16:55:20 INFO Signed narinfos id=36 count=25845worker2 # [ 97.469206] nix-grpc-daemon[1304]: time=2026-09-22T16:55:20.616Z level=INFO msg="Uploading 3 narinfos"5846lb # [ 97.480718] niks3-server[969]: 2026/09/22 16:55:20 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5847lb # [ 97.483449] niks3-server[969]: 2026/09/22 16:55:20 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5848lb # [ 97.492871] niks3-server[969]: 2026/09/22 16:55:20 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5849lb # [ 97.495237] niks3-server[969]: 2026/09/22 16:55:20 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5850lb # [ 97.498511] niks3-server[969]: 2026/09/22 16:55:20 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5851lb # [ 97.504566] niks3-server[969]: 2026/09/22 16:55:20 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5852lb # [ 97.506877] niks3-server[969]: 2026/09/22 16:55:20 INFO Completed upload id=355853lb # [ 97.508219] niks3-server[969]: 2026/09/22 16:55:20 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5854lb # [ 97.511396] niks3-server[969]: 2026/09/22 16:55:20 INFO Completed upload id=365855worker2 # [ 97.511453] nix-grpc-daemon[1304]: time=2026-09-22T16:55:20.660Z level=INFO msg="Upload complete. (254ms)"5856worker2 # [ 97.516814] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52938 duration_s=0 paths=2 nar_bytes_in=3125857worker2 # [ 97.531360] nix-daemon[865]: reaped child process 1302, status = succeeded5858client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.54 seconds)5859worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5860worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5861worker1 # [ 97.574078] systemd[1]: Listening on Nix worker-protocol over gRPC.5862worker1 # [ 97.577187] systemd[1]: Starting Nix worker-protocol over gRPC...5863worker1 # [ 97.653886] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5864worker1 # [ 97.655850] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5865worker1 # [ 97.658945] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15866worker1 # [ 97.663072] nix-grpc-daemon[1260]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5867worker1 # [ 97.664697] nix-grpc-daemon[1260]: E0000 00:00:1790096119.977663 1260 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5868worker1 # [ 97.670861] nix-grpc-daemon[1260]: E0000 00:00:1790096119.983161 1260 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5869worker1 # [ 97.674345] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5870worker1 # [ 97.678804] systemd[1]: Started Nix worker-protocol over gRPC.5871worker1 # [ 97.682107] nix-grpc-daemon[1260]: E0000 00:00:1790096119.995412 1300 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5872worker1 # [ 97.694535] nix-grpc-daemon[1260]: E0000 00:00:1790096120.009234 1300 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5873worker1 # [ 97.698309] nix-grpc-daemon[1260]: E0000 00:00:1790096120.011625 1300 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5874lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5875lb: must succeed: curl -sf localhost:9901/clusters5876worker1 # [ 97.713815] nix-grpc-daemon[1260]: E0000 00:00:1790096120.028495 1300 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5877worker1 # [ 97.723609] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:20Z level=info event=scheduler_connected addr=lb:500515878worker2 # [ 97.731179] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:20Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:529245879lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5880worker2 # [ 97.735923] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:20Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05881lb: must succeed: curl -sf localhost:9901/clusters5882lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5883lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.08 seconds)5884(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.53 seconds)5885subtest: a niks3 restart does not cost the leader its role5886lb: must succeed: curl -sf localhost:9901/clusters5887lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5888lb: must succeed: curl -sf localhost:9901/clusters5889lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5890lb # [ 98.853659] systemd[1]: Stopping niks3 server...5891lb # [ 98.855553] niks3-server[969]: 2026/09/22 16:55:22 INFO Shutdown signal received, draining in-flight requests timeout=10s5892lb # [ 98.858886] niks3-server[969]: 2026/09/22 16:55:22 INFO lead: released remote=192.168.1.4:501305893worker2 # [ 98.859928] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=info event=scheduler_lock_lost status=2005894worker1 # [ 98.855723] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:21Z level=info event=scheduler_lock_lost status=2005895lb # [ 98.874378] systemd[1]: niks3.service: Deactivated successfully.5896lb # [ 98.875875] systemd[1]: Stopped niks3 server.5897lb # [ 98.876693] systemd[1]: niks3.service: Consumed 672ms CPU time over 1min 19.253s wall clock time, 29M memory peak, 100.1K incoming IP traffic, 143.3K outgoing IP traffic.5898lb # [ 98.884518] systemd[1]: Starting niks3 server...5899lb # [ 98.936171] niks3-server[1111]: 2026/09/22 16:55:22 goose: no migrations to run. current version: 202609200000005900lb # [ 98.940883] niks3-server[1111]: 2026/09/22 16:55:22 OK 1_commit_pending_closure.sql (6.02ms)5901lb # [ 98.944453] niks3-server[1111]: 2026/09/22 16:55:22 OK 2_object_stats_trigger.sql (1.91ms)5902lb # [ 98.945561] niks3-server[1111]: 2026/09/22 16:55:22 goose: up to current file version: 25903lb # [ 98.947284] niks3-server[1111]: 2026/09/22 16:55:22 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5904lb # [ 98.958887] niks3-server[1111]: 2026/09/22 16:55:22 INFO Read proxy enabled — serving cache objects from S35905lb # [ 98.960186] niks3-server[1111]: 2026/09/22 16:55:22 INFO Using socket-activated listener address=0.0.0.0:57515906lb # [ 98.962146] systemd[1]: Started niks3 server.5907lb # [ 98.962810] niks3-server[1111]: 2026/09/22 16:55:22 INFO systemd watchdog enabled interval=15s5908lb # [ 98.964369] niks3-server[1111]: 2026/09/22 16:55:22 INFO Starting HTTP server address=0.0.0.0:57515909lb: waiting for unit niks3.service5910lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5911lb: must succeed: curl -sf localhost:9901/clusters5912lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5913worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5914worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5915lb: must succeed: curl -sf localhost:9901/clusters5916lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5917lb: must succeed: curl -sf localhost:9901/clusters5918lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5919worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5920worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5921client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25922worker2 # [ 99.290183] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:529385923client # don't know how to build these paths:5924client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5925worker2 # [ 99.298644] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529385926worker2 # [ 99.317588] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05927worker2 # [ 99.322917] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15928worker2 # [ 99.361098] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529385929worker2 # [ 99.371838] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529385930client # copying 1 paths to 'lb:50051'...5931client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5932worker2 # [ 99.382413] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529385933worker2 # [ 99.393340] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5934worker2 # [ 99.500168] nix-daemon[865]: accepted connection from pid 1336, user nix-grpc-daemon (trusted)5935worker2 # [ 99.514835] nix-daemon[865]: reaped child process 1339, status = succeeded5936lb # [ 99.522250] niks3-server[1111]: 2026/09/22 16:55:22 INFO Received uploads request method=POST path=/api/pending_closures5937worker2 # [ 99.527726] nix-grpc-daemon[1304]: time=2026-09-22T16:55:22.676Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5938worker2 # [ 99.529970] nix-grpc-daemon[1304]: time=2026-09-22T16:55:22.678Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5939lb # [ 99.547873] niks3-server[1111]: 2026/09/22 16:55:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5940lb # [ 99.555760] niks3-server[1111]: 2026/09/22 16:55:22 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5941lb # [ 99.575060] niks3-server[1111]: 2026/09/22 16:55:22 INFO Signed narinfos id=37 count=15942worker2 # [ 99.576863] nix-grpc-daemon[1304]: time=2026-09-22T16:55:22.725Z level=INFO msg="Uploading 1 narinfos"5943lb # [ 99.581678] niks3-server[1111]: 2026/09/22 16:55:22 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5944lb # [ 99.584421] niks3-server[1111]: 2026/09/22 16:55:22 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5945lb # [ 99.590401] niks3-server[1111]: 2026/09/22 16:55:22 INFO Completed upload id=375946worker2 # [ 99.590374] nix-grpc-daemon[1304]: time=2026-09-22T16:55:22.739Z level=INFO msg="Upload complete. (184ms)"5947lb # [ 99.594552] niks3-server[1111]: 2026/09/22 16:55:22 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5948worker2 # [ 99.594735] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52938 duration_s=0 paths=1 nar_bytes_in=5365949worker2 # [ 99.600086] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529385950worker2 # [ 99.610042] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5951worker2 # [ 99.615279] nix-daemon[865]: reaped child process 1334, status = succeeded5952worker2 # [ 99.619694] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5953client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5954client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5955lb # [ 99.867657] niks3-server[1111]: 2026/09/22 16:55:23 INFO lead: acquired remote=192.168.1.4:502485956client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5957client # farm-a> LOG-farm-a-after-niks3-restart5958worker2 # [ 100.660160] nix-daemon[865]: reaped child process 1343, status = succeeded5959worker2 # [ 100.745100] nix-daemon[865]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5960worker2 # [ 100.760343] nix-daemon[865]: reaped child process 1364, status = succeeded5961lb # [ 100.764224] niks3-server[1111]: 2026/09/22 16:55:23 INFO Received uploads request method=POST path=/api/pending_closures5962worker2 # [ 100.769635] nix-grpc-daemon[1304]: time=2026-09-22T16:55:23.918Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5963worker2 # [ 100.772058] nix-grpc-daemon[1304]: time=2026-09-22T16:55:23.920Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5964lb # [ 100.787651] niks3-server[1111]: 2026/09/22 16:55:23 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5965lb # [ 100.793938] niks3-server[1111]: 2026/09/22 16:55:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5966worker2 # [ 100.798628] nix-grpc-daemon[1304]: time=2026-09-22T16:55:23.947Z level=INFO msg="Uploading 1 narinfos"5967lb # [ 100.799418] niks3-server[1111]: 2026/09/22 16:55:23 INFO Signed narinfos id=38 count=15968lb # [ 100.807696] niks3-server[1111]: 2026/09/22 16:55:24 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5969lb # [ 100.811619] niks3-server[1111]: 2026/09/22 16:55:24 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5970lb # [ 100.816830] niks3-server[1111]: 2026/09/22 16:55:24 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5971lb # [ 100.820580] niks3-server[1111]: 2026/09/22 16:55:24 INFO Completed upload id=385972worker2 # [ 100.820430] nix-grpc-daemon[1304]: time=2026-09-22T16:55:23.969Z level=INFO msg="Upload complete. (192ms)"5973lb # [ 100.822730] niks3-server[1111]: 2026/09/22 16:55:24 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5974worker2 # [ 100.823442] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52938 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15975worker2 # [ 100.831447] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:23Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05976worker2 # [ 100.833483] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:23Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15977worker2 # [ 100.838900] nix-daemon[865]: reaped child process 1341, status = succeeded5978worker2 # [ 100.875263] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529385979worker2 # [ 100.885678] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529385980worker2 # [ 100.898158] nix-grpc-daemon[1304]: time=2026-09-22T16:55:24.047Z level=INFO msg="All 1 paths already cached"5981client # copying 1 paths to 'lb:50051'...5982client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5983worker2 # [ 100.905894] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529385984worker2 # [ 100.912877] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5985worker2 # [ 101.011906] nix-daemon[865]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)5986worker2 # [ 101.026852] nix-daemon[865]: reaped child process 1373, status = succeeded5987lb # [ 101.034983] niks3-server[1111]: 2026/09/22 16:55:24 INFO Received uploads request method=POST path=/api/pending_closures5988worker2 # [ 101.042810] nix-grpc-daemon[1304]: time=2026-09-22T16:55:24.191Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5989worker2 # [ 101.044936] nix-grpc-daemon[1304]: time=2026-09-22T16:55:24.193Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5990lb # [ 101.067301] niks3-server[1111]: 2026/09/22 16:55:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5991lb # [ 101.079211] niks3-server[1111]: 2026/09/22 16:55:24 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5992lb # [ 101.087971] niks3-server[1111]: 2026/09/22 16:55:24 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5993worker2 # [ 101.099308] nix-grpc-daemon[1304]: time=2026-09-22T16:55:24.248Z level=INFO msg="Uploading 1 narinfos"5994lb # [ 101.099457] niks3-server[1111]: 2026/09/22 16:55:24 INFO Signed narinfos id=39 count=15995lb # [ 101.111337] niks3-server[1111]: 2026/09/22 16:55:24 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5996lb # [ 101.115246] niks3-server[1111]: 2026/09/22 16:55:24 INFO Completed upload id=395997worker2 # [ 101.115251] nix-grpc-daemon[1304]: time=2026-09-22T16:55:24.264Z level=INFO msg="Upload complete. (190ms)"5998lb # [ 101.120119] niks3-server[1111]: 2026/09/22 16:55:24 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5999worker2 # [ 101.119350] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52938 duration_s=0 paths=1 nar_bytes_in=6566000worker2 # [ 101.125748] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529386001worker2 # [ 101.136365] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)6002worker2 # [ 101.147231] nix-daemon[865]: reaped child process 1368, status = succeeded6003worker2 # [ 101.152390] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)6004client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6005client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6006client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6007client # farm-b> LOG-farm-b-after-niks3-restart6008worker2 # [ 101.991575] nix-daemon[865]: reaped child process 1377, status = succeeded6009worker2 # [ 102.080574] nix-daemon[865]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)6010worker2 # [ 102.095903] nix-daemon[865]: reaped child process 1396, status = succeeded6011lb # [ 102.104532] niks3-server[1111]: 2026/09/22 16:55:25 INFO Received uploads request method=POST path=/api/pending_closures6012worker2 # [ 102.115335] nix-grpc-daemon[1304]: time=2026-09-22T16:55:25.264Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6013worker2 # [ 102.117148] nix-grpc-daemon[1304]: time=2026-09-22T16:55:25.265Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6014lb # [ 102.144160] niks3-server[1111]: 2026/09/22 16:55:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6015worker2 # [ 102.158312] nix-grpc-daemon[1304]: time=2026-09-22T16:55:25.307Z level=INFO msg="Uploading 1 narinfos"6016lb # [ 102.158455] niks3-server[1111]: 2026/09/22 16:55:25 INFO Signed narinfos id=40 count=16017lb # [ 102.167636] niks3-server[1111]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6018lb # [ 102.170680] niks3-server[1111]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6019lb # [ 102.177251] niks3-server[1111]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6020lb # [ 102.185238] niks3-server[1111]: 2026/09/22 16:55:25 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6021lb # [ 102.190084] niks3-server[1111]: 2026/09/22 16:55:25 INFO Completed upload id=406022worker2 # [ 102.190090] nix-grpc-daemon[1304]: time=2026-09-22T16:55:25.339Z level=INFO msg="Upload complete. (230ms)"6023lb # [ 102.192513] niks3-server[1111]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6024worker2 # [ 102.199277] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52938 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16025worker2 # [ 102.207084] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:25Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06026worker2 # [ 102.212224] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:25Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=16027worker2 # [ 102.221190] nix-daemon[865]: reaped child process 1375, status = succeeded6028worker2 # [ 102.251709] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529386029worker2 # [ 102.264216] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529386030worker2 # [ 102.276817] nix-grpc-daemon[1304]: time=2026-09-22T16:55:25.425Z level=INFO msg="All 2 paths already cached"6031client # copying 1 paths to 'lb:50051'...6032client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6033worker2 # [ 102.285170] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529386034worker2 # [ 102.295164] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)6035worker2 # [ 102.396931] nix-daemon[865]: accepted connection from pid 1402, user nix-grpc-daemon (trusted)6036worker2 # [ 102.411697] nix-daemon[865]: reaped child process 1405, status = succeeded6037lb # [ 102.420105] niks3-server[1111]: 2026/09/22 16:55:25 INFO Received uploads request method=POST path=/api/pending_closures6038worker2 # [ 102.427976] nix-grpc-daemon[1304]: time=2026-09-22T16:55:25.577Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6039worker2 # [ 102.429593] nix-grpc-daemon[1304]: time=2026-09-22T16:55:25.578Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6040lb # [ 102.451451] niks3-server[1111]: 2026/09/22 16:55:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6041worker2 # [ 102.460819] nix-grpc-daemon[1304]: time=2026-09-22T16:55:25.609Z level=INFO msg="Uploading 1 narinfos"6042lb # [ 102.460981] niks3-server[1111]: 2026/09/22 16:55:25 INFO Signed narinfos id=41 count=16043lb # [ 102.466282] niks3-server[1111]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6044lb # [ 102.471318] niks3-server[1111]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6045lb # [ 102.484254] niks3-server[1111]: 2026/09/22 16:55:25 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6046worker2 # [ 102.490789] nix-grpc-daemon[1304]: time=2026-09-22T16:55:25.639Z level=INFO msg="Upload complete. (184ms)"6047lb # [ 102.491572] niks3-server[1111]: 2026/09/22 16:55:25 INFO Completed upload id=416048worker2 # [ 102.493504] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52938 duration_s=0 paths=1 nar_bytes_in=7846049lb # [ 102.494214] niks3-server[1111]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6050worker2 # [ 102.500173] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529386051worker2 # [ 102.507351] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)6052worker2 # [ 102.510215] nix-daemon[865]: reaped child process 1400, status = succeeded6053client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6054client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6055worker2 # [ 102.516298] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)6056client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6057client # farm-top> LOG-farm-top-after-niks3-restart6058worker2 # [ 103.534546] nix-daemon[865]: reaped child process 1409, status = succeeded6059worker2 # [ 103.617751] nix-daemon[865]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)6060worker2 # [ 103.632648] nix-daemon[865]: reaped child process 1429, status = succeeded6061lb # [ 103.640981] niks3-server[1111]: 2026/09/22 16:55:26 INFO Received uploads request method=POST path=/api/pending_closures6062worker2 # [ 103.651634] nix-grpc-daemon[1304]: time=2026-09-22T16:55:26.800Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6063worker2 # [ 103.654338] nix-grpc-daemon[1304]: time=2026-09-22T16:55:26.802Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6064lb # [ 103.683132] niks3-server[1111]: 2026/09/22 16:55:26 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6065lb # [ 103.687382] niks3-server[1111]: 2026/09/22 16:55:26 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6066lb # [ 103.694421] niks3-server[1111]: 2026/09/22 16:55:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6067lb # [ 103.700888] niks3-server[1111]: 2026/09/22 16:55:26 INFO Signed narinfos id=42 count=16068worker2 # [ 103.703263] nix-grpc-daemon[1304]: time=2026-09-22T16:55:26.852Z level=INFO msg="Uploading 1 narinfos"6069lb # [ 103.704799] niks3-server[1111]: 2026/09/22 16:55:26 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6070lb # [ 103.711493] niks3-server[1111]: 2026/09/22 16:55:26 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6071lb # [ 103.718085] niks3-server[1111]: 2026/09/22 16:55:26 INFO Completed upload id=426072worker2 # [ 103.717913] nix-grpc-daemon[1304]: time=2026-09-22T16:55:26.866Z level=INFO msg="Upload complete. (194ms)"6073lb # [ 103.719821] niks3-server[1111]: 2026/09/22 16:55:26 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6074worker2 # [ 103.724655] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52938 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16075worker2 # [ 103.740388] nix-daemon[865]: reaped child process 1407, status = succeeded6076client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 4.61 seconds)6077client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6078client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.11 seconds)6079(finished: subtest: a niks3 restart does not cost the leader its role, in 5.08 seconds)6080subtest: scheduler down: the other node takes the lock and keeps it6081lb: must succeed: curl -sf localhost:9901/clusters6082lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6083lb: must succeed: curl -sf localhost:9901/clusters6084lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6085lb: must succeed: curl -sf localhost:9901/clusters6086lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6087lb: must succeed: curl -sf localhost:9901/clusters6088lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6089worker2 # [ 104.015617] systemd[1]: Stopping Nix worker-protocol over gRPC...6090worker2 # [ 104.153452] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:27Z level=info event=signal_exit6091lb # [ 104.869305] niks3-server[1111]: 2026/09/22 16:55:28 INFO lead: released remote=192.168.1.4:502486092worker1 # [ 105.863648] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:28Z level=info event=scheduler_restarting addr=lb:500516093worker2 # [ 105.874569] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:29Z level=info event=scheduler_restarting6094worker2 # [ 105.878618] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:29Z level=info event=scheduler_restarting addr=lb:500516095worker2 # [ 105.882583] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:529246096worker2 # [ 105.885920] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:29Z level=info event=worker_gone addr=192.168.1.3:500516097worker2 # [ 105.889189] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:529246098worker2 # [ 105.892790] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:29Z level=info event=worker_gone addr=192.168.1.4:500516099worker2 # [ 105.911433] nix-daemon[865]: reaped child process 1296, status = succeeded6100worker1 # [ 106.064204] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:28Z level=info event=scheduler_connected addr=lb:500516101worker1 # [ 106.068602] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6102worker2 # [ 106.083797] nix-grpc-daemon[1244]: ts=2026-09-22T16:55:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6103worker2 # [ 106.191853] nix-grpc-daemon[1244]: E0000 00:00:1790096129.340806 1289 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6104worker2 # [ 106.197577] nix-grpc-daemon[1244]: E0000 00:00:1790096129.346500 1289 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6105worker1 # [ 106.268831] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:28Z level=info event=scheduler_connected addr=lb:500516106worker1 # [ 106.270570] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6107worker1 # [ 106.471444] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:28Z level=info event=scheduler_connected addr=lb:500516108worker1 # [ 106.475392] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6109lb # [ 106.633875] envoy[915]: conn peer=[2001:db8:1::1]:33232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1008 tx=304 ms=14096110lb # [ 106.639386] envoy[915]: conn peer=[2001:db8:1::1]:33238 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9718 tx=4375 ms=44496111lb # [ 106.642906] envoy[915]: conn peer=[2001:db8:1::4]:55274 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5880 tx=625 ms=128786112worker1 # [ 106.675756] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:28Z level=info event=scheduler_connected addr=lb:500516113worker1 # [ 106.677760] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6114worker1 # [ 106.878825] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:29Z level=info event=scheduler_connected addr=lb:500516115worker1 # [ 106.880794] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6116worker2 # [ 107.083472] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6117worker1 # [ 107.081366] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:29Z level=info event=scheduler_connected addr=lb:500516118worker2 # [ 107.087555] systemd[1]: Stopped Nix worker-protocol over gRPC.6119worker1 # [ 107.083134] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6120worker2 # [ 107.091443] systemd[1]: nix-grpc-daemon.service: Consumed 948ms CPU time over 14.042s wall clock time, 37.8M memory peak, 46.2K incoming IP traffic, 85K outgoing IP traffic.6121worker2 # [ 107.103297] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6122worker2 # [ 107.107547] systemd[1]: Closed Nix worker-protocol over gRPC.6123worker2 # [ 107.111083] systemd[1]: nix-grpc-daemon.socket: Consumed 324us CPU time over 14.879s wall clock time, 256K memory peak, 37.5K incoming IP traffic, 27.6K outgoing IP traffic.6124worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6125worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.03 seconds)6126worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6127worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6128worker1 # [ 107.283725] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:29Z level=info event=scheduler_connected addr=lb:500516129worker1 # [ 107.285582] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6130worker1 # [ 107.486595] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:29Z level=info event=scheduler_connected addr=lb:500516131worker1 # [ 107.490999] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6132worker1 # [ 107.691598] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_connected addr=lb:500516133worker1 # [ 107.695975] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6134worker1 # [ 107.897548] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_connected addr=lb:500516135worker1 # [ 107.901886] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6136worker1 # [ 108.102893] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_connected addr=lb:500516137worker1 # [ 108.106936] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6138worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6139worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6140worker1 # [ 108.307776] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_connected addr=lb:500516141worker1 # [ 108.311731] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6142worker1 # [ 108.512284] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_connected addr=lb:500516143worker1 # [ 108.514365] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6144worker1 # [ 108.714925] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_connected addr=lb:500516145worker1 # [ 108.716667] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6146worker1 # [ 108.918557] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_connected addr=lb:500516147worker1 # [ 108.922617] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6148worker1 # [ 109.122757] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_connected addr=lb:500516149worker1 # [ 109.124734] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6150worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6151worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6152worker1 # [ 109.325768] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_connected addr=lb:500516153worker1 # [ 109.329777] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6154worker1 # [ 109.530785] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_connected addr=lb:500516155worker1 # [ 109.534897] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6156worker1 # [ 109.735247] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=info event=scheduler_connected addr=lb:500516157worker1 # [ 109.737126] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6158worker1 # [ 109.862149] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=info event=scheduler_take_over6159lb # [ 109.868301] niks3-server[1111]: 2026/09/22 16:55:33 INFO lead: acquired remote=192.168.1.3:609106160worker1 # [ 109.939404] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=info event=scheduler_connected addr=lb:500516161worker1 # [ 109.942576] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6162worker1 # [ 110.140457] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=info event=scheduler_connected addr=lb:500516163worker1 # [ 110.147996] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:447026164worker1 # [ 110.151704] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06165worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6166worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6167client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26168worker1 # [ 110.417262] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:447146169worker1 # [ 110.429627] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6170client # don't know how to build these paths:6171client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6172worker1 # [ 110.443910] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447146173worker1 # [ 110.454902] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 28 ms (attempt 1/5)6174worker1 # [ 110.491121] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 2/5)6175worker1 # [ 110.559192] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 3/5)6176worker1 # [ 110.606234] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 4/5)6177worker1 # [ 110.631161] nix-grpc-daemon[1260]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6178worker1 # [ 110.635267] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6179worker1 # [ 110.661334] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06180worker1 # [ 110.667565] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:32Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=16181worker1 # [ 110.710671] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447146182worker1 # [ 110.723877] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447146183client # copying 1 paths to 'lb:50051'...6184client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6185worker1 # [ 110.734593] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447146186worker1 # [ 110.745107] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6187worker1 # [ 110.765879] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:33Z level=info event=niks3_push_started pid=13426188worker1 # [ 110.868329] nix-grpc-daemon[1342]: time=2026-09-22T16:55:33.182Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6189worker1 # [ 110.954607] nix-daemon[845]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)6190worker1 # [ 110.968427] nix-daemon[845]: reaped child process 1358, status = succeeded6191lb # [ 110.984513] niks3-server[1111]: 2026/09/22 16:55:34 INFO Received uploads request method=POST path=/api/pending_closures6192worker1 # [ 110.985460] nix-grpc-daemon[1342]: time=2026-09-22T16:55:33.300Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6193worker1 # [ 110.986994] nix-grpc-daemon[1342]: time=2026-09-22T16:55:33.301Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6194lb # [ 111.018568] niks3-server[1111]: 2026/09/22 16:55:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6195lb # [ 111.030732] niks3-server[1111]: 2026/09/22 16:55:34 INFO Signed narinfos id=43 count=16196worker1 # [ 111.027805] nix-grpc-daemon[1342]: time=2026-09-22T16:55:33.342Z level=INFO msg="Uploading 1 narinfos"6197lb # [ 111.041548] niks3-server[1111]: 2026/09/22 16:55:34 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6198lb # [ 111.058074] niks3-server[1111]: 2026/09/22 16:55:34 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6199lb # [ 111.063950] niks3-server[1111]: 2026/09/22 16:55:34 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6200lb # [ 111.067350] niks3-server[1111]: 2026/09/22 16:55:34 INFO Completed upload id=436201worker1 # [ 111.061579] nix-grpc-daemon[1342]: time=2026-09-22T16:55:33.376Z level=INFO msg="Upload complete. (189ms)"6202lb # [ 111.069504] niks3-server[1111]: 2026/09/22 16:55:34 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6203worker1 # [ 111.068378] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44714 duration_s=0 paths=1 nar_bytes_in=5206204worker1 # [ 111.073360] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447146205worker1 # [ 111.082205] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6206worker1 # [ 111.089341] nix-daemon[845]: reaped child process 1340, status = succeeded6207worker1 # [ 111.094174] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6208client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6209client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6210client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6211client # farm-a> LOG-farm-a-on-standby6212worker1 # [ 111.735595] nix-daemon[845]: reaped child process 1362, status = succeeded6213worker1 # [ 111.808350] nix-daemon[845]: accepted connection from pid 1378, user nix-grpc-daemon (trusted)6214worker1 # [ 111.820490] nix-daemon[845]: reaped child process 1381, status = succeeded6215lb # [ 111.833868] niks3-server[1111]: 2026/09/22 16:55:35 INFO Received uploads request method=POST path=/api/pending_closures6216worker1 # [ 111.835914] nix-grpc-daemon[1342]: time=2026-09-22T16:55:34.150Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6217worker1 # [ 111.837426] nix-grpc-daemon[1342]: time=2026-09-22T16:55:34.152Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6218lb # [ 111.872955] niks3-server[1111]: 2026/09/22 16:55:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6219worker1 # [ 111.876192] nix-grpc-daemon[1342]: time=2026-09-22T16:55:34.190Z level=INFO msg="Uploading 1 narinfos"6220lb # [ 111.882783] niks3-server[1111]: 2026/09/22 16:55:35 INFO Signed narinfos id=44 count=16221lb # [ 111.888807] niks3-server[1111]: 2026/09/22 16:55:35 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6222lb # [ 111.895732] niks3-server[1111]: 2026/09/22 16:55:35 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6223lb # [ 111.901824] niks3-server[1111]: 2026/09/22 16:55:35 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6224lb # [ 111.913893] niks3-server[1111]: 2026/09/22 16:55:35 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6225lb # [ 111.918658] niks3-server[1111]: 2026/09/22 16:55:35 INFO Completed upload id=446226worker1 # [ 111.912678] nix-grpc-daemon[1342]: time=2026-09-22T16:55:34.227Z level=INFO msg="Upload complete. (212ms)"6227lb # [ 111.920858] niks3-server[1111]: 2026/09/22 16:55:35 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6228worker1 # [ 111.915612] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44714 duration_s=0 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16229worker1 # [ 111.923379] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:34Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06230worker1 # [ 111.925888] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:34Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=16231worker1 # [ 111.932515] nix-daemon[845]: reaped child process 1360, status = succeeded6232worker1 # [ 111.966552] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447146233worker1 # [ 111.974070] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447146234worker1 # [ 111.982666] nix-grpc-daemon[1342]: time=2026-09-22T16:55:34.297Z level=INFO msg="All 1 paths already cached"6235client # copying 1 paths to 'lb:50051'...6236client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6237worker1 # [ 111.987055] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447146238worker1 # [ 111.991704] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6239worker1 # [ 112.073779] nix-daemon[845]: accepted connection from pid 1387, user nix-grpc-daemon (trusted)6240worker1 # [ 112.087062] nix-daemon[845]: reaped child process 1390, status = succeeded6241lb # [ 112.099603] niks3-server[1111]: 2026/09/22 16:55:35 INFO Received uploads request method=POST path=/api/pending_closures6242worker1 # [ 112.102536] nix-grpc-daemon[1342]: time=2026-09-22T16:55:34.417Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6243worker1 # [ 112.103926] nix-grpc-daemon[1342]: time=2026-09-22T16:55:34.418Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6244lb # [ 112.129888] niks3-server[1111]: 2026/09/22 16:55:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6245worker1 # [ 112.131340] nix-grpc-daemon[1342]: time=2026-09-22T16:55:34.445Z level=INFO msg="Uploading 1 narinfos"6246lb # [ 112.137408] niks3-server[1111]: 2026/09/22 16:55:35 INFO Signed narinfos id=45 count=16247lb # [ 112.146137] niks3-server[1111]: 2026/09/22 16:55:35 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6248lb # [ 112.160349] niks3-server[1111]: 2026/09/22 16:55:35 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6249lb # [ 112.165549] niks3-server[1111]: 2026/09/22 16:55:35 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6250lb # [ 112.175729] niks3-server[1111]: 2026/09/22 16:55:35 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6251lb # [ 112.179332] niks3-server[1111]: 2026/09/22 16:55:35 INFO Completed upload id=456252worker1 # [ 112.173400] nix-grpc-daemon[1342]: time=2026-09-22T16:55:34.488Z level=INFO msg="Upload complete. (177ms)"6253worker1 # [ 112.175669] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44714 duration_s=0 paths=1 nar_bytes_in=6406254worker1 # [ 112.178426] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447146255worker1 # [ 112.184731] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6256worker1 # [ 112.188621] nix-daemon[845]: reaped child process 1385, status = succeeded6257worker1 # [ 112.193418] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6258client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6259client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6260client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6261client # farm-b> LOG-farm-b-on-standby6262worker1 # [ 113.207320] nix-daemon[845]: reaped child process 1394, status = succeeded6263worker1 # [ 113.284087] nix-daemon[845]: accepted connection from pid 1410, user nix-grpc-daemon (trusted)6264worker1 # [ 113.297471] nix-daemon[845]: reaped child process 1413, status = succeeded6265lb # [ 113.308278] niks3-server[1111]: 2026/09/22 16:55:36 INFO Received uploads request method=POST path=/api/pending_closures6266worker1 # [ 113.312376] nix-grpc-daemon[1342]: time=2026-09-22T16:55:35.626Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6267worker1 # [ 113.314877] nix-grpc-daemon[1342]: time=2026-09-22T16:55:35.628Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6268lb # [ 113.349536] niks3-server[1111]: 2026/09/22 16:55:36 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6269lb # [ 113.355679] niks3-server[1111]: 2026/09/22 16:55:36 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6270lb # [ 113.362928] niks3-server[1111]: 2026/09/22 16:55:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6271worker1 # [ 113.363729] nix-grpc-daemon[1342]: time=2026-09-22T16:55:35.678Z level=INFO msg="Uploading 1 narinfos"6272lb # [ 113.369517] niks3-server[1111]: 2026/09/22 16:55:36 INFO Signed narinfos id=46 count=16273lb # [ 113.377595] niks3-server[1111]: 2026/09/22 16:55:36 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6274lb # [ 113.386672] niks3-server[1111]: 2026/09/22 16:55:36 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6275lb # [ 113.392100] niks3-server[1111]: 2026/09/22 16:55:36 INFO Completed upload id=466276lb # [ 113.394302] niks3-server[1111]: 2026/09/22 16:55:36 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6277worker1 # [ 113.386296] nix-grpc-daemon[1342]: time=2026-09-22T16:55:35.700Z level=INFO msg="Upload complete. (186ms)"6278worker1 # [ 113.392832] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44714 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16279worker1 # [ 113.406103] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:35Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06280worker1 # [ 113.410221] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:35Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16281worker1 # [ 113.419590] nix-daemon[845]: reaped child process 1392, status = succeeded6282worker1 # [ 113.449861] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447146283worker1 # [ 113.462790] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447146284worker1 # [ 113.476144] nix-grpc-daemon[1342]: time=2026-09-22T16:55:35.790Z level=INFO msg="All 2 paths already cached"6285client # copying 1 paths to 'lb:50051'...6286client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6287worker1 # [ 113.485144] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447146288worker1 # [ 113.493344] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6289worker1 # [ 113.592050] nix-daemon[845]: accepted connection from pid 1419, user nix-grpc-daemon (trusted)6290worker1 # [ 113.604623] nix-daemon[845]: reaped child process 1422, status = succeeded6291lb # [ 113.617747] niks3-server[1111]: 2026/09/22 16:55:36 INFO Received uploads request method=POST path=/api/pending_closures6292worker1 # [ 113.616416] nix-grpc-daemon[1342]: time=2026-09-22T16:55:35.931Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6293worker1 # [ 113.618095] nix-grpc-daemon[1342]: time=2026-09-22T16:55:35.932Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6294lb # [ 113.636635] niks3-server[1111]: 2026/09/22 16:55:36 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6295lb # [ 113.639466] niks3-server[1111]: 2026/09/22 16:55:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6296lb # [ 113.642399] niks3-server[1111]: 2026/09/22 16:55:36 INFO Signed narinfos id=47 count=16297worker1 # [ 113.636432] nix-grpc-daemon[1342]: time=2026-09-22T16:55:35.951Z level=INFO msg="Uploading 1 narinfos"6298lb # [ 113.646198] niks3-server[1111]: 2026/09/22 16:55:36 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6299lb # [ 113.650779] niks3-server[1111]: 2026/09/22 16:55:36 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6300lb # [ 113.655086] niks3-server[1111]: 2026/09/22 16:55:36 INFO Completed upload id=476301worker1 # [ 113.650469] nix-grpc-daemon[1342]: time=2026-09-22T16:55:35.965Z level=INFO msg="Upload complete. (144ms)"6302lb # [ 113.656471] niks3-server[1111]: 2026/09/22 16:55:36 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6303worker1 # [ 113.652833] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44714 duration_s=0 paths=1 nar_bytes_in=7686304worker1 # [ 113.655941] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447146305worker1 # [ 113.664364] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6306worker1 # [ 113.665722] nix-daemon[845]: reaped child process 1417, status = succeeded6307worker1 # [ 113.670853] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6308client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6309client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6310client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6311client # farm-top> LOG-farm-top-on-standby6312worker1 # [ 114.713365] nix-daemon[845]: reaped child process 1426, status = succeeded6313worker1 # [ 114.783832] nix-daemon[845]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)6314worker1 # [ 114.799766] nix-daemon[845]: reaped child process 1445, status = succeeded6315lb # [ 114.805543] niks3-server[1111]: 2026/09/22 16:55:38 INFO Received uploads request method=POST path=/api/pending_closures6316worker1 # [ 114.804583] nix-grpc-daemon[1342]: time=2026-09-22T16:55:37.119Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6317worker1 # [ 114.806176] nix-grpc-daemon[1342]: time=2026-09-22T16:55:37.120Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6318lb # [ 114.827464] niks3-server[1111]: 2026/09/22 16:55:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6319lb # [ 114.830673] niks3-server[1111]: 2026/09/22 16:55:38 INFO Signed narinfos id=48 count=16320worker1 # [ 114.826217] nix-grpc-daemon[1342]: time=2026-09-22T16:55:37.140Z level=INFO msg="Uploading 1 narinfos"6321lb # [ 114.832323] niks3-server[1111]: 2026/09/22 16:55:38 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6322lb # [ 114.835197] niks3-server[1111]: 2026/09/22 16:55:38 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6323lb # [ 114.838701] niks3-server[1111]: 2026/09/22 16:55:38 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6324lb # [ 114.844424] niks3-server[1111]: 2026/09/22 16:55:38 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6325lb # [ 114.847744] niks3-server[1111]: 2026/09/22 16:55:38 INFO Completed upload id=486326worker1 # [ 114.843095] nix-grpc-daemon[1342]: time=2026-09-22T16:55:37.157Z level=INFO msg="Upload complete. (164ms)"6327lb # [ 114.850365] niks3-server[1111]: 2026/09/22 16:55:38 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6328worker1 # [ 114.846260] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44714 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16329worker1 # [ 114.855783] nix-daemon[845]: reaped child process 1424, status = succeeded6330client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 4.61 seconds)6331client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6332client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)6333worker2 # [ 115.053744] systemd[1]: Listening on Nix worker-protocol over gRPC.6334worker2 # [ 115.060907] systemd[1]: Starting scheduler-token.service...6335worker2 # [ 115.110127] systemd[1]: scheduler-token.service: Deactivated successfully.6336worker2 # [ 115.111683] systemd[1]: Finished scheduler-token.service.6337worker2 # [ 115.113108] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 50ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6338worker2 # [ 115.117922] systemd[1]: Starting Nix worker-protocol over gRPC...6339worker2 # [ 115.201192] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:38Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6340worker2 # [ 115.203234] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:38Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6341worker2 # [ 115.207047] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:38Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16342worker2 # [ 115.211058] nix-grpc-daemon[1450]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6343worker2 # [ 115.212373] nix-grpc-daemon[1450]: E0000 00:00:1790096138.359917 1450 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6344worker2 # [ 115.220060] nix-grpc-daemon[1450]: E0000 00:00:1790096138.365138 1450 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6345worker2 # [ 115.222275] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:38Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6346worker2 # [ 115.223952] systemd[1]: Started Nix worker-protocol over gRPC.6347worker2 # [ 115.234376] nix-grpc-daemon[1450]: E0000 00:00:1790096138.381403 1492 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6348lb: must succeed: curl -sf localhost:9901/clusters6349worker2 # [ 115.247443] nix-grpc-daemon[1450]: E0000 00:00:1790096138.396583 1492 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6350worker2 # [ 115.251156] nix-grpc-daemon[1450]: E0000 00:00:1790096138.398832 1492 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6351worker2 # [ 115.261532] nix-grpc-daemon[1450]: E0000 00:00:1790096138.410617 1492 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6352worker2 # [ 115.272455] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:38Z level=info event=scheduler_connected addr=lb:500516353worker1 # [ 115.273952] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:37Z level=info event=oidc_jwks provider=mock keys=16354worker1 # [ 115.275705] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:447026355worker1 # [ 115.277701] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06356lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6357worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6358worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6359lb: must succeed: curl -sf localhost:9901/clusters6360lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6361lb: must succeed: curl -sf localhost:9901/clusters6362lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6363worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6364worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6365(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.55 seconds)6366subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6367client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6368client # [ 115.480120] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6369client # Running as unit: rs.service; invocation ID: 76dfa4c0e1aa421c8d07c4f57fa3b9156370client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6371worker2 # [ 115.609177] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:591906372worker2 # [ 115.620528] nix-daemon[865]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)6373client # [ 115.655442] nix[1364]: don't know how to build these paths:6374client # [ 115.656446] nix[1364]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6375worker1 # [ 115.632912] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:37Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06376worker1 # [ 115.635087] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:37Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16377worker1 # [ 115.677522] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447146378worker1 # [ 115.686594] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:447146379client # [ 115.721230] nix[1364]: copying 1 paths to 'lb:50051'...6380worker1 # [ 115.694126] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:447146381client # [ 115.723419] nix[1364]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6382worker1 # [ 115.701881] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6383worker1 # [ 115.786316] nix-daemon[845]: accepted connection from pid 1458, user nix-grpc-daemon (trusted)6384worker1 # [ 115.799845] nix-daemon[845]: reaped child process 1461, status = succeeded6385lb # [ 115.812609] niks3-server[1111]: 2026/09/22 16:55:39 INFO Received uploads request method=POST path=/api/pending_closures6386worker1 # [ 115.814792] nix-grpc-daemon[1342]: time=2026-09-22T16:55:38.129Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6387worker1 # [ 115.816798] nix-grpc-daemon[1342]: time=2026-09-22T16:55:38.131Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6388lb # [ 115.843946] niks3-server[1111]: 2026/09/22 16:55:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6389worker1 # [ 115.844289] nix-grpc-daemon[1342]: time=2026-09-22T16:55:38.158Z level=INFO msg="Uploading 1 narinfos"6390lb # [ 115.850124] niks3-server[1111]: 2026/09/22 16:55:39 INFO Signed narinfos id=49 count=16391lb # [ 115.856526] niks3-server[1111]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6392lb # [ 115.865158] niks3-server[1111]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6393lb # [ 115.874232] niks3-server[1111]: 2026/09/22 16:55:39 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6394worker1 # [ 115.876899] nix-grpc-daemon[1342]: time=2026-09-22T16:55:38.191Z level=INFO msg="Upload complete. (169ms)"6395lb # [ 115.883146] niks3-server[1111]: 2026/09/22 16:55:39 INFO Completed upload id=496396lb # [ 115.886040] niks3-server[1111]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6397worker1 # [ 115.879422] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44714 duration_s=0 paths=1 nar_bytes_in=4166398worker1 # [ 115.882751] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:447146399worker1 # [ 115.891171] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6400worker1 # [ 115.894134] nix-daemon[845]: reaped child process 1456, status = succeeded6401worker1 # [ 115.899155] nix-daemon[845]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)6402client # [ 115.926621] nix[1364]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6403lb # [ 116.634691] envoy[915]: conn peer=[2001:db8:1::1]:57434 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9586 tx=4349 ms=44436404lb: must succeed: curl -sf localhost:9901/clusters6405lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6406lb: must succeed: curl -sf localhost:9901/clusters6407lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6408lb: must succeed: curl -sf localhost:9901/clusters6409lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6410lb: must succeed: curl -sf localhost:9901/clusters6411lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6412worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6413worker2 # [ 118.069789] systemd-journald[282]: Received client request to rotate journal, rotating.6414worker2 # [ 118.088210] systemd-journald[282]: Vacuuming done, freed 0B of archived journals from /var/log/journal/c46ffd053bdd4e86b1ce82876f3e8346.6415worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6416worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6417worker1 # [ 117.992437] systemd[1]: Stopping Nix worker-protocol over gRPC...6418worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6419client: must succeed: sleep 5.06420worker1 # [ 118.675634] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:40Z level=info event=draining6421client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6422worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6423worker1 # [ 123.349361] nix-daemon[845]: reaped child process 1465, status = succeeded6424worker1 # [ 123.421363] nix-daemon[845]: accepted connection from pid 1506, user nix-grpc-daemon (trusted)6425worker1 # [ 123.434399] nix-daemon[845]: reaped child process 1509, status = succeeded6426lb # [ 123.447226] niks3-server[1111]: 2026/09/22 16:55:46 INFO Received uploads request method=POST path=/api/pending_closures6427worker1 # [ 123.449454] nix-grpc-daemon[1342]: time=2026-09-22T16:55:45.763Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6428worker1 # [ 123.451371] nix-grpc-daemon[1342]: time=2026-09-22T16:55:45.765Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6429lb # [ 123.493285] niks3-server[1111]: 2026/09/22 16:55:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6430worker1 # [ 123.497908] nix-grpc-daemon[1342]: time=2026-09-22T16:55:45.812Z level=INFO msg="Uploading 1 narinfos"6431lb # [ 123.503800] niks3-server[1111]: 2026/09/22 16:55:46 INFO Signed narinfos id=50 count=16432lb # [ 123.508969] niks3-server[1111]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6433lb # [ 123.514216] niks3-server[1111]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6434lb # [ 123.518336] niks3-server[1111]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6435lb # [ 123.530121] niks3-server[1111]: 2026/09/22 16:55:46 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6436lb # [ 123.536265] niks3-server[1111]: 2026/09/22 16:55:46 INFO Completed upload id=506437worker1 # [ 123.530918] nix-grpc-daemon[1342]: time=2026-09-22T16:55:45.845Z level=INFO msg="Upload complete. (218ms)"6438lb # [ 123.538852] niks3-server[1111]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6439worker1 # [ 123.539616] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44714 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16440worker1 # [ 123.554609] nix-daemon[845]: reaped child process 1463, status = succeeded6441client # [ 123.584877] systemd[1]: rs.service: Deactivated successfully.6442client # [ 123.586479] systemd[1]: rs.service: Consumed 133ms CPU time over 8.106s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.6443worker1 # [ 123.677189] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:45Z level=info event=signal_exit6444lb # [ 123.875354] niks3-server[1111]: 2026/09/22 16:55:47 INFO lead: released remote=192.168.1.3:609106445worker2 # [ 124.877953] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=scheduler_restarting addr=lb:500516446worker1 # [ 124.873986] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:47Z level=info event=scheduler_restarting6447worker1 # [ 124.877924] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:447026448worker1 # [ 124.882119] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:47Z level=info event=worker_gone addr=192.168.1.4:500516449worker1 # [ 124.885406] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:47Z level=info event=scheduler_restarting addr=lb:500516450worker1 # [ 124.888764] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:447026451worker1 # [ 124.892406] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:47Z level=info event=worker_gone addr=192.168.1.3:500516452worker1 # [ 124.910402] nix-daemon[845]: reaped child process 1332, status = succeeded6453worker1 # [ 124.986827] nix-grpc-daemon[1260]: E0000 00:00:1790096147.301277 1305 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6454worker1 # [ 124.992183] nix-grpc-daemon[1260]: E0000 00:00:1790096147.301729 1305 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6455worker2 # [ 125.078439] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=scheduler_connected addr=lb:500516456worker2 # [ 125.082574] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6457worker1 # [ 125.085286] nix-grpc-daemon[1260]: ts=2026-09-22T16:55:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6458worker2 # [ 125.232941] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=scheduler_take_over6459lb # [ 125.233603] niks3-server[1111]: 2026/09/22 16:55:48 INFO lead: acquired remote=192.168.1.4:543426460worker2 # [ 125.286744] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=scheduler_connected addr=lb:500516461worker2 # [ 125.298215] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=oidc_jwks provider=mock keys=16462worker2 # [ 125.301518] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:465946463worker2 # [ 125.306856] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06464worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.13 seconds)6465worker1 # [ 125.339374] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6466client: waiting for success: ! systemctl is-active rs6467worker1 # [ 125.343374] systemd[1]: Stopped Nix worker-protocol over gRPC.6468worker1 # [ 125.346810] systemd[1]: nix-grpc-daemon.service: Consumed 1.022s CPU time over 27.764s wall clock time, 37.5M memory peak, 50.7K incoming IP traffic, 102.1K outgoing IP traffic.6469client: (finished: waiting for success: ! systemctl is-active rs, in 0.02 seconds)6470client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6471worker1 # [ 125.361158] systemd[1]: Starting Nix worker-protocol over gRPC...6472client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.02 seconds)6473client: must fail: journalctl -u rs | grep -q 'reconnecting'6474client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6475lb: must succeed: curl -sf localhost:9901/clusters6476lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6477worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6478worker1 # [ 125.452669] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6479worker1 # [ 125.458449] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6480worker1 # [ 125.462843] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16481worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6482worker1 # [ 125.467145] nix-grpc-daemon[1516]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6483worker1 # [ 125.469971] nix-grpc-daemon[1516]: E0000 00:00:1790096147.772623 1516 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6484worker1 # [ 125.474103] nix-grpc-daemon[1516]: E0000 00:00:1790096147.776146 1516 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6485worker1 # [ 125.477102] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6486worker1 # [ 125.480120] systemd[1]: Started Nix worker-protocol over gRPC.6487worker1 # [ 125.480922] nix-grpc-daemon[1516]: E0000 00:00:1790096147.787262 1558 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6488worker1 # [ 125.489190] nix-grpc-daemon[1516]: E0000 00:00:1790096147.803896 1558 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6489worker1 # [ 125.493052] nix-grpc-daemon[1516]: E0000 00:00:1790096147.806196 1558 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6490worker1 # [ 125.503438] nix-grpc-daemon[1516]: E0000 00:00:1790096147.818074 1558 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6491worker1 # [ 125.511178] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:47Z level=info event=scheduler_connected addr=lb:500516492worker2 # [ 125.518331] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:465946493worker2 # [ 125.522783] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06494lb: must succeed: curl -sf localhost:9901/clusters6495lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6496worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6497worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6498(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.15 seconds)6499subtest: scheduler restart mid-build: build finishes, no double build6500client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6501client # [ 126.626085] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6502client # Running as unit: mid.service; invocation ID: d1970bcd16a140c7891540fa85cabd566503client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6504lb # [ 126.635334] envoy[915]: conn peer=[2001:db8:1::1]:36974 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=1515 ms=79526505lb # [ 126.639356] envoy[915]: conn peer=[2001:db8:1::3]:55838 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5619 tx=1351 ms=273556506worker1 # [ 126.747799] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:523826507worker1 # [ 126.759511] nix-daemon[845]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6508client # [ 126.799457] nix[1400]: don't know how to build these paths:6509client # [ 126.800407] nix[1400]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6510worker2 # [ 126.790694] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:49Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06511worker2 # [ 126.794808] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:49Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16512worker2 # [ 126.833383] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591906513worker2 # [ 126.845098] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 1/5)6514worker2 # [ 126.864374] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 106 ms (attempt 2/5)6515worker2 # [ 126.972675] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 3/5)6516worker2 # [ 127.022692] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 736 ms (attempt 4/5)6517worker2 # [ 127.761275] nix-grpc-daemon[1450]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6518worker2 # [ 127.765465] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6519worker2 # [ 127.786688] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:591906520client # [ 127.818783] nix[1400]: copying 1 paths to 'lb:50051'...6521client # [ 127.823136] nix[1400]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6522worker2 # [ 127.803297] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:591906523worker2 # [ 127.816216] nix-daemon[865]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)6524worker2 # [ 127.837659] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:50Z level=info event=niks3_push_started pid=15766525worker2 # [ 127.940955] nix-grpc-daemon[1576]: time=2026-09-22T16:55:51.089Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6526worker2 # [ 128.033581] nix-daemon[865]: accepted connection from pid 1589, user nix-grpc-daemon (trusted)6527worker2 # [ 128.049041] nix-daemon[865]: reaped child process 1592, status = succeeded6528lb # [ 128.056857] niks3-server[1111]: 2026/09/22 16:55:51 INFO Received uploads request method=POST path=/api/pending_closures6529worker2 # [ 128.064687] nix-grpc-daemon[1576]: time=2026-09-22T16:55:51.213Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6530worker2 # [ 128.066301] nix-grpc-daemon[1576]: time=2026-09-22T16:55:51.215Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6531lb # [ 128.096230] niks3-server[1111]: 2026/09/22 16:55:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6532lb # [ 128.102596] niks3-server[1111]: 2026/09/22 16:55:51 INFO Signed narinfos id=51 count=16533worker2 # [ 128.106079] nix-grpc-daemon[1576]: time=2026-09-22T16:55:51.254Z level=INFO msg="Uploading 1 narinfos"6534lb # [ 128.110569] niks3-server[1111]: 2026/09/22 16:55:51 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6535lb # [ 128.116224] niks3-server[1111]: 2026/09/22 16:55:51 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6536lb # [ 128.128157] niks3-server[1111]: 2026/09/22 16:55:51 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6537lb # [ 128.135088] niks3-server[1111]: 2026/09/22 16:55:51 INFO Completed upload id=516538worker2 # [ 128.137270] nix-grpc-daemon[1576]: time=2026-09-22T16:55:51.286Z level=INFO msg="Upload complete. (192ms)"6539worker2 # [ 128.139667] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59190 duration_s=0 paths=1 nar_bytes_in=4246540lb # [ 128.141085] niks3-server[1111]: 2026/09/22 16:55:51 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6541worker2 # [ 128.143174] nix-grpc-daemon[1450]: ts=2026-09-22T16:55:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:591906542worker2 # [ 128.151328] nix-daemon[865]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)6543worker2 # [ 128.153830] nix-daemon[865]: reaped child process 1574, status = succeeded6544worker2 # [ 128.159333] nix-daemon[865]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)6545client # [ 128.180966] nix[1400]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6546lb: must succeed: curl -sf localhost:9901/clusters6547lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6548lb: must succeed: curl -sf localhost:9901/clusters6549lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6550worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6551worker2 # [ 129.144405] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1450 (nix-grpc-daemon) on client request.6552worker2 # [ 129.150453] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6553lb # [ 129.159297] niks3-server[1111]: 2026/09/22 16:55:52 INFO lead: released remote=192.168.1.4:543426554client # [ 129.188623] nix[1400]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6555worker2 # [ 129.170129] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6556worker1 # [ 129.160863] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:51Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6557worker2 # [ 129.171504] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6558worker2 # [ 129.172695] systemd[1]: nix-grpc-daemon.service: Consumed 372ms CPU time over 14.048s wall clock time, 35.8M memory peak, 15.2K incoming IP traffic, 29.3K outgoing IP traffic.6559client # [ 129.193530] nix[1400]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6560worker2 # [ 129.179847] systemd[1]: Starting scheduler-token.service...6561worker2 # [ 129.204221] nix-daemon[865]: reaped child process 1594, status = succeeded6562worker2 # [ 129.206105] nix-daemon[865]: reaped child process 1509, status = succeeded6563worker2 # [ 129.212413] nix-daemon[1596]: unexpected Nix daemon error: error: interrupted by the user6564worker2 # [ 129.220075] nix-daemon[865]: reaped child process 1596, status = failed with exit code 16565worker2 # [ 129.236629] systemd[1]: scheduler-token.service: Deactivated successfully.6566worker2 # [ 129.237845] systemd[1]: Finished scheduler-token.service.6567worker2 # [ 129.239300] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 58ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6568worker2 # [ 129.242144] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6569worker2 # [ 129.245868] systemd[1]: Starting scheduler-token.service...6570worker2 # [ 129.281249] systemd[1]: scheduler-token.service: Deactivated successfully.6571worker2 # [ 129.282869] systemd[1]: Finished scheduler-token.service.6572worker2 # [ 129.284379] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 36ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6573worker2 # [ 129.289451] systemd[1]: Starting Nix worker-protocol over gRPC...6574worker1 # [ 129.361092] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:51Z level=info event=scheduler_connected addr=lb:500516575worker2 # [ 129.373363] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6576worker2 # [ 129.377861] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6577worker2 # [ 129.382121] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16578worker2 # [ 129.384142] nix-grpc-daemon[1635]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6579worker2 # [ 129.385473] nix-grpc-daemon[1635]: E0000 00:00:1790096152.529840 1635 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6580worker2 # [ 129.395861] nix-grpc-daemon[1635]: E0000 00:00:1790096152.537389 1635 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6581worker2 # [ 129.399127] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6582worker2 # [ 129.403484] systemd[1]: Started Nix worker-protocol over gRPC.6583lb # [ 129.405768] niks3-server[1111]: 2026/09/22 16:55:52 INFO lead: acquired remote=192.168.1.4:369126584worker2 # [ 129.408082] nix-grpc-daemon[1635]: E0000 00:00:1790096152.554034 1675 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6585worker2 # [ 129.410368] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=scheduler_take_over6586worker2 # [ 129.417081] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:466286587worker2 # [ 129.419113] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06588worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.31 seconds)6589client: must succeed: sleep 5.06590worker2 # [ 129.426381] nix-grpc-daemon[1635]: E0000 00:00:1790096152.575521 1675 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6591worker2 # [ 129.430129] nix-grpc-daemon[1635]: E0000 00:00:1790096152.577692 1675 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6592worker2 # [ 129.440493] nix-grpc-daemon[1635]: E0000 00:00:1790096152.589633 1675 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6593worker2 # [ 129.450138] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=scheduler_connected addr=lb:500516594worker2 # [ 129.451685] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6595worker2 # [ 129.653245] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=scheduler_connected addr=lb:500516596worker2 # [ 129.657348] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6597client # [ 129.694910] nix[1400]: scheduler at lb:50051: no healthy upstream, reconnecting6598worker2 # [ 129.858104] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=scheduler_connected addr=lb:500516599worker2 # [ 129.860000] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6600worker2 # [ 130.061081] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=scheduler_connected addr=lb:500516601worker2 # [ 130.062957] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6602worker2 # [ 130.264494] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=scheduler_connected addr=lb:500516603worker2 # [ 130.268575] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6604worker2 # [ 130.472784] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=scheduler_connected addr=lb:500516605worker2 # [ 130.480137] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=oidc_jwks provider=mock keys=16606worker2 # [ 130.483970] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:466286607worker2 # [ 130.487921] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06608worker2 # [ 130.681555] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06609worker2 # [ 130.683475] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:53Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16610worker1 # [ 130.721352] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:523826611worker1 # [ 130.733738] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)6612worker1 # [ 130.756204] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 2/5)6613worker1 # [ 130.883574] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 3/5)6614worker1 # [ 130.906161] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 241 ms (attempt 4/5)6615worker1 # [ 131.149177] nix-grpc-daemon[1516]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6616worker1 # [ 131.151836] nix-grpc-daemon[1516]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6617worker1 # [ 131.165952] nix-grpc-daemon[1516]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6618worker1 # [ 131.187824] nix-daemon[845]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6619worker1 # [ 131.198349] nix-daemon[845]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6620client # [ 131.226198] nix[1400]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6621client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6622worker1 # [ 134.613931] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:56Z level=info event=niks3_push_started pid=16186623client: waiting for success: ! systemctl is-active mid6624worker1 # [ 134.631562] nix-daemon[845]: reaped child process 1593, status = succeeded6625worker1 # [ 134.705768] nix-grpc-daemon[1618]: time=2026-09-22T16:55:57.018Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6626worker1 # [ 134.788983] nix-daemon[845]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6627worker1 # [ 134.802141] nix-daemon[845]: reaped child process 1633, status = succeeded6628lb # [ 134.815765] niks3-server[1111]: 2026/09/22 16:55:58 INFO Received uploads request method=POST path=/api/pending_closures6629worker1 # [ 134.819622] nix-grpc-daemon[1618]: time=2026-09-22T16:55:57.134Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6630worker1 # [ 134.821211] nix-grpc-daemon[1618]: time=2026-09-22T16:55:57.135Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6631lb # [ 134.863351] niks3-server[1111]: 2026/09/22 16:55:58 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6632lb # [ 134.869630] niks3-server[1111]: 2026/09/22 16:55:58 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6633lb # [ 134.875462] niks3-server[1111]: 2026/09/22 16:55:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6634worker1 # [ 134.877636] nix-grpc-daemon[1618]: time=2026-09-22T16:55:57.191Z level=INFO msg="Uploading 1 narinfos"6635lb # [ 134.882906] niks3-server[1111]: 2026/09/22 16:55:58 INFO Signed narinfos id=52 count=16636lb # [ 134.895340] niks3-server[1111]: 2026/09/22 16:55:58 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6637lb # [ 134.902775] niks3-server[1111]: 2026/09/22 16:55:58 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6638lb # [ 134.910573] niks3-server[1111]: 2026/09/22 16:55:58 INFO Completed upload id=526639worker1 # [ 134.904778] nix-grpc-daemon[1618]: time=2026-09-22T16:55:57.219Z level=INFO msg="Upload complete. (200ms)"6640lb # [ 134.916762] niks3-server[1111]: 2026/09/22 16:55:58 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6641worker1 # [ 134.910090] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52382 duration_s=4 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16642worker1 # [ 134.924623] nix-daemon[845]: reaped child process 1591, status = succeeded6643client # [ 134.970577] systemd[1]: mid.service: Deactivated successfully.6644client # [ 134.973770] systemd[1]: mid.service: Consumed 152ms CPU time over 8.344s wall clock time, 43.4M memory peak, 8.9K incoming IP traffic, 10.7K outgoing IP traffic.6645client: (finished: waiting for success: ! systemctl is-active mid, in 1.06 seconds)6646client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6647client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)6648(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.16 seconds)6649subtest: interrupting the client stops the build on the worker6650client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6651client # [ 135.766938] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6652client # Running as unit: intr.service; invocation ID: 12ee3bda439a4c85a1819306e994eaef6653client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6654worker1 # [ 135.864286] nix-grpc-daemon[1516]: ts=2026-09-22T16:55:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:523826655client # [ 135.898662] nix[1436]: don't know how to build these paths:6656client # [ 135.899548] nix[1436]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6657worker2 # [ 135.887983] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:59Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06658worker2 # [ 135.892261] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:59Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16659worker2 # [ 135.934088] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966660worker2 # [ 135.946395] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6661worker2 # [ 135.962767] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 1/5)6662worker2 # [ 136.051082] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 2/5)6663worker2 # [ 136.128358] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 379 ms (attempt 3/5)6664worker2 # [ 136.509702] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 280 ms (attempt 4/5)6665lb # [ 136.636348] envoy[915]: conn peer=[2001:db8:1::4]:58572 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2871 tx=376 ms=138986666lb # [ 136.640294] envoy[915]: conn peer=[2001:db8:1::1]:54986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4076 tx=1997 ms=81796667worker2 # [ 136.792386] nix-grpc-daemon[1635]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6668worker2 # [ 136.796367] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6669worker2 # [ 136.812426] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352966670client # [ 136.845202] nix[1436]: copying 1 paths to 'lb:50051'...6671worker2 # [ 136.823777] nix-grpc-daemon[1635]: ts=2026-09-22T16:55:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352966672client # [ 136.848187] nix[1436]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6673worker2 # [ 136.834827] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6674worker2 # [ 136.856406] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:00Z level=info event=niks3_push_started pid=17116675worker2 # [ 136.971600] nix-grpc-daemon[1711]: time=2026-09-22T16:56:00.118Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6676worker2 # [ 137.063980] nix-daemon[865]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6677worker2 # [ 137.080985] nix-daemon[865]: reaped child process 1730, status = succeeded6678lb # [ 137.083712] niks3-server[1111]: 2026/09/22 16:56:00 INFO Received uploads request method=POST path=/api/pending_closures6679worker2 # [ 137.093778] nix-grpc-daemon[1711]: time=2026-09-22T16:56:00.242Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6680worker2 # [ 137.096697] nix-grpc-daemon[1711]: time=2026-09-22T16:56:00.244Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6681lb # [ 137.125137] niks3-server[1111]: 2026/09/22 16:56:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6682worker2 # [ 137.136498] nix-grpc-daemon[1711]: time=2026-09-22T16:56:00.284Z level=INFO msg="Uploading 1 narinfos"6683lb # [ 137.135982] niks3-server[1111]: 2026/09/22 16:56:00 INFO Signed narinfos id=53 count=16684lb # [ 137.144062] niks3-server[1111]: 2026/09/22 16:56:00 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6685lb # [ 137.154449] niks3-server[1111]: 2026/09/22 16:56:00 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6686lb # [ 137.160220] niks3-server[1111]: 2026/09/22 16:56:00 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6687lb # [ 137.167429] niks3-server[1111]: 2026/09/22 16:56:00 INFO Completed upload id=536688worker2 # [ 137.167191] nix-grpc-daemon[1711]: time=2026-09-22T16:56:00.316Z level=INFO msg="Upload complete. (197ms)"6689lb # [ 137.169150] niks3-server[1111]: 2026/09/22 16:56:00 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6690worker2 # [ 137.172291] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=0 paths=1 nar_bytes_in=4246691worker2 # [ 137.177872] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966692worker2 # [ 137.188558] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6693worker2 # [ 137.194131] nix-daemon[865]: reaped child process 1709, status = succeeded6694client # [ 137.219538] nix[1436]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6695worker2 # [ 137.198635] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6696client: must succeed: systemctl kill -s INT intr6697client # [ 139.233047] systemd[1]: intr.service: Sent signal SIGINT to main process 1436 (nix) on client request.6698client # [ 139.235282] nix[1436]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6699worker2 # [ 139.213888] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:02Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6700client: (finished: must succeed: systemctl kill -s INT intr, in 0.02 seconds)6701client # [ 139.238316] nix[1436]: error: interrupted by the user6702client # [ 139.255316] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6703client # [ 139.256690] systemd[1]: intr.service: Failed with result 'exit-code'.6704client # [ 139.258848] systemd[1]: intr.service: Consumed 117ms CPU time over 3.491s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6705worker2 # [ 139.418639] nix-daemon[865]: reaped child process 1732, status = succeeded6706worker2 # [ 139.421279] nix-daemon[1734]: unexpected Nix daemon error: error: interrupted by the user6707worker2 # [ 139.426686] nix-daemon[865]: reaped child process 1734, status = failed with exit code 16708(finished: subtest: interrupting the client stops the build on the worker, in 4.67 seconds)6709subtest: a deploy mid-build drains: the build finishes, the new generation takes over6710client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6711client # [ 140.464179] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6712client # Running as unit: sw.service; invocation ID: b6ba30b09faf4c56a13b486aa2fb89b16713client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6714client # [ 140.611768] nix[1456]: don't know how to build these paths:6715client # [ 140.612752] nix[1456]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6716worker1 # [ 140.581970] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:523826717worker2 # [ 140.598391] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:03Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06718worker2 # [ 140.602549] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:03Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16719worker1 # [ 140.635782] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:523826720worker1 # [ 140.648875] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:523826721client # [ 140.687153] nix[1456]: copying 1 paths to 'lb:50051'...6722client # [ 140.689189] nix[1456]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6723worker1 # [ 140.660479] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:523826724worker1 # [ 140.672260] nix-daemon[845]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6725worker1 # [ 140.772596] nix-daemon[845]: accepted connection from pid 1665, user nix-grpc-daemon (trusted)6726worker1 # [ 140.785482] nix-daemon[845]: reaped child process 1668, status = succeeded6727lb # [ 140.800502] niks3-server[1111]: 2026/09/22 16:56:03 INFO Received uploads request method=POST path=/api/pending_closures6728worker1 # [ 140.801363] nix-grpc-daemon[1618]: time=2026-09-22T16:56:03.115Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6729worker1 # [ 140.803922] nix-grpc-daemon[1618]: time=2026-09-22T16:56:03.117Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6730lb # [ 140.829652] niks3-server[1111]: 2026/09/22 16:56:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6731worker1 # [ 140.834650] nix-grpc-daemon[1618]: time=2026-09-22T16:56:03.148Z level=INFO msg="Uploading 1 narinfos"6732lb # [ 140.840574] niks3-server[1111]: 2026/09/22 16:56:04 INFO Signed narinfos id=54 count=16733lb # [ 140.844371] niks3-server[1111]: 2026/09/22 16:56:04 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6734lb # [ 140.851452] niks3-server[1111]: 2026/09/22 16:56:04 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6735lb # [ 140.865262] niks3-server[1111]: 2026/09/22 16:56:04 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6736lb # [ 140.870116] niks3-server[1111]: 2026/09/22 16:56:04 INFO Completed upload id=546737worker1 # [ 140.866612] nix-grpc-daemon[1618]: time=2026-09-22T16:56:03.181Z level=INFO msg="Upload complete. (182ms)"6738lb # [ 140.875162] niks3-server[1111]: 2026/09/22 16:56:04 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6739worker1 # [ 140.870975] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52382 duration_s=0 paths=1 nar_bytes_in=4166740worker1 # [ 140.877318] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:523826741worker1 # [ 140.888109] nix-daemon[845]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6742worker1 # [ 140.895347] nix-daemon[845]: reaped child process 1663, status = succeeded6743worker1 # [ 140.900350] nix-daemon[845]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6744client # [ 140.927934] nix[1456]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6745worker1: must succeed: systemctl reload nix-grpc-daemon.service6746worker1 # [ 142.854462] systemd[1]: Reloading Nix worker-protocol over gRPC...6747worker1 # [ 142.881472] systemd[1]: Reloaded Nix worker-protocol over gRPC.6748worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6749worker1: must succeed: pgrep -f 'read -t [9]0 x'6750worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6751client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26752client # don't know how to build these paths:6753client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6754worker2 # [ 143.082956] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:352966755worker2 # [ 143.089345] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352966756worker2 # [ 143.112756] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:06Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06757worker2 # [ 143.118233] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:06Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16758worker2 # [ 143.162197] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966759worker2 # [ 143.174468] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352966760client # copying 1 paths to 'lb:50051'...6761worker2 # [ 143.186906] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352966762client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6763worker2 # [ 143.207900] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6764worker2 # [ 143.311964] nix-daemon[865]: accepted connection from pid 1789, user nix-grpc-daemon (trusted)6765worker2 # [ 143.324743] nix-daemon[865]: reaped child process 1792, status = succeeded6766lb # [ 143.333159] niks3-server[1111]: 2026/09/22 16:56:06 INFO Received uploads request method=POST path=/api/pending_closures6767worker2 # [ 143.342456] nix-grpc-daemon[1711]: time=2026-09-22T16:56:06.491Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6768worker2 # [ 143.344723] nix-grpc-daemon[1711]: time=2026-09-22T16:56:06.493Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6769lb # [ 143.371118] niks3-server[1111]: 2026/09/22 16:56:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6770worker2 # [ 143.377652] nix-grpc-daemon[1711]: time=2026-09-22T16:56:06.526Z level=INFO msg="Uploading 1 narinfos"6771lb # [ 143.378584] niks3-server[1111]: 2026/09/22 16:56:06 INFO Signed narinfos id=55 count=16772lb # [ 143.393043] niks3-server[1111]: 2026/09/22 16:56:06 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6773lb # [ 143.402528] niks3-server[1111]: 2026/09/22 16:56:06 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6774lb # [ 143.408882] niks3-server[1111]: 2026/09/22 16:56:06 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6775lb # [ 143.415289] niks3-server[1111]: 2026/09/22 16:56:06 INFO Completed upload id=556776worker2 # [ 143.415371] nix-grpc-daemon[1711]: time=2026-09-22T16:56:06.564Z level=INFO msg="Upload complete. (196ms)"6777lb # [ 143.417351] niks3-server[1111]: 2026/09/22 16:56:06 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6778worker2 # [ 143.420088] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=0 paths=1 nar_bytes_in=5286779worker2 # [ 143.426481] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966780worker2 # [ 143.440108] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6781worker2 # [ 143.449224] nix-daemon[865]: reaped child process 1787, status = succeeded6782worker2 # [ 143.455247] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6783client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6784client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6785worker1 # [ 143.471661] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:05Z level=info event=draining6786client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6787client # farm-a> LOG-farm-a-during-drain6788worker2 # [ 144.895300] nix-daemon[865]: reaped child process 1796, status = succeeded6789worker2 # [ 144.971080] nix-daemon[865]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6790worker2 # [ 144.984107] nix-daemon[865]: reaped child process 1815, status = succeeded6791lb # [ 144.989495] niks3-server[1111]: 2026/09/22 16:56:08 INFO Received uploads request method=POST path=/api/pending_closures6792worker2 # [ 144.997424] nix-grpc-daemon[1711]: time=2026-09-22T16:56:08.146Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6793worker2 # [ 145.000061] nix-grpc-daemon[1711]: time=2026-09-22T16:56:08.148Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6794lb # [ 145.029230] niks3-server[1111]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6795lb # [ 145.036320] niks3-server[1111]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6796lb # [ 145.043358] niks3-server[1111]: 2026/09/22 16:56:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6797worker2 # [ 145.050655] nix-grpc-daemon[1711]: time=2026-09-22T16:56:08.199Z level=INFO msg="Uploading 1 narinfos"6798lb # [ 145.050816] niks3-server[1111]: 2026/09/22 16:56:08 INFO Signed narinfos id=56 count=16799lb # [ 145.057283] niks3-server[1111]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6800lb # [ 145.066993] niks3-server[1111]: 2026/09/22 16:56:08 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6801lb # [ 145.071646] niks3-server[1111]: 2026/09/22 16:56:08 INFO Completed upload id=566802worker2 # [ 145.071548] nix-grpc-daemon[1711]: time=2026-09-22T16:56:08.220Z level=INFO msg="Upload complete. (208ms)"6803lb # [ 145.073188] niks3-server[1111]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6804worker2 # [ 145.077455] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16805worker2 # [ 145.091145] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:08Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06806worker2 # [ 145.094574] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:08Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16807worker2 # [ 145.103797] nix-daemon[865]: reaped child process 1794, status = succeeded6808worker2 # [ 145.133589] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966809worker2 # [ 145.145293] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352966810worker2 # [ 145.158182] nix-grpc-daemon[1711]: time=2026-09-22T16:56:08.307Z level=INFO msg="All 1 paths already cached"6811client # copying 1 paths to 'lb:50051'...6812worker2 # [ 145.165770] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352966813client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6814worker2 # [ 145.179334] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6815worker2 # [ 145.275413] nix-daemon[865]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)6816worker2 # [ 145.290514] nix-daemon[865]: reaped child process 1824, status = succeeded6817lb # [ 145.295318] niks3-server[1111]: 2026/09/22 16:56:08 INFO Received uploads request method=POST path=/api/pending_closures6818worker2 # [ 145.302710] nix-grpc-daemon[1711]: time=2026-09-22T16:56:08.451Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6819worker2 # [ 145.304313] nix-grpc-daemon[1711]: time=2026-09-22T16:56:08.453Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6820lb # [ 145.326337] niks3-server[1111]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6821lb # [ 145.333490] niks3-server[1111]: 2026/09/22 16:56:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6822lb # [ 145.340308] niks3-server[1111]: 2026/09/22 16:56:08 INFO Signed narinfos id=57 count=16823worker2 # [ 145.343442] nix-grpc-daemon[1711]: time=2026-09-22T16:56:08.492Z level=INFO msg="Uploading 1 narinfos"6824lb # [ 145.348527] niks3-server[1111]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6825lb # [ 145.360400] niks3-server[1111]: 2026/09/22 16:56:08 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6826lb # [ 145.367495] niks3-server[1111]: 2026/09/22 16:56:08 INFO Completed upload id=576827worker2 # [ 145.367382] nix-grpc-daemon[1711]: time=2026-09-22T16:56:08.516Z level=INFO msg="Upload complete. (178ms)"6828lb # [ 145.370979] niks3-server[1111]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6829worker2 # [ 145.371612] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=0 paths=1 nar_bytes_in=6406830worker2 # [ 145.376794] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966831worker2 # [ 145.387657] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6832worker2 # [ 145.392339] nix-daemon[865]: reaped child process 1819, status = succeeded6833client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6834client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6835worker2 # [ 145.397791] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6836client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6837client # farm-b> LOG-farm-b-during-drain6838worker2 # [ 146.238429] nix-daemon[865]: reaped child process 1828, status = succeeded6839worker2 # [ 146.314432] nix-daemon[865]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6840worker2 # [ 146.326900] nix-daemon[865]: reaped child process 1847, status = succeeded6841lb # [ 146.334976] niks3-server[1111]: 2026/09/22 16:56:09 INFO Received uploads request method=POST path=/api/pending_closures6842worker2 # [ 146.340250] nix-grpc-daemon[1711]: time=2026-09-22T16:56:09.489Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6843worker2 # [ 146.341786] nix-grpc-daemon[1711]: time=2026-09-22T16:56:09.490Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6844lb # [ 146.356469] niks3-server[1111]: 2026/09/22 16:56:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6845lb # [ 146.361614] niks3-server[1111]: 2026/09/22 16:56:09 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6846lb # [ 146.367559] niks3-server[1111]: 2026/09/22 16:56:09 INFO Signed narinfos id=58 count=16847worker2 # [ 146.370059] nix-grpc-daemon[1711]: time=2026-09-22T16:56:09.518Z level=INFO msg="Uploading 1 narinfos"6848lb # [ 146.375933] niks3-server[1111]: 2026/09/22 16:56:09 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6849lb # [ 146.377822] niks3-server[1111]: 2026/09/22 16:56:09 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6850lb # [ 146.383379] niks3-server[1111]: 2026/09/22 16:56:09 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6851lb # [ 146.387821] niks3-server[1111]: 2026/09/22 16:56:09 INFO Completed upload id=586852worker2 # [ 146.387561] nix-grpc-daemon[1711]: time=2026-09-22T16:56:09.536Z level=INFO msg="Upload complete. (183ms)"6853lb # [ 146.389394] niks3-server[1111]: 2026/09/22 16:56:09 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6854worker2 # [ 146.390246] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16855worker2 # [ 146.398553] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:09Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06856worker2 # [ 146.400986] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:09Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16857worker2 # [ 146.405141] nix-daemon[865]: reaped child process 1826, status = succeeded6858worker2 # [ 146.441962] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966859worker2 # [ 146.452370] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352966860worker2 # [ 146.464066] nix-grpc-daemon[1711]: time=2026-09-22T16:56:09.613Z level=INFO msg="All 2 paths already cached"6861client # copying 1 paths to 'lb:50051'...6862client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6863worker2 # [ 146.470821] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352966864worker2 # [ 146.477542] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6865worker2 # [ 146.566368] nix-daemon[865]: accepted connection from pid 1853, user nix-grpc-daemon (trusted)6866worker2 # [ 146.578801] nix-daemon[865]: reaped child process 1856, status = succeeded6867lb # [ 146.586702] niks3-server[1111]: 2026/09/22 16:56:09 INFO Received uploads request method=POST path=/api/pending_closures6868worker2 # [ 146.596627] nix-grpc-daemon[1711]: time=2026-09-22T16:56:09.745Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6869worker2 # [ 146.598184] nix-grpc-daemon[1711]: time=2026-09-22T16:56:09.747Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6870lb # [ 146.618790] niks3-server[1111]: 2026/09/22 16:56:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6871lb # [ 146.627171] niks3-server[1111]: 2026/09/22 16:56:09 INFO Signed narinfos id=59 count=16872worker2 # [ 146.630949] nix-grpc-daemon[1711]: time=2026-09-22T16:56:09.779Z level=INFO msg="Uploading 1 narinfos"6873lb # [ 146.636484] niks3-server[1111]: 2026/09/22 16:56:09 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6874lb # [ 146.643158] envoy[915]: conn peer=[2001:db8:1::1]:42078 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=828 ms=33556875lb # [ 146.648393] niks3-server[1111]: 2026/09/22 16:56:09 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6876lb # [ 146.659972] niks3-server[1111]: 2026/09/22 16:56:09 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6877lb # [ 146.665614] niks3-server[1111]: 2026/09/22 16:56:09 INFO Completed upload id=596878worker2 # [ 146.665481] nix-grpc-daemon[1711]: time=2026-09-22T16:56:09.814Z level=INFO msg="Upload complete. (177ms)"6879lb # [ 146.671506] niks3-server[1111]: 2026/09/22 16:56:09 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6880worker2 # [ 146.670250] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=0 paths=1 nar_bytes_in=7686881worker2 # [ 146.676350] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352966882worker2 # [ 146.688130] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6883worker2 # [ 146.696177] nix-daemon[865]: reaped child process 1851, status = succeeded6884client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6885client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6886worker2 # [ 146.704302] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6887client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6888client # farm-top> LOG-farm-top-during-drain6889worker2 # [ 147.743624] nix-daemon[865]: reaped child process 1860, status = succeeded6890worker2 # [ 147.830034] nix-daemon[865]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6891worker2 # [ 147.844680] nix-daemon[865]: reaped child process 1879, status = succeeded6892lb # [ 147.851793] niks3-server[1111]: 2026/09/22 16:56:11 INFO Received uploads request method=POST path=/api/pending_closures6893worker2 # [ 147.859978] nix-grpc-daemon[1711]: time=2026-09-22T16:56:11.008Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6894worker2 # [ 147.861527] nix-grpc-daemon[1711]: time=2026-09-22T16:56:11.010Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6895lb # [ 147.889467] niks3-server[1111]: 2026/09/22 16:56:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6896worker2 # [ 147.896865] nix-grpc-daemon[1711]: time=2026-09-22T16:56:11.045Z level=INFO msg="Uploading 1 narinfos"6897lb # [ 147.897263] niks3-server[1111]: 2026/09/22 16:56:11 INFO Signed narinfos id=60 count=16898lb # [ 147.906410] niks3-server[1111]: 2026/09/22 16:56:11 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6899lb # [ 147.915518] niks3-server[1111]: 2026/09/22 16:56:11 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6900lb # [ 147.921162] niks3-server[1111]: 2026/09/22 16:56:11 INFO Completed upload id=606901worker2 # [ 147.921202] nix-grpc-daemon[1711]: time=2026-09-22T16:56:11.070Z level=INFO msg="Upload complete. (209ms)"6902lb # [ 147.922888] niks3-server[1111]: 2026/09/22 16:56:11 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6903lb # [ 147.925503] niks3-server[1111]: 2026/09/22 16:56:11 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6904lb # [ 147.928256] niks3-server[1111]: 2026/09/22 16:56:11 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6905worker2 # [ 147.929186] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16906worker2 # [ 147.941969] nix-daemon[865]: reaped child process 1858, status = succeeded6907client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 5.03 seconds)6908client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6909client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.10 seconds)6910worker1: must succeed: pkill -f 'read -t [9]0 x'6911worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6912client: waiting for success: ! systemctl is-active sw6913worker1 # [ 148.122667] nix-daemon[845]: reaped child process 1672, status = succeeded6914worker1 # [ 148.196251] nix-daemon[845]: accepted connection from pid 1713, user nix-grpc-daemon (trusted)6915worker1 # [ 148.207949] nix-daemon[845]: reaped child process 1716, status = succeeded6916lb # [ 148.219798] niks3-server[1111]: 2026/09/22 16:56:11 INFO Received uploads request method=POST path=/api/pending_closures6917worker1 # [ 148.221374] nix-grpc-daemon[1618]: time=2026-09-22T16:56:10.535Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6918worker1 # [ 148.223311] nix-grpc-daemon[1618]: time=2026-09-22T16:56:10.537Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6919lb # [ 148.258183] niks3-server[1111]: 2026/09/22 16:56:11 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6920lb # [ 148.265102] niks3-server[1111]: 2026/09/22 16:56:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6921lb # [ 148.269104] niks3-server[1111]: 2026/09/22 16:56:11 INFO Signed narinfos id=61 count=16922worker1 # [ 148.266220] nix-grpc-daemon[1618]: time=2026-09-22T16:56:10.580Z level=INFO msg="Uploading 1 narinfos"6923lb # [ 148.276572] niks3-server[1111]: 2026/09/22 16:56:11 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6924lb # [ 148.284787] niks3-server[1111]: 2026/09/22 16:56:11 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6925lb # [ 148.292275] niks3-server[1111]: 2026/09/22 16:56:11 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6926lb # [ 148.299604] niks3-server[1111]: 2026/09/22 16:56:11 INFO Completed upload id=616927worker1 # [ 148.293625] nix-grpc-daemon[1618]: time=2026-09-22T16:56:10.608Z level=INFO msg="Upload complete. (179ms)"6928lb # [ 148.304674] niks3-server[1111]: 2026/09/22 16:56:11 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6929worker1 # [ 148.299718] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52382 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16930worker1 # [ 148.314273] nix-daemon[845]: reaped child process 1670, status = succeeded6931client # [ 148.347614] systemd[1]: sw.service: Deactivated successfully.6932client # [ 148.349294] systemd[1]: sw.service: Consumed 124ms CPU time over 7.885s wall clock time, 43.4M memory peak, 7.1K incoming IP traffic, 8.2K outgoing IP traffic.6933worker1 # [ 148.472991] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:10Z level=info event=signal_exit6934client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6935client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6936client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6937worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6938worker2 # [ 149.485941] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:466286939worker2 # [ 149.489779] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:12Z level=info event=worker_gone addr=192.168.1.3:500516940worker1 # [ 149.483535] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:11Z level=info event=scheduler_restarting6941worker1 # [ 149.493540] nix-grpc-daemon[1516]: ts=2026-09-22T16:56:11Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6942worker1 # [ 149.512070] nix-daemon[845]: reaped child process 1572, status = succeeded6943worker1 # [ 150.386900] nix-grpc-daemon[1516]: E0000 00:00:1790096172.701440 1561 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6944worker1 # [ 150.392467] nix-grpc-daemon[1516]: E0000 00:00:1790096172.706838 1561 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6945worker1 # [ 151.501559] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6946worker1 # [ 151.505606] systemd[1]: nix-grpc-daemon.service: Consumed 557ms CPU time over 26.143s wall clock time, 34.9M memory peak, 21.1K incoming IP traffic, 27.9K outgoing IP traffic.6947worker1 # [ 151.522978] systemd[1]: Starting Nix worker-protocol over gRPC...6948worker1 # [ 151.611029] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6949worker1 # [ 151.615651] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6950worker1 # [ 151.618497] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16951worker1 # [ 151.621796] nix-grpc-daemon[1730]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6952worker1 # [ 151.626711] nix-grpc-daemon[1730]: E0000 00:00:1790096173.931201 1730 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6953worker1 # [ 151.630535] nix-grpc-daemon[1730]: E0000 00:00:1790096173.936310 1730 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6954worker1 # [ 151.633850] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:13Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6955worker1 # [ 151.636557] systemd[1]: Started Nix worker-protocol over gRPC.6956worker1 # [ 151.639139] nix-grpc-daemon[1730]: E0000 00:00:1790096173.950404 1771 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6957worker1 # [ 151.651068] nix-grpc-daemon[1730]: E0000 00:00:1790096173.964806 1771 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6958worker1 # [ 151.654131] nix-grpc-daemon[1730]: E0000 00:00:1790096173.964913 1771 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6959worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.48 seconds)6960lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6961lb: must succeed: curl -sf localhost:9901/clusters6962worker1 # [ 151.665232] nix-grpc-daemon[1730]: E0000 00:00:1790096173.979893 1771 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6963worker1 # [ 151.675670] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:13Z level=info event=scheduler_connected addr=lb:500516964worker2 # [ 151.683498] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:14Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:466286965worker2 # [ 151.688810] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:14Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06966lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6967lb: must succeed: curl -sf localhost:9901/clusters6968lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6969lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)6970lb: must succeed: curl -sf localhost:9901/clusters6971lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6972worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6973worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6974(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 12.44 seconds)6975subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6976client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16977worker2 # [ 152.961325] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:352966978worker1 # [ 152.967796] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340086979worker1 # [ 152.979711] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)6980worker1 # [ 152.995415] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5)6981worker1 # [ 153.082090] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 2/5)6982worker1 # [ 153.167157] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 3/5)6983worker1 # [ 153.325215] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 4/5)6984worker1 # [ 153.344108] nix-grpc-daemon[1730]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6985worker1 # [ 153.347296] nix-grpc-daemon[1730]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6986worker2 # [ 153.401286] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:16Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06987worker2 # [ 153.405514] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:16Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16988worker2 # [ 153.409944] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:16Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06989worker2 # [ 153.414208] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:16Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06990worker1 # [ 153.440574] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340086991worker1 # [ 153.453793] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340086992worker1 # [ 153.465279] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340086993worker1 # [ 153.473555] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)6994worker1 # [ 153.496422] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:15Z level=info event=niks3_push_started pid=17906995worker1 # [ 153.601773] nix-grpc-daemon[1790]: time=2026-09-22T16:56:15.916Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6996worker1 # [ 153.688589] nix-daemon[845]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6997worker1 # [ 153.701381] nix-daemon[845]: reaped child process 1805, status = succeeded6998lb # [ 153.716312] niks3-server[1111]: 2026/09/22 16:56:16 INFO Received uploads request method=POST path=/api/pending_closures6999worker1 # [ 153.719847] nix-grpc-daemon[1790]: time=2026-09-22T16:56:16.034Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7000worker1 # [ 153.721317] nix-grpc-daemon[1790]: time=2026-09-22T16:56:16.036Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7001lb # [ 153.760324] niks3-server[1111]: 2026/09/22 16:56:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7002lb # [ 153.768402] niks3-server[1111]: 2026/09/22 16:56:16 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7003worker1 # [ 153.767887] nix-grpc-daemon[1790]: time=2026-09-22T16:56:16.082Z level=INFO msg="Uploading 1 narinfos"7004lb # [ 153.773866] niks3-server[1111]: 2026/09/22 16:56:16 INFO Signed narinfos id=62 count=17005lb # [ 153.782410] niks3-server[1111]: 2026/09/22 16:56:16 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7006lb # [ 153.794115] niks3-server[1111]: 2026/09/22 16:56:16 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7007lb # [ 153.798514] niks3-server[1111]: 2026/09/22 16:56:16 INFO Completed upload id=627008worker1 # [ 153.792713] nix-grpc-daemon[1790]: time=2026-09-22T16:56:16.107Z level=INFO msg="Upload complete. (187ms)"7009lb # [ 153.800562] niks3-server[1111]: 2026/09/22 16:56:16 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7010worker1 # [ 153.796756] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 paths=1 nar_bytes_in=5047011worker1 # [ 153.802084] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087012worker1 # [ 153.812896] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7013worker1 # [ 153.817213] nix-daemon[845]: reaped child process 1788, status = succeeded7014worker1 # [ 153.821362] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7015worker1 # [ 154.836401] nix-daemon[845]: reaped child process 1809, status = succeeded7016worker1 # [ 154.911298] nix-daemon[845]: accepted connection from pid 1825, user nix-grpc-daemon (trusted)7017worker1 # [ 154.923311] nix-daemon[845]: reaped child process 1828, status = succeeded7018lb # [ 154.938503] niks3-server[1111]: 2026/09/22 16:56:18 INFO Received uploads request method=POST path=/api/pending_closures7019worker1 # [ 154.940927] nix-grpc-daemon[1790]: time=2026-09-22T16:56:17.255Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7020worker1 # [ 154.942732] nix-grpc-daemon[1790]: time=2026-09-22T16:56:17.257Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7021lb # [ 154.975268] niks3-server[1111]: 2026/09/22 16:56:18 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7022lb # [ 154.986791] niks3-server[1111]: 2026/09/22 16:56:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7023lb # [ 154.993338] niks3-server[1111]: 2026/09/22 16:56:18 INFO Signed narinfos id=63 count=17024worker1 # [ 154.992739] nix-grpc-daemon[1790]: time=2026-09-22T16:56:17.306Z level=INFO msg="Uploading 1 narinfos"7025lb # [ 155.002484] niks3-server[1111]: 2026/09/22 16:56:18 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7026lb # [ 155.007532] niks3-server[1111]: 2026/09/22 16:56:18 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7027lb # [ 155.018219] niks3-server[1111]: 2026/09/22 16:56:18 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7028lb # [ 155.021695] niks3-server[1111]: 2026/09/22 16:56:18 INFO Completed upload id=637029worker1 # [ 155.015927] nix-grpc-daemon[1790]: time=2026-09-22T16:56:17.330Z level=INFO msg="Upload complete. (187ms)"7030lb # [ 155.024681] niks3-server[1111]: 2026/09/22 16:56:18 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7031worker1 # [ 155.021748] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=17032worker2 # [ 155.029296] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:18Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17033worker1 # [ 155.030917] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087034worker2 # [ 155.037923] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:18Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07035worker1 # [ 155.041591] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087036worker1 # [ 155.048539] nix-daemon[845]: reaped child process 1807, status = succeeded7037worker1 # [ 155.050608] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340087038worker1 # [ 155.054424] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7039worker1 # [ 155.139159] nix-daemon[845]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)7040worker1 # [ 155.152402] nix-daemon[845]: reaped child process 1835, status = succeeded7041lb # [ 155.164962] niks3-server[1111]: 2026/09/22 16:56:18 INFO Received uploads request method=POST path=/api/pending_closures7042worker1 # [ 155.166142] nix-grpc-daemon[1790]: time=2026-09-22T16:56:17.480Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7043worker1 # [ 155.168145] nix-grpc-daemon[1790]: time=2026-09-22T16:56:17.482Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7044lb # [ 155.192227] niks3-server[1111]: 2026/09/22 16:56:18 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7045lb # [ 155.201185] niks3-server[1111]: 2026/09/22 16:56:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7046lb # [ 155.205749] niks3-server[1111]: 2026/09/22 16:56:18 INFO Signed narinfos id=64 count=17047worker1 # [ 155.203108] nix-grpc-daemon[1790]: time=2026-09-22T16:56:17.517Z level=INFO msg="Uploading 1 narinfos"7048lb # [ 155.221185] niks3-server[1111]: 2026/09/22 16:56:18 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7049lb # [ 155.227074] niks3-server[1111]: 2026/09/22 16:56:18 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7050lb # [ 155.234252] niks3-server[1111]: 2026/09/22 16:56:18 INFO Completed upload id=647051lb # [ 155.236224] niks3-server[1111]: 2026/09/22 16:56:18 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7052worker1 # [ 155.230277] nix-grpc-daemon[1790]: time=2026-09-22T16:56:17.544Z level=INFO msg="Upload complete. (170ms)"7053worker1 # [ 155.234642] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 paths=1 nar_bytes_in=5047054worker1 # [ 155.240139] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087055worker1 # [ 155.248870] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7056worker1 # [ 155.256439] nix-daemon[845]: reaped child process 1830, status = succeeded7057worker1 # [ 155.261919] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7058worker1 # [ 156.475747] nix-daemon[845]: reaped child process 1839, status = succeeded7059worker1 # [ 156.551830] nix-daemon[845]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)7060worker1 # [ 156.564198] nix-daemon[845]: reaped child process 1858, status = succeeded7061lb # [ 156.578730] niks3-server[1111]: 2026/09/22 16:56:19 INFO Received uploads request method=POST path=/api/pending_closures7062worker1 # [ 156.581883] nix-grpc-daemon[1790]: time=2026-09-22T16:56:18.896Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7063worker1 # [ 156.583945] nix-grpc-daemon[1790]: time=2026-09-22T16:56:18.898Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7064lb # [ 156.615220] niks3-server[1111]: 2026/09/22 16:56:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7065lb # [ 156.621117] niks3-server[1111]: 2026/09/22 16:56:19 INFO Signed narinfos id=65 count=17066worker1 # [ 156.618161] nix-grpc-daemon[1790]: time=2026-09-22T16:56:18.932Z level=INFO msg="Uploading 1 narinfos"7067lb # [ 156.633131] niks3-server[1111]: 2026/09/22 16:56:19 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7068lb # [ 156.640887] niks3-server[1111]: 2026/09/22 16:56:19 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7069lb # [ 156.645943] envoy[915]: conn peer=[2001:db8:1::1]:38322 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9525 tx=4398 ms=48667070lb # [ 156.649342] envoy[915]: conn peer=[2001:db8:1::1]:42082 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2945 tx=1498 ms=77307071lb # [ 156.652423] envoy[915]: conn peer=[2001:db8:1::3]:44236 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1114 tx=448 ms=239767072lb # [ 156.659899] niks3-server[1111]: 2026/09/22 16:56:19 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7073lb # [ 156.662480] niks3-server[1111]: 2026/09/22 16:56:19 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7074lb # [ 156.665032] niks3-server[1111]: 2026/09/22 16:56:19 INFO Completed upload id=657075lb # [ 156.667090] niks3-server[1111]: 2026/09/22 16:56:19 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7076worker1 # [ 156.659415] nix-grpc-daemon[1790]: time=2026-09-22T16:56:18.973Z level=INFO msg="Upload complete. (191ms)"7077worker2 # [ 156.672890] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:19Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17078worker1 # [ 156.665577] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=17079worker1 # [ 156.674555] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087080worker2 # [ 156.682491] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:19Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07081worker1 # [ 156.686605] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087082worker1 # [ 156.692358] nix-daemon[845]: reaped child process 1837, status = succeeded7083worker1 # [ 156.697485] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340087084worker1 # [ 156.705041] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7085worker1 # [ 156.789036] nix-daemon[845]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)7086worker1 # [ 156.802476] nix-daemon[845]: reaped child process 1865, status = succeeded7087lb # [ 156.814266] niks3-server[1111]: 2026/09/22 16:56:20 INFO Received uploads request method=POST path=/api/pending_closures7088worker1 # [ 156.813692] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.128Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7089worker1 # [ 156.815741] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.129Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7090lb # [ 156.831817] niks3-server[1111]: 2026/09/22 16:56:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7091lb # [ 156.834042] niks3-server[1111]: 2026/09/22 16:56:20 INFO Signed narinfos id=66 count=17092worker1 # [ 156.829692] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.144Z level=INFO msg="Uploading 1 narinfos"7093lb # [ 156.835605] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7094lb # [ 156.842159] niks3-server[1111]: 2026/09/22 16:56:20 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7095lb # [ 156.847867] niks3-server[1111]: 2026/09/22 16:56:20 INFO Completed upload id=667096worker1 # [ 156.841731] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.156Z level=INFO msg="Upload complete. (131ms)"7097lb # [ 156.849659] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7098worker1 # [ 156.844197] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 paths=1 nar_bytes_in=5047099lb # [ 156.852303] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7100worker1 # [ 156.848704] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087101worker1 # [ 156.855259] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7102worker1 # [ 156.857439] nix-daemon[845]: reaped child process 1860, status = succeeded7103worker1 # [ 156.862531] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7104worker1 # [ 157.276362] nix-daemon[845]: reaped child process 1869, status = succeeded7105worker1 # [ 157.348673] nix-daemon[845]: accepted connection from pid 1885, user nix-grpc-daemon (trusted)7106worker1 # [ 157.362769] nix-daemon[845]: reaped child process 1888, status = succeeded7107lb # [ 157.372595] niks3-server[1111]: 2026/09/22 16:56:20 INFO Received uploads request method=POST path=/api/pending_closures7108worker1 # [ 157.372065] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.685Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7109worker1 # [ 157.373500] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.685Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7110lb # [ 157.391982] niks3-server[1111]: 2026/09/22 16:56:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7111worker1 # [ 157.388514] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.703Z level=INFO msg="Uploading 1 narinfos"7112lb # [ 157.394659] niks3-server[1111]: 2026/09/22 16:56:20 INFO Signed narinfos id=67 count=17113lb # [ 157.396726] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7114lb # [ 157.401085] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7115lb # [ 157.403523] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7116lb # [ 157.410215] niks3-server[1111]: 2026/09/22 16:56:20 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7117lb # [ 157.412867] niks3-server[1111]: 2026/09/22 16:56:20 INFO Completed upload id=677118worker1 # [ 157.406919] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.721Z level=INFO msg="Upload complete. (138ms)"7119lb # [ 157.415797] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7120worker1 # [ 157.409265] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=17121worker1 # [ 157.416398] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087122worker2 # [ 157.418878] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:20Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17123worker1 # [ 157.419572] nix-daemon[845]: reaped child process 1867, status = succeeded7124worker1 # [ 157.423463] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087125worker2 # [ 157.428546] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:20Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07126worker1 # [ 157.429200] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.743Z level=INFO msg="All 1 paths already cached"7127worker1 # [ 157.433119] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340087128worker1 # [ 157.436824] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7129worker1 # [ 157.520185] nix-daemon[845]: accepted connection from pid 1892, user nix-grpc-daemon (trusted)7130worker1 # [ 157.533546] nix-daemon[845]: reaped child process 1895, status = succeeded7131lb # [ 157.545276] niks3-server[1111]: 2026/09/22 16:56:20 INFO Received uploads request method=POST path=/api/pending_closures7132worker1 # [ 157.547506] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.861Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7133worker1 # [ 157.548873] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.861Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7134lb # [ 157.573126] niks3-server[1111]: 2026/09/22 16:56:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7135lb # [ 157.580312] niks3-server[1111]: 2026/09/22 16:56:20 INFO Signed narinfos id=68 count=17136worker1 # [ 157.577353] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.891Z level=INFO msg="Uploading 1 narinfos"7137lb # [ 157.585359] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7138lb # [ 157.595428] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7139lb # [ 157.605151] niks3-server[1111]: 2026/09/22 16:56:20 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7140lb # [ 157.612940] niks3-server[1111]: 2026/09/22 16:56:20 INFO Completed upload id=687141worker1 # [ 157.609071] nix-grpc-daemon[1790]: time=2026-09-22T16:56:19.921Z level=INFO msg="Upload complete. (164ms)"7142lb # [ 157.615703] niks3-server[1111]: 2026/09/22 16:56:20 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7143worker1 # [ 157.610649] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 paths=1 nar_bytes_in=6247144worker1 # [ 157.613143] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087145worker1 # [ 157.619944] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7146worker1 # [ 157.623939] nix-daemon[845]: reaped child process 1890, status = succeeded7147worker1 # [ 157.628600] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7148worker1 # [ 159.263734] nix-daemon[845]: reaped child process 1899, status = succeeded7149worker1 # [ 159.348082] nix-daemon[845]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)7150worker1 # [ 159.359601] nix-daemon[845]: reaped child process 1918, status = succeeded7151lb # [ 159.372968] niks3-server[1111]: 2026/09/22 16:56:22 INFO Received uploads request method=POST path=/api/pending_closures7152worker1 # [ 159.375298] nix-grpc-daemon[1790]: time=2026-09-22T16:56:21.689Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7153worker1 # [ 159.376923] nix-grpc-daemon[1790]: time=2026-09-22T16:56:21.691Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7154lb # [ 159.410693] niks3-server[1111]: 2026/09/22 16:56:22 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7155lb # [ 159.424337] niks3-server[1111]: 2026/09/22 16:56:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7156lb # [ 159.431363] niks3-server[1111]: 2026/09/22 16:56:22 INFO Signed narinfos id=69 count=17157worker1 # [ 159.428296] nix-grpc-daemon[1790]: time=2026-09-22T16:56:21.742Z level=INFO msg="Uploading 1 narinfos"7158lb # [ 159.438722] niks3-server[1111]: 2026/09/22 16:56:22 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7159lb # [ 159.443562] niks3-server[1111]: 2026/09/22 16:56:22 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7160lb # [ 159.455567] niks3-server[1111]: 2026/09/22 16:56:22 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7161lb # [ 159.459981] niks3-server[1111]: 2026/09/22 16:56:22 INFO Completed upload id=697162worker1 # [ 159.454337] nix-grpc-daemon[1790]: time=2026-09-22T16:56:21.768Z level=INFO msg="Upload complete. (215ms)"7163lb # [ 159.461943] niks3-server[1111]: 2026/09/22 16:56:22 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7164worker1 # [ 159.460451] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=17165worker2 # [ 159.470447] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:22Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17166worker1 # [ 159.469328] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087167worker2 # [ 159.477360] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:22Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07168worker1 # [ 159.480607] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087169worker1 # [ 159.487895] nix-daemon[845]: reaped child process 1897, status = succeeded7170worker1 # [ 159.493181] nix-grpc-daemon[1790]: time=2026-09-22T16:56:21.807Z level=INFO msg="All 1 paths already cached"7171worker1 # [ 159.498708] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340087172worker1 # [ 159.506438] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7173worker1 # [ 159.589074] nix-daemon[845]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)7174worker1 # [ 159.601348] nix-daemon[845]: reaped child process 1925, status = succeeded7175lb # [ 159.615148] niks3-server[1111]: 2026/09/22 16:56:22 INFO Received uploads request method=POST path=/api/pending_closures7176worker1 # [ 159.617070] nix-grpc-daemon[1790]: time=2026-09-22T16:56:21.931Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7177worker1 # [ 159.618488] nix-grpc-daemon[1790]: time=2026-09-22T16:56:21.933Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7178lb # [ 159.645401] niks3-server[1111]: 2026/09/22 16:56:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7179worker1 # [ 159.645955] nix-grpc-daemon[1790]: time=2026-09-22T16:56:21.960Z level=INFO msg="Uploading 1 narinfos"7180lb # [ 159.652244] niks3-server[1111]: 2026/09/22 16:56:22 INFO Signed narinfos id=70 count=17181lb # [ 159.658843] niks3-server[1111]: 2026/09/22 16:56:22 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7182lb # [ 159.667425] niks3-server[1111]: 2026/09/22 16:56:22 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7183lb # [ 159.678679] niks3-server[1111]: 2026/09/22 16:56:22 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7184lb # [ 159.686249] niks3-server[1111]: 2026/09/22 16:56:22 INFO Completed upload id=707185worker1 # [ 159.680556] nix-grpc-daemon[1790]: time=2026-09-22T16:56:21.995Z level=INFO msg="Upload complete. (169ms)"7186lb # [ 159.687807] niks3-server[1111]: 2026/09/22 16:56:22 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7187worker1 # [ 159.684560] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 paths=1 nar_bytes_in=6247188worker1 # [ 159.689356] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087189worker1 # [ 159.698650] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7190worker1 # [ 159.706632] nix-daemon[845]: reaped child process 1920, status = succeeded7191worker1 # [ 159.711223] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7192worker1 # [ 160.950427] nix-daemon[845]: reaped child process 1929, status = succeeded7193worker1 # [ 161.030691] nix-daemon[845]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)7194worker1 # [ 161.043336] nix-daemon[845]: reaped child process 1948, status = succeeded7195lb # [ 161.057349] niks3-server[1111]: 2026/09/22 16:56:24 INFO Received uploads request method=POST path=/api/pending_closures7196worker1 # [ 161.059357] nix-grpc-daemon[1790]: time=2026-09-22T16:56:23.373Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7197worker1 # [ 161.060827] nix-grpc-daemon[1790]: time=2026-09-22T16:56:23.375Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7198lb # [ 161.104961] niks3-server[1111]: 2026/09/22 16:56:24 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7199lb # [ 161.110189] niks3-server[1111]: 2026/09/22 16:56:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7200lb # [ 161.115379] niks3-server[1111]: 2026/09/22 16:56:24 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7201worker1 # [ 161.114957] nix-grpc-daemon[1790]: time=2026-09-22T16:56:23.429Z level=INFO msg="Uploading 1 narinfos"7202lb # [ 161.120662] niks3-server[1111]: 2026/09/22 16:56:24 INFO Signed narinfos id=71 count=17203lb # [ 161.129115] niks3-server[1111]: 2026/09/22 16:56:24 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7204lb # [ 161.137150] niks3-server[1111]: 2026/09/22 16:56:24 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7205lb # [ 161.140199] niks3-server[1111]: 2026/09/22 16:56:24 INFO Completed upload id=717206worker1 # [ 161.135948] nix-grpc-daemon[1790]: time=2026-09-22T16:56:23.450Z level=INFO msg="Upload complete. (216ms)"7207lb # [ 161.143941] niks3-server[1111]: 2026/09/22 16:56:24 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7208worker2 # [ 161.148809] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:24Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17209worker1 # [ 161.144080] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=17210worker1 # [ 161.151479] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087211worker2 # [ 161.158128] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:24Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07212worker1 # [ 161.162419] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087213worker1 # [ 161.170723] nix-daemon[845]: reaped child process 1927, status = succeeded7214worker1 # [ 161.173698] nix-grpc-daemon[1790]: time=2026-09-22T16:56:23.487Z level=INFO msg="All 1 paths already cached"7215worker1 # [ 161.179223] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340087216worker1 # [ 161.185804] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7217worker1 # [ 161.267666] nix-daemon[845]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)7218worker1 # [ 161.280475] nix-daemon[845]: reaped child process 1955, status = succeeded7219lb # [ 161.292831] niks3-server[1111]: 2026/09/22 16:56:24 INFO Received uploads request method=POST path=/api/pending_closures7220worker1 # [ 161.295206] nix-grpc-daemon[1790]: time=2026-09-22T16:56:23.609Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7221worker1 # [ 161.297155] nix-grpc-daemon[1790]: time=2026-09-22T16:56:23.611Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7222lb # [ 161.322408] niks3-server[1111]: 2026/09/22 16:56:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7223worker1 # [ 161.326690] nix-grpc-daemon[1790]: time=2026-09-22T16:56:23.641Z level=INFO msg="Uploading 1 narinfos"7224lb # [ 161.332182] niks3-server[1111]: 2026/09/22 16:56:24 INFO Signed narinfos id=72 count=17225lb # [ 161.338363] niks3-server[1111]: 2026/09/22 16:56:24 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7226lb # [ 161.342520] niks3-server[1111]: 2026/09/22 16:56:24 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7227lb # [ 161.354771] niks3-server[1111]: 2026/09/22 16:56:24 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7228worker1 # [ 161.355410] nix-grpc-daemon[1790]: time=2026-09-22T16:56:23.670Z level=INFO msg="Upload complete. (164ms)"7229lb # [ 161.361456] niks3-server[1111]: 2026/09/22 16:56:24 INFO Completed upload id=727230worker1 # [ 161.357460] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 paths=1 nar_bytes_in=6247231lb # [ 161.364923] niks3-server[1111]: 2026/09/22 16:56:24 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7232worker1 # [ 161.362536] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087233worker1 # [ 161.370116] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7234worker1 # [ 161.376780] nix-daemon[845]: reaped child process 1950, status = succeeded7235worker1 # [ 161.380942] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7236worker1 # [ 162.812853] nix-daemon[845]: reaped child process 1959, status = succeeded7237worker1 # [ 162.900632] nix-daemon[845]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)7238worker1 # [ 162.913080] nix-daemon[845]: reaped child process 1978, status = succeeded7239lb # [ 162.926913] niks3-server[1111]: 2026/09/22 16:56:26 INFO Received uploads request method=POST path=/api/pending_closures7240worker1 # [ 162.929161] nix-grpc-daemon[1790]: time=2026-09-22T16:56:25.243Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7241worker1 # [ 162.930847] nix-grpc-daemon[1790]: time=2026-09-22T16:56:25.245Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7242lb # [ 162.968607] niks3-server[1111]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7243lb # [ 162.975726] niks3-server[1111]: 2026/09/22 16:56:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7244lb # [ 162.983979] niks3-server[1111]: 2026/09/22 16:56:26 INFO Signed narinfos id=73 count=17245worker1 # [ 162.981863] nix-grpc-daemon[1790]: time=2026-09-22T16:56:25.296Z level=INFO msg="Uploading 1 narinfos"7246lb # [ 162.988823] niks3-server[1111]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7247lb # [ 162.996091] niks3-server[1111]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7248lb # [ 163.006461] niks3-server[1111]: 2026/09/22 16:56:26 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7249lb # [ 163.011096] niks3-server[1111]: 2026/09/22 16:56:26 INFO Completed upload id=737250worker1 # [ 163.005131] nix-grpc-daemon[1790]: time=2026-09-22T16:56:25.319Z level=INFO msg="Upload complete. (213ms)"7251lb # [ 163.015237] niks3-server[1111]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7252worker1 # [ 163.010613] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=17253worker2 # [ 163.018834] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:26Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17254worker1 # [ 163.019501] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087255worker2 # [ 163.028741] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:26Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07256worker1 # [ 163.033159] nix-daemon[845]: reaped child process 1957, status = succeeded7257worker1 # [ 163.060277] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087258worker1 # [ 163.071327] nix-grpc-daemon[1790]: time=2026-09-22T16:56:25.385Z level=INFO msg="All 2 paths already cached"7259worker1 # [ 163.078988] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340087260worker1 # [ 163.088160] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7261worker1 # [ 163.178574] nix-daemon[845]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)7262worker1 # [ 163.190835] nix-daemon[845]: reaped child process 1985, status = succeeded7263lb # [ 163.204588] niks3-server[1111]: 2026/09/22 16:56:26 INFO Received uploads request method=POST path=/api/pending_closures7264worker1 # [ 163.205801] nix-grpc-daemon[1790]: time=2026-09-22T16:56:25.520Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7265worker1 # [ 163.207701] nix-grpc-daemon[1790]: time=2026-09-22T16:56:25.521Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7266lb # [ 163.232425] niks3-server[1111]: 2026/09/22 16:56:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7267lb # [ 163.238557] niks3-server[1111]: 2026/09/22 16:56:26 INFO Signed narinfos id=74 count=17268worker1 # [ 163.239814] nix-grpc-daemon[1790]: time=2026-09-22T16:56:25.554Z level=INFO msg="Uploading 1 narinfos"7269lb # [ 163.247205] niks3-server[1111]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7270lb # [ 163.255701] niks3-server[1111]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7271lb # [ 163.267220] niks3-server[1111]: 2026/09/22 16:56:26 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7272lb # [ 163.272831] niks3-server[1111]: 2026/09/22 16:56:26 INFO Completed upload id=747273worker1 # [ 163.267110] nix-grpc-daemon[1790]: time=2026-09-22T16:56:25.581Z level=INFO msg="Upload complete. (168ms)"7274lb # [ 163.277840] niks3-server[1111]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7275worker1 # [ 163.270981] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 paths=1 nar_bytes_in=7527276worker1 # [ 163.276792] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087277worker1 # [ 163.287107] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7278worker1 # [ 163.291214] nix-daemon[845]: reaped child process 1980, status = succeeded7279worker1 # [ 163.296221] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7280worker1 # [ 164.310925] nix-daemon[845]: reaped child process 1989, status = succeeded7281worker1 # [ 164.384457] nix-daemon[845]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)7282worker1 # [ 164.397613] nix-daemon[845]: reaped child process 2008, status = succeeded7283lb # [ 164.410991] niks3-server[1111]: 2026/09/22 16:56:27 INFO Received uploads request method=POST path=/api/pending_closures7284worker1 # [ 164.413182] nix-grpc-daemon[1790]: time=2026-09-22T16:56:26.727Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7285worker1 # [ 164.414964] nix-grpc-daemon[1790]: time=2026-09-22T16:56:26.729Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7286lb # [ 164.454166] niks3-server[1111]: 2026/09/22 16:56:27 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7287lb # [ 164.462862] niks3-server[1111]: 2026/09/22 16:56:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7288lb # [ 164.470447] niks3-server[1111]: 2026/09/22 16:56:27 INFO Signed narinfos id=75 count=17289worker1 # [ 164.468794] nix-grpc-daemon[1790]: time=2026-09-22T16:56:26.783Z level=INFO msg="Uploading 1 narinfos"7290lb # [ 164.478250] niks3-server[1111]: 2026/09/22 16:56:27 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7291lb # [ 164.485044] niks3-server[1111]: 2026/09/22 16:56:27 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7292lb # [ 164.492595] niks3-server[1111]: 2026/09/22 16:56:27 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7293lb # [ 164.496529] niks3-server[1111]: 2026/09/22 16:56:27 INFO Completed upload id=757294lb # [ 164.499158] niks3-server[1111]: 2026/09/22 16:56:27 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7295worker1 # [ 164.492098] nix-grpc-daemon[1790]: time=2026-09-22T16:56:26.806Z level=INFO msg="Upload complete. (187ms)"7296worker2 # [ 164.504653] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:27Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17297worker1 # [ 164.498931] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=17298worker1 # [ 164.513707] nix-daemon[845]: reaped child process 1987, status = succeeded7299worker1 # [ 164.542516] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087300worker1 # [ 164.555114] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087301worker1 # [ 164.567917] nix-grpc-daemon[1790]: time=2026-09-22T16:56:26.882Z level=INFO msg="All 2 paths already cached"7302worker1 # [ 164.576617] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340087303worker1 # [ 164.586156] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7304worker1 # [ 164.680783] nix-daemon[845]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)7305worker1 # [ 164.693735] nix-daemon[845]: reaped child process 2015, status = succeeded7306lb # [ 164.706806] niks3-server[1111]: 2026/09/22 16:56:27 INFO Received uploads request method=POST path=/api/pending_closures7307worker1 # [ 164.708484] nix-grpc-daemon[1790]: time=2026-09-22T16:56:27.023Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7308worker1 # [ 164.709916] nix-grpc-daemon[1790]: time=2026-09-22T16:56:27.024Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7309lb # [ 164.741335] niks3-server[1111]: 2026/09/22 16:56:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7310lb # [ 164.747364] niks3-server[1111]: 2026/09/22 16:56:27 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7311lb # [ 164.752230] niks3-server[1111]: 2026/09/22 16:56:27 INFO Signed narinfos id=76 count=17312worker1 # [ 164.750227] nix-grpc-daemon[1790]: time=2026-09-22T16:56:27.064Z level=INFO msg="Uploading 1 narinfos"7313lb # [ 164.758249] niks3-server[1111]: 2026/09/22 16:56:27 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7314lb # [ 164.773124] niks3-server[1111]: 2026/09/22 16:56:27 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7315lb # [ 164.780854] niks3-server[1111]: 2026/09/22 16:56:27 INFO Completed upload id=767316worker1 # [ 164.774955] nix-grpc-daemon[1790]: time=2026-09-22T16:56:27.089Z level=INFO msg="Upload complete. (177ms)"7317lb # [ 164.786340] niks3-server[1111]: 2026/09/22 16:56:27 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7318worker1 # [ 164.778948] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 paths=1 nar_bytes_in=7527319worker1 # [ 164.784216] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087320worker1 # [ 164.795109] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7321worker1 # [ 164.804375] nix-daemon[845]: reaped child process 2010, status = succeeded7322worker1 # [ 164.807027] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7323worker1 # [ 165.634134] nix-daemon[845]: reaped child process 2019, status = succeeded7324worker1 # [ 165.708814] nix-daemon[845]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)7325worker1 # [ 165.723592] nix-daemon[845]: reaped child process 2038, status = succeeded7326lb # [ 165.733780] niks3-server[1111]: 2026/09/22 16:56:28 INFO Received uploads request method=POST path=/api/pending_closures7327worker1 # [ 165.735993] nix-grpc-daemon[1790]: time=2026-09-22T16:56:28.050Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7328worker1 # [ 165.738145] nix-grpc-daemon[1790]: time=2026-09-22T16:56:28.052Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7329lb # [ 165.771257] niks3-server[1111]: 2026/09/22 16:56:28 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7330lb # [ 165.783599] niks3-server[1111]: 2026/09/22 16:56:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7331worker1 # [ 165.782457] nix-grpc-daemon[1790]: time=2026-09-22T16:56:28.096Z level=INFO msg="Uploading 1 narinfos"7332lb # [ 165.790135] niks3-server[1111]: 2026/09/22 16:56:28 INFO Signed narinfos id=77 count=17333lb # [ 165.795888] niks3-server[1111]: 2026/09/22 16:56:28 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7334lb # [ 165.802826] niks3-server[1111]: 2026/09/22 16:56:28 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7335lb # [ 165.812270] niks3-server[1111]: 2026/09/22 16:56:29 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7336lb # [ 165.820082] niks3-server[1111]: 2026/09/22 16:56:29 INFO Completed upload id=777337lb # [ 165.821829] niks3-server[1111]: 2026/09/22 16:56:29 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7338worker1 # [ 165.814274] nix-grpc-daemon[1790]: time=2026-09-22T16:56:28.128Z level=INFO msg="Upload complete. (199ms)"7339worker2 # [ 165.826694] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:28Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17340worker1 # [ 165.820694] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=17341worker1 # [ 165.836527] nix-daemon[845]: reaped child process 2017, status = succeeded7342worker1 # [ 165.870426] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087343worker1 # [ 165.883170] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087344worker1 # [ 165.895396] nix-grpc-daemon[1790]: time=2026-09-22T16:56:28.209Z level=INFO msg="All 2 paths already cached"7345worker1 # [ 165.903466] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:340087346worker1 # [ 165.911728] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7347worker1 # [ 166.008310] nix-daemon[845]: accepted connection from pid 2042, user nix-grpc-daemon (trusted)7348worker1 # [ 166.021273] nix-daemon[845]: reaped child process 2045, status = succeeded7349lb # [ 166.033714] niks3-server[1111]: 2026/09/22 16:56:29 INFO Received uploads request method=POST path=/api/pending_closures7350worker1 # [ 166.038584] nix-grpc-daemon[1790]: time=2026-09-22T16:56:28.353Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7351worker1 # [ 166.040470] nix-grpc-daemon[1790]: time=2026-09-22T16:56:28.355Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7352lb # [ 166.067349] niks3-server[1111]: 2026/09/22 16:56:29 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7353lb # [ 166.077312] niks3-server[1111]: 2026/09/22 16:56:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7354lb # [ 166.082269] niks3-server[1111]: 2026/09/22 16:56:29 INFO Signed narinfos id=78 count=17355worker1 # [ 166.079044] nix-grpc-daemon[1790]: time=2026-09-22T16:56:28.393Z level=INFO msg="Uploading 1 narinfos"7356lb # [ 166.092133] niks3-server[1111]: 2026/09/22 16:56:29 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7357lb # [ 166.103867] niks3-server[1111]: 2026/09/22 16:56:29 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7358lb # [ 166.110822] niks3-server[1111]: 2026/09/22 16:56:29 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7359lb # [ 166.113592] niks3-server[1111]: 2026/09/22 16:56:29 INFO Completed upload id=787360worker1 # [ 166.107865] nix-grpc-daemon[1790]: time=2026-09-22T16:56:28.422Z level=INFO msg="Upload complete. (182ms)"7361worker1 # [ 166.112519] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=0 paths=1 nar_bytes_in=7527362worker1 # [ 166.117929] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:340087363worker1 # [ 166.130110] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7364worker1 # [ 166.137380] nix-daemon[845]: reaped child process 2040, status = succeeded7365worker1 # [ 166.142450] nix-daemon[845]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)7366worker1 # [ 167.180559] nix-daemon[845]: reaped child process 2049, status = succeeded7367worker1 # [ 167.264464] nix-daemon[845]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)7368worker1 # [ 167.277030] nix-daemon[845]: reaped child process 2068, status = succeeded7369lb # [ 167.290991] niks3-server[1111]: 2026/09/22 16:56:30 INFO Received uploads request method=POST path=/api/pending_closures7370worker1 # [ 167.293731] nix-grpc-daemon[1790]: time=2026-09-22T16:56:29.608Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7371worker1 # [ 167.295341] nix-grpc-daemon[1790]: time=2026-09-22T16:56:29.609Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7372lb # [ 167.331844] niks3-server[1111]: 2026/09/22 16:56:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7373worker1 # [ 167.331912] nix-grpc-daemon[1790]: time=2026-09-22T16:56:29.646Z level=INFO msg="Uploading 1 narinfos"7374lb # [ 167.338195] niks3-server[1111]: 2026/09/22 16:56:30 INFO Signed narinfos id=79 count=17375lb # [ 167.346172] niks3-server[1111]: 2026/09/22 16:56:30 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7376lb # [ 167.351648] niks3-server[1111]: 2026/09/22 16:56:30 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7377lb # [ 167.364763] niks3-server[1111]: 2026/09/22 16:56:30 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7378lb # [ 167.371156] niks3-server[1111]: 2026/09/22 16:56:30 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7379lb # [ 167.373899] niks3-server[1111]: 2026/09/22 16:56:30 INFO Completed upload id=797380lb # [ 167.375250] niks3-server[1111]: 2026/09/22 16:56:30 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7381worker1 # [ 167.368177] nix-grpc-daemon[1790]: time=2026-09-22T16:56:29.682Z level=INFO msg="Upload complete. (216ms)"7382worker1 # [ 167.374783] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34008 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=17383worker1 # [ 167.389347] nix-daemon[845]: reaped child process 2047, status = succeeded7384client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.58 seconds)7385worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7386worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)7387client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7388client # [ 167.517888] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7389client # Running as unit: gpu.service; invocation ID: 0528216dbdef42adb7d67f91b8f1b56d7390client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.05 seconds)7391lb: must succeed: curl -sf localhost:9901/clusters7392lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7393lb: must succeed: curl -sf localhost:9901/clusters7394lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7395worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7396worker2 # [ 167.633797] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:352967397worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7398client # [ 167.661659] nix[1536]: don't know how to build these paths:7399client # [ 167.662653] nix[1536]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7400worker1 # [ 167.634641] nix-grpc-daemon[1730]: ts=2026-09-22T16:56:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087401worker2 # [ 167.654182] nix-grpc-daemon[1635]: ts=2026-09-22T16:56:30Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07402client # [ 167.676824] nix[1536]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7403lb: must succeed: curl -sf localhost:9901/clusters7404lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7405lb: must succeed: curl -sf localhost:9901/clusters7406lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7407worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7408worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7409client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7410lb # [ 176.645520] envoy[915]: conn peer=[2001:db8:1::1]:38708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27029 tx=12344 ms=144357411client # [ 197.679202] nix[1536]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7412client # [ 197.686657] nix[1536]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7413client # [ 197.718649] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7414client # [ 197.722511] systemd[1]: gpu.service: Failed with result 'exit-code'.7415client # [ 197.727084] systemd[1]: gpu.service: Consumed 138ms CPU time over 30.204s wall clock time, 35.6M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7416client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.11 seconds)7417client: must succeed: journalctl -u gpu -o cat7418client: (finished: must succeed: journalctl -u gpu -o cat, in 0.01 seconds)7419lb: must succeed: curl -sf localhost:9901/clusters7420lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7421lb: must succeed: curl -sf localhost:9901/clusters7422lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7423worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7424worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7425(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.07 seconds)7426subtest: low disk drains a worker and builds go to the other7427worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7428worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.89 seconds)7429worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7430worker1 # [ 199.638988] nix-grpc-daemon[1730]: ts=2026-09-22T16:57:01Z level=info event=unhealthy reason=min_free7431worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.06 seconds)7432client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27433client # don't know how to build these paths:7434worker1 # [ 199.989502] nix-grpc-daemon[1730]: ts=2026-09-22T16:57:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:340087435client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7436worker1 # [ 199.993402] nix-grpc-daemon[1730]: ts=2026-09-22T16:57:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:340087437worker2 # [ 200.018910] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:03Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07438worker2 # [ 200.023841] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:03Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17439worker2 # [ 200.063250] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352967440worker2 # [ 200.078286] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352967441client # copying 1 paths to 'lb:50051'...7442client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7443worker2 # [ 200.093302] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352967444worker2 # [ 200.105284] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7445worker2 # [ 200.212337] nix-daemon[865]: accepted connection from pid 1914, user nix-grpc-daemon (trusted)7446worker2 # [ 200.228415] nix-daemon[865]: reaped child process 1917, status = succeeded7447lb # [ 200.234842] niks3-server[1111]: 2026/09/22 16:57:03 INFO Received uploads request method=POST path=/api/pending_closures7448worker2 # [ 200.239454] nix-grpc-daemon[1711]: time=2026-09-22T16:57:03.387Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7449worker2 # [ 200.240868] nix-grpc-daemon[1711]: time=2026-09-22T16:57:03.388Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7450lb # [ 200.249480] niks3-server[1111]: 2026/09/22 16:57:03 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7451lb # [ 200.252585] niks3-server[1111]: 2026/09/22 16:57:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7452lb # [ 200.255142] niks3-server[1111]: 2026/09/22 16:57:03 INFO Signed narinfos id=80 count=17453worker2 # [ 200.256371] nix-grpc-daemon[1711]: time=2026-09-22T16:57:03.405Z level=INFO msg="Uploading 1 narinfos"7454lb # [ 200.261337] niks3-server[1111]: 2026/09/22 16:57:03 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7455lb # [ 200.265466] niks3-server[1111]: 2026/09/22 16:57:03 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7456lb # [ 200.269287] niks3-server[1111]: 2026/09/22 16:57:03 INFO Completed upload id=807457worker2 # [ 200.268804] nix-grpc-daemon[1711]: time=2026-09-22T16:57:03.417Z level=INFO msg="Upload complete. (151ms)"7458worker2 # [ 200.271078] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=0 paths=1 nar_bytes_in=5127459lb # [ 200.272402] niks3-server[1111]: 2026/09/22 16:57:03 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7460worker2 # [ 200.274568] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352967461worker2 # [ 200.281819] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7462worker2 # [ 200.284183] nix-daemon[865]: reaped child process 1911, status = succeeded7463client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7464worker2 # [ 200.289501] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7465client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7466client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7467client # farm-a> LOG-farm-a-drain7468worker2 # [ 200.927059] nix-daemon[865]: reaped child process 1921, status = succeeded7469worker2 # [ 200.987099] nix-daemon[865]: accepted connection from pid 1937, user nix-grpc-daemon (trusted)7470worker2 # [ 201.001859] nix-daemon[865]: reaped child process 1940, status = succeeded7471lb # [ 201.004978] niks3-server[1111]: 2026/09/22 16:57:04 INFO Received uploads request method=POST path=/api/pending_closures7472worker2 # [ 201.012825] nix-grpc-daemon[1711]: time=2026-09-22T16:57:04.161Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7473worker2 # [ 201.014360] nix-grpc-daemon[1711]: time=2026-09-22T16:57:04.163Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7474lb # [ 201.043548] niks3-server[1111]: 2026/09/22 16:57:04 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7475lb # [ 201.054872] niks3-server[1111]: 2026/09/22 16:57:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7476lb # [ 201.060384] niks3-server[1111]: 2026/09/22 16:57:04 INFO Signed narinfos id=81 count=17477worker2 # [ 201.063718] nix-grpc-daemon[1711]: time=2026-09-22T16:57:04.212Z level=INFO msg="Uploading 1 narinfos"7478lb # [ 201.069590] niks3-server[1111]: 2026/09/22 16:57:04 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7479lb # [ 201.076479] niks3-server[1111]: 2026/09/22 16:57:04 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7480lb # [ 201.087291] niks3-server[1111]: 2026/09/22 16:57:04 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7481lb # [ 201.090460] niks3-server[1111]: 2026/09/22 16:57:04 INFO Completed upload id=817482worker2 # [ 201.090312] nix-grpc-daemon[1711]: time=2026-09-22T16:57:04.239Z level=INFO msg="Upload complete. (193ms)"7483lb # [ 201.093270] niks3-server[1111]: 2026/09/22 16:57:04 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7484worker2 # [ 201.096602] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=0 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=17485worker2 # [ 201.107421] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:04Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07486worker2 # [ 201.113210] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:04Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17487worker2 # [ 201.117360] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352967488worker2 # [ 201.128754] nix-daemon[865]: reaped child process 1919, status = succeeded7489worker2 # [ 201.130151] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352967490worker2 # [ 201.137456] nix-grpc-daemon[1711]: time=2026-09-22T16:57:04.286Z level=INFO msg="All 1 paths already cached"7491client # copying 1 paths to 'lb:50051'...7492worker2 # [ 201.143406] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352967493client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7494worker2 # [ 201.149573] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7495worker2 # [ 201.235844] nix-daemon[865]: accepted connection from pid 1946, user nix-grpc-daemon (trusted)7496worker2 # [ 201.250762] nix-daemon[865]: reaped child process 1949, status = succeeded7497lb # [ 201.259098] niks3-server[1111]: 2026/09/22 16:57:04 INFO Received uploads request method=POST path=/api/pending_closures7498worker2 # [ 201.264898] nix-grpc-daemon[1711]: time=2026-09-22T16:57:04.413Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7499worker2 # [ 201.266952] nix-grpc-daemon[1711]: time=2026-09-22T16:57:04.415Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7500lb # [ 201.286590] niks3-server[1111]: 2026/09/22 16:57:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7501worker2 # [ 201.296921] nix-grpc-daemon[1711]: time=2026-09-22T16:57:04.445Z level=INFO msg="Uploading 1 narinfos"7502lb # [ 201.298204] niks3-server[1111]: 2026/09/22 16:57:04 INFO Signed narinfos id=82 count=17503lb # [ 201.301823] niks3-server[1111]: 2026/09/22 16:57:04 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7504lb # [ 201.308188] niks3-server[1111]: 2026/09/22 16:57:04 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7505lb # [ 201.320216] niks3-server[1111]: 2026/09/22 16:57:04 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7506lb # [ 201.327566] niks3-server[1111]: 2026/09/22 16:57:04 INFO Completed upload id=827507worker2 # [ 201.327278] nix-grpc-daemon[1711]: time=2026-09-22T16:57:04.476Z level=INFO msg="Upload complete. (172ms)"7508lb # [ 201.329694] niks3-server[1111]: 2026/09/22 16:57:04 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7509worker2 # [ 201.331337] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=0 paths=1 nar_bytes_in=6247510worker2 # [ 201.337147] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352967511worker2 # [ 201.345703] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7512worker2 # [ 201.352912] nix-daemon[865]: reaped child process 1944, status = succeeded7513worker2 # [ 201.359066] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7514client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7515client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7516client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7517client # farm-b> LOG-farm-b-drain7518worker2 # [ 202.187256] nix-daemon[865]: reaped child process 1953, status = succeeded7519worker2 # [ 202.264096] nix-daemon[865]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)7520worker2 # [ 202.277408] nix-daemon[865]: reaped child process 1972, status = succeeded7521lb # [ 202.283719] niks3-server[1111]: 2026/09/22 16:57:05 INFO Received uploads request method=POST path=/api/pending_closures7522worker2 # [ 202.292414] nix-grpc-daemon[1711]: time=2026-09-22T16:57:05.441Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7523worker2 # [ 202.295991] nix-grpc-daemon[1711]: time=2026-09-22T16:57:05.443Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7524lb # [ 202.328357] niks3-server[1111]: 2026/09/22 16:57:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7525lb # [ 202.334121] niks3-server[1111]: 2026/09/22 16:57:05 INFO Signed narinfos id=83 count=17526worker2 # [ 202.337105] nix-grpc-daemon[1711]: time=2026-09-22T16:57:05.485Z level=INFO msg="Uploading 1 narinfos"7527lb # [ 202.340474] niks3-server[1111]: 2026/09/22 16:57:05 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7528lb # [ 202.343412] niks3-server[1111]: 2026/09/22 16:57:05 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7529lb # [ 202.345417] niks3-server[1111]: 2026/09/22 16:57:05 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7530lb # [ 202.351449] niks3-server[1111]: 2026/09/22 16:57:05 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7531lb # [ 202.354290] niks3-server[1111]: 2026/09/22 16:57:05 INFO Completed upload id=837532worker2 # [ 202.354084] nix-grpc-daemon[1711]: time=2026-09-22T16:57:05.503Z level=INFO msg="Upload complete. (189ms)"7533lb # [ 202.357518] niks3-server[1111]: 2026/09/22 16:57:05 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7534worker2 # [ 202.359932] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=17535worker2 # [ 202.370081] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:05Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07536worker2 # [ 202.372532] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:05Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17537worker2 # [ 202.377978] nix-daemon[865]: reaped child process 1951, status = succeeded7538worker2 # [ 202.415338] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352967539worker2 # [ 202.428612] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:352967540worker2 # [ 202.443730] nix-grpc-daemon[1711]: time=2026-09-22T16:57:05.592Z level=INFO msg="All 2 paths already cached"7541client # copying 1 paths to 'lb:50051'...7542client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7543worker2 # [ 202.453658] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:352967544worker2 # [ 202.462174] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7545worker2 # [ 202.559530] nix-daemon[865]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)7546worker2 # [ 202.573743] nix-daemon[865]: reaped child process 1981, status = succeeded7547lb # [ 202.579359] niks3-server[1111]: 2026/09/22 16:57:05 INFO Received uploads request method=POST path=/api/pending_closures7548worker2 # [ 202.588094] nix-grpc-daemon[1711]: time=2026-09-22T16:57:05.737Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7549worker2 # [ 202.589556] nix-grpc-daemon[1711]: time=2026-09-22T16:57:05.738Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7550lb # [ 202.614889] niks3-server[1111]: 2026/09/22 16:57:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7551lb # [ 202.620904] niks3-server[1111]: 2026/09/22 16:57:05 INFO Signed narinfos id=84 count=17552worker2 # [ 202.623141] nix-grpc-daemon[1711]: time=2026-09-22T16:57:05.772Z level=INFO msg="Uploading 1 narinfos"7553lb # [ 202.627520] niks3-server[1111]: 2026/09/22 16:57:05 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7554lb # [ 202.638913] niks3-server[1111]: 2026/09/22 16:57:05 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7555lb # [ 202.645133] niks3-server[1111]: 2026/09/22 16:57:05 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7556lb # [ 202.649530] niks3-server[1111]: 2026/09/22 16:57:05 INFO Completed upload id=847557worker2 # [ 202.649382] nix-grpc-daemon[1711]: time=2026-09-22T16:57:05.798Z level=INFO msg="Upload complete. (174ms)"7558lb # [ 202.652105] niks3-server[1111]: 2026/09/22 16:57:05 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7559worker2 # [ 202.653169] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=0 paths=1 nar_bytes_in=7527560worker2 # [ 202.658302] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:352967561worker2 # [ 202.668266] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7562worker2 # [ 202.674580] nix-daemon[865]: reaped child process 1976, status = succeeded7563worker2 # [ 202.678842] nix-daemon[865]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7564client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7565client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7566client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7567client # farm-top> LOG-farm-top-drain7568worker2 # [ 203.518632] nix-daemon[865]: reaped child process 1985, status = succeeded7569worker2 # [ 203.591095] nix-daemon[865]: accepted connection from pid 2001, user nix-grpc-daemon (trusted)7570worker2 # [ 203.605772] nix-daemon[865]: reaped child process 2004, status = succeeded7571lb # [ 203.610297] niks3-server[1111]: 2026/09/22 16:57:06 INFO Received uploads request method=POST path=/api/pending_closures7572worker2 # [ 203.619592] nix-grpc-daemon[1711]: time=2026-09-22T16:57:06.768Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7573worker2 # [ 203.622162] nix-grpc-daemon[1711]: time=2026-09-22T16:57:06.770Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7574lb # [ 203.655505] niks3-server[1111]: 2026/09/22 16:57:06 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7575lb # [ 203.662936] niks3-server[1111]: 2026/09/22 16:57:06 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7576lb # [ 203.670850] niks3-server[1111]: 2026/09/22 16:57:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7577lb # [ 203.674717] niks3-server[1111]: 2026/09/22 16:57:06 INFO Signed narinfos id=85 count=17578worker2 # [ 203.677069] nix-grpc-daemon[1711]: time=2026-09-22T16:57:06.825Z level=INFO msg="Uploading 1 narinfos"7579lb # [ 203.680397] niks3-server[1111]: 2026/09/22 16:57:06 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7580lb # [ 203.691157] niks3-server[1111]: 2026/09/22 16:57:06 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7581lb # [ 203.694814] niks3-server[1111]: 2026/09/22 16:57:06 INFO Completed upload id=857582worker2 # [ 203.696217] nix-grpc-daemon[1711]: time=2026-09-22T16:57:06.845Z level=INFO msg="Upload complete. (210ms)"7583lb # [ 203.698238] niks3-server[1111]: 2026/09/22 16:57:06 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7584worker2 # [ 203.703521] nix-grpc-daemon[1635]: ts=2026-09-22T16:57:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35296 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=17585worker2 # [ 203.717313] nix-daemon[865]: reaped child process 1983, status = succeeded7586client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 3.92 seconds)7587client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7588client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7589worker1: must succeed: rm /nix/.rw-store/fill7590worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7591worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7592worker1 # [ 204.642587] nix-grpc-daemon[1730]: ts=2026-09-22T16:57:06Z level=info event=healthy reason=min_free7593worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)7594(finished: subtest: low disk drains a worker and builds go to the other, in 7.24 seconds)7595(finished: run the VM test script, in 206.07 seconds)7596test script finished in 206.88s7597cleanup7598kill QemuMachine (pid 49)7599client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7600client # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7601client # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7602client # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7603kill QemuMachine (pid 51)7604lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7605lb # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7606lb # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7607lb # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7608kill QemuMachine (pid 50)7609worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7610worker1 # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7611worker1 # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7612worker1 # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7613kill QemuMachine (pid 52)7614worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7615worker2 # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7616worker2 # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7617worker2 # [2026-09-22T16:57:09Z INFO virtiofsd] Client disconnected, shutting down7618(finished: cleanup, in 1.08 seconds)7619additionally exposed symbols:7620 client, lb, worker1, worker2,7621 vlan1,7622 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh