vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #282
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb # Formatting '/build/vm-state-lb/tmp.KhRKvH7f16', fmt=raw size=107374182414lb: QEMU running (pid 50)15lb # mke2fs 1.47.4 (6-Mar-2025)16worker2 # Disk image does not exist, creating the virtualisation disk image...17client # Disk image does not exist, creating the virtualisation disk image...18client: QEMU running (pid 49)19worker2 # Formatting '/build/vm-state-worker2/tmp.v5pbrLkDiB', fmt=raw size=107374182420worker1 # Disk image does not exist, creating the virtualisation disk image...21worker1 # Formatting '/build/vm-state-worker1/tmp.imYgJCWQcu', fmt=raw size=107374182422lb # Discarding device blocks: 0/262144 done23client # Formatting '/build/vm-state-client/tmp.uqg48iJL46', fmt=raw size=107374182424client # mke2fs 1.47.4 (6-Mar-2025)25worker1: QEMU running (pid 51)26worker1 # mke2fs 1.47.4 (6-Mar-2025)27worker2: QEMU running (pid 52)28worker2 # mke2fs 1.47.4 (6-Mar-2025)29worker1 # Discarding device blocks: 0/262144 done30lb # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes32(finished: start all VMs, in 0.20 seconds)33client # Discarding device blocks: 0/262144 done34worker1 # Filesystem UUID: b59ff217-9035-4f24-a079-898ae0c70fff35client: waiting for unit network-addresses-eth1.service36client # Creating filesystem with 262144 4k blocks and 65536 inodes37worker1 # Superblock backups stored on blocks:38client: waiting for the VM to finish booting39client # Filesystem UUID: 9a5a30e2-7ed0-427e-80e7-5dadfe8c3a4240worker1 # 32768, 98304, 163840, 22937641worker2 # Discarding device blocks: 0/262144 done42lb # Filesystem UUID: 6ca02bf2-b5b7-4203-bde3-1bf937006a8943client # Superblock backups stored on blocks:44worker1 # 45worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes46lb # Superblock backups stored on blocks:47client # 32768, 98304, 163840, 22937648worker1 # Allocating group tables: 0/8 done49worker2 # Filesystem UUID: 485cee4e-5dbd-4079-9bfb-b56fc6ba2fe350lb # 32768, 98304, 163840, 22937651worker1 # Writing inode tables: 0/8 done52client # 53worker2 # Superblock backups stored on blocks:54worker2 # 32768, 98304, 163840, 22937655worker1 # Creating journal (8192 blocks): done56client # Allocating group tables: 0/8 done57lb # 58worker2 # 59worker1 # Writing superblocks and filesystem accounting information: 0/8 done60lb # Allocating group tables: 0/8 done61client # Writing inode tables: 0/8 done62worker2 # Allocating group tables: 0/8 done63lb # Writing inode tables: 0/8 done64worker1 # 65client # Creating journal (8192 blocks): done66worker2 # Writing inode tables: 0/8 done67lb # Creating journal (8192 blocks): done68worker1 # Virtualisation disk image created.69client # Writing superblocks and filesystem accounting information: 0/8 done70worker2 # Creating journal (8192 blocks): done71client # 72worker2 # Writing superblocks and filesystem accounting information: 0/8 done73client # Virtualisation disk image created.74worker2 # 75lb # Writing superblocks and filesystem accounting information: 0/8 done76worker2 # Virtualisation disk image created.77lb # 78lb # Virtualisation disk image created.79worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)80worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)81client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)82lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)83worker2 # 84worker2 # 85worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0086worker1 # 87worker1 # 88worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA0089worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...90worker2 # 91worker2 # 92worker2 # 93worker2 # 94worker2 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB0095worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...96worker1 # 97worker1 # 98worker1 # 99worker1 # 100worker1 # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00101client # 102client # 103client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00104worker2 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...105worker2 # 106worker2 # 107worker1 # Press Ctrl-B to configure iPXE (PCI 00:08.0)...108worker1 # 109worker1 # 110client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...111client # 112client # 113client # 114client # 115client # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00116client # Press Ctrl-B to configure iPXE (PCI 00:08.0)...117client # 118client # 119lb # 120lb # 121lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00122lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...123lb # 124lb # 125lb # 126lb # 127lb # iPXE (http://ipxe.org) 00:08.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00128lb # Press Ctrl-B to configure iPXE (PCI 00:08.0)...129lb # 130lb # 131worker2 # Booting from ROM...132worker1 # Booting from ROM...133client # Booting from ROM...134lb # Booting from ROM...135worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026136worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026137worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hvv0rrwgpqzhm46a4280mwzmbapcmqq9-nixos-system-worker1-test/init regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration console=ttyS0,115200n8 console=tty0138worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5qgax5h37ybfdjp7r7k01yx9abl6wij-nixos-system-worker2-test/init regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration console=ttyS0,115200n8 console=tty0139worker1 # [ 0.000000] BIOS-provided physical RAM map:140worker2 # [ 0.000000] BIOS-provided physical RAM map:141worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable142worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable143worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved144worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved145worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved146worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved147worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable148worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable149worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved150client # Probing EDD (edd=off to disable)... o[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026151worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved152worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved153worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved154worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved155worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved156worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved157worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved158worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved159worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved160client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test/init regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration console=ttyS0,115200n8 console=tty0161worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved162client # [ 0.000000] BIOS-provided physical RAM map:163worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved164worker1 # [ 0.000000] NX (Execute Disable) protection: active165worker1 # [ 0.000000] APIC: Static calls initialized166worker2 # [ 0.000000] NX (Execute Disable) protection: active167client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable168worker1 # [ 0.000000] SMBIOS 2.8 present.169worker2 # [ 0.000000] APIC: Static calls initialized170client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved171worker2 # [ 0.000000] SMBIOS 2.8 present.172worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014173client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved174worker1 # [ 0.000000] DMI: Memory slots populated: 1/1175worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014176client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable177worker1 # [ 0.000000] Hypervisor detected: KVM178worker2 # [ 0.000000] DMI: Memory slots populated: 1/1179worker2 # [ 0.000000] Hypervisor detected: KVM180worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000181client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved182worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000183worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00184client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved185worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00186worker1 # [ 0.000000] kvm-clock: using sched offset of 464043290 cycles187client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved188worker2 # [ 0.000000] kvm-clock: using sched offset of 465329704 cycles189worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns190client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved191worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor192worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns193client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved194worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor195worker1 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000196client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved197worker2 # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000198worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs199client # [ 0.000000] NX (Execute Disable) protection: active200worker2 # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs201client # [ 0.000000] APIC: Static calls initialized202worker1 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT203client # [ 0.000000] SMBIOS 2.8 present.204worker2 # [ 0.000833] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT205worker1 # [ 0.002732] found SMP MP-table at [mem 0x000f5450-0x000f545f]206worker1 # [ 0.002743] Using GB pages for direct mapping207worker2 # [ 0.002738] found SMP MP-table at [mem 0x000f5450-0x000f545f]208client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014209worker1 # [ 0.002897] RAMDISK: [mem 0x5e367000-0x5ffcffff]210worker2 # [ 0.002749] Using GB pages for direct mapping211client # [ 0.000000] DMI: Memory slots populated: 1/1212client # [ 0.000000] Hypervisor detected: KVM213worker2 # [ 0.002844] RAMDISK: [mem 0x5e367000-0x5ffcffff]214worker1 # [ 0.002906] ACPI: Early table checksum verification disabled215client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000216worker2 # [ 0.002852] ACPI: Early table checksum verification disabled217worker1 # [ 0.002908] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )218client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00219worker2 # [ 0.002855] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )220worker1 # [ 0.002912] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)221client # [ 0.000000] kvm-clock: using sched offset of 465492273 cycles222worker2 # [ 0.002859] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)223worker1 # [ 0.002916] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)224client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns225worker2 # [ 0.002863] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)226client # [ 0.000005] tsc: Detected 2400.012 MHz processor227worker1 # [ 0.002923] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)228lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 7 15:23:00 UTC 2026229worker1 # [ 0.002924] ACPI: FACS 0x000000005FFE0000 000040230client # [ 0.000815] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000231worker2 # [ 0.002870] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)232worker2 # [ 0.002872] ACPI: FACS 0x000000005FFE0000 000040233worker1 # [ 0.002926] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)234client # [ 0.000842] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs235worker2 # [ 0.002873] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)236client # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT237worker1 # [ 0.002928] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)238client # [ 0.002750] found SMP MP-table at [mem 0x000f5450-0x000f545f]239worker2 # [ 0.002875] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)240worker1 # [ 0.002929] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)241client # [ 0.002761] Using GB pages for direct mapping242lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r7i2qny6zxqjgksiw8bnyjrpmwmq2adf-nixos-system-lb-test/init regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration console=ttyS0,115200n8 console=tty0243client # [ 0.002871] RAMDISK: [mem 0x5e366000-0x5ffcffff]244worker2 # [ 0.002876] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)245lb # [ 0.000000] BIOS-provided physical RAM map:246worker1 # [ 0.002931] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)247client # [ 0.002879] ACPI: Early table checksum verification disabled248lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable249worker2 # [ 0.002878] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)250worker1 # [ 0.002932] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]251client # [ 0.002882] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )252lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved253worker2 # [ 0.002879] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]254worker1 # [ 0.002933] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]255client # [ 0.002886] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)256lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved257worker2 # [ 0.002880] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]258worker1 # [ 0.002933] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]259lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable260client # [ 0.002890] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)261worker2 # [ 0.002881] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]262worker1 # [ 0.002934] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]263lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved264client # [ 0.002896] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)265worker2 # [ 0.002881] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]266worker1 # [ 0.002934] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]267client # [ 0.002899] ACPI: FACS 0x000000005FFE0000 000040268lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved269worker2 # [ 0.002882] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]270worker1 # [ 0.002935] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]271client # [ 0.002900] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)272lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved273worker2 # [ 0.002882] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]274worker1 # [ 0.002935] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]275worker1 # [ 0.003158] No NUMA configuration found276lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved277worker2 # [ 0.002883] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]278client # [ 0.002902] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)279worker2 # [ 0.003115] No NUMA configuration found280worker1 # [ 0.003159] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]281lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved282client # [ 0.002903] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)283worker2 # [ 0.003116] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]284worker1 # [ 0.003161] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]285lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved286worker1 # [ 0.005246] Zone ranges:287worker2 # [ 0.003119] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]288lb # [ 0.000000] NX (Execute Disable) protection: active289client # [ 0.002905] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)290worker1 # [ 0.005246] DMA [mem 0x0000000000001000-0x0000000000ffffff]291worker2 # [ 0.005259] Zone ranges:292lb # [ 0.000000] APIC: Static calls initialized293client # [ 0.002906] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]294lb # [ 0.000000] SMBIOS 2.8 present.295worker1 # [ 0.005248] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]296worker2 # [ 0.005260] DMA [mem 0x0000000000001000-0x0000000000ffffff]297worker1 # [ 0.005249] Normal empty298worker1 # [ 0.005250] Device empty299worker2 # [ 0.005262] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]300client # [ 0.002907] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]301worker2 # [ 0.005263] Normal empty302lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014303worker1 # [ 0.005250] Movable zone start for each node304worker2 # [ 0.005264] Device empty305worker1 # [ 0.005251] Early memory node ranges306client # [ 0.002908] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]307lb # [ 0.000000] DMI: Memory slots populated: 1/1308worker2 # [ 0.005264] Movable zone start for each node309lb # [ 0.000000] Hypervisor detected: KVM310worker2 # [ 0.005265] Early memory node ranges311worker1 # [ 0.005251] node 0: [mem 0x0000000000001000-0x000000000009efff]312client # [ 0.002908] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]313lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000314worker1 # [ 0.005252] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]315worker2 # [ 0.005266] node 0: [mem 0x0000000000001000-0x000000000009efff]316client # [ 0.002909] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]317lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00318worker2 # [ 0.005267] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]319worker1 # [ 0.005253] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]320lb # [ 0.000001] kvm-clock: using sched offset of 467768280 cycles321client # [ 0.002909] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]322worker1 # [ 0.005272] On node 0, zone DMA: 1 pages in unavailable ranges323worker2 # [ 0.005268] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]324client # [ 0.002910] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]325lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns326worker1 # [ 0.005518] On node 0, zone DMA: 97 pages in unavailable ranges327client # [ 0.003133] No NUMA configuration found328worker2 # [ 0.005286] On node 0, zone DMA: 1 pages in unavailable ranges329lb # [ 0.000005] tsc: Detected 2400.012 MHz processor330worker1 # [ 0.030686] On node 0, zone DMA32: 40 pages in unavailable ranges331worker2 # [ 0.005532] On node 0, zone DMA: 97 pages in unavailable ranges332client # [ 0.003134] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]333lb # [ 0.000985] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000334worker1 # [ 0.031139] ACPI: PM-Timer IO Port: 0x608335worker2 # [ 0.031038] On node 0, zone DMA32: 40 pages in unavailable ranges336client # [ 0.003136] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]337worker1 # [ 0.031148] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])338client # [ 0.005221] Zone ranges:339worker2 # [ 0.031504] ACPI: PM-Timer IO Port: 0x608340lb # [ 0.001027] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs341client # [ 0.005222] DMA [mem 0x0000000000001000-0x0000000000ffffff]342worker1 # [ 0.031171] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23343worker2 # [ 0.031515] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])344lb # [ 0.001031] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT345worker1 # [ 0.031174] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)346worker2 # [ 0.031541] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23347client # [ 0.005223] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]348lb # [ 0.002961] found SMP MP-table at [mem 0x000f5450-0x000f545f]349client # [ 0.005224] Normal empty350client # [ 0.005225] Device empty351worker2 # [ 0.031544] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)352worker1 # [ 0.031176] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)353lb # [ 0.002973] Using GB pages for direct mapping354client # [ 0.005225] Movable zone start for each node355lb # [ 0.003120] RAMDISK: [mem 0x5e367000-0x5ffcffff]356worker1 # [ 0.031177] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)357client # [ 0.005226] Early memory node ranges358worker2 # [ 0.031546] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)359lb # [ 0.003140] ACPI: Early table checksum verification disabled360client # [ 0.005226] node 0: [mem 0x0000000000001000-0x000000000009efff]361worker2 # [ 0.031547] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)362worker1 # [ 0.031178] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)363lb # [ 0.003143] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )364client # [ 0.005227] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]365worker1 # [ 0.031178] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)366worker2 # [ 0.031548] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)367lb # [ 0.003147] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)368client # [ 0.005228] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]369worker1 # [ 0.031181] ACPI: Using ACPI (MADT) for SMP configuration information370worker2 # [ 0.031548] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)371client # [ 0.005248] On node 0, zone DMA: 1 pages in unavailable ranges372worker1 # [ 0.031181] ACPI: HPET id: 0x8086a201 base: 0xfed00000373lb # [ 0.003151] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)374worker2 # [ 0.031551] ACPI: Using ACPI (MADT) for SMP configuration information375worker1 # [ 0.031184] TSC deadline timer available376client # [ 0.005637] On node 0, zone DMA: 97 pages in unavailable ranges377worker2 # [ 0.031552] ACPI: HPET id: 0x8086a201 base: 0xfed00000378worker1 # [ 0.031189] CPU topo: Max. logical packages: 1379lb # [ 0.003158] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)380worker2 # [ 0.031555] TSC deadline timer available381client # [ 0.030552] On node 0, zone DMA32: 40 pages in unavailable ranges382worker1 # [ 0.031189] CPU topo: Max. logical dies: 1383lb # [ 0.003161] ACPI: FACS 0x000000005FFE0000 000040384worker2 # [ 0.031559] CPU topo: Max. logical packages: 1385client # [ 0.031009] ACPI: PM-Timer IO Port: 0x608386worker1 # [ 0.031190] CPU topo: Max. dies per package: 1387worker2 # [ 0.031560] CPU topo: Max. logical dies: 1388client # [ 0.031020] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])389worker1 # [ 0.031193] CPU topo: Max. threads per core: 1390lb # [ 0.003162] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)391worker2 # [ 0.031560] CPU topo: Max. dies per package: 1392worker1 # [ 0.031194] CPU topo: Num. cores per package: 1393client # [ 0.031046] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23394worker2 # [ 0.031564] CPU topo: Max. threads per core: 1395lb # [ 0.003164] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)396worker1 # [ 0.031194] CPU topo: Num. threads per package: 1397worker2 # [ 0.031565] CPU topo: Num. cores per package: 1398client # [ 0.031049] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)399worker1 # [ 0.031194] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs400worker2 # [ 0.031565] CPU topo: Num. threads per package: 1401lb # [ 0.003165] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)402client # [ 0.031050] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)403worker1 # [ 0.031211] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()404worker2 # [ 0.031565] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs405client # [ 0.031052] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)406lb # [ 0.003167] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)407worker2 # [ 0.031582] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()408worker1 # [ 0.031239] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]409client # [ 0.031052] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)410lb # [ 0.003168] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]411worker1 # [ 0.031240] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]412worker2 # [ 0.031611] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]413client # [ 0.031053] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)414lb # [ 0.003169] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]415worker1 # [ 0.031241] [mem 0x60000000-0xafffffff] available for PCI devices416worker2 # [ 0.031612] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]417client # [ 0.031055] ACPI: Using ACPI (MADT) for SMP configuration information418lb # [ 0.003170] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]419worker1 # [ 0.031243] Booting paravirtualized kernel on KVM420client # [ 0.031056] ACPI: HPET id: 0x8086a201 base: 0xfed00000421worker2 # [ 0.031613] [mem 0x60000000-0xafffffff] available for PCI devices422lb # [ 0.003170] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]423client # [ 0.031060] TSC deadline timer available424worker2 # [ 0.031615] Booting paravirtualized kernel on KVM425worker1 # [ 0.031245] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns426client # [ 0.031064] CPU topo: Max. logical packages: 1427lb # [ 0.003171] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]428client # [ 0.031065] CPU topo: Max. logical dies: 1429worker1 # [ 0.035684] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1430worker2 # [ 0.031617] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns431client # [ 0.031065] CPU topo: Max. dies per package: 1432lb # [ 0.003171] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]433client # [ 0.031069] CPU topo: Max. threads per core: 1434worker1 # [ 0.037725] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152435worker2 # [ 0.036064] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1436lb # [ 0.003172] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]437client # [ 0.031069] CPU topo: Num. cores per package: 1438worker1 # [ 0.037764] kvm-guest: PV spinlocks disabled, single CPU439lb # [ 0.003419] No NUMA configuration found440worker2 # [ 0.038113] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152441client # [ 0.031070] CPU topo: Num. threads per package: 1442lb # [ 0.003420] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]443worker2 # [ 0.038155] kvm-guest: PV spinlocks disabled, single CPU444client # [ 0.031070] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs445lb # [ 0.003423] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]446lb # [ 0.005548] Zone ranges:447client # [ 0.031088] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()448lb # [ 0.005549] DMA [mem 0x0000000000001000-0x0000000000ffffff]449client # [ 0.031119] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]450lb # [ 0.005550] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]451worker1 # [ 0.037765] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/hvv0rrwgpqzhm46a4280mwzmbapcmqq9-nixos-system-worker1-test/init regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration console=ttyS0,115200n8 console=tty0452lb # [ 0.005551] Normal empty453client # [ 0.031120] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]454lb # [ 0.005552] Device empty455client # [ 0.031121] [mem 0x60000000-0xafffffff] available for PCI devices456lb # [ 0.005552] Movable zone start for each node457worker2 # [ 0.038156] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x5qgax5h37ybfdjp7r7k01yx9abl6wij-nixos-system-worker2-test/init regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration console=ttyS0,115200n8 console=tty0458lb # [ 0.005553] Early memory node ranges459client # [ 0.031123] Booting paravirtualized kernel on KVM460worker1 # [ 0.037858] Unknown kernel command line parameters "regInfo=/nix/store/vhr1b62dfjs04ywz0rz399x3gf2shwsl-closure-info/registration", will be passed to user space.461worker1 # [ 0.037871] random: crng init done462lb # [ 0.005554] node 0: [mem 0x0000000000001000-0x000000000009efff]463client # [ 0.031126] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns464worker2 # [ 0.038250] Unknown kernel command line parameters "regInfo=/nix/store/dq71v172s5jppb7nnfi3vkq85w35rkbj-closure-info/registration", will be passed to user space.465lb # [ 0.005555] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]466worker1 # [ 0.037872] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes467worker2 # [ 0.038263] random: crng init done468client # [ 0.035603] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1469lb # [ 0.005556] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]470worker2 # [ 0.038264] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes471worker1 # [ 0.039879] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)472client # [ 0.037647] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152473lb # [ 0.005575] On node 0, zone DMA: 1 pages in unavailable ranges474worker1 # [ 0.040880] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)475worker2 # [ 0.040295] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)476client # [ 0.037687] kvm-guest: PV spinlocks disabled, single CPU477lb # [ 0.005829] On node 0, zone DMA: 97 pages in unavailable ranges478worker1 # [ 0.040908] Fallback order for Node 0: 0479worker2 # [ 0.041314] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)480lb # [ 0.030893] On node 0, zone DMA32: 40 pages in unavailable ranges481worker1 # [ 0.040911] Built 1 zonelists, mobility grouping on. Total pages: 393078482worker2 # [ 0.041346] Fallback order for Node 0: 0483lb # [ 0.031358] ACPI: PM-Timer IO Port: 0x608484worker1 # [ 0.040912] Policy zone: DMA32485lb # [ 0.031368] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])486worker2 # [ 0.041348] Built 1 zonelists, mobility grouping on. Total pages: 393078487worker1 # [ 0.043479] mem auto-init: stack:all(zero), heap alloc:on, heap free:off488worker2 # [ 0.041349] Policy zone: DMA32489lb # [ 0.031411] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23490worker1 # [ 0.046403] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1491worker2 # [ 0.043871] mem auto-init: stack:all(zero), heap alloc:on, heap free:off492worker1 # [ 0.049524] allocated 3145728 bytes of page_ext493lb # [ 0.031414] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)494client # [ 0.037688] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test/init regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration console=ttyS0,115200n8 console=tty0495worker2 # [ 0.046858] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1496worker1 # [ 0.059142] ftrace: allocating 48731 entries in 192 pages497lb # [ 0.031415] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)498worker2 # [ 0.049977] allocated 3145728 bytes of page_ext499worker1 # [ 0.059144] ftrace: allocated 192 pages with 2 groups500worker1 # [ 0.059931] Dynamic Preempt: lazy501lb # [ 0.031416] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)502worker2 # [ 0.059698] ftrace: allocating 48731 entries in 192 pages503client # [ 0.037779] Unknown kernel command line parameters "regInfo=/nix/store/9p1idx4s91wv7zwkal5pb5lr899hddrn-closure-info/registration", will be passed to user space.504worker1 # [ 0.060046] rcu: Preemptible hierarchical RCU implementation.505worker2 # [ 0.059700] ftrace: allocated 192 pages with 2 groups506lb # [ 0.031417] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)507client # [ 0.037792] random: crng init done508worker2 # [ 0.060508] Dynamic Preempt: lazy509worker1 # [ 0.060047] rcu: RCU event tracing is enabled.510client # [ 0.037793] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes511lb # [ 0.031418] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)512worker2 # [ 0.060620] rcu: Preemptible hierarchical RCU implementation.513worker1 # [ 0.060047] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.514lb # [ 0.031420] ACPI: Using ACPI (MADT) for SMP configuration information515worker2 # [ 0.060621] rcu: RCU event tracing is enabled.516client # [ 0.039792] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)517worker1 # [ 0.060049] Trampoline variant of Tasks RCU enabled.518lb # [ 0.031421] ACPI: HPET id: 0x8086a201 base: 0xfed00000519worker2 # [ 0.060621] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.520worker1 # [ 0.060049] Rude variant of Tasks RCU enabled.521client # [ 0.040798] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)522lb # [ 0.031425] TSC deadline timer available523worker1 # [ 0.060049] Tracing variant of Tasks RCU enabled.524worker2 # [ 0.060623] Trampoline variant of Tasks RCU enabled.525client # [ 0.040828] Fallback order for Node 0: 0526lb # [ 0.031429] CPU topo: Max. logical packages: 1527worker2 # [ 0.060623] Rude variant of Tasks RCU enabled.528lb # [ 0.031430] CPU topo: Max. logical dies: 1529worker1 # [ 0.060050] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.530client # [ 0.040831] Built 1 zonelists, mobility grouping on. Total pages: 393078531worker2 # [ 0.060624] Tracing variant of Tasks RCU enabled.532client # [ 0.040832] Policy zone: DMA32533lb # [ 0.031431] CPU topo: Max. dies per package: 1534worker1 # [ 0.060051] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1535lb # [ 0.031434] CPU topo: Max. threads per core: 1536worker2 # [ 0.060624] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.537client # [ 0.043186] mem auto-init: stack:all(zero), heap alloc:on, heap free:off538lb # [ 0.031434] CPU topo: Num. cores per package: 1539worker1 # [ 0.060106] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.540worker2 # [ 0.060625] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1541client # [ 0.046282] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1542lb # [ 0.031435] CPU topo: Num. threads per package: 1543client # [ 0.049406] allocated 3145728 bytes of page_ext544worker1 # [ 0.060108] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.545lb # [ 0.031435] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs546worker2 # [ 0.060645] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.547client # [ 0.059040] ftrace: allocating 48731 entries in 192 pages548lb # [ 0.031452] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()549client # [ 0.059042] ftrace: allocated 192 pages with 2 groups550worker1 # [ 0.060109] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.551worker2 # [ 0.060647] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.552client # [ 0.059830] Dynamic Preempt: lazy553worker1 # [ 0.064386] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16554lb # [ 0.031483] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]555client # [ 0.059961] rcu: Preemptible hierarchical RCU implementation.556worker2 # [ 0.060648] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.557client # [ 0.059962] rcu: RCU event tracing is enabled.558worker1 # [ 0.064647] rcu: srcu_init: Setting srcu_struct sizes based on contention.559lb # [ 0.031484] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]560worker2 # [ 0.065060] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16561client # [ 0.059962] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.562lb # [ 0.031486] [mem 0x60000000-0xafffffff] available for PCI devices563worker1 # [ 0.064653] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns564worker2 # [ 0.065331] rcu: srcu_init: Setting srcu_struct sizes based on contention.565lb # [ 0.031487] Booting paravirtualized kernel on KVM566client # [ 0.059964] Trampoline variant of Tasks RCU enabled.567client # [ 0.059964] Rude variant of Tasks RCU enabled.568worker2 # [ 0.065337] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns569worker1 # [ 0.064761] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)570client # [ 0.059964] Tracing variant of Tasks RCU enabled.571lb # [ 0.031490] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns572worker1 # [ 0.068282] Console: colour VGA+ 80x25573client # [ 0.059965] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.574worker1 # [ 0.068284] printk: legacy console [tty0] enabled575worker2 # [ 0.065437] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)576lb # [ 0.035962] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1577worker1 # [ 0.097736] printk: legacy console [ttyS0] enabled578worker2 # [ 0.068936] Console: colour VGA+ 80x25579client # [ 0.059966] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1580lb # [ 0.038034] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152581worker2 # [ 0.068939] printk: legacy console [tty0] enabled582worker1 # [ 0.194173] ACPI: Core revision 20250807583lb # [ 0.038072] kvm-guest: PV spinlocks disabled, single CPU584worker2 # [ 0.098582] printk: legacy console [ttyS0] enabled585client # [ 0.060012] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.586worker2 # [ 0.195694] ACPI: Core revision 20250807587worker1 # [ 0.195038] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns588client # [ 0.060014] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.589worker1 # [ 0.196563] APIC: Switch to symmetric I/O mode setup590client # [ 0.060015] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.591worker2 # [ 0.196713] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns592worker1 # [ 0.197490] x2apic enabled593client # [ 0.064322] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16594worker2 # [ 0.198214] APIC: Switch to symmetric I/O mode setup595lb # [ 0.038074] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/r7i2qny6zxqjgksiw8bnyjrpmwmq2adf-nixos-system-lb-test/init regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration console=ttyS0,115200n8 console=tty0596client # [ 0.064591] rcu: srcu_init: Setting srcu_struct sizes based on contention.597worker1 # [ 0.198228] APIC: Switched APIC routing to: physical x2apic598worker2 # [ 0.199166] x2apic enabled599client # [ 0.064597] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns600worker2 # [ 0.199932] APIC: Switched APIC routing to: physical x2apic601lb # [ 0.038166] Unknown kernel command line parameters "regInfo=/nix/store/nsd6mwhnd72yzbh9syf2drsax74mqqk0-closure-info/registration", will be passed to user space.602worker1 # [ 0.200028] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1603lb # [ 0.038179] random: crng init done604client # [ 0.064696] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)605client # [ 0.068204] Console: colour VGA+ 80x25606lb # [ 0.038180] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes607worker2 # [ 0.201720] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1608client # [ 0.068207] printk: legacy console [tty0] enabled609worker1 # [ 0.200992] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns610lb # [ 0.040201] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)611client # [ 0.097709] printk: legacy console [ttyS0] enabled612client # [ 0.193940] ACPI: Core revision 20250807613lb # [ 0.041414] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)614worker1 # [ 0.202666] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)615worker2 # [ 0.202707] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns616lb # [ 0.041448] Fallback order for Node 0: 0617worker1 # [ 0.203750] x86/cpu: User Mode Instruction Prevention (UMIP) activated618lb # [ 0.041451] Built 1 zonelists, mobility grouping on. Total pages: 393078619client # [ 0.194762] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns620worker2 # [ 0.204381] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)621lb # [ 0.041452] Policy zone: DMA32622client # [ 0.196311] APIC: Switch to symmetric I/O mode setup623worker1 # [ 0.204829] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127624lb # [ 0.043802] mem auto-init: stack:all(zero), heap alloc:on, heap free:off625worker2 # [ 0.205463] x86/cpu: User Mode Instruction Prevention (UMIP) activated626client # [ 0.197247] x2apic enabled627lb # [ 0.046730] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1628worker1 # [ 0.205662] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0629worker2 # [ 0.206504] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127630lb # [ 0.049927] allocated 3145728 bytes of page_ext631client # [ 0.198001] APIC: Switched APIC routing to: physical x2apic632lb # [ 0.059671] ftrace: allocating 48731 entries in 192 pages633worker2 # [ 0.207378] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0634lb # [ 0.059673] ftrace: allocated 192 pages with 2 groups635worker1 # [ 0.206666] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto636lb # [ 0.060518] Dynamic Preempt: lazy637client # [ 0.199837] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1638lb # [ 0.060819] rcu: Preemptible hierarchical RCU implementation.639worker2 # [ 0.208381] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto640worker1 # [ 0.208663] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl641lb # [ 0.060819] rcu: RCU event tracing is enabled.642worker1 # [ 0.209663] Transient Scheduler Attacks: Vulnerable: No microcode643lb # [ 0.060820] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.644client # [ 0.200790] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns645worker2 # [ 0.210377] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl646lb # [ 0.060821] Trampoline variant of Tasks RCU enabled.647worker1 # [ 0.211360] Spectre V2 : Mitigation: Enhanced / Automatic IBRS648lb # [ 0.060822] Rude variant of Tasks RCU enabled.649worker2 # [ 0.211378] Transient Scheduler Attacks: Vulnerable: No microcode650client # [ 0.202430] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)651lb # [ 0.060822] Tracing variant of Tasks RCU enabled.652worker1 # [ 0.211663] Speculative Return Stack Overflow: Mitigation: Safe RET653worker2 # [ 0.212378] Spectre V2 : Mitigation: Enhanced / Automatic IBRS654lb # [ 0.060823] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.655client # [ 0.203513] x86/cpu: User Mode Instruction Prevention (UMIP) activated656worker2 # [ 0.213378] Speculative Return Stack Overflow: Mitigation: Safe RET657lb # [ 0.060823] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1658client # [ 0.204589] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127659lb # [ 0.060838] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.660client # [ 0.205428] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0661lb # [ 0.060840] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.662worker1 # [ 0.212663] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization663worker1 # [ 0.214670] Spectre V2 : Enabling IBPB for BPF664client # [ 0.206431] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto665lb # [ 0.060841] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.666worker2 # [ 0.214377] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization667lb # [ 0.065173] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16668worker1 # [ 0.215664] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier669worker2 # [ 0.216385] Spectre V2 : Enabling IBPB for BPF670client # [ 0.208428] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl671lb # [ 0.065449] rcu: srcu_init: Setting srcu_struct sizes based on contention.672worker1 # [ 0.216663] active return thunk: srso_alias_return_thunk673client # [ 0.209428] Transient Scheduler Attacks: Vulnerable: No microcode674worker2 # [ 0.217378] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier675lb # [ 0.065454] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns676client # [ 0.211128] Spectre V2 : Mitigation: Enhanced / Automatic IBRS677worker1 # [ 0.217684] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'678worker2 # [ 0.218378] active return thunk: srso_alias_return_thunk679lb # [ 0.065558] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)680client # [ 0.211428] Speculative Return Stack Overflow: Mitigation: Safe RET681worker1 # [ 0.218663] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'682lb # [ 0.069105] Console: colour VGA+ 80x25683worker2 # [ 0.219399] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'684lb # [ 0.069108] printk: legacy console [tty0] enabled685worker1 # [ 0.219663] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'686lb # [ 0.098906] printk: legacy console [ttyS0] enabled687worker2 # [ 0.220377] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'688worker1 # [ 0.220663] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'689lb # [ 0.195791] ACPI: Core revision 20250807690worker2 # [ 0.221383] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'691worker1 # [ 0.221663] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'692worker2 # [ 0.222384] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'693lb # [ 0.196643] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns694client # [ 0.212427] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization695worker1 # [ 0.222663] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'696worker2 # [ 0.223377] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'697client # [ 0.214435] Spectre V2 : Enabling IBPB for BPF698lb # [ 0.198169] APIC: Switch to symmetric I/O mode setup699worker1 # [ 0.223663] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'700lb # [ 0.199172] x2apic enabled701worker2 # [ 0.224378] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'702client # [ 0.215428] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier703worker1 # [ 0.225663] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256704worker2 # [ 0.225377] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'705client # [ 0.216428] active return thunk: srso_alias_return_thunk706lb # [ 0.200081] APIC: Switched APIC routing to: physical x2apic707worker1 # [ 0.226663] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64708worker2 # [ 0.227275] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256709client # [ 0.217448] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'710worker1 # [ 0.227593] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512711worker2 # [ 0.228103] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64712lb # [ 0.202192] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1713client # [ 0.218427] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'714worker1 # [ 0.228370] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024715worker2 # [ 0.229087] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512716client # [ 0.220151] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'717worker1 # [ 0.229374] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8718worker2 # [ 0.230112] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024719lb # [ 0.203157] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns720client # [ 0.221143] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'721worker2 # [ 0.231096] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8722worker1 # [ 0.230361] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.723lb # [ 0.204810] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)724client # [ 0.221428] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'725worker2 # [ 0.232125] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.726client # [ 0.222428] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'727lb # [ 0.205894] x86/cpu: User Mode Instruction Prevention (UMIP) activated728lb # [ 0.206949] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127729client # [ 0.223427] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'730lb # [ 0.207807] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0731client # [ 0.225428] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256732client # [ 0.226428] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64733lb # [ 0.208810] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto734client # [ 0.227427] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512735client # [ 0.228427] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024736lb # [ 0.209809] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl737client # [ 0.229427] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8738lb # [ 0.210807] Transient Scheduler Attacks: Vulnerable: No microcode739lb # [ 0.211806] Spectre V2 : Mitigation: Enhanced / Automatic IBRS740client # [ 0.230427] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.741lb # [ 0.212807] Speculative Return Stack Overflow: Mitigation: Safe RET742lb # [ 0.213807] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization743lb # [ 0.215817] Spectre V2 : Enabling IBPB for BPF744lb # [ 0.217375] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier745lb # [ 0.218807] active return thunk: srso_alias_return_thunk746lb # [ 0.219828] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'747lb # [ 0.220807] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'748lb # [ 0.221806] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'749lb # [ 0.222806] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'750lb # [ 0.223807] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'751lb # [ 0.224806] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'752lb # [ 0.225806] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'753lb # [ 0.227807] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256754lb # [ 0.228794] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64755lb # [ 0.229806] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512756lb # [ 0.230807] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024757lb # [ 0.231807] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8758lb # [ 0.232807] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.759worker1 # [ 0.263987] Freeing SMP alternatives memory: 44K760worker1 # [ 0.264666] pid_max: default: 32768 minimum: 301761worker1 # [ 0.265691] LSM: initializing lsm=capability,landlock,yama,bpf,ima762worker1 # [ 0.266756] landlock: Up and running.763worker1 # [ 0.267397] Yama: becoming mindful.764worker1 # [ 0.268328] LSM support for eBPF active765worker2 # [ 0.269151] Freeing SMP alternatives memory: 44K766worker2 # [ 0.269984] pid_max: default: 32768 minimum: 301767worker1 # [ 0.268918] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)768worker2 # [ 0.270471] LSM: initializing lsm=capability,landlock,yama,bpf,ima769worker1 # [ 0.269698] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)770worker2 # [ 0.271490] landlock: Up and running.771worker2 # [ 0.272378] Yama: becoming mindful.772worker2 # [ 0.273542] LSM support for eBPF active773client # [ 0.264196] Freeing SMP alternatives memory: 44K774worker1 # [ 0.272809] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)775client # [ 0.264430] pid_max: default: 32768 minimum: 301776worker2 # [ 0.274313] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)777worker1 # [ 0.274193] Performance Events: Fam17h+ core perfctr, AMD PMU driver.778client # [ 0.265510] LSM: initializing lsm=capability,landlock,yama,bpf,ima779worker2 # [ 0.275411] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)780worker1 # [ 0.274672] ... version: 2781client # [ 0.266550] landlock: Up and running.782client # [ 0.267428] Yama: becoming mindful.783worker1 # [ 0.275359] ... bit width: 48784worker1 # [ 0.275665] ... generic counters: 6785client # [ 0.268220] LSM support for eBPF active786worker2 # [ 0.277729] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)787worker1 # [ 0.276397] ... generic bitmap: 000000000000003f788client # [ 0.269032] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)789worker1 # [ 0.276665] ... fixed-purpose counters: 0790worker2 # [ 0.278915] Performance Events: Fam17h+ core perfctr, AMD PMU driver.791worker1 # [ 0.277397] ... fixed-purpose bitmap: 0000000000000000792client # [ 0.270444] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)793worker2 # [ 0.279387] ... version: 2794worker1 # [ 0.277665] ... value mask: 0000ffffffffffff795worker2 # [ 0.280113] ... bit width: 48796worker1 # [ 0.278561] ... max period: 00007fffffffffff797worker2 # [ 0.280380] ... generic counters: 6798client # [ 0.272558] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)799worker1 # [ 0.279342] ... global_ctrl mask: 000000000000003f800worker2 # [ 0.281062] ... generic bitmap: 000000000000003f801worker1 # [ 0.279773] signal: max sigframe size: 3376802worker2 # [ 0.281380] ... fixed-purpose counters: 0803client # [ 0.273967] Performance Events: Fam17h+ core perfctr, AMD PMU driver.804worker1 # [ 0.280588] rcu: Hierarchical SRCU implementation.805worker2 # [ 0.282104] ... fixed-purpose bitmap: 0000000000000000806client # [ 0.274436] ... version: 2807worker1 # [ 0.281271] rcu: Max phase no-delay instances is 400.808worker2 # [ 0.282379] ... value mask: 0000ffffffffffff809client # [ 0.275118] ... bit width: 48810client # [ 0.275430] ... generic counters: 6811worker2 # [ 0.283256] ... max period: 00007fffffffffff812client # [ 0.276142] ... generic bitmap: 000000000000003f813worker2 # [ 0.284077] ... global_ctrl mask: 000000000000003f814client # [ 0.276430] ... fixed-purpose counters: 0815worker2 # [ 0.284489] signal: max sigframe size: 3376816client # [ 0.277110] ... fixed-purpose bitmap: 0000000000000000817worker2 # [ 0.285260] rcu: Hierarchical SRCU implementation.818client # [ 0.277430] ... value mask: 0000ffffffffffff819worker2 # [ 0.285985] rcu: Max phase no-delay instances is 400.820client # [ 0.278307] ... max period: 00007fffffffffff821worker1 # [ 0.286074] smp: Bringing up secondary CPUs ...822worker1 # [ 0.286679] smp: Brought up 1 node, 1 CPU823client # [ 0.279102] ... global_ctrl mask: 000000000000003f824client # [ 0.279543] signal: max sigframe size: 3376825worker1 # [ 0.287370] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)826client # [ 0.280321] rcu: Hierarchical SRCU implementation.827client # [ 0.281025] rcu: Max phase no-delay instances is 400.828worker1 # [ 0.287850] Memory: 1454560K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)829worker2 # [ 0.290811] smp: Bringing up secondary CPUs ...830worker1 # [ 0.288878] devtmpfs: initialized831worker2 # [ 0.291394] smp: Brought up 1 node, 1 CPU832worker1 # [ 0.289688] x86/mm: Memory block size: 128MB833worker2 # [ 0.292091] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)834worker1 # [ 0.291410] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)835client # [ 0.285864] smp: Bringing up secondary CPUs ...836worker2 # [ 0.292534] Memory: 1454592K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)837client # [ 0.286443] smp: Brought up 1 node, 1 CPU838worker2 # [ 0.293604] devtmpfs: initialized839worker1 # [ 0.292591] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).840client # [ 0.287091] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)841worker2 # [ 0.294486] x86/mm: Memory block size: 128MB842worker1 # [ 0.293753] pinctrl core: initialized pinctrl subsystem843worker1 # [ 0.294864] PM: RTC time: 09:33:11, date: 2026-09-20844client # [ 0.287638] Memory: 1454564K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109348K reserved, 0K cma-reserved)845lb # [ 0.272358] Freeing SMP alternatives memory: 44K846worker2 # [ 0.296285] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)847client # [ 0.288645] devtmpfs: initialized848lb # [ 0.272812] pid_max: default: 32768 minimum: 301849worker2 # [ 0.297292] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).850client # [ 0.289379] x86/mm: Memory block size: 128MB851lb # [ 0.273929] LSM: initializing lsm=capability,landlock,yama,bpf,ima852worker2 # [ 0.298459] pinctrl core: initialized pinctrl subsystem853worker1 # [ 0.298268] NET: Registered PF_NETLINK/PF_ROUTE protocol family854lb # [ 0.275855] landlock: Up and running.855worker2 # [ 0.299640] PM: RTC time: 09:33:11, date: 2026-09-20856client # [ 0.290919] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)857lb # [ 0.276708] Yama: becoming mindful.858worker1 # [ 0.299152] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations859client # [ 0.291455] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).860lb # [ 0.277589] LSM support for eBPF active861worker1 # [ 0.299686] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations862client # [ 0.292509] pinctrl core: initialized pinctrl subsystem863lb # [ 0.278512] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)864client # [ 0.293645] PM: RTC time: 09:33:11, date: 2026-09-20865worker2 # [ 0.302970] NET: Registered PF_NETLINK/PF_ROUTE protocol family866worker1 # [ 0.300922] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations867worker1 # [ 0.301675] audit: initializing netlink subsys (disabled)868lb # [ 0.279866] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)869worker2 # [ 0.303860] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations870worker1 # [ 0.302821] thermal_sys: Registered thermal governor 'fair_share'871worker2 # [ 0.304398] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations872worker1 # [ 0.302823] thermal_sys: Registered thermal governor 'bang_bang'873client # [ 0.297153] NET: Registered PF_NETLINK/PF_ROUTE protocol family874worker1 # [ 0.303666] thermal_sys: Registered thermal governor 'step_wise'875lb # [ 0.282978] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)876worker2 # [ 0.305642] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations877worker1 # [ 0.304638] thermal_sys: Registered thermal governor 'user_space'878client # [ 0.297897] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations879worker2 # [ 0.306390] audit: initializing netlink subsys (disabled)880lb # [ 0.284462] Performance Events: Fam17h+ core perfctr, AMD PMU driver.881worker1 # [ 0.305392] audit: type=2000 audit(1789896792.461:1): state=initialized audit_enabled=0 res=1882client # [ 0.298450] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations883worker2 # [ 0.307488] thermal_sys: Registered thermal governor 'fair_share'884lb # [ 0.284820] ... version: 2885worker1 # [ 0.306669] thermal_sys: Registered thermal governor 'power_allocator'886worker2 # [ 0.307490] thermal_sys: Registered thermal governor 'bang_bang'887lb # [ 0.285646] ... bit width: 48888client # [ 0.299689] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations889worker1 # [ 0.306683] cpuidle: using governor menu890worker2 # [ 0.308381] thermal_sys: Registered thermal governor 'step_wise'891lb # [ 0.286428] ... generic counters: 6892client # [ 0.300450] audit: initializing netlink subsys (disabled)893worker2 # [ 0.309309] thermal_sys: Registered thermal governor 'user_space'894lb # [ 0.286811] ... generic bitmap: 000000000000003f895worker1 # [ 0.308887] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5896client # [ 0.301591] thermal_sys: Registered thermal governor 'fair_share'897lb # [ 0.287811] ... fixed-purpose counters: 0898worker2 # [ 0.310110] audit: type=2000 audit(1789896792.464:1): state=initialized audit_enabled=0 res=1899client # [ 0.301593] thermal_sys: Registered thermal governor 'bang_bang'900worker1 # [ 0.309892] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]901lb # [ 0.288634] ... fixed-purpose bitmap: 0000000000000000902worker2 # [ 0.311382] thermal_sys: Registered thermal governor 'power_allocator'903client # [ 0.302431] thermal_sys: Registered thermal governor 'step_wise'904worker1 # [ 0.310668] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry905worker2 # [ 0.311395] cpuidle: using governor menu906lb # [ 0.289565] ... value mask: 0000ffffffffffff907client # [ 0.303360] thermal_sys: Registered thermal governor 'user_space'908worker1 # [ 0.311677] PCI: Using configuration type 1 for base access909lb # [ 0.290622] ... max period: 00007fffffffffff910worker2 # [ 0.313573] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5911client # [ 0.304158] audit: type=2000 audit(1789896792.469:1): state=initialized audit_enabled=0 res=1912lb # [ 0.291624] ... global_ctrl mask: 000000000000003f913worker1 # [ 0.312781] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.914client # [ 0.305432] thermal_sys: Registered thermal governor 'power_allocator'915worker2 # [ 0.314652] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]916lb # [ 0.292743] signal: max sigframe size: 3376917client # [ 0.305446] cpuidle: using governor menu918worker2 # [ 0.315383] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry919lb # [ 0.293603] rcu: Hierarchical SRCU implementation.920client # [ 0.307647] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5921worker2 # [ 0.316392] PCI: Using configuration type 1 for base access922lb # [ 0.294531] rcu: Max phase no-delay instances is 400.923worker2 # [ 0.317485] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.924client # [ 0.308656] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]925client # [ 0.309433] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry926worker1 # [ 0.317891] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages927client # [ 0.310442] PCI: Using configuration type 1 for base access928worker1 # [ 0.318667] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page929client # [ 0.311515] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.930lb # [ 0.300156] smp: Bringing up secondary CPUs ...931worker2 # [ 0.322498] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages932lb # [ 0.300830] smp: Brought up 1 node, 1 CPU933worker2 # [ 0.323382] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page934lb # [ 0.301633] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)935worker1 # [ 0.323666] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages936worker1 # [ 0.324666] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page937client # [ 0.316523] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages938lb # [ 0.302902] Memory: 1454576K/1572312K available (17223K kernel code, 2726K rwdata, 13584K rodata, 3644K init, 2988K bss, 109344K reserved, 0K cma-reserved)939client # [ 0.317431] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page940lb # [ 0.304031] devtmpfs: initialized941lb # [ 0.304851] x86/mm: Memory block size: 128MB942worker2 # [ 0.328381] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages943worker2 # [ 0.329380] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page944lb # [ 0.306651] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)945client # [ 0.322432] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages946lb # [ 0.307790] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).947client # [ 0.323430] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page948lb # [ 0.308888] pinctrl core: initialized pinctrl subsystem949lb # [ 0.310010] PM: RTC time: 09:33:11, date: 2026-09-20950worker1 # [ 0.335034] ACPI: Added _OSI(Module Device)951worker1 # [ 0.335668] ACPI: Added _OSI(Processor Device)952worker1 # [ 0.336451] ACPI: Added _OSI(Processor Aggregator Device)953lb # [ 0.313397] NET: Registered PF_NETLINK/PF_ROUTE protocol family954lb # [ 0.314311] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations955worker2 # [ 0.339712] ACPI: Added _OSI(Module Device)956worker2 # [ 0.340382] ACPI: Added _OSI(Processor Device)957lb # [ 0.314829] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations958worker2 # [ 0.341135] ACPI: Added _OSI(Processor Aggregator Device)959lb # [ 0.316074] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations960client # [ 0.333804] ACPI: Added _OSI(Module Device)961lb # [ 0.316820] audit: initializing netlink subsys (disabled)962client # [ 0.334432] ACPI: Added _OSI(Processor Device)963client # [ 0.335157] ACPI: Added _OSI(Processor Aggregator Device)964lb # [ 0.317812] audit: type=2000 audit(1789896792.503:1): state=initialized audit_enabled=0 res=1965lb # [ 0.318963] thermal_sys: Registered thermal governor 'fair_share'966worker1 # [ 0.345201] ACPI: 1 ACPI AML tables successfully acquired and loaded967lb # [ 0.318965] thermal_sys: Registered thermal governor 'bang_bang'968lb # [ 0.319810] thermal_sys: Registered thermal governor 'step_wise'969lb # [ 0.320791] thermal_sys: Registered thermal governor 'user_space'970lb # [ 0.321629] thermal_sys: Registered thermal governor 'power_allocator'971worker2 # [ 0.349544] ACPI: 1 ACPI AML tables successfully acquired and loaded972lb # [ 0.322659] cpuidle: using governor menu973worker1 # [ 0.349029] ACPI: Interpreter enabled974worker1 # [ 0.349681] ACPI: PM: (supports S0 S3 S4 S5)975worker1 # [ 0.350423] ACPI: Using IOAPIC for interrupt routing976lb # [ 0.324872] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5977client # [ 0.343648] ACPI: 1 ACPI AML tables successfully acquired and loaded978worker1 # [ 0.350713] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug979lb # [ 0.326022] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]980worker2 # [ 0.353777] ACPI: Interpreter enabled981worker1 # [ 0.351666] PCI: Using E820 reservations for host bridge windows982lb # [ 0.326812] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry983worker2 # [ 0.354396] ACPI: PM: (supports S0 S3 S4 S5)984lb # [ 0.327821] PCI: Using configuration type 1 for base access985worker2 # [ 0.355119] ACPI: Using IOAPIC for interrupt routing986lb # [ 0.328931] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.987client # [ 0.347843] ACPI: Interpreter enabled988worker1 # [ 0.354759] ACPI: Enabled 2 GPEs in block 00 to 3F989client # [ 0.348446] ACPI: PM: (supports S0 S3 S4 S5)990client # [ 0.349129] ACPI: Using IOAPIC for interrupt routing991worker2 # [ 0.357414] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug992worker2 # [ 0.358380] PCI: Using E820 reservations for host bridge windows993client # [ 0.351493] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug994lb # [ 0.333899] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages995worker2 # [ 0.361516] ACPI: Enabled 2 GPEs in block 00 to 3F996client # [ 0.352442] PCI: Using E820 reservations for host bridge windows997lb # [ 0.334810] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page998client # [ 0.355511] ACPI: Enabled 2 GPEs in block 00 to 3F999worker1 # [ 0.363434] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1000worker1 # [ 0.364447] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1001lb # [ 0.339811] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1002worker1 # [ 0.366189] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1003lb # [ 0.340809] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1004worker2 # [ 0.368985] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1005worker1 # [ 0.366788] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1006worker1 # [ 0.368002] PCI host bridge to bus 0000:001007worker2 # [ 0.369387] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1008worker1 # [ 0.368669] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1009worker2 # [ 0.370459] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1010client # [ 0.363068] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1011worker1 # [ 0.369666] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1012worker2 # [ 0.371502] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1013worker1 # [ 0.370666] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1014client # [ 0.363436] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1015worker2 # [ 0.372731] PCI host bridge to bus 0000:001016worker1 # [ 0.371666] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1017client # [ 0.364509] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1018worker2 # [ 0.373385] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1019worker1 # [ 0.372666] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1020client # [ 0.365554] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1021worker2 # [ 0.374380] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1022client # [ 0.366808] PCI host bridge to bus 0000:001023worker1 # [ 0.373665] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1024worker2 # [ 0.375380] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1025lb # [ 0.351140] ACPI: Added _OSI(Module Device)1026worker1 # [ 0.374671] pci_bus 0000:00: root bus resource [bus 00-ff]1027client # [ 0.367436] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1028worker2 # [ 0.376380] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1029lb # [ 0.351811] ACPI: Added _OSI(Processor Device)1030client # [ 0.368431] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1031lb # [ 0.352526] ACPI: Added _OSI(Processor Aggregator Device)1032worker1 # [ 0.375644] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1033worker2 # [ 0.377380] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1034client # [ 0.369430] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1035worker2 # [ 0.378380] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1036worker1 # [ 0.377148] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1037client # [ 0.370430] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1038worker2 # [ 0.379380] pci_bus 0000:00: root bus resource [bus 00-ff]1039client # [ 0.371430] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1040worker2 # [ 0.380325] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1041client # [ 0.372430] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1042client # [ 0.373431] pci_bus 0000:00: root bus resource [bus 00-ff]1043worker2 # [ 0.381821] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1044client # [ 0.374382] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1045worker1 # [ 0.379695] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1046client # [ 0.375924] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1047lb # [ 0.361032] ACPI: 1 ACPI AML tables successfully acquired and loaded1048worker1 # [ 0.380678] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1049worker1 # [ 0.381671] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1050worker1 # [ 0.382881] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1051worker2 # [ 0.384526] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1052lb # [ 0.365211] ACPI: Interpreter enabled1053worker2 # [ 0.385393] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1054lb # [ 0.365826] ACPI: PM: (supports S0 S3 S4 S5)1055worker2 # [ 0.386423] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1056worker1 # [ 0.384376] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1057lb # [ 0.366521] ACPI: Using IOAPIC for interrupt routing1058client # [ 0.378468] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1059worker2 # [ 0.387691] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1060client # [ 0.379443] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1061client # [ 0.380414] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1062worker1 # [ 0.387674] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1063worker2 # [ 0.389388] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1064lb # [ 0.368872] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1065worker1 # [ 0.388543] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1066client # [ 0.381551] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1067lb # [ 0.369810] PCI: Using E820 reservations for host bridge windows1068worker1 # [ 0.389398] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1069worker1 # [ 0.389672] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1070client # [ 0.383161] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1071worker2 # [ 0.392397] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1072lb # [ 0.372885] ACPI: Enabled 2 GPEs in block 00 to 3F1073worker2 # [ 0.393270] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1074worker1 # [ 0.391227] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1075worker2 # [ 0.394114] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1076client # [ 0.386466] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1077worker2 # [ 0.394386] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1078client # [ 0.387320] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1079worker2 # [ 0.395916] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1080worker1 # [ 0.393755] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1081client # [ 0.388175] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1082client # [ 0.388437] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1083worker1 # [ 0.394673] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1084worker1 # [ 0.395687] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1085client # [ 0.389959] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1086worker2 # [ 0.398387] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1087lb # [ 0.380408] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1088worker1 # [ 0.397237] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1089worker2 # [ 0.399238] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1090lb # [ 0.380816] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1091worker2 # [ 0.400122] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1092lb # [ 0.381889] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1093worker1 # [ 0.398702] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1094client # [ 0.392545] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1095worker2 # [ 0.400929] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1096lb # [ 0.382931] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1097client # [ 0.393437] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1098worker1 # [ 0.399617] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1099lb # [ 0.384207] PCI host bridge to bus 0000:001100client # [ 0.394364] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1101worker1 # [ 0.400393] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1102lb # [ 0.384819] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1103worker2 # [ 0.402389] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1104client # [ 0.395803] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1105worker2 # [ 0.403257] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1106lb # [ 0.385810] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1107worker1 # [ 0.401761] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1108lb # [ 0.386833] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1109worker2 # [ 0.404098] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1110lb # [ 0.387809] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1111client # [ 0.398379] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1112worker1 # [ 0.403701] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1113client # [ 0.399070] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1114lb # [ 0.388809] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1115worker2 # [ 0.404927] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1116worker1 # [ 0.404573] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1117client # [ 0.399453] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1118lb # [ 0.389809] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1119worker1 # [ 0.405394] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1120lb # [ 0.390810] pci_bus 0000:00: root bus resource [bus 00-ff]1121worker2 # [ 0.406389] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1122worker1 # [ 0.406210] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1123worker2 # [ 0.407267] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1124client # [ 0.401458] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1125lb # [ 0.391881] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1126worker2 # [ 0.408096] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1127lb # [ 0.393258] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1128client # [ 0.403468] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1129worker2 # [ 0.408932] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1130worker1 # [ 0.408522] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1131client # [ 0.404299] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1132worker1 # [ 0.409673] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1133client # [ 0.405159] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1134worker1 # [ 0.410639] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1135client # [ 0.406011] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1136worker2 # [ 0.410697] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1137worker1 # [ 0.412153] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1138worker2 # [ 0.412017] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1139worker2 # [ 0.412402] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1140lb # [ 0.395863] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1141lb # [ 0.396821] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1142client # [ 0.407466] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1143worker1 # [ 0.413702] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1144worker2 # [ 0.413964] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1145lb # [ 0.397776] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1146client # [ 0.408438] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1147worker1 # [ 0.414565] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1148client # [ 0.409407] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1149worker1 # [ 0.415403] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1150lb # [ 0.398751] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1151worker2 # [ 0.415389] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1152client # [ 0.410799] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1153worker2 # [ 0.416279] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1154lb # [ 0.400696] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1155worker1 # [ 0.416244] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1156worker2 # [ 0.417132] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1157client # [ 0.412459] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1158worker1 # [ 0.418673] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1159client # [ 0.413286] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1160lb # [ 0.403848] pci 0000:00:02.0: BAR 0 [io 0xc140-0xc15f]1161worker2 # [ 0.417980] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1162worker1 # [ 0.419545] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1163client # [ 0.414137] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1164lb # [ 0.404670] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1165worker1 # [ 0.420401] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1166lb # [ 0.405606] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1167worker2 # [ 0.420388] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1168worker1 # [ 0.420672] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1169lb # [ 0.406667] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1170client # [ 0.415047] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1171worker2 # [ 0.421229] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1172worker2 # [ 0.422138] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1173worker1 # [ 0.422221] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1174lb # [ 0.408183] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1175worker2 # [ 0.422386] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1176client # [ 0.416466] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1177client # [ 0.417288] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1178worker2 # [ 0.423898] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1179worker1 # [ 0.423678] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1180client # [ 0.418138] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1181worker1 # [ 0.424688] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1182client # [ 0.418438] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1183lb # [ 0.411680] pci 0000:00:03.0: BAR 0 [io 0xc160-0xc17f]1184lb # [ 0.412466] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1185worker2 # [ 0.425395] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1186worker1 # [ 0.426225] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1187client # [ 0.419942] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1188lb # [ 0.412831] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1189worker2 # [ 0.426403] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1190client # [ 0.421443] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1191worker1 # [ 0.427686] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1192worker2 # [ 0.427912] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1193lb # [ 0.414420] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1194worker1 # [ 0.428493] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1195client # [ 0.422386] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1196worker1 # [ 0.429440] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1197worker2 # [ 0.429399] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1198lb # [ 0.416844] pci 0000:00:04.0: BAR 0 [io 0xc080-0xc0bf]1199client # [ 0.423997] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1200worker2 # [ 0.430296] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1201worker1 # [ 0.431125] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1202lb # [ 0.417705] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1203worker2 # [ 0.431091] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1204client # [ 0.425438] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1205lb # [ 0.418564] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1206worker2 # [ 0.431927] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1207client # [ 0.426296] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1208worker1 # [ 0.433700] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1209client # [ 0.427133] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1210worker1 # [ 0.434555] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1211lb # [ 0.419389] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1212worker1 # [ 0.435375] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1213worker2 # [ 0.434387] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1214client # [ 0.428009] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1215worker2 # [ 0.435279] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1216lb # [ 0.421602] pci 0000:00:05.0: BAR 0 [io 0xc180-0xc19f]1217worker1 # [ 0.436240] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1218worker2 # [ 0.436149] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1219lb # [ 0.422474] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1220worker1 # [ 0.437315] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1221client # [ 0.429837] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1222lb # [ 0.422834] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1223worker2 # [ 0.437792] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1224client # [ 0.431066] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1225worker1 # [ 0.437878] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1226worker2 # [ 0.439013] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1227lb # [ 0.424391] pci 0000:00:06.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1228client # [ 0.431480] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1229worker1 # [ 0.439297] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1230worker2 # [ 0.439577] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1231client # [ 0.432995] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1232worker2 # [ 0.441026] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1233client # [ 0.434154] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1234lb # [ 0.426845] pci 0000:00:06.0: BAR 0 [io 0xc1a0-0xc1bf]1235lb # [ 0.427763] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1236client # [ 0.434632] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1237lb # [ 0.428588] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1238client # [ 0.436065] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1239lb # [ 0.430288] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1240lb # [ 0.432846] pci 0000:00:07.0: BAR 0 [io 0xc000-0xc07f]1241lb # [ 0.433743] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1242lb # [ 0.434618] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1243worker1 # [ 0.439863] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1244lb # [ 0.436304] pci 0000:00:08.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1245worker2 # [ 0.442224] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1246worker1 # [ 0.443317] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1247worker1 # [ 0.443973] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1248lb # [ 0.438587] pci 0000:00:08.0: BAR 0 [io 0xc1c0-0xc1df]1249worker2 # [ 0.444990] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1250lb # [ 0.439479] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1251client # [ 0.436634] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1252worker1 # [ 0.445546] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1253lb # [ 0.439833] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1254worker2 # [ 0.445616] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1255lb # [ 0.440815] pci 0000:00:08.0: ROM [mem 0xfeb80000-0xfebbffff pref]1256worker1 # [ 0.446682] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1257worker2 # [ 0.447321] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1258client # [ 0.440034] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1259worker1 # [ 0.447952] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1260lb # [ 0.442378] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1261worker2 # [ 0.448419] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1262client # [ 0.440668] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1263worker1 # [ 0.448922] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1264worker2 # [ 0.449666] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1265lb # [ 0.443853] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1266client # [ 0.442473] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1267worker2 # [ 0.450632] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1268lb # [ 0.444832] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1269worker1 # [ 0.451348] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1270client # [ 0.443692] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1271worker1 # [ 0.451672] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1272lb # [ 0.446387] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1273client # [ 0.444748] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1274worker2 # [ 0.453019] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1275worker1 # [ 0.452973] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1276worker2 # [ 0.453387] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1277client # [ 0.445676] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1278worker1 # [ 0.454425] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1279lb # [ 0.448720] pci 0000:00:0a.0: BAR 0 [io 0xc0c0-0xc0ff]1280worker2 # [ 0.454691] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1281lb # [ 0.449490] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1282client # [ 0.448099] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1283worker2 # [ 0.456154] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1284client # [ 0.448438] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1285lb # [ 0.449831] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1286worker1 # [ 0.457611] ACPI: PCI: Interrupt link LNKA configured for IRQ 101287client # [ 0.449688] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1288lb # [ 0.451360] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1289worker1 # [ 0.458552] ACPI: PCI: Interrupt link LNKB configured for IRQ 101290client # [ 0.451187] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1291worker2 # [ 0.459345] ACPI: PCI: Interrupt link LNKA configured for IRQ 101292worker1 # [ 0.459554] ACPI: PCI: Interrupt link LNKC configured for IRQ 111293worker2 # [ 0.460232] ACPI: PCI: Interrupt link LNKB configured for IRQ 101294worker1 # [ 0.460516] ACPI: PCI: Interrupt link LNKD configured for IRQ 111295lb # [ 0.453501] pci 0000:00:0b.0: BAR 0 [io 0xc1e0-0xc1ff]1296worker2 # [ 0.461199] ACPI: PCI: Interrupt link LNKC configured for IRQ 111297worker1 # [ 0.461510] ACPI: PCI: Interrupt link LNKE configured for IRQ 101298lb # [ 0.453817] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1299worker2 # [ 0.462180] ACPI: PCI: Interrupt link LNKD configured for IRQ 111300worker1 # [ 0.462483] ACPI: PCI: Interrupt link LNKF configured for IRQ 101301lb # [ 0.454746] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1302client # [ 0.454398] ACPI: PCI: Interrupt link LNKA configured for IRQ 101303worker2 # [ 0.463176] ACPI: PCI: Interrupt link LNKE configured for IRQ 101304worker1 # [ 0.463494] ACPI: PCI: Interrupt link LNKG configured for IRQ 111305client # [ 0.455234] ACPI: PCI: Interrupt link LNKB configured for IRQ 101306lb # [ 0.456218] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1307worker2 # [ 0.464223] ACPI: PCI: Interrupt link LNKF configured for IRQ 101308worker1 # [ 0.464547] ACPI: PCI: Interrupt link LNKH configured for IRQ 111309client # [ 0.456537] ACPI: PCI: Interrupt link LNKC configured for IRQ 111310worker2 # [ 0.465179] ACPI: PCI: Interrupt link LNKG configured for IRQ 111311worker1 # [ 0.465412] ACPI: PCI: Interrupt link GSIA configured for IRQ 161312client # [ 0.457529] ACPI: PCI: Interrupt link LNKD configured for IRQ 111313worker1 # [ 0.465681] ACPI: PCI: Interrupt link GSIB configured for IRQ 171314worker2 # [ 0.466213] ACPI: PCI: Interrupt link LNKH configured for IRQ 111315lb # [ 0.457555] pci 0000:00:1d.0: BAR 4 [io 0xc200-0xc21f]1316client # [ 0.458526] ACPI: PCI: Interrupt link LNKE configured for IRQ 101317worker2 # [ 0.467124] ACPI: PCI: Interrupt link GSIA configured for IRQ 161318worker1 # [ 0.466638] ACPI: PCI: Interrupt link GSIC configured for IRQ 181319client # [ 0.459526] ACPI: PCI: Interrupt link LNKF configured for IRQ 101320worker2 # [ 0.467395] ACPI: PCI: Interrupt link GSIB configured for IRQ 171321lb # [ 0.458656] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1322worker1 # [ 0.467578] ACPI: PCI: Interrupt link GSID configured for IRQ 191323worker2 # [ 0.468313] ACPI: PCI: Interrupt link GSIC configured for IRQ 181324client # [ 0.460532] ACPI: PCI: Interrupt link LNKG configured for IRQ 111325worker1 # [ 0.468455] ACPI: PCI: Interrupt link GSIE configured for IRQ 201326lb # [ 0.460655] pci 0000:00:1d.1: BAR 4 [io 0xc220-0xc23f]1327client # [ 0.461530] ACPI: PCI: Interrupt link LNKH configured for IRQ 111328worker2 # [ 0.469106] ACPI: PCI: Interrupt link GSID configured for IRQ 191329worker1 # [ 0.469495] ACPI: PCI: Interrupt link GSIF configured for IRQ 211330client # [ 0.462470] ACPI: PCI: Interrupt link GSIA configured for IRQ 161331worker2 # [ 0.469395] ACPI: PCI: Interrupt link GSIE configured for IRQ 201332worker1 # [ 0.470405] ACPI: PCI: Interrupt link GSIG configured for IRQ 221333client # [ 0.463405] ACPI: PCI: Interrupt link GSIB configured for IRQ 171334worker2 # [ 0.470394] ACPI: PCI: Interrupt link GSIF configured for IRQ 211335worker1 # [ 0.470678] ACPI: PCI: Interrupt link GSIH configured for IRQ 231336client # [ 0.464173] ACPI: PCI: Interrupt link GSIC configured for IRQ 181337worker2 # [ 0.471357] ACPI: PCI: Interrupt link GSIG configured for IRQ 221338client # [ 0.464443] ACPI: PCI: Interrupt link GSID configured for IRQ 191339worker2 # [ 0.472150] ACPI: PCI: Interrupt link GSIH configured for IRQ 231340worker1 # [ 0.472740] iommu: Default domain type: Translated1341client # [ 0.465448] ACPI: PCI: Interrupt link GSIE configured for IRQ 201342worker1 # [ 0.473566] iommu: DMA domain TLB invalidation policy: lazy mode1343worker2 # [ 0.473352] iommu: Default domain type: Translated1344client # [ 0.466443] ACPI: PCI: Interrupt link GSIF configured for IRQ 211345worker1 # [ 0.474711] ACPI: bus type USB registered1346worker2 # [ 0.473981] iommu: DMA domain TLB invalidation policy: lazy mode1347client # [ 0.467410] ACPI: PCI: Interrupt link GSIG configured for IRQ 221348worker1 # [ 0.475473] usbcore: registered new interface driver usbfs1349worker2 # [ 0.474620] ACPI: bus type USB registered1350client # [ 0.468156] ACPI: PCI: Interrupt link GSIH configured for IRQ 231351worker1 # [ 0.476376] usbcore: registered new interface driver hub1352worker2 # [ 0.475361] usbcore: registered new interface driver usbfs1353worker1 # [ 0.476676] usbcore: registered new device driver usb1354worker2 # [ 0.476306] usbcore: registered new interface driver hub1355client # [ 0.469500] iommu: Default domain type: Translated1356worker1 # [ 0.478432] NetLabel: Initializing1357worker2 # [ 0.477062] usbcore: registered new device driver usb1358client # [ 0.470297] iommu: DMA domain TLB invalidation policy: lazy mode1359worker1 # [ 0.478698] NetLabel: domain hash size = 1281360client # [ 0.471405] ACPI: bus type USB registered1361worker2 # [ 0.478327] NetLabel: Initializing1362worker1 # [ 0.479469] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1363lb # [ 0.461659] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1364client # [ 0.471979] usbcore: registered new interface driver usbfs1365worker2 # [ 0.478820] NetLabel: domain hash size = 1281366worker1 # [ 0.480517] NetLabel: unlabeled traffic allowed by default1367client # [ 0.472446] usbcore: registered new interface driver hub1368worker2 # [ 0.479379] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1369worker1 # [ 0.481356] PCI: Using ACPI for IRQ routing1370client # [ 0.473270] usbcore: registered new device driver usb1371lb # [ 0.463586] pci 0000:00:1d.2: BAR 4 [io 0xc240-0xc25f]1372worker2 # [ 0.480309] NetLabel: unlabeled traffic allowed by default1373worker2 # [ 0.481090] PCI: Using ACPI for IRQ routing1374client # [ 0.474843] NetLabel: Initializing1375lb # [ 0.464048] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1376client # [ 0.475431] NetLabel: domain hash size = 1281377client # [ 0.476156] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1378client # [ 0.476467] NetLabel: unlabeled traffic allowed by default1379lb # [ 0.465845] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1380client # [ 0.477402] PCI: Using ACPI for IRQ routing1381lb # [ 0.467065] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1382lb # [ 0.468130] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1383lb # [ 0.469065] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1384lb # [ 0.470837] pci 0000:00:1f.2: BAR 4 [io 0xc260-0xc27f]1385lb # [ 0.471741] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1386lb # [ 0.472946] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1387lb # [ 0.474637] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1388lb # [ 0.478461] ACPI: PCI: Interrupt link LNKA configured for IRQ 101389lb # [ 0.478913] ACPI: PCI: Interrupt link LNKB configured for IRQ 101390lb # [ 0.479909] ACPI: PCI: Interrupt link LNKC configured for IRQ 111391lb # [ 0.480909] ACPI: PCI: Interrupt link LNKD configured for IRQ 111392lb # [ 0.481916] ACPI: PCI: Interrupt link LNKE configured for IRQ 101393lb # [ 0.482907] ACPI: PCI: Interrupt link LNKF configured for IRQ 101394lb # [ 0.483913] ACPI: PCI: Interrupt link LNKG configured for IRQ 111395lb # [ 0.484906] ACPI: PCI: Interrupt link LNKH configured for IRQ 111396lb # [ 0.485856] ACPI: PCI: Interrupt link GSIA configured for IRQ 161397lb # [ 0.486824] ACPI: PCI: Interrupt link GSIB configured for IRQ 171398lb # [ 0.487838] ACPI: PCI: Interrupt link GSIC configured for IRQ 181399lb # [ 0.488888] ACPI: PCI: Interrupt link GSID configured for IRQ 191400lb # [ 0.490016] ACPI: PCI: Interrupt link GSIE configured for IRQ 201401lb # [ 0.490823] ACPI: PCI: Interrupt link GSIF configured for IRQ 211402lb # [ 0.491822] ACPI: PCI: Interrupt link GSIG configured for IRQ 221403lb # [ 0.492822] ACPI: PCI: Interrupt link GSIH configured for IRQ 231404lb # [ 0.494853] iommu: Default domain type: Translated1405lb # [ 0.495706] iommu: DMA domain TLB invalidation policy: lazy mode1406lb # [ 0.496885] ACPI: bus type USB registered1407lb # [ 0.497674] usbcore: registered new interface driver usbfs1408lb # [ 0.498537] usbcore: registered new interface driver hub1409lb # [ 0.498820] usbcore: registered new device driver usb1410worker1 # [ 0.524839] pci 0000:00:01.0: vgaarb: setting as boot VGA device1411lb # [ 0.500741] NetLabel: Initializing1412worker1 # [ 0.525662] pci 0000:00:01.0: vgaarb: bridge control possible1413worker2 # [ 0.525021] pci 0000:00:01.0: vgaarb: setting as boot VGA device1414worker2 # [ 0.525376] pci 0000:00:01.0: vgaarb: bridge control possible1415worker1 # [ 0.525662] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1416worker1 # [ 0.525671] vgaarb: loaded1417lb # [ 0.501261] NetLabel: domain hash size = 1281418worker2 # [ 0.525376] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1419lb # [ 0.501811] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1420worker1 # [ 0.526429] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01421worker2 # [ 0.525385] vgaarb: loaded1422lb # [ 0.502784] NetLabel: unlabeled traffic allowed by default1423worker1 # [ 0.526668] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1424worker2 # [ 0.526113] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01425lb # [ 0.503496] PCI: Using ACPI for IRQ routing1426client # [ 0.522802] pci 0000:00:01.0: vgaarb: setting as boot VGA device1427worker2 # [ 0.526382] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1428client # [ 0.523426] pci 0000:00:01.0: vgaarb: bridge control possible1429client # [ 0.523426] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1430client # [ 0.523435] vgaarb: loaded1431worker1 # [ 0.529747] clocksource: Switched to clocksource kvm-clock1432client # [ 0.524189] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01433client # [ 0.524432] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1434worker2 # [ 0.531463] clocksource: Switched to clocksource kvm-clock1435worker1 # [ 0.533232] VFS: Disk quotas dquot_6.6.01436worker1 # [ 0.533921] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1437worker1 # [ 0.535349] pnp: PnP ACPI init1438worker1 # [ 0.536147] ACPI: IRQ 4 override to edge(!), high(!)1439worker2 # [ 0.534904] VFS: Disk quotas dquot_6.6.01440client # [ 0.529520] clocksource: Switched to clocksource kvm-clock1441worker2 # [ 0.535601] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1442worker1 # [ 0.537145] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1443worker2 # [ 0.536934] pnp: PnP ACPI init1444worker1 # [ 0.538784] pnp: PnP ACPI: found 6 devices1445worker2 # [ 0.537756] ACPI: IRQ 4 override to edge(!), high(!)1446client # [ 0.532974] VFS: Disk quotas dquot_6.6.01447worker2 # [ 0.538754] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1448client # [ 0.533653] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1449worker2 # [ 0.540309] pnp: PnP ACPI: found 6 devices1450client # [ 0.534970] pnp: PnP ACPI init1451client # [ 0.535717] ACPI: IRQ 4 override to edge(!), high(!)1452client # [ 0.536666] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1453client # [ 0.538221] pnp: PnP ACPI: found 6 devices1454worker1 # [ 0.546359] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1455worker1 # [ 0.547882] clocksource: Switched to clocksource acpi_pm1456worker1 # [ 0.548891] NET: Registered PF_INET protocol family1457worker2 # [ 0.547871] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1458worker1 # [ 0.550104] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1459worker2 # [ 0.549380] clocksource: Switched to clocksource acpi_pm1460worker2 # [ 0.550361] NET: Registered PF_INET protocol family1461worker2 # [ 0.551546] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1462client # [ 0.545811] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1463client # [ 0.547235] clocksource: Switched to clocksource acpi_pm1464client # [ 0.548186] NET: Registered PF_INET protocol family1465client # [ 0.549333] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1466worker1 # [ 0.565934] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1467worker1 # [ 0.567545] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1468worker1 # [ 0.568905] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1469worker2 # [ 0.567593] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1470lb # [ 0.546635] pci 0000:00:01.0: vgaarb: setting as boot VGA device1471worker1 # [ 0.570294] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1472worker2 # [ 0.569139] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1473lb # [ 0.546805] pci 0000:00:01.0: vgaarb: bridge control possible1474worker1 # [ 0.571527] TCP: Hash tables configured (established 16384 bind 16384)1475worker2 # [ 0.570436] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1476lb # [ 0.546805] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1477worker1 # [ 0.572724] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1478lb # [ 0.546815] vgaarb: loaded1479client # [ 0.565488] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1480worker2 # [ 0.571785] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1481lb # [ 0.547528] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01482worker1 # [ 0.574028] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1483client # [ 0.566953] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1484worker2 # [ 0.572991] TCP: Hash tables configured (established 16384 bind 16384)1485lb # [ 0.547810] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1486worker1 # [ 0.575136] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1487worker2 # [ 0.574067] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1488client # [ 0.568232] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1489worker1 # [ 0.576346] NET: Registered PF_UNIX/PF_LOCAL protocol family1490worker2 # [ 0.575295] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1491worker1 # [ 0.577300] NET: Registered PF_XDP protocol family1492client # [ 0.569552] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1493worker2 # [ 0.576367] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1494worker1 # [ 0.578114] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1495client # [ 0.570857] TCP: Hash tables configured (established 16384 bind 16384)1496worker2 # [ 0.577553] NET: Registered PF_UNIX/PF_LOCAL protocol family1497worker1 # [ 0.579085] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1498lb # [ 0.550899] clocksource: Switched to clocksource kvm-clock1499client # [ 0.571957] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1500worker2 # [ 0.578483] NET: Registered PF_XDP protocol family1501worker1 # [ 0.580046] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1502client # [ 0.573201] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1503worker2 # [ 0.579313] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1504worker1 # [ 0.581122] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1505client # [ 0.574295] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1506worker2 # [ 0.580311] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1507worker1 # [ 0.582270] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1508lb # [ 0.554341] VFS: Disk quotas dquot_6.6.01509client # [ 0.575518] NET: Registered PF_UNIX/PF_LOCAL protocol family1510worker2 # [ 0.581297] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1511worker1 # [ 0.583367] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1512lb # [ 0.555026] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1513client # [ 0.576449] NET: Registered PF_XDP protocol family1514worker2 # [ 0.582352] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1515lb # [ 0.556319] pnp: PnP ACPI init1516client # [ 0.577280] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1517worker2 # [ 0.583418] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1518worker1 # [ 0.585268] ACPI: \_SB_.GSIA: Enabled at IRQ 161519lb # [ 0.557107] ACPI: IRQ 4 override to edge(!), high(!)1520client # [ 0.578303] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1521worker2 # [ 0.584482] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1522client # [ 0.579318] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1523lb # [ 0.558129] system 00:05: [mem 0xb0000000-0xbfffffff window] has been reserved1524worker2 # [ 0.586166] ACPI: \_SB_.GSIA: Enabled at IRQ 161525worker1 # [ 0.587630] ACPI: \_SB_.GSIB: Enabled at IRQ 171526lb # [ 0.559651] pnp: PnP ACPI: found 6 devices1527client # [ 0.580423] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1528client # [ 0.581502] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1529worker1 # [ 0.589532] ACPI: \_SB_.GSIC: Enabled at IRQ 181530worker2 # [ 0.588564] ACPI: \_SB_.GSIB: Enabled at IRQ 171531client # [ 0.582659] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1532worker1 # [ 0.591459] ACPI: \_SB_.GSID: Enabled at IRQ 191533client # [ 0.584405] ACPI: \_SB_.GSIA: Enabled at IRQ 161534worker2 # [ 0.590419] ACPI: \_SB_.GSIC: Enabled at IRQ 181535worker1 # [ 0.593225] PCI: CLS 0 bytes, default 641536worker2 # [ 0.592307] ACPI: \_SB_.GSID: Enabled at IRQ 191537client # [ 0.586901] ACPI: \_SB_.GSIB: Enabled at IRQ 171538worker1 # [ 0.594139] Trying to unpack rootfs image as initramfs...1539worker2 # [ 0.593915] PCI: CLS 0 bytes, default 641540client # [ 0.588714] ACPI: \_SB_.GSIC: Enabled at IRQ 181541lb # [ 0.567196] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1542worker2 # [ 0.594801] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1543lb # [ 0.568672] clocksource: Switched to clocksource acpi_pm1544client # [ 0.590605] ACPI: \_SB_.GSID: Enabled at IRQ 191545lb # [ 0.569672] NET: Registered PF_INET protocol family1546worker2 # [ 0.596553] Trying to unpack rootfs image as initramfs...1547worker1 # [ 0.598246] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1548client # [ 0.592229] PCI: CLS 0 bytes, default 641549lb # [ 0.570822] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1550client # [ 0.593146] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1551client # [ 0.594951] Trying to unpack rootfs image as initramfs...1552lb # [ 0.586842] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1553lb # [ 0.588436] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1554lb # [ 0.589816] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1555lb # [ 0.591229] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1556lb # [ 0.592565] TCP: Hash tables configured (established 16384 bind 16384)1557lb # [ 0.593756] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1558lb # [ 0.595136] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1559lb # [ 0.596264] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1560lb # [ 0.597588] NET: Registered PF_UNIX/PF_LOCAL protocol family1561lb # [ 0.598601] NET: Registered PF_XDP protocol family1562lb # [ 0.599495] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1563lb # [ 0.600555] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1564lb # [ 0.601618] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1565lb # [ 0.602777] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1566lb # [ 0.603957] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1567lb # [ 0.605143] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1568lb # [ 0.606885] ACPI: \_SB_.GSIA: Enabled at IRQ 161569lb # [ 0.609675] ACPI: \_SB_.GSIB: Enabled at IRQ 171570lb # [ 0.611634] ACPI: \_SB_.GSIC: Enabled at IRQ 181571worker1 # [ 0.641452] Initialise system trusted keyrings1572worker2 # [ 0.640150] Initialise system trusted keyrings1573lb # [ 0.613580] ACPI: \_SB_.GSID: Enabled at IRQ 191574worker1 # [ 0.642389] workingset: timestamp_bits=40 max_order=19 bucket_order=01575lb # [ 0.615172] PCI: CLS 0 bytes, default 641576worker2 # [ 0.643110] workingset: timestamp_bits=40 max_order=19 bucket_order=01577lb # [ 0.616060] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1578lb # [ 0.617814] Trying to unpack rootfs image as initramfs...1579client # [ 0.639694] Initialise system trusted keyrings1580client # [ 0.642899] workingset: timestamp_bits=40 max_order=19 bucket_order=01581worker2 # [ 0.664241] Key type asymmetric registered1582worker1 # [ 0.666153] Key type asymmetric registered1583worker2 # [ 0.664967] Asymmetric key parser 'x509' registered1584worker1 # [ 0.666876] Asymmetric key parser 'x509' registered1585worker2 # [ 0.665810] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1586worker1 # [ 0.667807] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1587worker1 # [ 0.671142] io scheduler mq-deadline registered1588client # [ 0.664249] Key type asymmetric registered1589worker1 # [ 0.671948] io scheduler kyber registered1590worker2 # [ 0.671183] io scheduler mq-deadline registered1591worker2 # [ 0.671955] io scheduler kyber registered1592client # [ 0.666881] Asymmetric key parser 'x509' registered1593client # [ 0.667740] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1594worker1 # [ 0.675609] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1595worker2 # [ 0.675593] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1596worker1 # [ 0.676906] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1597client # [ 0.670949] io scheduler mq-deadline registered1598worker2 # [ 0.676879] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1599client # [ 0.671719] io scheduler kyber registered1600worker1 # [ 0.680804] Linux agpgart interface v0.1031601worker1 # [ 0.681639] ACPI: bus type drm_connector registered1602worker2 # [ 0.680870] Linux agpgart interface v0.1031603client # [ 0.675393] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1604worker2 # [ 0.681658] ACPI: bus type drm_connector registered1605client # [ 0.676684] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1606worker1 # [ 0.684446] usbcore: registered new interface driver usbserial_generic1607worker1 # [ 0.685551] usbserial: USB Serial support registered for generic1608client # [ 0.680622] Linux agpgart interface v0.1031609worker2 # [ 0.686492] usbcore: registered new interface driver usbserial_generic1610client # [ 0.681415] ACPI: bus type drm_connector registered1611worker2 # [ 0.687575] usbserial: USB Serial support registered for generic1612worker1 # [ 0.689039] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1613worker1 # [ 0.689039] Please enable it if your BIOS has the CPPC option.1614lb # [ 0.662522] Initialise system trusted keyrings1615worker1 # [ 0.691315] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1616client # [ 0.684262] usbcore: registered new interface driver usbserial_generic1617client # [ 0.685361] usbserial: USB Serial support registered for generic1618worker2 # [ 0.691077] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1619worker2 # [ 0.691077] Please enable it if your BIOS has the CPPC option.1620lb # [ 0.665416] workingset: timestamp_bits=40 max_order=19 bucket_order=01621worker2 # [ 0.693331] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1622worker1 # [ 0.696163] drop_monitor: Initializing network drop monitor service1623worker2 # [ 0.694751] drop_monitor: Initializing network drop monitor service1624client # [ 0.688869] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1625worker1 # [ 0.697335] NET: Registered PF_INET6 protocol family1626worker2 # [ 0.695899] NET: Registered PF_INET6 protocol family1627client # [ 0.688869] Please enable it if your BIOS has the CPPC option.1628client # [ 0.691075] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1629worker1 # [ 0.700555] Segment Routing with IPv61630worker2 # [ 0.699529] Segment Routing with IPv61631client # [ 0.694995] drop_monitor: Initializing network drop monitor service1632worker2 # [ 0.702081] In-situ OAM (IOAM) with IPv61633client # [ 0.696361] NET: Registered PF_INET6 protocol family1634worker1 # [ 0.704048] In-situ OAM (IOAM) with IPv61635worker2 # [ 0.703112] IPI shorthand broadcast: enabled1636worker1 # [ 0.705030] IPI shorthand broadcast: enabled1637client # [ 0.700378] Segment Routing with IPv61638client # [ 0.703864] In-situ OAM (IOAM) with IPv61639client # [ 0.704922] IPI shorthand broadcast: enabled1640worker2 # [ 0.711194] sched_clock: Marking stable (576013680, 135040205)->(788205059, -77151174)1641worker1 # [ 0.713267] sched_clock: Marking stable (579014728, 134009520)->(788401797, -75377549)1642lb # [ 0.686745] Key type asymmetric registered1643lb # [ 0.687503] Asymmetric key parser 'x509' registered1644worker2 # [ 0.716290] registered taskstats version 11645worker1 # [ 0.718108] registered taskstats version 11646worker2 # [ 0.717255] Loading compiled-in X.509 certificates1647worker1 # [ 0.719073] Loading compiled-in X.509 certificates1648lb # [ 0.690456] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1649client # [ 0.713059] sched_clock: Marking stable (579014679, 133829521)->(786375298, -73531098)1650lb # [ 0.693429] io scheduler mq-deadline registered1651lb # [ 0.694214] io scheduler kyber registered1652client # [ 0.715926] registered taskstats version 11653client # [ 0.716905] Loading compiled-in X.509 certificates1654lb # [ 0.697933] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1655lb # [ 0.699263] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1656lb # [ 0.704533] Linux agpgart interface v0.1031657lb # [ 0.705326] ACPI: bus type drm_connector registered1658worker2 # [ 0.733656] Demotion targets for Node 0: null1659worker1 # [ 0.735420] Demotion targets for Node 0: null1660lb # [ 0.708611] usbcore: registered new interface driver usbserial_generic1661worker2 # [ 0.736113] Key type .fscrypt registered1662worker1 # [ 0.738072] Key type .fscrypt registered1663worker2 # [ 0.736789] Key type fscrypt-provisioning registered1664lb # [ 0.709734] usbserial: USB Serial support registered for generic1665worker1 # [ 0.738720] Key type fscrypt-provisioning registered1666worker2 # [ 0.737708] ima: No TPM chip found, activating TPM-bypass!1667worker1 # [ 0.739688] ima: No TPM chip found, activating TPM-bypass!1668worker1 # [ 0.742042] ima: Allocated hash algorithm: sha11669lb # [ 0.713386] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1670worker1 # [ 0.742854] ima: No architecture policies found1671client # [ 0.735864] Demotion targets for Node 0: null1672lb # [ 0.713386] Please enable it if your BIOS has the CPPC option.1673worker2 # [ 0.742073] ima: Allocated hash algorithm: sha11674worker2 # [ 0.742837] ima: No architecture policies found1675lb # [ 0.715634] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1676worker2 # [ 0.743809] PM: Magic number: 10:504:5761677client # [ 0.739158] Key type .fscrypt registered1678worker1 # [ 0.747061] PM: Magic number: 10:504:5761679client # [ 0.739902] Key type fscrypt-provisioning registered1680lb # [ 0.719529] drop_monitor: Initializing network drop monitor service1681worker1 # [ 0.748627] RAS: Correctable Errors collector initialized.1682worker2 # [ 0.747425] RAS: Correctable Errors collector initialized.1683lb # [ 0.720809] NET: Registered PF_INET6 protocol family1684client # [ 0.742893] ima: No TPM chip found, activating TPM-bypass!1685client # [ 0.743858] ima: Allocated hash algorithm: sha11686client # [ 0.744684] ima: No architecture policies found1687lb # [ 0.723829] Segment Routing with IPv61688client # [ 0.747119] PM: Magic number: 10:504:5761689lb # [ 0.726403] In-situ OAM (IOAM) with IPv61690lb # [ 0.727433] IPI shorthand broadcast: enabled1691worker2 # [ 0.756087] clk: Disabling unused clocks1692worker1 # [ 0.757711] clk: Disabling unused clocks1693worker2 # [ 0.756780] PM: genpd: Disabling unused power domains1694client # [ 0.750970] RAS: Correctable Errors collector initialized.1695worker1 # [ 0.760037] PM: genpd: Disabling unused power domains1696lb # [ 0.735481] sched_clock: Marking stable (600013871, 135351653)->(811223034, -75857510)1697client # [ 0.759669] clk: Disabling unused clocks1698client # [ 0.760384] PM: genpd: Disabling unused power domains1699lb # [ 0.739598] registered taskstats version 11700lb # [ 0.740579] Loading compiled-in X.509 certificates1701lb # [ 0.759384] Demotion targets for Node 0: null1702lb # [ 0.760298] Key type .fscrypt registered1703lb # [ 0.761010] Key type fscrypt-provisioning registered1704lb # [ 0.764486] ima: No TPM chip found, activating TPM-bypass!1705lb # [ 0.765450] ima: Allocated hash algorithm: sha11706lb # [ 0.766268] ima: No architecture policies found1707lb # [ 0.770561] PM: Magic number: 10:504:5761708lb # [ 0.772159] RAS: Correctable Errors collector initialized.1709lb # [ 0.780127] clk: Disabling unused clocks1710lb # [ 0.784378] PM: genpd: Disabling unused power domains1711worker1 # [ 0.912833] Freeing initrd memory: 29092K1712worker2 # [ 0.912493] Freeing initrd memory: 29092K1713worker1 # [ 0.915970] Freeing unused decrypted memory: 2028K1714worker2 # [ 0.915578] Freeing unused decrypted memory: 2028K1715worker1 # [ 0.918522] Freeing unused kernel image (initmem) memory: 3644K1716worker1 # [ 0.919627] Write protecting the kernel read-only data: 32768k1717worker2 # [ 0.918154] Freeing unused kernel image (initmem) memory: 3644K1718worker2 # [ 0.919256] Write protecting the kernel read-only data: 32768k1719worker1 # [ 0.921521] Freeing unused kernel image (text/rodata gap) memory: 1208K1720worker2 # [ 0.921132] Freeing unused kernel image (text/rodata gap) memory: 1208K1721worker1 # [ 0.923080] Freeing unused kernel image (rodata/data gap) memory: 752K1722worker2 # [ 0.922655] Freeing unused kernel image (rodata/data gap) memory: 752K1723client # [ 0.944865] Freeing initrd memory: 29096K1724client # [ 0.948181] Freeing unused decrypted memory: 2028K1725client # [ 0.950988] Freeing unused kernel image (initmem) memory: 3644K1726client # [ 0.952147] Write protecting the kernel read-only data: 32768k1727client # [ 0.954123] Freeing unused kernel image (text/rodata gap) memory: 1208K1728client # [ 0.955729] Freeing unused kernel image (rodata/data gap) memory: 752K1729lb # [ 0.939451] Freeing initrd memory: 29092K1730lb # [ 0.942544] Freeing unused decrypted memory: 2028K1731lb # [ 0.945133] Freeing unused kernel image (initmem) memory: 3644K1732worker1 # [ 0.974323] x86/mm: Checked W+X mappings: passed, no W+X pages found.1733lb # [ 0.946243] Write protecting the kernel read-only data: 32768k1734worker1 # [ 0.975374] Run /init as init process1735worker2 # [ 0.973725] x86/mm: Checked W+X mappings: passed, no W+X pages found.1736worker2 # [ 0.974774] Run /init as init process1737lb # [ 0.948052] Freeing unused kernel image (text/rodata gap) memory: 1208K1738lb # [ 0.949600] Freeing unused kernel image (rodata/data gap) memory: 752K1739worker1 # [ 0.985137] systemd[1]: Inserted module 'autofs4'1740worker2 # [ 0.984677] systemd[1]: Inserted module 'autofs4'1741worker1 # [ 1.001462] fuse: init (API version 7.45)1742worker2 # [ 1.000706] fuse: init (API version 7.45)1743worker1 # [ 1.007252] ACPI: \_SB_.GSIG: Enabled at IRQ 221744worker2 # [ 1.006442] ACPI: \_SB_.GSIG: Enabled at IRQ 221745worker1 # [ 1.009777] ACPI: \_SB_.GSIH: Enabled at IRQ 231746worker2 # [ 1.009001] ACPI: \_SB_.GSIH: Enabled at IRQ 231747worker1 # [ 1.013109] ACPI: \_SB_.GSIE: Enabled at IRQ 201748worker2 # [ 1.012278] ACPI: \_SB_.GSIE: Enabled at IRQ 201749client # [ 1.006930] x86/mm: Checked W+X mappings: passed, no W+X pages found.1750client # [ 1.008123] Run /init as init process1751worker1 # [ 1.015434] ACPI: \_SB_.GSIF: Enabled at IRQ 211752worker2 # [ 1.014641] ACPI: \_SB_.GSIF: Enabled at IRQ 211753client # [ 1.018989] systemd[1]: Inserted module 'autofs4'1754lb # [ 1.000675] x86/mm: Checked W+X mappings: passed, no W+X pages found.1755lb # [ 1.001788] Run /init as init process1756lb # [ 1.011777] systemd[1]: Inserted module 'autofs4'1757client # [ 1.036108] fuse: init (API version 7.45)1758worker2 # [ 1.045765] systemd[1]: Successfully made /usr/ read-only.1759worker1 # [ 1.048236] systemd[1]: Successfully made /usr/ read-only.1760client # [ 1.042372] ACPI: \_SB_.GSIG: Enabled at IRQ 221761client # [ 1.044782] ACPI: \_SB_.GSIH: Enabled at IRQ 231762client # [ 1.047955] ACPI: \_SB_.GSIE: Enabled at IRQ 201763lb # [ 1.028059] fuse: init (API version 7.45)1764client # [ 1.050204] ACPI: \_SB_.GSIF: Enabled at IRQ 211765lb # [ 1.033868] ACPI: \_SB_.GSIG: Enabled at IRQ 221766lb # [ 1.036488] ACPI: \_SB_.GSIH: Enabled at IRQ 231767lb # [ 1.040242] ACPI: \_SB_.GSIE: Enabled at IRQ 201768lb # [ 1.042682] ACPI: \_SB_.GSIF: Enabled at IRQ 211769client # [ 1.083934] systemd[1]: Successfully made /usr/ read-only.1770lb # [ 1.070170] systemd[1]: Successfully made /usr/ read-only.1771worker1 # [ 1.384099] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1772worker1 # [ 1.389647] systemd[1]: Detected virtualization kvm.1773worker1 # [ 1.390637] systemd[1]: Detected architecture x86-64.1774worker1 # [ 1.391625] systemd[1]: Running in initrd.1775worker1 # [ 1.393041] systemd[1]: Initializing machine ID from random generator.1776worker1 # [ 1.394455] systemd[1]: Hostname set to <worker1>.1777worker2 # [ 1.382356] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1778worker2 # [ 1.394424] systemd[1]: Detected virtualization kvm.1779worker2 # [ 1.396540] systemd[1]: Detected architecture x86-64.1780worker2 # [ 1.398680] systemd[1]: Running in initrd.1781worker2 # [ 1.401191] systemd[1]: Initializing machine ID from random generator.1782worker2 # [ 1.403991] systemd[1]: Hostname set to <worker2>.1783client # [ 1.419600] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1784client # [ 1.424389] systemd[1]: Detected virtualization kvm.1785client # [ 1.425253] systemd[1]: Detected architecture x86-64.1786client # [ 1.426089] systemd[1]: Running in initrd.1787client # [ 1.427303] systemd[1]: Initializing machine ID from random generator.1788client # [ 1.428466] systemd[1]: Hostname set to <client>.1789lb # [ 1.405658] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1790lb # [ 1.410669] systemd[1]: Detected virtualization kvm.1791lb # [ 1.411581] systemd[1]: Detected architecture x86-64.1792lb # [ 1.412489] systemd[1]: Running in initrd.1793lb # [ 1.413609] systemd[1]: Initializing machine ID from random generator.1794lb # [ 1.414849] systemd[1]: Hostname set to <lb>.1795worker1 # [ 1.608445] systemd[1]: bpf-restrict-fs: LSM BPF program attached1796client # [ 1.617307] systemd[1]: bpf-restrict-fs: LSM BPF program attached1797lb # [ 1.603436] systemd[1]: bpf-restrict-fs: LSM BPF program attached1798worker2 # [ 1.628948] systemd[1]: bpf-restrict-fs: LSM BPF program attached1799worker1 # [ 1.663702] systemd[1]: Queued start job for default target Initrd Default Target.1800client # [ 1.656943] systemd[1]: Queued start job for default target Initrd Default Target.1801client # [ 1.660223] systemd[1]: Created slice Slice /system/modprobe.1802worker1 # [ 1.668424] systemd[1]: Created slice Slice /system/modprobe.1803client # [ 1.661437] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1804worker1 # [ 1.669600] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1805lb # [ 1.641289] systemd[1]: Queued start job for default target Initrd Default Target.1806client # [ 1.662927] systemd[1]: Expecting device /dev/disk/by-label/nixos...1807worker1 # [ 1.670941] systemd[1]: Expecting device /dev/disk/by-label/nixos...1808client # [ 1.663994] systemd[1]: Reached target Path Units.1809worker1 # [ 1.672080] systemd[1]: Reached target Path Units.1810client # [ 1.664892] systemd[1]: Reached target Slice Units.1811worker1 # [ 1.672918] systemd[1]: Reached target Slice Units.1812client # [ 1.665733] systemd[1]: Reached target Swaps.1813worker1 # [ 1.673764] systemd[1]: Reached target Swaps.1814client # [ 1.666538] systemd[1]: Reached target Timer Units.1815lb # [ 1.645667] systemd[1]: Created slice Slice /system/modprobe.1816worker1 # [ 1.674588] systemd[1]: Reached target Timer Units.1817client # [ 1.667520] systemd[1]: Listening on D-Bus System Message Bus Socket.1818lb # [ 1.646831] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1819worker1 # [ 1.675593] systemd[1]: Listening on D-Bus System Message Bus Socket.1820client # [ 1.668720] systemd[1]: Listening on Journal Socket (/dev/log).1821lb # [ 1.648213] systemd[1]: Expecting device /dev/disk/by-label/nixos...1822worker1 # [ 1.676768] systemd[1]: Listening on Journal Socket (/dev/log).1823client # [ 1.669883] systemd[1]: Listening on Journal Sockets.1824lb # [ 1.649264] systemd[1]: Reached target Path Units.1825worker1 # [ 1.677888] systemd[1]: Listening on Journal Sockets.1826client # [ 1.670811] systemd[1]: Listening on udev Control Socket.1827lb # [ 1.650145] systemd[1]: Reached target Slice Units.1828worker1 # [ 1.678895] systemd[1]: Listening on udev Control Socket.1829client # [ 1.671805] systemd[1]: Listening on udev Kernel Socket.1830lb # [ 1.651038] systemd[1]: Reached target Swaps.1831client # [ 1.672711] systemd[1]: Reached target Socket Units.1832worker1 # [ 1.679885] systemd[1]: Listening on udev Kernel Socket.1833lb # [ 1.651846] systemd[1]: Reached target Timer Units.1834worker1 # [ 1.680809] systemd[1]: Reached target Socket Units.1835lb # [ 1.652821] systemd[1]: Listening on D-Bus System Message Bus Socket.1836client # [ 1.674566] systemd[1]: Starting Create List of Static Device Nodes...1837lb # [ 1.654080] systemd[1]: Listening on Journal Socket (/dev/log).1838worker1 # [ 1.682692] systemd[1]: Starting Create List of Static Device Nodes...1839lb # [ 1.655237] systemd[1]: Listening on Journal Sockets.1840lb # [ 1.656246] systemd[1]: Listening on udev Control Socket.1841lb # [ 1.657265] systemd[1]: Listening on udev Kernel Socket.1842client # [ 1.678916] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1843lb # [ 1.658265] systemd[1]: Reached target Socket Units.1844worker1 # [ 1.687111] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1845lb # [ 1.660040] systemd[1]: Starting Create List of Static Device Nodes...1846worker2 # [ 1.688385] systemd[1]: Queued start job for default target Initrd Default Target.1847client # [ 1.685929] systemd[1]: Starting Load Kernel Module configfs...1848worker1 # [ 1.693787] systemd[1]: Starting Load Kernel Module configfs...1849lb # [ 1.665444] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1850worker2 # [ 1.692534] systemd[1]: Created slice Slice /system/modprobe.1851worker2 # [ 1.693713] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1852worker2 # [ 1.695230] systemd[1]: Expecting device /dev/disk/by-label/nixos...1853worker2 # [ 1.696415] systemd[1]: Reached target Path Units.1854worker2 # [ 1.697389] systemd[1]: Reached target Slice Units.1855worker2 # [ 1.698377] systemd[1]: Reached target Swaps.1856worker2 # [ 1.699290] systemd[1]: Reached target Timer Units.1857lb # [ 1.672434] systemd[1]: Starting Load Kernel Module configfs...1858worker2 # [ 1.700361] systemd[1]: Listening on D-Bus System Message Bus Socket.1859worker2 # [ 1.701691] systemd[1]: Listening on Journal Socket (/dev/log).1860worker2 # [ 1.702994] systemd[1]: Listening on Journal Sockets.1861worker2 # [ 1.704123] systemd[1]: Listening on udev Control Socket.1862worker2 # [ 1.705240] systemd[1]: Listening on udev Kernel Socket.1863client # [ 1.700043] systemd[1]: Starting Journal Service...1864worker1 # [ 1.708101] systemd[1]: Starting Journal Service...1865worker2 # [ 1.706411] systemd[1]: Reached target Socket Units.1866worker2 # [ 1.708699] systemd[1]: Starting Create List of Static Device Nodes...1867lb # [ 1.685672] systemd[1]: Starting Journal Service...1868worker2 # [ 1.715147] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1869worker1 # [ 1.720102] systemd[1]: Starting Load Kernel Modules...1870client # [ 1.713924] systemd[1]: Starting Load Kernel Modules...1871worker2 # [ 1.722124] systemd[1]: Starting Load Kernel Module configfs...1872lb # [ 1.699262] systemd[1]: Starting Load Kernel Modules...1873client # [ 1.718928] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1874worker2 # [ 1.738263] systemd[1]: Starting Journal Service...1875worker1 # [ 1.729120] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1876client # [ 1.736902] netfs: FS-Cache loaded1877lb # [ 1.704478] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1878worker1 # [ 1.746431] netfs: FS-Cache loaded1879client # [ 1.741790] 9pnet: Installing 9P2000 support1880lb # [ 1.721812] netfs: FS-Cache loaded1881worker2 # [ 1.750163] systemd[1]: Starting Load Kernel Modules...1882worker1 # [ 1.753736] 9pnet: Installing 9P2000 support1883lb # [ 1.727014] 9pnet: Installing 9P2000 support1884client # [ 1.752924] systemd[1]: Starting Coldplug All udev Devices...1885worker1 # [ 1.766701] systemd[1]: Starting Coldplug All udev Devices...1886lb # [ 1.741655] systemd[1]: Starting Coldplug All udev Devices...1887worker2 # [ 1.757127] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1888worker2 # [ 1.773492] netfs: FS-Cache loaded1889client # [ 1.767986] systemd[1]: Finished Create List of Static Device Nodes.1890worker1 # [ 1.778664] systemd-journald[67]: Collecting audit messages is disabled.1891worker2 # [ 1.778348] 9pnet: Installing 9P2000 support1892worker1 # [ 1.782124] systemd[1]: Finished Create List of Static Device Nodes.1893client # [ 1.775580] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1894lb # [ 1.757566] systemd-journald[67]: Collecting audit messages is disabled.1895lb # [ 1.758812] systemd[1]: Finished Create List of Static Device Nodes.1896worker1 # [ 1.787942] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1897worker2 # [ 1.787250] systemd[1]: Starting Coldplug All udev Devices...1898client # [ 1.783360] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1899lb # [ 1.764097] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1900worker1 # [ 1.795440] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1901client # [ 1.788924] systemd-journald[67]: Collecting audit messages is disabled.1902client # [ 1.790532] systemd[1]: modprobe@configfs.service: Deactivated successfully.1903lb # [ 1.771459] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1904worker1 # [ 1.801460] systemd[1]: modprobe@configfs.service: Deactivated successfully.1905worker2 # [ 1.802129] systemd[1]: Finished Create List of Static Device Nodes.1906client # [ 1.798235] systemd[1]: Finished Load Kernel Module configfs.1907lb # [ 1.777786] systemd[1]: modprobe@configfs.service: Deactivated successfully.1908worker1 # [ 1.807416] systemd[1]: Finished Load Kernel Module configfs.1909worker2 # [ 1.807606] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1910client # [ 1.802307] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1911client # [ 1.804780] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1912worker1 # [ 1.812515] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1913lb # [ 1.785729] systemd[1]: Finished Load Kernel Module configfs.1914worker2 # [ 1.815455] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1915worker1 # [ 1.816461] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1916client # [ 1.812118] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1917lb # [ 1.791832] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1918worker2 # [ 1.820584] systemd-journald[67]: Collecting audit messages is disabled.1919client # [ 1.815932] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1920worker2 # [ 1.822124] systemd[1]: modprobe@configfs.service: Deactivated successfully.1921worker1 # [ 1.824156] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1922worker1 # [ 1.825719] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1923lb # [ 1.796682] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1924worker2 # [ 1.827402] systemd[1]: Finished Load Kernel Module configfs.1925lb # [ 1.802309] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1926worker2 # [ 1.831421] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1927lb # [ 1.806390] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1928worker2 # [ 1.833784] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1929worker2 # [ 1.842240] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1930worker2 # [ 1.845184] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1931client # [ 1.843396] systemd[1]: Finished Load Kernel Modules.1932worker1 # [ 1.854542] systemd[1]: Finished Load Kernel Modules.1933client # [ 1.852044] systemd[1]: Starting Apply Kernel Variables...1934worker1 # [ 1.861208] systemd[1]: Starting Apply Kernel Variables...1935lb # [ 1.837024] systemd[1]: Finished Load Kernel Modules.1936client # [ 1.864940] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1937worker1 # [ 1.872893] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1938lb # [ 1.845629] systemd[1]: Starting Apply Kernel Variables...1939worker2 # [ 1.873569] systemd[1]: Finished Load Kernel Modules.1940client # [ 1.874060] systemd[1]: Starting Create Static Device Nodes in /dev...1941worker2 # [ 1.880280] systemd[1]: Starting Apply Kernel Variables...1942lb # [ 1.855512] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1943worker1 # [ 1.885232] systemd[1]: Starting Create Static Device Nodes in /dev...1944worker2 # [ 1.890175] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1945client # [ 1.887446] systemd[1]: Finished Apply Kernel Variables.1946lb # [ 1.866645] systemd[1]: Starting Create Static Device Nodes in /dev...1947worker1 # [ 1.763297] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.899144] systemd[1]: Finished Apply Kernel Variables.1948worker1 # 1949worker2 # [ 1.900137] systemd[1]: Starting Create Static Device Nodes in /dev...1950worker1 # [ 1.768234] systemd-modules-load[68]: Inserted module 'virtio_balloon'1951worker1 # [ 1.769261] systemd-modules-load[68]: Inserted module 'virtio_gpu'1952worker1 # [ 1.905122] systemd[1]: Started Journal Service.1953lb # [ 1.743063] systemd-modules-load[68]: Inserted module 'dm_mod'1954lb # [ 1.744339] systemd-modules-load[68]: Inserted module 'virtio_balloon'1955lb # [ 1.745501] systemd-modules-load[68]: Inserted module 'virtio_gpu'1956lb # [ 1.882069] systemd[1]: Started Journal Service.1957client # [ 1.769606] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.904508] systemd[1]: Started Journal Service.1958client # 1959client # [ 1.774489] systemd-modules-load[68]: Inserted module 'virtio_balloon'1960client # [ 1.775543] systemd-modules-load[68]: Inserted module 'virtio_gpu'1961worker2 # [ 1.917152] systemd[1]: Finished Apply Kernel Variables.1962lb # [ 1.755578] systemd[1]: Finished Apply Kernel Variables.1963worker1 # [ 1.785675] systemd[1]: Finished Create Static Device Nodes in /dev.1964client # [ 1.780168] systemd[1]: Finished Create Static Device Nodes in /dev.1965worker1 # [ 1.788780] systemd[1]: Reached target Preparation for Local File Systems.1966worker1 # [ 1.789820] systemd[1]: Reached target Local File Systems.1967worker1 # [ 1.791060] systemd[1]: Starting Create System Files and Directories...1968client # [ 1.784960] systemd[1]: Reached target Preparation for Local File Systems.1969client # [ 1.787953] systemd[1]: Reached target Local File Systems.1970worker1 # [ 1.796069] systemd[1]: Starting Rule-based Manager for Device Events and Files...1971client # [ 1.789276] systemd[1]: Starting Create System Files and Directories...1972lb # [ 1.771112] systemd[1]: Finished Create Static Device Nodes in /dev.1973lb # [ 1.772094] systemd[1]: Reached target Preparation for Local File Systems.1974worker2 # [ 1.934570] systemd[1]: Finished Create Static Device Nodes in /dev.1975lb # [ 1.773040] systemd[1]: Reached target Local File Systems.1976client # [ 1.796453] systemd[1]: Starting Rule-based Manager for Device Events and Files...1977worker2 # [ 1.801063] systemd-modules-load[68]: Inserted module 'dm_mod'1978worker2 # [ 1.938233] systemd[1]: Started Journal Service.1979worker2 # [ 1.804346] systemd-modules-load[68]: Inserted module 'virtio_balloon'1980lb # [ 1.778094] systemd[1]: Starting Create System Files and Directories...1981worker2 # [ 1.805456] systemd-modules-load[68]: Inserted module 'virtio_gpu'1982lb # [ 1.784074] systemd[1]: Starting Rule-based Manager for Device Events and Files...1983worker2 # [ 1.815100] systemd[1]: Reached target Preparation for Local File Systems.1984worker2 # [ 1.816718] systemd[1]: Reached target Local File Systems.1985worker1 # [ 1.820521] systemd[1]: Finished Create System Files and Directories.1986worker2 # [ 1.819974] systemd[1]: Starting Create System Files and Directories...1987worker2 # [ 1.825129] systemd[1]: Starting Rule-based Manager for Device Events and Files...1988client # [ 1.821403] systemd[1]: Finished Create System Files and Directories.1989lb # [ 1.809182] systemd[1]: Finished Create System Files and Directories.1990worker1 # [ 1.844361] systemd-udevd[83]: Using default interface naming scheme 'v261'.1991client # [ 1.843409] systemd-udevd[83]: Using default interface naming scheme 'v261'.1992worker2 # [ 1.852443] systemd[1]: Finished Create System Files and Directories.1993lb # [ 1.832338] systemd-udevd[84]: Using default interface naming scheme 'v261'.1994worker1 # [ 1.868840] systemd[1]: Started Rule-based Manager for Device Events and Files.1995worker2 # [ 1.872307] systemd-udevd[82]: Using default interface naming scheme 'v261'.1996client # [ 1.870073] systemd[1]: Started Rule-based Manager for Device Events and Files.1997lb # [ 1.856074] systemd[1]: Started Rule-based Manager for Device Events and Files.1998worker2 # [ 1.895837] systemd[1]: Started Rule-based Manager for Device Events and Files.1999client # [ 1.915084] systemd[1]: Finished Coldplug All udev Devices.2000client # [ 1.916330] systemd[1]: Reached target System Initialization.
loading 1,958 hidden lines…
3959client # [ 16.255976] systemd[1]: Startup finished in 877ms (kernel) + 4.115s (initrd) + 11.262s (userspace) = 16.255s.3960worker2 # [ 16.879936] dhcpcd[692]: eth0: leased 10.0.2.15 for 86400 seconds3961worker2 # [ 16.882630] dhcpcd[692]: eth0: adding route to 10.0.2.0/243962worker2 # [ 16.884968] dhcpcd[692]: eth0: adding default route via 10.0.2.23963worker2 # [ 17.003513] systemd[1]: Started DHCP Client.3964worker2 # [ 17.005564] systemd[1]: Reached target Multi-User System.3965worker2 # [ 17.006461] systemd[1]: Startup finished in 843ms (kernel) + 5.063s (initrd) + 11.098s (userspace) = 17.005s.3966worker1 # [ 17.124902] dhcpcd[694]: eth0: leased 10.0.2.15 for 86400 seconds3967worker1 # [ 17.127574] dhcpcd[694]: eth0: adding route to 10.0.2.0/243968worker1 # [ 17.130202] dhcpcd[694]: eth0: adding default route via 10.0.2.23969worker1 # [ 17.233453] systemd[1]: Started DHCP Client.3970worker1 # [ 17.235297] systemd[1]: Reached target Multi-User System.3971worker1 # [ 17.236108] systemd[1]: Startup finished in 844ms (kernel) + 4.064s (initrd) + 12.325s (userspace) = 17.234s.3972lb # [ 18.031893] dhcpcd[715]: eth0: leased 10.0.2.15 for 86400 seconds3973lb # [ 18.032823] dhcpcd[715]: eth0: adding route to 10.0.2.0/243974lb # [ 18.033553] dhcpcd[715]: eth0: adding default route via 10.0.2.23975lb # [ 18.129684] systemd[1]: Started DHCP Client.3976lb # [ 18.131510] systemd[1]: Reached target Network is Online.3977lb # [ 18.135711] systemd[1]: Started Envoy reverse proxy.3978lb # [ 18.140613] systemd[1]: Started mock-oidc.service.3979lb # [ 18.232811] mock-oidc-server[929]: Mock OIDC Server running3980lb # [ 18.233872] mock-oidc-server[929]: OIDC Address: 192.168.1.2:80803981lb # [ 18.235095] mock-oidc-server[929]: Issue Address: 0.0.0.0:80813982lb # [ 18.236206] mock-oidc-server[929]: Issuer: http://192.168.1.2:8080/oidc3983lb # [ 18.237272] mock-oidc-server[929]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3984lb # [ 18.238395] mock-oidc-server[929]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3985lb # [ 18.239902] mock-oidc-server[929]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3986lb # [ 18.669662] envoy[928]: [2026-09-20 09:33:30.704][928][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.104)3987lb # [ 18.671468] envoy[928]: [2026-09-20 09:33:30.704][928][info][main] [source/server/server.cc:469] statically linked extensions:3988lb # [ 18.672841] envoy[928]: [2026-09-20 09:33:30.704][928][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3989lb # [ 18.675187] envoy[928]: [2026-09-20 09:33:30.704][928][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13990lb # [ 18.677373] envoy[928]: [2026-09-20 09:33:30.704][928][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3991lb # [ 18.691477] envoy[928]: [2026-09-20 09:33:30.704][928][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3992lb # [ 18.694132] envoy[928]: [2026-09-20 09:33:30.704][928][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3993lb # [ 18.696554] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3994lb # [ 18.700390] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3995lb # [ 18.704528] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3996lb # [ 18.706214] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3997lb # [ 18.707951] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3998lb # [ 18.710123] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3999lb # [ 18.711849] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche4000lb # [ 18.713799] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts4001lb # [ 18.716591] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator4002lb # [ 18.719369] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value4003lb # [ 18.722489] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain4004lb # [ 18.724166] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed4005lb # [ 18.726258] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file4006lb # [ 18.727951] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json4007lb # [ 18.729473] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction4008lb # [ 18.733479] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based4009lb # [ 18.737127] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name4010lb # [ 18.739967] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls4011lb # [ 18.744189] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default4012lb # [ 18.746070] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config4013lb # [ 18.749262] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe4014lb # [ 18.751716] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff4015lb # [ 18.753968] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua4016lb # [ 18.755724] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic4017lb # [ 18.758085] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest4018lb # [ 18.762739] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response4019lb # [ 18.764691] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd4020lb # [ 18.768968] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr4021lb # [ 18.770593] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter4022lb # [ 18.772369] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override4023lb # [ 18.775114] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector4024lb # [ 18.780466] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default4025lb # [ 18.781935] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory4026lb # [ 18.784619] envoy[928]: [2026-09-20 09:33:30.710][928][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule4027lb # [ 18.787304] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats4028lb # [ 18.789684] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router4029lb # [ 18.815045] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router4030lb # [ 18.816707] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso4031lb # [ 18.818769] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor4032lb # [ 18.821498] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy4033lb # [ 18.832417] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation4034lb # [ 18.834643] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme4035lb # [ 18.837528] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service4036lb # [ 18.841673] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert4037lb # [ 18.843467] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth24038lb # [ 18.845694] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable4039lb # [ 18.847652] envoy[928]: [2026-09-20 09:33:30.712][928][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection4040lb # [ 18.849574] envoy[928]: [2026-09-20 09:33:30.718][928][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override4041lb # [ 18.852390] envoy[928]: [2026-09-20 09:33:30.718][928][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple4042lb # [ 18.854532] envoy[928]: [2026-09-20 09:33:30.718][928][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit4043lb # [ 18.857072] envoy[928]: [2026-09-20 09:33:30.718][928][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid4044lb # [ 18.858572] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router4045lb # [ 18.861357] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default4046lb # [ 18.863506] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re24047lb # [ 18.865168] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal4048lb # [ 18.867056] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4049lb # [ 18.869200] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory4050lb # [ 18.871700] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian24051lb # [ 18.873427] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple4052lb # [ 18.875869] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v84053lb # [ 18.877760] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query4054lb # [ 18.881428] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4055lb # [ 18.883730] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm4056lb # [ 18.889437] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed4057lb # [ 18.891059] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo4058lb # [ 18.892406] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy4059lb # [ 18.894691] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san4060lb # [ 18.900446] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case4061lb # [ 18.902662] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default4062lb # [ 18.904904] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name4063lb # [ 18.907483] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated4064lb # [ 18.909354] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls4065lb # [ 18.914436] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log4066lb # [ 18.920690] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router4067lb # [ 18.922481] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action4068lb # [ 18.924522] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder4069lb # [ 18.926662] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret4070lb # [ 18.928912] envoy[928]: [2026-09-20 09:33:30.719][928][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift4071lb # [ 18.931871] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params4072lb # [ 18.941122] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher4073lb # [ 18.943660] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle4074lb # [ 18.946097] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based4075lb # [ 18.947612] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator4076lb # [ 18.949701] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default4077lb # [ 18.951472] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy4078lb # [ 18.953725] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection4079lb # [ 18.958321] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default4080lb # [ 18.960076] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input4081lb # [ 18.962506] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state4082lb # [ 18.965171] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port4083lb # [ 18.972202] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource4084lb # [ 18.975215] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind4085lb # [ 18.977512] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher4086lb # [ 18.979849] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin4087lb # [ 18.983400] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher4088lb # [ 18.985170] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header4089lb # [ 18.987423] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default4090lb # [ 18.989087] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input4091lb # [ 18.990994] envoy[928]: [2026-09-20 09:33:30.720][928][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter4092lb # [ 18.992902] envoy[928]: [2026-09-20 09:33:30.867][928][info][main] [source/server/server.cc:531] HTTP header map info:4093lb # [ 18.994599] envoy[928]: [2026-09-20 09:33:30.906][928][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id4094lb # [ 19.008127] envoy[928]: [2026-09-20 09:33:30.906][928][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:4095lb # [ 19.009527] envoy[928]: [2026-09-20 09:33:30.906][928][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id4096lb # [ 19.016831] envoy[928]: [2026-09-20 09:33:30.906][928][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status4097lb # [ 19.058388] envoy[928]: [2026-09-20 09:33:31.093][928][info][main] [source/server/server.cc:968] runtime: {}4098lb # [ 19.065970] envoy[928]: [2026-09-20 09:33:31.101][928][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.4099lb # [ 19.068063] envoy[928]: [2026-09-20 09:33:31.103][928][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99014100lb # [ 19.070706] envoy[928]: [2026-09-20 09:33:31.105][928][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99014101lb # [ 19.074414] envoy[928]: [2026-09-20 09:33:31.109][928][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration4102lb # [ 19.076248] envoy[928]: [2026-09-20 09:33:31.111][928][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)4103lb # [ 19.077742] envoy[928]: [2026-09-20 09:33:31.111][928][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)4104lb # [ 19.124134] envoy[928]: [2026-09-20 09:33:31.159][928][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)4105worker1 # [ 19.158767] systemd[1]: Starting Nix worker-protocol over gRPC...4106worker2 # [ 19.159844] systemd[1]: Starting Nix worker-protocol over gRPC...4107lb # [ 19.143222] envoy[928]: [2026-09-20 09:33:31.178][928][info][config] [source/server/configuration_impl.cc:164] loading stats configuration4108lb # [ 19.145602] envoy[928]: [2026-09-20 09:33:31.180][928][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.4109lb # [ 19.148734] envoy[928]: [2026-09-20 09:33:31.180][928][info][main] [source/server/server.cc:1077] starting main dispatch loop4110worker2 # [ 19.349690] nix-grpc-daemon[785]: ts=2026-09-20T09:33:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4111worker1 # [ 19.361449] nix-grpc-daemon[786]: ts=2026-09-20T09:33:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4112worker2 # [ 19.360404] nix-grpc-daemon[785]: ts=2026-09-20T09:33:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14113worker1 # [ 19.373678] nix-grpc-daemon[786]: ts=2026-09-20T09:33:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14114worker2 # [ 19.397302] nix-grpc-daemon[785]: ts=2026-09-20T09:33:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4115worker2 # [ 19.400519] systemd[1]: Started Nix worker-protocol over gRPC.4116worker1 # [ 19.414751] nix-grpc-daemon[786]: ts=2026-09-20T09:33:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4117worker1 # [ 19.417853] systemd[1]: Started Nix worker-protocol over gRPC.4118lb # [ 19.422180] envoy[928]: [2026-09-20 09:33:31.457][928][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization4119lb # [ 19.423966] envoy[928]: [2026-09-20 09:33:31.459][928][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized4120lb # [ 19.425639] envoy[928]: [2026-09-20 09:33:31.459][928][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager4121lb # [ 19.427545] envoy[928]: [2026-09-20 09:33:31.460][928][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers4122worker2 # [ 19.470851] nix-grpc-daemon[785]: ts=2026-09-20T09:33:31Z level=info event=scheduler_connected addr=lb:500514123worker2 # [ 19.479769] nix-grpc-daemon[785]: ts=2026-09-20T09:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4124worker1 # [ 19.491129] nix-grpc-daemon[786]: ts=2026-09-20T09:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4125worker2 # [ 19.680771] nix-grpc-daemon[785]: ts=2026-09-20T09:33:31Z level=info event=scheduler_connected addr=lb:500514126worker2 # [ 19.684847] nix-grpc-daemon[785]: ts=2026-09-20T09:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4127worker1 # [ 19.692398] nix-grpc-daemon[786]: ts=2026-09-20T09:33:31Z level=info event=scheduler_connected addr=lb:500514128worker1 # [ 19.696188] nix-grpc-daemon[786]: ts=2026-09-20T09:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4129worker2 # [ 19.885633] nix-grpc-daemon[785]: ts=2026-09-20T09:33:31Z level=info event=scheduler_connected addr=lb:500514130worker2 # [ 19.889646] nix-grpc-daemon[785]: ts=2026-09-20T09:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4131worker1 # [ 19.893998] nix-grpc-daemon[786]: ts=2026-09-20T09:33:31Z level=info event=scheduler_connected addr=lb:500514132worker1 # [ 19.897856] nix-grpc-daemon[786]: ts=2026-09-20T09:33:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4133worker2 # [ 20.090560] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514134worker2 # [ 20.094561] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4135worker1 # [ 20.098652] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514136worker1 # [ 20.101262] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4137worker2 # [ 20.295467] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514138worker1 # [ 20.301826] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514139worker1 # [ 20.303601] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4140worker2 # [ 20.299414] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4141worker2 # [ 20.501493] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514142worker1 # [ 20.505826] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514143worker2 # [ 20.504768] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4144worker1 # [ 20.508984] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 20.702750] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514146worker1 # [ 20.706352] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514147worker2 # [ 20.706283] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4148worker1 # [ 20.711401] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4149worker2 # [ 20.907230] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514150worker1 # [ 20.912436] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_connected addr=lb:500514151worker2 # [ 20.911260] nix-grpc-daemon[785]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4152worker1 # [ 20.916248] nix-grpc-daemon[786]: ts=2026-09-20T09:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4153worker2 # [ 21.112102] nix-grpc-daemon[785]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500514154worker1 # [ 21.116608] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500514155worker1 # [ 21.118470] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4156worker2 # [ 21.116127] nix-grpc-daemon[785]: ts=2026-09-20T09:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4157worker1 # [ 21.319025] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500514158worker1 # [ 21.320848] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4159worker2 # [ 21.316874] nix-grpc-daemon[785]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500514160worker2 # [ 21.320799] nix-grpc-daemon[785]: ts=2026-09-20T09:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4161worker1 # [ 21.521640] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500514162worker1 # [ 21.523294] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4163worker2 # [ 21.522642] nix-grpc-daemon[785]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500514164worker2 # [ 21.525719] nix-grpc-daemon[785]: ts=2026-09-20T09:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4165worker1 # [ 21.724396] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500514166worker2 # [ 21.723431] nix-grpc-daemon[785]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500514167worker1 # [ 21.728281] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4168worker2 # [ 21.727329] nix-grpc-daemon[785]: ts=2026-09-20T09:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4169worker1 # [ 21.928974] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_connected addr=lb:500514170worker2 # [ 21.928347] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514171worker1 # [ 21.933136] nix-grpc-daemon[786]: ts=2026-09-20T09:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4172worker2 # [ 21.932317] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4173worker1 # [ 22.134335] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514174worker2 # [ 22.133085] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514175worker1 # [ 22.138431] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4176worker2 # [ 22.137220] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker2 # [ 22.337555] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 22.338929] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514179worker2 # [ 22.339405] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4180worker1 # [ 22.343150] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker2 # [ 22.540245] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 22.542068] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4183worker1 # [ 22.545197] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514184worker1 # [ 22.548337] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185worker2 # [ 22.742480] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514186worker1 # [ 22.745609] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514187worker2 # [ 22.744238] nix-grpc-daemon[785]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4188worker1 # [ 22.747386] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4189worker2 # [ 22.945227] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514190worker1 # [ 22.948504] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_connected addr=lb:500514191worker2 # [ 22.949189] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4192worker1 # [ 22.952729] nix-grpc-daemon[786]: ts=2026-09-20T09:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4193worker2 # [ 23.150094] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514194worker1 # [ 23.153433] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514195worker2 # [ 23.154351] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4196worker1 # [ 23.157318] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4197worker2 # [ 23.354938] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514198worker1 # [ 23.358187] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514199worker2 # [ 23.358975] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4200worker1 # [ 23.362200] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4201worker1 # [ 23.563709] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514202worker2 # [ 23.560920] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514203worker1 # [ 23.567538] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4204worker2 # [ 23.565294] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4205worker1 # [ 23.768323] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514206worker2 # [ 23.766361] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514207worker1 # [ 23.772363] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4208worker2 # [ 23.770434] nix-grpc-daemon[785]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4209worker2 # [ 23.970883] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514210worker1 # [ 23.973155] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_connected addr=lb:500514211worker2 # [ 23.972594] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4212worker1 # [ 23.977336] nix-grpc-daemon[786]: ts=2026-09-20T09:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4213worker2 # [ 24.173593] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514214worker1 # [ 24.178350] nix-grpc-daemon[786]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514215worker2 # [ 24.177601] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4216worker1 # [ 24.182133] nix-grpc-daemon[786]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4217worker2 # [ 24.378422] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514218worker1 # [ 24.382743] nix-grpc-daemon[786]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514219worker2 # [ 24.382443] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4220worker1 # [ 24.386709] nix-grpc-daemon[786]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4221worker2 # [ 24.584547] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514222worker1 # [ 24.588814] nix-grpc-daemon[786]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514223worker2 # [ 24.587740] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4224worker1 # [ 24.592152] nix-grpc-daemon[786]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4225worker2 # [ 24.785286] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514226worker1 # [ 24.789358] nix-grpc-daemon[786]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514227worker2 # [ 24.789279] nix-grpc-daemon[785]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4228worker1 # [ 24.793181] nix-grpc-daemon[786]: ts=2026-09-20T09:33:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4229worker1 # [ 24.991082] nix-grpc-daemon[786]: ts=2026-09-20T09:33:36Z level=info event=scheduler_connected addr=lb:500514230worker2 # [ 24.989878] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500514231worker1 # [ 24.995330] nix-grpc-daemon[786]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4232worker2 # [ 24.993829] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4233worker1 # [ 25.196166] nix-grpc-daemon[786]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500514234worker2 # [ 25.194627] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500514235worker1 # [ 25.200233] nix-grpc-daemon[786]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4236worker2 # [ 25.198457] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4237worker1 # [ 25.401124] nix-grpc-daemon[786]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500514238worker2 # [ 25.399293] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500514239worker1 # [ 25.405304] nix-grpc-daemon[786]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4240worker2 # [ 25.403379] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4241worker1 # [ 25.606664] nix-grpc-daemon[786]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500514242worker2 # [ 25.604750] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500514243worker1 # [ 25.610677] nix-grpc-daemon[786]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4244worker2 # [ 25.608566] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4245worker1 # [ 25.811665] nix-grpc-daemon[786]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500514246worker2 # [ 25.809419] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_connected addr=lb:500514247worker2 # [ 25.813402] nix-grpc-daemon[785]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4248worker1 # [ 25.815893] nix-grpc-daemon[786]: ts=2026-09-20T09:33:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4249worker2 # [ 26.013881] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514250worker1 # [ 26.016710] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514251worker2 # [ 26.017113] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4252worker1 # [ 26.020582] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4253worker1 # [ 26.221383] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514254worker2 # [ 26.218075] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514255worker1 # [ 26.224838] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4256worker2 # [ 26.221992] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4257worker1 # [ 26.424887] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514258worker1 # [ 26.426565] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4259worker2 # [ 26.422552] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514260worker2 # [ 26.426393] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4261worker1 # [ 26.628557] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514262worker2 # [ 26.628458] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514263worker1 # [ 26.632286] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4264worker2 # [ 26.631524] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4265worker1 # [ 26.830290] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514266worker2 # [ 26.828990] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_connected addr=lb:500514267worker1 # [ 26.834340] nix-grpc-daemon[786]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4268worker2 # [ 26.832995] nix-grpc-daemon[785]: ts=2026-09-20T09:33:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4269worker1 # [ 27.035359] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514270worker2 # [ 27.034162] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514271worker1 # [ 27.039430] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4272worker2 # [ 27.038277] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4273worker1 # [ 27.240257] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514274worker2 # [ 27.239459] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514275worker1 # [ 27.244181] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4276worker2 # [ 27.243401] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4277worker1 # [ 27.444942] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514278worker2 # [ 27.444263] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514279worker1 # [ 27.448864] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4280worker2 # [ 27.448089] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4281worker1 # [ 27.650490] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514282worker2 # [ 27.649802] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514283worker1 # [ 27.654215] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4284worker2 # [ 27.653547] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4285worker1 # [ 27.855149] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514286worker2 # [ 27.854488] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_connected addr=lb:500514287worker1 # [ 27.859381] nix-grpc-daemon[786]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4288worker2 # [ 27.858421] nix-grpc-daemon[785]: ts=2026-09-20T09:33:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4289worker1 # [ 28.060257] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514290worker2 # [ 28.056384] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514291worker1 # [ 28.064137] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4292worker2 # [ 28.061820] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4293worker1 # [ 28.265156] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514294worker2 # [ 28.262547] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514295worker1 # [ 28.269094] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4296worker2 # [ 28.266680] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4297worker1 # [ 28.469863] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514298worker2 # [ 28.467727] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514299worker1 # [ 28.473864] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4300worker2 # [ 28.471733] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4301worker1 # [ 28.675689] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514302worker2 # [ 28.673434] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514303worker1 # [ 28.679501] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4304worker2 # [ 28.677280] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4305worker1 # [ 28.879942] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514306worker1 # [ 28.882183] nix-grpc-daemon[786]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4307worker2 # [ 28.878069] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_connected addr=lb:500514308worker2 # [ 28.881940] nix-grpc-daemon[785]: ts=2026-09-20T09:33:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4309worker1 # [ 29.082934] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514310worker1 # [ 29.084781] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4311worker2 # [ 29.082878] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514312worker2 # [ 29.086669] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4313lb # [ 29.152766] rustfs-bucket-start[1038]: mb s3://farm4314lb # [ 29.157745] systemd[1]: Finished rustfs-bucket.service.4315lb # [ 29.164192] systemd[1]: Starting niks3 server...4316worker1 # [ 29.285288] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514317worker1 # [ 29.287228] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4318worker2 # [ 29.287578] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514319worker2 # [ 29.291401] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4320lb # [ 29.384335] postgres[1053]: [1053] ERROR: relation "goose_db_version" does not exist at character 364321lb # [ 29.385789] postgres[1053]: [1053] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4322lb # [ 29.422402] niks3-server[1047]: 2026/09/20 09:33:41 OK 20241026095416_initial_model.sql (20.46ms)4323lb # [ 29.425813] niks3-server[1047]: 2026/09/20 09:33:41 OK 20251210153512_drop_unused_gin_index.sql (3.42ms)4324lb # [ 29.430880] niks3-server[1047]: 2026/09/20 09:33:41 OK 20251218171726_add_pins.sql (5.04ms)4325worker1 # [ 29.487598] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514326worker1 # [ 29.489829] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4327worker2 # [ 29.491947] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514328worker2 # [ 29.495645] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4329lb # [ 29.471635] niks3-server[1047]: 2026/09/20 09:33:41 OK 20260628120000_add_object_size_and_stats.sql (40.71ms)4330lb # [ 29.476726] niks3-server[1047]: 2026/09/20 09:33:41 OK 20260905000000_add_claims.sql (3.42ms)4331lb # [ 29.481694] niks3-server[1047]: 2026/09/20 09:33:41 OK 20260920000000_drop_claims.sql (4.89ms)4332lb # [ 29.483088] niks3-server[1047]: 2026/09/20 09:33:41 goose: successfully migrated database to version: 202609200000004333lb # [ 29.488904] niks3-server[1047]: 2026/09/20 09:33:41 OK 1_commit_pending_closure.sql (7.17ms)4334lb # [ 29.491639] niks3-server[1047]: 2026/09/20 09:33:41 OK 2_object_stats_trigger.sql (2.73ms)4335lb # [ 29.492804] niks3-server[1047]: 2026/09/20 09:33:41 goose: up to current file version: 24336lb # [ 29.499189] niks3-server[1047]: 2026/09/20 09:33:41 INFO Loaded signing key name=farm-test-1 path=/nix/store/c745bsy529madd2s6hz4z5is2436ni04-key4337lb # [ 29.575487] niks3-server[1047]: 2026/09/20 09:33:41 INFO Read proxy enabled — serving cache objects from S34338lb # [ 29.577718] niks3-server[1047]: 2026/09/20 09:33:41 INFO Using socket-activated listener address=0.0.0.0:57514339lb # [ 29.581636] systemd[1]: Started niks3 server.4340lb # [ 29.583702] systemd[1]: Reached target Multi-User System.4341lb # [ 29.585161] systemd[1]: Startup finished in 869ms (kernel) + 4.983s (initrd) + 23.730s (userspace) = 29.584s.4342lb # [ 29.587617] niks3-server[1047]: 2026/09/20 09:33:41 INFO Starting HTTP server address=0.0.0.0:57514343lb # [ 29.594603] niks3-server[1047]: 2026/09/20 09:33:41 INFO systemd watchdog enabled interval=15s4344lb # [ 29.595820] niks3-server[1047]: 2026/09/20 09:33:41 INFO lead: acquired remote=192.168.1.4:505164345worker2 # [ 29.630495] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_take_over4346worker1 # [ 29.690714] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514347worker1 # [ 29.692404] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4348worker2 # [ 29.694826] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514349worker2 # [ 29.697924] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4350worker1 # [ 29.892998] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514351worker1 # [ 29.894878] nix-grpc-daemon[786]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4352worker2 # [ 29.895623] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_connected addr=lb:500514353worker2 # [ 29.899505] nix-grpc-daemon[785]: ts=2026-09-20T09:33:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4354lb: (finished: waiting for unit niks3.service, in 18.82 seconds)4355lb: waiting for TCP port 5751 on localhost4356lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4357lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds)4358lb: waiting for unit envoy.service4359worker1 # [ 30.095777] nix-grpc-daemon[786]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500514360worker1 # [ 30.097604] nix-grpc-daemon[786]: ts=2026-09-20T09:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4361worker2 # [ 30.099920] nix-grpc-daemon[785]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500514362worker2 # [ 30.101878] nix-grpc-daemon[785]: ts=2026-09-20T09:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4363lb: (finished: waiting for unit envoy.service, in 0.06 seconds)4364lb: waiting for TCP port 50051 on localhost4365lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4366lb # [ 30.134879] envoy[928]: conn peer=[::1]:57946 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=34367lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)4368lb: waiting until exactly no endpoints of x86_64-linux are unhealthy4369lb: must succeed: curl -sf localhost:9901/clusters4370worker1 # [ 30.298711] nix-grpc-daemon[786]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500514371worker1 # [ 30.302573] nix-grpc-daemon[786]: ts=2026-09-20T09:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4372worker2 # [ 30.302378] nix-grpc-daemon[785]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500514373worker2 # [ 30.304134] nix-grpc-daemon[785]: ts=2026-09-20T09:33:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4374lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4375lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.09 seconds)4376subtest: one node schedules and both builders hold a WorkerSession on it4377lb: must succeed: curl -sf localhost:9901/clusters4378lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4379worker1 # [ 30.503228] nix-grpc-daemon[786]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500514380worker2 # [ 30.504663] nix-grpc-daemon[785]: ts=2026-09-20T09:33:42Z level=info event=scheduler_connected addr=lb:500514381worker2 # [ 30.515851] nix-grpc-daemon[785]: ts=2026-09-20T09:33:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04382worker2 # [ 30.518409] nix-grpc-daemon[785]: ts=2026-09-20T09:33:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04383lb: must succeed: curl -sf localhost:9901/clusters4384lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4385worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4386worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.09 seconds)4387(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 1.19 seconds)4388subtest: balancer auth: client cert, bearer token, nothing, unknown CN4389worker1 # [ 31.899808] nix-grpc-daemon[786]: ts=2026-09-20T09:33:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466524390worker1 # [ 31.911829] systemd[1]: Started Nix Daemon.4391worker1 # [ 32.118217] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4392worker1 # [ 32.165500] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5)4393worker1 # [ 32.180870] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 155 ms (attempt 2/5)4394worker1 # [ 32.338636] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 3/5)4395worker1 # [ 32.496685] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 707 ms (attempt 4/5)4396worker1 # [ 33.215469] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4397worker2 # [ 33.251993] nix-grpc-daemon[785]: ts=2026-09-20T09:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:368244398worker2 # [ 33.259590] systemd[1]: Started Nix Daemon.4399worker2 # [ 33.436097] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4400worker2 # [ 33.469987] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5)4401worker2 # [ 33.475729] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 2/5)4402worker2 # [ 33.542347] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 183 ms (attempt 3/5)4403worker2 # [ 33.728242] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 609 ms (attempt 4/5)4404worker2 # [ 34.344966] nix-grpc-daemon[785]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4405worker2 # [ 34.363761] nix-grpc-daemon[785]: ts=2026-09-20T09:33:46Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4406worker2 # [ 34.373843] nix-grpc-daemon[785]: ts=2026-09-20T09:33:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:368244407worker2 # [ 34.382664] nix-grpc-daemon[785]: ts=2026-09-20T09:33:46Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4408client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4409client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)4410client: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4411client: (finished: must succeed: install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)4412worker2 # [ 34.773168] nix-grpc-daemon[785]: ts=2026-09-20T09:33:46Z level=info event=oidc_jwks provider=mock keys=14413worker2 # [ 34.778481] nix-grpc-daemon[785]: ts=2026-09-20T09:33:46Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:368244414worker1 # [ 34.814322] nix-grpc-daemon[786]: ts=2026-09-20T09:33:46Z level=info event=oidc_jwks provider=mock keys=14415worker1 # [ 34.817870] nix-grpc-daemon[786]: ts=2026-09-20T09:33:46Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:466524416worker1 # [ 34.835381] nix-grpc-daemon[786]: ts=2026-09-20T09:33:46Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4417worker1 # [ 34.844530] nix-grpc-daemon[786]: ts=2026-09-20T09:33:46Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:466524418worker1 # [ 34.853584] nix-grpc-daemon[786]: ts=2026-09-20T09:33:46Z level=debug event=path_info_miss path=akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test error="error: path '/nix/store/akn52nnm33csynnyhwrqayy7c3050vjf-nixos-system-client-test' is required, but there is no substituter that can build it"4419client: must succeed: rm -r /var/lib/nix-grpc-store4420client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds)4421worker1 # [ 35.105607] nix-grpc-daemon[786]: ts=2026-09-20T09:33:47Z level=info event=denied method=QueryMissing cn=- role=none4422worker2 # [ 35.328259] nix-grpc-daemon[785]: ts=2026-09-20T09:33:47Z level=info event=denied method=QueryMissing cn=stranger role=none4423(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.80 seconds)4424subtest: DAG build is scheduled across workers and lands in the cache4425client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24426worker1 # [ 35.611916] nix-grpc-daemon[786]: ts=2026-09-20T09:33:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466524427client # don't know how to build these paths:4428client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4429worker1 # [ 35.624968] nix-grpc-daemon[786]: ts=2026-09-20T09:33:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524430worker2 # [ 35.655345] nix-grpc-daemon[785]: ts=2026-09-20T09:33:47Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04431worker2 # [ 35.659450] nix-grpc-daemon[785]: ts=2026-09-20T09:33:47Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14432worker1 # [ 35.709665] nix-grpc-daemon[786]: ts=2026-09-20T09:33:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524433worker1 # [ 35.732223] nix-grpc-daemon[786]: ts=2026-09-20T09:33:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524434client # copying 1 paths to 'lb:50051'...4435worker1 # [ 35.747294] nix-grpc-daemon[786]: ts=2026-09-20T09:33:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466524436client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4437worker1 # [ 35.776757] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4438worker1 # [ 35.812193] nix-grpc-daemon[786]: ts=2026-09-20T09:33:47Z level=info event=niks3_push_started pid=8544439worker1 # [ 36.254606] nix-daemon[844]: accepted connection from pid 867, user nix-grpc-daemon (trusted)4440worker1 # [ 36.272507] nix-daemon[844]: reaped child process 870, status = succeeded4441worker1 # [ 36.428039] nix-daemon[844]: accepted connection from pid 872, user nix-grpc-daemon (trusted)4442worker1 # [ 36.439754] nix-daemon[844]: reaped child process 875, status = succeeded4443lb # [ 36.425317] niks3-server[1047]: 2026/09/20 09:33:48 INFO Received uploads request method=POST path=/api/pending_closures4444worker1 # [ 36.482126] nix-grpc-daemon[854]: time=2026-09-20T09:33:48.488Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4445worker1 # [ 36.484837] nix-grpc-daemon[854]: time=2026-09-20T09:33:48.491Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4446lb # [ 36.495105] niks3-server[1047]: 2026/09/20 09:33:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4447lb # [ 36.503502] niks3-server[1047]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4448lb # [ 36.513323] niks3-server[1047]: 2026/09/20 09:33:48 INFO Signed narinfos id=1 count=14449worker1 # [ 36.544783] nix-grpc-daemon[854]: time=2026-09-20T09:33:48.551Z level=INFO msg="Uploading 1 narinfos"4450lb # [ 36.523556] niks3-server[1047]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4451lb # [ 36.526657] niks3-server[1047]: 2026/09/20 09:33:48 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4452lb # [ 36.532047] niks3-server[1047]: 2026/09/20 09:33:48 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4453lb # [ 36.540637] niks3-server[1047]: 2026/09/20 09:33:48 INFO Completed upload id=14454worker1 # [ 36.570711] nix-grpc-daemon[854]: time=2026-09-20T09:33:48.577Z level=INFO msg="Upload complete. (488ms)"4455worker1 # [ 36.573362] nix-grpc-daemon[786]: ts=2026-09-20T09:33:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_in=5044456worker1 # [ 36.577986] nix-grpc-daemon[786]: ts=2026-09-20T09:33:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524457worker1 # [ 36.585779] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4458worker1 # [ 36.590249] nix-daemon[844]: reaped child process 852, status = succeeded4459client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4460worker1 # [ 36.595874] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4461client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4462client # farm-a> LOG-farm-a-t14463worker1 # [ 38.022502] nix-daemon[844]: reaped child process 879, status = succeeded4464worker1 # [ 38.178368] nix-daemon[844]: accepted connection from pid 894, user nix-grpc-daemon (trusted)4465worker1 # [ 38.189408] nix-daemon[844]: reaped child process 897, status = succeeded4466lb # [ 38.176664] niks3-server[1047]: 2026/09/20 09:33:50 INFO Received uploads request method=POST path=/api/pending_closures4467worker1 # [ 38.217262] nix-grpc-daemon[854]: time=2026-09-20T09:33:50.224Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4468worker1 # [ 38.218830] nix-grpc-daemon[854]: time=2026-09-20T09:33:50.225Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4469lb # [ 38.224200] niks3-server[1047]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4470lb # [ 38.234793] niks3-server[1047]: 2026/09/20 09:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4471lb # [ 38.241132] niks3-server[1047]: 2026/09/20 09:33:50 INFO Signed narinfos id=2 count=14472worker1 # [ 38.274641] nix-grpc-daemon[854]: time=2026-09-20T09:33:50.281Z level=INFO msg="Uploading 1 narinfos"4473lb # [ 38.250095] niks3-server[1047]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4474lb # [ 38.255989] niks3-server[1047]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4475lb # [ 38.264415] niks3-server[1047]: 2026/09/20 09:33:50 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4476lb # [ 38.269129] niks3-server[1047]: 2026/09/20 09:33:50 INFO Completed upload id=24477lb # [ 38.271064] niks3-server[1047]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4478worker1 # [ 38.300482] nix-grpc-daemon[854]: time=2026-09-20T09:33:50.307Z level=INFO msg="Upload complete. (293ms)"4479worker1 # [ 38.307551] nix-grpc-daemon[786]: ts=2026-09-20T09:33:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14480worker1 # [ 38.320094] nix-daemon[844]: reaped child process 877, status = succeeded4481worker2 # [ 38.322792] nix-grpc-daemon[785]: ts=2026-09-20T09:33:50Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04482worker2 # [ 38.327074] nix-grpc-daemon[785]: ts=2026-09-20T09:33:50Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14483worker1 # [ 38.369946] nix-grpc-daemon[786]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524484worker1 # [ 38.384178] nix-grpc-daemon[786]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524485worker1 # [ 38.400624] nix-grpc-daemon[854]: time=2026-09-20T09:33:50.407Z level=INFO msg="All 1 paths already cached"4486client # copying 1 paths to 'lb:50051'...4487client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4488worker1 # [ 38.408526] nix-grpc-daemon[786]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466524489worker1 # [ 38.420290] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4490worker1 # [ 38.603124] nix-daemon[844]: accepted connection from pid 902, user nix-grpc-daemon (trusted)4491worker1 # [ 38.617089] nix-daemon[844]: reaped child process 905, status = succeeded4492worker1 # [ 38.786907] nix-daemon[844]: accepted connection from pid 907, user nix-grpc-daemon (trusted)4493worker1 # [ 38.798832] nix-daemon[844]: reaped child process 910, status = succeeded4494lb # [ 38.784674] niks3-server[1047]: 2026/09/20 09:33:50 INFO Received uploads request method=POST path=/api/pending_closures4495worker1 # [ 38.821464] nix-grpc-daemon[854]: time=2026-09-20T09:33:50.828Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4496worker1 # [ 38.823181] nix-grpc-daemon[854]: time=2026-09-20T09:33:50.830Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4497lb # [ 38.807382] niks3-server[1047]: 2026/09/20 09:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4498lb # [ 38.810780] niks3-server[1047]: 2026/09/20 09:33:50 INFO Signed narinfos id=3 count=14499worker1 # [ 38.840803] nix-grpc-daemon[854]: time=2026-09-20T09:33:50.848Z level=INFO msg="Uploading 1 narinfos"4500lb # [ 38.812173] niks3-server[1047]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4501lb # [ 38.814900] niks3-server[1047]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4502lb # [ 38.819728] niks3-server[1047]: 2026/09/20 09:33:50 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4503lb # [ 38.825285] niks3-server[1047]: 2026/09/20 09:33:50 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4504lb # [ 38.828196] niks3-server[1047]: 2026/09/20 09:33:50 INFO Completed upload id=34505worker1 # [ 38.858143] nix-grpc-daemon[854]: time=2026-09-20T09:33:50.865Z level=INFO msg="Upload complete. (422ms)"4506worker1 # [ 38.860246] nix-grpc-daemon[786]: ts=2026-09-20T09:33:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_in=6244507worker1 # [ 38.863234] nix-grpc-daemon[786]: ts=2026-09-20T09:33:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524508worker1 # [ 38.868551] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4509worker1 # [ 38.872663] nix-daemon[844]: reaped child process 899, status = succeeded4510worker1 # [ 38.877921] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4511client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4512client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4513lb # [ 39.132767] envoy[928]: conn peer=[2001:db8:1::1]:36696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=25444514lb # [ 39.136421] envoy[928]: conn peer=[2001:db8:1::1]:55856 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=1034515lb # [ 39.139804] envoy[928]: conn peer=[2001:db8:1::1]:55860 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134516lb # [ 39.143133] envoy[928]: conn peer=[2001:db8:1::1]:55876 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134517worker1 # [ 40.289962] nix-daemon[844]: reaped child process 914, status = succeeded4518client # farm-b> LOG-farm-b-t14519worker1 # [ 40.407633] nix-daemon[844]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4520worker1 # [ 40.419745] nix-daemon[844]: reaped child process 933, status = succeeded4521lb # [ 40.401679] niks3-server[1047]: 2026/09/20 09:33:52 INFO Received uploads request method=POST path=/api/pending_closures4522worker1 # [ 40.443053] nix-grpc-daemon[854]: time=2026-09-20T09:33:52.449Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4523worker1 # [ 40.444646] nix-grpc-daemon[854]: time=2026-09-20T09:33:52.449Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4524lb # [ 40.437820] niks3-server[1047]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4525lb # [ 40.448884] niks3-server[1047]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4526lb # [ 40.456799] niks3-server[1047]: 2026/09/20 09:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4527worker1 # [ 40.493878] nix-grpc-daemon[854]: time=2026-09-20T09:33:52.500Z level=INFO msg="Uploading 1 narinfos"4528lb # [ 40.464393] niks3-server[1047]: 2026/09/20 09:33:52 INFO Signed narinfos id=4 count=14529lb # [ 40.469965] niks3-server[1047]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4530lb # [ 40.479114] niks3-server[1047]: 2026/09/20 09:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4531lb # [ 40.482988] niks3-server[1047]: 2026/09/20 09:33:52 INFO Completed upload id=44532lb # [ 40.484619] niks3-server[1047]: 2026/09/20 09:33:52 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4533worker1 # [ 40.514339] nix-grpc-daemon[854]: time=2026-09-20T09:33:52.521Z level=INFO msg="Upload complete. (228ms)"4534worker1 # [ 40.521342] nix-grpc-daemon[786]: ts=2026-09-20T09:33:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14535worker2 # [ 40.524386] nix-grpc-daemon[785]: ts=2026-09-20T09:33:52Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04536worker2 # [ 40.528963] nix-grpc-daemon[785]: ts=2026-09-20T09:33:52Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14537worker1 # [ 40.534150] nix-daemon[844]: reaped child process 912, status = succeeded4538worker1 # [ 40.571471] nix-grpc-daemon[786]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524539worker1 # [ 40.583996] nix-grpc-daemon[786]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524540worker1 # [ 40.598454] nix-grpc-daemon[854]: time=2026-09-20T09:33:52.605Z level=INFO msg="All 2 paths already cached"4541client # copying 1 paths to 'lb:50051'...4542client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4543worker1 # [ 40.606295] nix-grpc-daemon[786]: ts=2026-09-20T09:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466524544worker1 # [ 40.617226] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4545worker1 # [ 40.774331] nix-daemon[844]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4546worker1 # [ 40.786135] nix-daemon[844]: reaped child process 940, status = succeeded4547worker1 # [ 40.942813] nix-daemon[844]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4548worker1 # [ 40.954814] nix-daemon[844]: reaped child process 945, status = succeeded4549lb # [ 40.940888] niks3-server[1047]: 2026/09/20 09:33:52 INFO Received uploads request method=POST path=/api/pending_closures4550worker1 # [ 40.979438] nix-grpc-daemon[854]: time=2026-09-20T09:33:52.986Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4551worker1 # [ 40.981446] nix-grpc-daemon[854]: time=2026-09-20T09:33:52.988Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4552lb # [ 40.970744] niks3-server[1047]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4553lb # [ 40.980446] niks3-server[1047]: 2026/09/20 09:33:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4554lb # [ 40.985369] niks3-server[1047]: 2026/09/20 09:33:53 INFO Signed narinfos id=5 count=14555worker1 # [ 41.019544] nix-grpc-daemon[854]: time=2026-09-20T09:33:53.026Z level=INFO msg="Uploading 1 narinfos"4556lb # [ 40.990356] niks3-server[1047]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4557lb # [ 41.001892] niks3-server[1047]: 2026/09/20 09:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4558lb # [ 41.007748] niks3-server[1047]: 2026/09/20 09:33:53 INFO Completed upload id=54559worker1 # [ 41.040441] nix-grpc-daemon[854]: time=2026-09-20T09:33:53.047Z level=INFO msg="Upload complete. (411ms)"4560lb # [ 41.012283] niks3-server[1047]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4561worker1 # [ 41.044145] nix-grpc-daemon[786]: ts=2026-09-20T09:33:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_in=7524562worker1 # [ 41.050115] nix-grpc-daemon[786]: ts=2026-09-20T09:33:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524563worker1 # [ 41.058194] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4564worker1 # [ 41.062889] nix-daemon[844]: reaped child process 935, status = succeeded4565client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4566client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4567worker1 # [ 41.068819] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4568client # farm-top> LOG-farm-top-t14569worker1 # [ 41.686711] nix-daemon[844]: reaped child process 949, status = succeeded4570worker1 # [ 41.846788] nix-daemon[844]: accepted connection from pid 964, user nix-grpc-daemon (trusted)4571worker1 # [ 41.857041] nix-daemon[844]: reaped child process 967, status = succeeded4572lb # [ 41.844224] niks3-server[1047]: 2026/09/20 09:33:53 INFO Received uploads request method=POST path=/api/pending_closures4573worker1 # [ 41.881876] nix-grpc-daemon[854]: time=2026-09-20T09:33:53.888Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4574worker1 # [ 41.883569] nix-grpc-daemon[854]: time=2026-09-20T09:33:53.890Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4575lb # [ 41.873291] niks3-server[1047]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4576lb # [ 41.879054] niks3-server[1047]: 2026/09/20 09:33:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4577lb # [ 41.882147] niks3-server[1047]: 2026/09/20 09:33:53 INFO Signed narinfos id=6 count=14578worker1 # [ 41.911884] nix-grpc-daemon[854]: time=2026-09-20T09:33:53.919Z level=INFO msg="Uploading 1 narinfos"4579lb # [ 41.883596] niks3-server[1047]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4580lb # [ 41.887235] niks3-server[1047]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4581lb # [ 41.891524] niks3-server[1047]: 2026/09/20 09:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4582lb # [ 41.897411] niks3-server[1047]: 2026/09/20 09:33:53 INFO Completed upload id=64583worker1 # [ 41.927290] nix-grpc-daemon[854]: time=2026-09-20T09:33:53.934Z level=INFO msg="Upload complete. (253ms)"4584lb # [ 41.899095] niks3-server[1047]: 2026/09/20 09:33:53 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4585worker1 # [ 41.929652] nix-grpc-daemon[786]: ts=2026-09-20T09:33:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14586worker1 # [ 41.935808] nix-daemon[844]: reaped child process 947, status = succeeded4587client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 6.63 seconds)4588client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4589client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4590client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4591client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.02 seconds)4592client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4593client # copying 3 paths...4594client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4595client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4596client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4597client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4598(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.21 seconds)4599subtest: repeat is answered Cached by the scheduler without building4600worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4601worker1 # finding garbage collector roots...4602worker1 # deleting specified paths...4603worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4604worker1 # deleting unused links...4605worker1 # note: hard linking is currently saving 0.0 KiB4606worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.26 seconds)4607worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4608worker2 # finding garbage collector roots...4609worker2 # deleting specified paths...4610worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4611worker2 # deleting unused links...4612worker2 # note: hard linking is currently saving 0.0 KiB4613worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.31 seconds)4614client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&24615client # don't know how to build these paths:4616client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4617worker2 # [ 43.358580] nix-grpc-daemon[785]: ts=2026-09-20T09:33:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:368244618worker2 # [ 43.363597] nix-grpc-daemon[785]: ts=2026-09-20T09:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368244619worker2 # [ 43.391219] nix-grpc-daemon[785]: ts=2026-09-20T09:33:55Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14620client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 >&2, in 0.29 seconds)4621client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath4622client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 outPath, in 0.20 seconds)4623lb: must succeed: curl -sf localhost:9901/clusters4624lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4625lb: must succeed: curl -sf localhost:9901/clusters4626lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4627worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true4628worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true, in 0.07 seconds)4629(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.31 seconds)4630subtest: inputs already in the cache are not sent to the scheduler4631lb: must succeed: curl -sf localhost:9901/clusters4632lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4633lb: must succeed: curl -sf localhost:9901/clusters4634lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4635worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4636worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4637client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&24638worker1 # [ 44.214643] nix-grpc-daemon[786]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466524639client # don't know how to build these paths:4640client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4641worker1 # [ 44.222897] nix-grpc-daemon[786]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524642worker1 # [ 44.232412] nix-grpc-daemon[854]: time=2026-09-20T09:33:56.239Z level=INFO msg="All 2 paths already cached"4643worker2 # [ 44.239908] nix-grpc-daemon[785]: ts=2026-09-20T09:33:56Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04644worker2 # [ 44.241948] nix-grpc-daemon[785]: ts=2026-09-20T09:33:56Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14645worker1 # [ 44.286105] nix-grpc-daemon[786]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524646worker1 # [ 44.297634] nix-grpc-daemon[786]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524647worker1 # [ 44.309685] nix-grpc-daemon[854]: time=2026-09-20T09:33:56.316Z level=INFO msg="All 2 paths already cached"4648client # copying 1 paths to 'lb:50051'...4649client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4650worker1 # [ 44.316353] nix-grpc-daemon[786]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466524651worker1 # [ 44.325071] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4652worker1 # [ 44.488224] nix-daemon[844]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4653worker1 # [ 44.500741] nix-daemon[844]: reaped child process 988, status = succeeded4654worker1 # [ 44.669202] nix-daemon[844]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4655worker1 # [ 44.680521] nix-daemon[844]: reaped child process 993, status = succeeded4656lb # [ 44.665643] niks3-server[1047]: 2026/09/20 09:33:56 INFO Received uploads request method=POST path=/api/pending_closures4657worker1 # [ 44.702502] nix-grpc-daemon[854]: time=2026-09-20T09:33:56.709Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4658worker1 # [ 44.703911] nix-grpc-daemon[854]: time=2026-09-20T09:33:56.709Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4659lb # [ 44.681809] niks3-server[1047]: 2026/09/20 09:33:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4660worker1 # [ 44.715231] nix-grpc-daemon[854]: time=2026-09-20T09:33:56.722Z level=INFO msg="Uploading 1 narinfos"4661lb # [ 44.685499] niks3-server[1047]: 2026/09/20 09:33:56 INFO Signed narinfos id=7 count=14662lb # [ 44.686842] niks3-server[1047]: 2026/09/20 09:33:56 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4663lb # [ 44.690621] niks3-server[1047]: 2026/09/20 09:33:56 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4664lb # [ 44.695243] niks3-server[1047]: 2026/09/20 09:33:56 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4665lb # [ 44.698977] niks3-server[1047]: 2026/09/20 09:33:56 INFO Completed upload id=74666worker1 # [ 44.728861] nix-grpc-daemon[854]: time=2026-09-20T09:33:56.736Z level=INFO msg="Upload complete. (390ms)"4667lb # [ 44.700598] niks3-server[1047]: 2026/09/20 09:33:56 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4668worker1 # [ 44.730698] nix-grpc-daemon[786]: ts=2026-09-20T09:33:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_in=7524669worker1 # [ 44.734408] nix-grpc-daemon[786]: ts=2026-09-20T09:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524670worker1 # [ 44.741137] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4671worker1 # [ 44.742362] nix-daemon[844]: reaped child process 983, status = succeeded4672client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4673client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4674worker1 # [ 44.747494] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4675client # farm-top> LOG-farm-top-t1b4676worker1 # [ 45.961161] nix-daemon[844]: reaped child process 997, status = succeeded4677worker1 # [ 46.101232] nix-daemon[844]: accepted connection from pid 1012, user nix-grpc-daemon (trusted)4678worker1 # [ 46.114091] nix-daemon[844]: reaped child process 1015, status = succeeded4679lb # [ 46.100949] niks3-server[1047]: 2026/09/20 09:33:58 INFO Received uploads request method=POST path=/api/pending_closures4680worker1 # [ 46.143293] nix-grpc-daemon[854]: time=2026-09-20T09:33:58.149Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4681worker1 # [ 46.144651] nix-grpc-daemon[854]: time=2026-09-20T09:33:58.149Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4682lb # [ 46.145599] niks3-server[1047]: 2026/09/20 09:33:58 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4683lb # [ 46.154148] niks3-server[1047]: 2026/09/20 09:33:58 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4684lb # [ 46.165179] niks3-server[1047]: 2026/09/20 09:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4685lb # [ 46.169850] niks3-server[1047]: 2026/09/20 09:33:58 INFO Signed narinfos id=8 count=14686worker1 # [ 46.203456] nix-grpc-daemon[854]: time=2026-09-20T09:33:58.210Z level=INFO msg="Uploading 1 narinfos"4687lb # [ 46.183883] niks3-server[1047]: 2026/09/20 09:33:58 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4688lb # [ 46.189448] niks3-server[1047]: 2026/09/20 09:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4689lb # [ 46.195498] niks3-server[1047]: 2026/09/20 09:33:58 INFO Completed upload id=84690lb # [ 46.197862] niks3-server[1047]: 2026/09/20 09:33:58 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4691worker1 # [ 46.227687] nix-grpc-daemon[854]: time=2026-09-20T09:33:58.234Z level=INFO msg="Upload complete. (273ms)"4692worker1 # [ 46.234157] nix-grpc-daemon[786]: ts=2026-09-20T09:33:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14693worker1 # [ 46.246468] nix-daemon[844]: reaped child process 995, status = succeeded4694client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.23 seconds)4695client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath4696client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds)4697lb: must succeed: curl -sf localhost:9901/clusters4698lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4699lb: must succeed: curl -sf localhost:9901/clusters4700lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4701worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4702worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4703(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.70 seconds)4704subtest: two clients wanting the same drv build it once4705client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4706client # [ 46.639432] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4707client # Running as unit: dedup1.service; invocation ID: 0d5fdaf00f8f40268cbfa903cb071aac4708client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 0.08 seconds)4709client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup4710worker1 # [ 46.868668] nix-grpc-daemon[786]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466524711client # [ 46.867739] nix[888]: don't know how to build these paths:4712client # [ 46.868764] nix[888]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4713worker2 # [ 46.881740] nix-grpc-daemon[785]: ts=2026-09-20T09:33:58Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04714worker2 # [ 46.887212] nix-grpc-daemon[785]: ts=2026-09-20T09:33:58Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14715worker2 # [ 46.891704] nix-grpc-daemon[785]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368244716worker2 # [ 46.907255] nix-grpc-daemon[785]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368244717client # [ 46.912214] nix[888]: copying 1 paths to 'lb:50051'...4718client # [ 46.913202] nix[888]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4719worker2 # [ 46.917833] nix-grpc-daemon[785]: ts=2026-09-20T09:33:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368244720worker2 # [ 46.929258] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4721worker2 # [ 46.947691] nix-grpc-daemon[785]: ts=2026-09-20T09:33:59Z level=info event=niks3_push_started pid=9014722worker2 # [ 47.356417] nix-daemon[848]: accepted connection from pid 914, user nix-grpc-daemon (trusted)4723worker2 # [ 47.371610] nix-daemon[848]: reaped child process 917, status = succeeded4724worker2 # [ 47.548493] nix-daemon[848]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4725worker2 # [ 47.560748] nix-daemon[848]: reaped child process 922, status = succeeded4726lb # [ 47.551355] niks3-server[1047]: 2026/09/20 09:33:59 INFO Received uploads request method=POST path=/api/pending_closures4727worker2 # [ 47.590587] nix-grpc-daemon[901]: time=2026-09-20T09:33:59.665Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4728worker2 # [ 47.594609] nix-grpc-daemon[901]: time=2026-09-20T09:33:59.669Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4729lb # [ 47.592622] niks3-server[1047]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4730lb # [ 47.604236] niks3-server[1047]: 2026/09/20 09:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4731worker2 # [ 47.638311] nix-grpc-daemon[901]: time=2026-09-20T09:33:59.714Z level=INFO msg="Uploading 1 narinfos"4732lb # [ 47.610804] niks3-server[1047]: 2026/09/20 09:33:59 INFO Signed narinfos id=9 count=14733lb # [ 47.623717] niks3-server[1047]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4734lb # [ 47.630876] niks3-server[1047]: 2026/09/20 09:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4735worker2 # [ 47.664053] nix-grpc-daemon[901]: time=2026-09-20T09:33:59.740Z level=INFO msg="Upload complete. (495ms)"4736lb # [ 47.636637] niks3-server[1047]: 2026/09/20 09:33:59 INFO Completed upload id=94737worker2 # [ 47.665897] nix-grpc-daemon[785]: ts=2026-09-20T09:33:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36824 duration_s=0 paths=1 nar_bytes_in=4324738lb # [ 47.641487] niks3-server[1047]: 2026/09/20 09:33:59 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4739worker2 # [ 47.672270] nix-daemon[848]: reaped child process 899, status = succeeded4740worker2 # [ 47.673598] nix-grpc-daemon[785]: ts=2026-09-20T09:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368244741worker2 # [ 47.677210] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4742client # [ 47.677968] nix[888]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4743worker2 # [ 47.683795] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)4744client # [ 47.712093] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup.4745client # Running as unit: dedup2.service; invocation ID: 28e79490a2b34a3da42b4dbf275773af4746client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag dedup, in 1.07 seconds)4747worker2 # [ 47.960555] nix-grpc-daemon[785]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:368244748client # [ 47.965799] nix[904]: this derivation will be built:4749worker1 # [ 47.976432] nix-grpc-daemon[786]: ts=2026-09-20T09:33:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:466524750worker1 # [ 47.985336] nix-grpc-daemon[786]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' from 'http://lb:5751'...4751client # [ 47.997066] nix[904]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4752worker2 # [ 48.011574] nix-grpc-daemon[785]: ts=2026-09-20T09:34:00Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04753client # [ 48.012723] nix[904]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4754worker2 # [ 48.016914] nix-grpc-daemon[785]: ts=2026-09-20T09:34:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368244755lb # [ 49.132867] envoy[928]: conn peer=[2001:db8:1::1]:55880 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9511 tx=3807 ms=63724756lb # [ 49.134427] envoy[928]: conn peer=[2001:db8:1::1]:41356 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=494757lb # [ 49.135782] envoy[928]: conn peer=[2001:db8:1::1]:41372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4152 tx=1687 ms=20314758client: must succeed: sleep 24759client: (finished: must succeed: sleep 2, in 2.05 seconds)4760client: waiting for success: ! systemctl is-active dedup1 dedup24761worker2 # [ 52.299674] nix-daemon[848]: reaped child process 926, status = succeeded4762worker2 # [ 52.453607] nix-daemon[848]: accepted connection from pid 961, user nix-grpc-daemon (trusted)4763worker2 # [ 52.465062] nix-daemon[848]: reaped child process 964, status = succeeded4764lb # [ 52.454319] niks3-server[1047]: 2026/09/20 09:34:04 INFO Received uploads request method=POST path=/api/pending_closures4765worker2 # [ 52.488684] nix-grpc-daemon[901]: time=2026-09-20T09:34:04.564Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4766worker2 # [ 52.490334] nix-grpc-daemon[901]: time=2026-09-20T09:34:04.564Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4767lb # [ 52.483798] niks3-server[1047]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4768lb # [ 52.486094] niks3-server[1047]: 2026/09/20 09:34:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4769lb # [ 52.489304] niks3-server[1047]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4770lb # [ 52.491662] niks3-server[1047]: 2026/09/20 09:34:04 INFO Signed narinfos id=10 count=14771worker2 # [ 52.520556] nix-grpc-daemon[901]: time=2026-09-20T09:34:04.596Z level=INFO msg="Uploading 1 narinfos"4772lb # [ 52.493336] niks3-server[1047]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4773lb # [ 52.498724] niks3-server[1047]: 2026/09/20 09:34:04 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4774lb # [ 52.502519] niks3-server[1047]: 2026/09/20 09:34:04 INFO Completed upload id=104775worker2 # [ 52.531371] nix-grpc-daemon[901]: time=2026-09-20T09:34:04.607Z level=INFO msg="Upload complete. (235ms)"4776lb # [ 52.504544] niks3-server[1047]: 2026/09/20 09:34:04 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4777worker2 # [ 52.534694] nix-grpc-daemon[785]: ts=2026-09-20T09:34:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36824 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14778worker2 # [ 52.537955] nix-grpc-daemon[785]: ts=2026-09-20T09:34:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36824 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14779worker2 # [ 52.543637] nix-daemon[848]: reaped child process 924, status = succeeded4780client # [ 52.571457] systemd[1]: dedup2.service: Deactivated successfully.4781client # [ 52.573076] systemd[1]: dedup2.service: Consumed 134ms CPU time over 4.860s wall clock time, 33.1M memory peak, 7.9K incoming IP traffic, 6.7K outgoing IP traffic.4782client # [ 52.602634] systemd[1]: dedup1.service: Deactivated successfully.4783client # [ 52.604061] systemd[1]: dedup1.service: Consumed 117ms CPU time over 5.966s wall clock time, 39.3M memory peak, 8.6K incoming IP traffic, 8.9K outgoing IP traffic.4784client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.10 seconds)4785client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4786client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4787worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4788worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4789worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4790worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4791worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4792worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.08 seconds)4793worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4794worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.07 seconds)4795(finished: subtest: two clients wanting the same drv build it once, in 7.09 seconds)4796subtest: build hook: nix-daemon with builders = grpc://lb4797client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log4798client # [ 53.852672] systemd[1]: Started Nix Daemon.4799client # [ 54.014524] nix-daemon[946]: accepted connection from pid 943, user root (trusted)4800worker2 # [ 54.208206] nix-grpc-daemon[785]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:368244801worker1 # [ 54.217474] nix-grpc-daemon[786]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524802worker2 # [ 54.225618] nix-grpc-daemon[785]: ts=2026-09-20T09:34:06Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=04803worker2 # [ 54.229477] nix-grpc-daemon[785]: ts=2026-09-20T09:34:06Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=14804worker1 # [ 54.272317] nix-grpc-daemon[786]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524805worker1 # [ 54.284110] nix-grpc-daemon[786]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524806worker1 # [ 54.295435] nix-grpc-daemon[786]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466524807worker1 # [ 54.306194] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4808worker1 # [ 54.469265] nix-daemon[844]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)4809worker1 # [ 54.482048] nix-daemon[844]: reaped child process 1050, status = succeeded4810worker1 # [ 54.639117] nix-daemon[844]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)4811worker1 # [ 54.653171] nix-daemon[844]: reaped child process 1055, status = succeeded4812lb # [ 54.638633] niks3-server[1047]: 2026/09/20 09:34:06 INFO Received uploads request method=POST path=/api/pending_closures4813worker1 # [ 54.676846] nix-grpc-daemon[854]: time=2026-09-20T09:34:06.683Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4814worker1 # [ 54.678265] nix-grpc-daemon[854]: time=2026-09-20T09:34:06.683Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"4815lb # [ 54.666835] niks3-server[1047]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst4816lb # [ 54.676528] niks3-server[1047]: 2026/09/20 09:34:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4817worker1 # [ 54.712525] nix-grpc-daemon[854]: time=2026-09-20T09:34:06.719Z level=INFO msg="Uploading 1 narinfos"4818lb # [ 54.682620] niks3-server[1047]: 2026/09/20 09:34:06 INFO Signed narinfos id=11 count=14819lb # [ 54.689242] niks3-server[1047]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls4820lb # [ 54.702278] niks3-server[1047]: 2026/09/20 09:34:06 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4821lb # [ 54.707560] niks3-server[1047]: 2026/09/20 09:34:06 INFO Completed upload id=114822worker1 # [ 54.737574] nix-grpc-daemon[854]: time=2026-09-20T09:34:06.744Z level=INFO msg="Upload complete. (420ms)"4823lb # [ 54.710787] niks3-server[1047]: 2026/09/20 09:34:06 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo4824worker1 # [ 54.742132] nix-grpc-daemon[786]: ts=2026-09-20T09:34:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_in=5124825worker1 # [ 54.746545] nix-grpc-daemon[786]: ts=2026-09-20T09:34:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524826worker1 # [ 54.756131] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4827worker1 # [ 54.760515] nix-daemon[844]: reaped child process 1045, status = succeeded4828worker1 # [ 54.765311] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4829worker1 # [ 55.589711] nix-daemon[844]: reaped child process 1059, status = succeeded4830worker1 # [ 55.730089] nix-daemon[844]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4831worker1 # [ 55.740864] nix-daemon[844]: reaped child process 1077, status = succeeded4832lb # [ 55.725164] niks3-server[1047]: 2026/09/20 09:34:07 INFO Received uploads request method=POST path=/api/pending_closures4833worker1 # [ 55.759888] nix-grpc-daemon[854]: time=2026-09-20T09:34:07.766Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4834worker1 # [ 55.762424] nix-grpc-daemon[854]: time=2026-09-20T09:34:07.768Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"4835lb # [ 55.751067] niks3-server[1047]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4836lb # [ 55.753611] niks3-server[1047]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv4837lb # [ 55.756993] niks3-server[1047]: 2026/09/20 09:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4838lb # [ 55.759107] niks3-server[1047]: 2026/09/20 09:34:07 INFO Signed narinfos id=12 count=14839worker1 # [ 55.789072] nix-grpc-daemon[854]: time=2026-09-20T09:34:07.796Z level=INFO msg="Uploading 1 narinfos"4840lb # [ 55.762554] niks3-server[1047]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls4841lb # [ 55.767712] niks3-server[1047]: 2026/09/20 09:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4842lb # [ 55.771612] niks3-server[1047]: 2026/09/20 09:34:07 INFO Completed upload id=124843worker1 # [ 55.801531] nix-grpc-daemon[854]: time=2026-09-20T09:34:07.808Z level=INFO msg="Upload complete. (229ms)"4844lb # [ 55.773252] niks3-server[1047]: 2026/09/20 09:34:07 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo4845worker1 # [ 55.803839] nix-grpc-daemon[786]: ts=2026-09-20T09:34:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=14846worker1 # [ 55.810245] nix-daemon[844]: reaped child process 1057, status = succeeded4847worker1 # [ 55.849065] nix-grpc-daemon[786]: ts=2026-09-20T09:34:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:466524848worker1 # [ 55.861396] nix-grpc-daemon[786]: ts=2026-09-20T09:34:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:466524849worker1 # [ 55.866681] nix-grpc-daemon[786]: ts=2026-09-20T09:34:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_out=1284850worker1 # [ 56.109171] nix-grpc-daemon[786]: ts=2026-09-20T09:34:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:466524851worker2 # [ 56.110341] nix-grpc-daemon[785]: ts=2026-09-20T09:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368244852worker2 # [ 56.128187] nix-grpc-daemon[785]: ts=2026-09-20T09:34:08Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=04853worker2 # [ 56.133084] nix-grpc-daemon[785]: ts=2026-09-20T09:34:08Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=14854worker1 # [ 56.137289] nix-grpc-daemon[786]: ts=2026-09-20T09:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524855worker1 # [ 56.146666] nix-grpc-daemon[786]: ts=2026-09-20T09:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524856worker1 # [ 56.161391] nix-grpc-daemon[854]: time=2026-09-20T09:34:08.168Z level=INFO msg="All 1 paths already cached"4857worker1 # [ 56.167988] nix-grpc-daemon[786]: ts=2026-09-20T09:34:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466524858worker1 # [ 56.177178] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4859worker1 # [ 56.350043] nix-daemon[844]: accepted connection from pid 1081, user nix-grpc-daemon (trusted)4860worker1 # [ 56.363432] nix-daemon[844]: reaped child process 1084, status = succeeded4861worker1 # [ 56.517981] nix-daemon[844]: accepted connection from pid 1086, user nix-grpc-daemon (trusted)4862worker1 # [ 56.529712] nix-daemon[844]: reaped child process 1089, status = succeeded4863lb # [ 56.515616] niks3-server[1047]: 2026/09/20 09:34:08 INFO Received uploads request method=POST path=/api/pending_closures4864worker1 # [ 56.552994] nix-grpc-daemon[854]: time=2026-09-20T09:34:08.560Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4865worker1 # [ 56.554478] nix-grpc-daemon[854]: time=2026-09-20T09:34:08.560Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"4866lb # [ 56.539611] niks3-server[1047]: 2026/09/20 09:34:08 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst4867lb # [ 56.550921] niks3-server[1047]: 2026/09/20 09:34:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4868lb # [ 56.557686] niks3-server[1047]: 2026/09/20 09:34:08 INFO Signed narinfos id=13 count=14869worker1 # [ 56.590641] nix-grpc-daemon[854]: time=2026-09-20T09:34:08.597Z level=INFO msg="Uploading 1 narinfos"4870lb # [ 56.562889] niks3-server[1047]: 2026/09/20 09:34:08 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls4871lb # [ 56.574483] niks3-server[1047]: 2026/09/20 09:34:08 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4872lb # [ 56.577695] niks3-server[1047]: 2026/09/20 09:34:08 INFO Completed upload id=134873worker1 # [ 56.608872] nix-grpc-daemon[854]: time=2026-09-20T09:34:08.616Z level=INFO msg="Upload complete. (419ms)"4874lb # [ 56.579597] niks3-server[1047]: 2026/09/20 09:34:08 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo4875worker1 # [ 56.610988] nix-grpc-daemon[786]: ts=2026-09-20T09:34:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_in=6244876worker1 # [ 56.613727] nix-grpc-daemon[786]: ts=2026-09-20T09:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524877worker1 # [ 56.620518] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4878worker1 # [ 56.625082] nix-daemon[844]: reaped child process 1079, status = succeeded4879worker1 # [ 56.628733] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4880worker1 # [ 57.239204] nix-daemon[844]: reaped child process 1093, status = succeeded4881worker1 # [ 57.374751] nix-daemon[844]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)4882worker1 # [ 57.386655] nix-daemon[844]: reaped child process 1111, status = succeeded4883lb # [ 57.372760] niks3-server[1047]: 2026/09/20 09:34:09 INFO Received uploads request method=POST path=/api/pending_closures4884worker1 # [ 57.410828] nix-grpc-daemon[854]: time=2026-09-20T09:34:09.417Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4885worker1 # [ 57.413107] nix-grpc-daemon[854]: time=2026-09-20T09:34:09.419Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"4886lb # [ 57.406138] niks3-server[1047]: 2026/09/20 09:34:09 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst4887lb # [ 57.413632] niks3-server[1047]: 2026/09/20 09:34:09 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv4888lb # [ 57.424159] niks3-server[1047]: 2026/09/20 09:34:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4889worker1 # [ 57.458927] nix-grpc-daemon[854]: time=2026-09-20T09:34:09.466Z level=INFO msg="Uploading 1 narinfos"4890lb # [ 57.429115] niks3-server[1047]: 2026/09/20 09:34:09 INFO Signed narinfos id=14 count=14891lb # [ 57.435203] niks3-server[1047]: 2026/09/20 09:34:09 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls4892lb # [ 57.443866] niks3-server[1047]: 2026/09/20 09:34:09 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4893worker1 # [ 57.479303] nix-grpc-daemon[854]: time=2026-09-20T09:34:09.486Z level=INFO msg="Upload complete. (246ms)"4894lb # [ 57.449345] niks3-server[1047]: 2026/09/20 09:34:09 INFO Completed upload id=144895lb # [ 57.453579] niks3-server[1047]: 2026/09/20 09:34:09 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo4896worker1 # [ 57.483935] nix-grpc-daemon[786]: ts=2026-09-20T09:34:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=14897worker1 # [ 57.492329] nix-grpc-daemon[786]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:466524898worker1 # [ 57.499088] nix-grpc-daemon[786]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:466524899worker1 # [ 57.502076] nix-grpc-daemon[786]: ts=2026-09-20T09:34:09Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_out=1764900worker1 # [ 57.508599] nix-daemon[844]: reaped child process 1091, status = succeeded4901worker1 # [ 57.740708] nix-grpc-daemon[786]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:466524902worker2 # [ 57.742357] nix-grpc-daemon[785]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368244903worker2 # [ 57.762929] nix-grpc-daemon[785]: ts=2026-09-20T09:34:09Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=04904worker1 # [ 57.768065] nix-grpc-daemon[786]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524905worker2 # [ 57.767444] nix-grpc-daemon[785]: ts=2026-09-20T09:34:09Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=14906worker1 # [ 57.779124] nix-grpc-daemon[786]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524907worker1 # [ 57.791457] nix-grpc-daemon[854]: time=2026-09-20T09:34:09.798Z level=INFO msg="All 2 paths already cached"4908worker1 # [ 57.797639] nix-grpc-daemon[786]: ts=2026-09-20T09:34:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466524909worker1 # [ 57.808214] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4910worker1 # [ 57.965493] nix-daemon[844]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4911worker1 # [ 57.976905] nix-daemon[844]: reaped child process 1118, status = succeeded4912worker1 # [ 58.130051] nix-daemon[844]: accepted connection from pid 1120, user nix-grpc-daemon (trusted)4913worker1 # [ 58.141553] nix-daemon[844]: reaped child process 1123, status = succeeded4914lb # [ 58.125672] niks3-server[1047]: 2026/09/20 09:34:10 INFO Received uploads request method=POST path=/api/pending_closures4915worker1 # [ 58.159647] nix-grpc-daemon[854]: time=2026-09-20T09:34:10.166Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4916worker1 # [ 58.160986] nix-grpc-daemon[854]: time=2026-09-20T09:34:10.166Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"4917lb # [ 58.137321] niks3-server[1047]: 2026/09/20 09:34:10 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst4918lb # [ 58.140613] niks3-server[1047]: 2026/09/20 09:34:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4919lb # [ 58.142333] niks3-server[1047]: 2026/09/20 09:34:10 INFO Signed narinfos id=15 count=14920worker1 # [ 58.172404] nix-grpc-daemon[854]: time=2026-09-20T09:34:10.179Z level=INFO msg="Uploading 1 narinfos"4921lb # [ 58.147540] niks3-server[1047]: 2026/09/20 09:34:10 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls4922lb # [ 58.150161] niks3-server[1047]: 2026/09/20 09:34:10 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4923lb # [ 58.153511] niks3-server[1047]: 2026/09/20 09:34:10 INFO Completed upload id=154924worker1 # [ 58.183398] nix-grpc-daemon[854]: time=2026-09-20T09:34:10.190Z level=INFO msg="Upload complete. (364ms)"4925lb # [ 58.155054] niks3-server[1047]: 2026/09/20 09:34:10 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo4926worker1 # [ 58.185259] nix-grpc-daemon[786]: ts=2026-09-20T09:34:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_in=7524927worker1 # [ 58.188629] nix-grpc-daemon[786]: ts=2026-09-20T09:34:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466524928worker1 # [ 58.194459] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4929worker1 # [ 58.196612] nix-daemon[844]: reaped child process 1113, status = succeeded4930worker1 # [ 58.201600] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)4931lb # [ 59.133642] envoy[928]: conn peer=[2001:db8:1::1]:41394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1589 tx=1487 ms=45914932lb # [ 59.137393] envoy[928]: conn peer=[2001:db8:1::1]:41380 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2955 tx=1355 ms=57254933lb # [ 59.141176] envoy[928]: conn peer=[2001:db8:1::1]:41236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=333 ms=524934lb # [ 59.144629] envoy[928]: conn peer=[2001:db8:1::1]:41234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3410 tx=1636 ms=17084935lb # [ 59.148162] envoy[928]: conn peer=[2001:db8:1::1]:41262 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=474936lb # [ 59.152248] envoy[928]: conn peer=[2001:db8:1::1]:41248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3850 tx=1860 ms=14414937worker1 # [ 59.615559] nix-daemon[844]: reaped child process 1127, status = succeeded4938worker1 # [ 59.752896] nix-daemon[844]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)4939worker1 # [ 59.764829] nix-daemon[844]: reaped child process 1145, status = succeeded4940lb # [ 59.751278] niks3-server[1047]: 2026/09/20 09:34:11 INFO Received uploads request method=POST path=/api/pending_closures4941worker1 # [ 59.790035] nix-grpc-daemon[854]: time=2026-09-20T09:34:11.797Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4942worker1 # [ 59.791553] nix-grpc-daemon[854]: time=2026-09-20T09:34:11.798Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"4943lb # [ 59.787150] niks3-server[1047]: 2026/09/20 09:34:11 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst4944lb # [ 59.801619] niks3-server[1047]: 2026/09/20 09:34:11 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv4945lb # [ 59.811859] niks3-server[1047]: 2026/09/20 09:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4946lb # [ 59.818952] niks3-server[1047]: 2026/09/20 09:34:11 INFO Signed narinfos id=16 count=14947worker1 # [ 59.851753] nix-grpc-daemon[854]: time=2026-09-20T09:34:11.858Z level=INFO msg="Uploading 1 narinfos"4948lb # [ 59.823434] niks3-server[1047]: 2026/09/20 09:34:11 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls4949lb # [ 59.833888] niks3-server[1047]: 2026/09/20 09:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4950lb # [ 59.839236] niks3-server[1047]: 2026/09/20 09:34:11 INFO Completed upload id=164951worker1 # [ 59.869320] nix-grpc-daemon[854]: time=2026-09-20T09:34:11.876Z level=INFO msg="Upload complete. (260ms)"4952lb # [ 59.841450] niks3-server[1047]: 2026/09/20 09:34:11 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo4953worker1 # [ 59.876689] nix-grpc-daemon[786]: ts=2026-09-20T09:34:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=14954worker2 # [ 59.876458] nix-grpc-daemon[785]: ts=2026-09-20T09:34:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:368244955worker1 # [ 59.889353] nix-daemon[844]: reaped child process 1125, status = succeeded4956worker2 # [ 59.889226] nix-grpc-daemon[785]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...4957worker2 # [ 59.903224] nix-grpc-daemon[785]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...4958worker2 # [ 59.912659] nix-grpc-daemon[785]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...4959worker2 # [ 59.930492] nix-grpc-daemon[785]: ts=2026-09-20T09:34:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:368244960worker2 # [ 59.935915] nix-grpc-daemon[785]: ts=2026-09-20T09:34:12Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36824 duration_s=0 paths=1 nar_bytes_out=2324961client # [ 60.022414] nix-daemon[946]: reaped child process 948, status = succeeded4962client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.36 seconds)4963client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top4964client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.03 seconds)4965client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4966client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4967client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4968client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4969(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.45 seconds)4970subtest: an input only one worker has still reaches the builder4971client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only4972client # this derivation will be built:4973client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv4974client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...4975client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag w1only, in 0.33 seconds)4976client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure4977client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.19 seconds)4978worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4979worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4980worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only4981worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)4982client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&24983client # [ 61.064393] nix-daemon[946]: accepted connection from pid 1028, user root (trusted)4984client # this derivation will be built:4985client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv4986client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...4987worker1 # [ 61.262498] nix-grpc-daemon[786]: ts=2026-09-20T09:34:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:466524988worker1 # [ 61.268205] nix-grpc-daemon[786]: ts=2026-09-20T09:34:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466524989worker1 # [ 61.437323] nix-daemon[844]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)4990worker1 # [ 61.447453] nix-daemon[844]: reaped child process 1155, status = succeeded4991lb # [ 61.435037] niks3-server[1047]: 2026/09/20 09:34:13 INFO Received uploads request method=POST path=/api/pending_closures4992worker1 # [ 61.471228] nix-grpc-daemon[854]: time=2026-09-20T09:34:13.478Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4993worker1 # [ 61.472593] nix-grpc-daemon[854]: time=2026-09-20T09:34:13.478Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"4994lb # [ 61.459859] niks3-server[1047]: 2026/09/20 09:34:13 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4995lb # [ 61.470380] niks3-server[1047]: 2026/09/20 09:34:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4996lb # [ 61.475915] niks3-server[1047]: 2026/09/20 09:34:13 INFO Signed narinfos id=17 count=14997worker1 # [ 61.509223] nix-grpc-daemon[854]: time=2026-09-20T09:34:13.516Z level=INFO msg="Uploading 1 narinfos"4998lb # [ 61.485758] niks3-server[1047]: 2026/09/20 09:34:13 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls4999lb # [ 61.495940] niks3-server[1047]: 2026/09/20 09:34:13 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5000lb # [ 61.502682] niks3-server[1047]: 2026/09/20 09:34:13 INFO Completed upload id=175001worker1 # [ 61.532336] nix-grpc-daemon[854]: time=2026-09-20T09:34:13.539Z level=INFO msg="Upload complete. (265ms)"5002lb # [ 61.504301] niks3-server[1047]: 2026/09/20 09:34:13 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5003worker2 # [ 61.549654] nix-grpc-daemon[785]: ts=2026-09-20T09:34:13Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05004worker2 # [ 61.553871] nix-grpc-daemon[785]: ts=2026-09-20T09:34:13Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15005worker2 # [ 61.591911] nix-grpc-daemon[785]: ts=2026-09-20T09:34:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368245006worker2 # [ 61.605133] nix-grpc-daemon[785]: ts=2026-09-20T09:34:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368245007client # copying 2 paths to 'lb:50051'...5008client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5009worker2 # [ 61.621175] nix-grpc-daemon[785]: ts=2026-09-20T09:34:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368245010client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5011worker2 # [ 61.638955] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5012worker2 # [ 61.813617] nix-daemon[848]: accepted connection from pid 993, user nix-grpc-daemon (trusted)5013worker2 # [ 61.825403] nix-daemon[848]: reaped child process 996, status = succeeded5014worker2 # [ 61.952928] nix-daemon[848]: accepted connection from pid 998, user nix-grpc-daemon (trusted)5015worker2 # [ 61.964544] nix-daemon[848]: reaped child process 1001, status = succeeded5016lb # [ 61.947625] niks3-server[1047]: 2026/09/20 09:34:13 INFO Received uploads request method=POST path=/api/pending_closures5017lb # [ 61.955812] niks3-server[1047]: 2026/09/20 09:34:13 INFO Received uploads request method=POST path=/api/pending_closures5018worker2 # [ 61.990742] nix-grpc-daemon[901]: time=2026-09-20T09:34:14.066Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5019worker2 # [ 61.993039] nix-grpc-daemon[901]: time=2026-09-20T09:34:14.068Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5020worker2 # [ 61.994591] nix-grpc-daemon[901]: time=2026-09-20T09:34:14.068Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5021lb # [ 61.993877] niks3-server[1047]: 2026/09/20 09:34:14 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5022lb # [ 61.996179] niks3-server[1047]: 2026/09/20 09:34:14 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5023lb # [ 61.999681] niks3-server[1047]: 2026/09/20 09:34:14 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5024lb # [ 62.001679] niks3-server[1047]: 2026/09/20 09:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5025lb # [ 62.003727] niks3-server[1047]: 2026/09/20 09:34:14 INFO Signed narinfos id=18 count=15026lb # [ 62.005278] niks3-server[1047]: 2026/09/20 09:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5027lb # [ 62.007501] niks3-server[1047]: 2026/09/20 09:34:14 INFO Signed narinfos id=19 count=25028worker2 # [ 62.035417] nix-grpc-daemon[901]: time=2026-09-20T09:34:14.111Z level=INFO msg="Uploading 3 narinfos"5029lb # [ 62.010373] niks3-server[1047]: 2026/09/20 09:34:14 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5030lb # [ 62.022237] niks3-server[1047]: 2026/09/20 09:34:14 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5031lb # [ 62.024812] niks3-server[1047]: 2026/09/20 09:34:14 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5032lb # [ 62.028095] niks3-server[1047]: 2026/09/20 09:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5033lb # [ 62.031044] niks3-server[1047]: 2026/09/20 09:34:14 INFO Completed upload id=185034lb # [ 62.032845] niks3-server[1047]: 2026/09/20 09:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5035lb # [ 62.035129] niks3-server[1047]: 2026/09/20 09:34:14 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5036lb # [ 62.037342] niks3-server[1047]: 2026/09/20 09:34:14 INFO Completed upload id=195037worker2 # [ 62.065080] nix-grpc-daemon[901]: time=2026-09-20T09:34:14.141Z level=INFO msg="Upload complete. (407ms)"5038worker2 # [ 62.069162] nix-grpc-daemon[785]: ts=2026-09-20T09:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36824 duration_s=0 paths=2 nar_bytes_in=9765039worker2 # [ 62.074490] nix-grpc-daemon[785]: ts=2026-09-20T09:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368245040worker2 # [ 62.082145] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5041worker2 # [ 62.091511] nix-daemon[848]: reaped child process 991, status = succeeded5042worker2 # [ 62.095867] nix-grpc-daemon[785]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5043worker2 # [ 62.102819] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5044client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5045client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5046worker2 # [ 63.316227] nix-daemon[848]: reaped child process 1006, status = succeeded5047worker2 # [ 63.469343] nix-daemon[848]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)5048worker2 # [ 63.480614] nix-daemon[848]: reaped child process 1024, status = succeeded5049lb # [ 63.470428] niks3-server[1047]: 2026/09/20 09:34:15 INFO Received uploads request method=POST path=/api/pending_closures5050worker2 # [ 63.506415] nix-grpc-daemon[901]: time=2026-09-20T09:34:15.582Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5051lb # [ 63.501110] niks3-server[1047]: 2026/09/20 09:34:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5052worker2 # [ 63.537929] nix-grpc-daemon[901]: time=2026-09-20T09:34:15.614Z level=INFO msg="Uploading 1 narinfos"5053lb # [ 63.511080] niks3-server[1047]: 2026/09/20 09:34:15 INFO Signed narinfos id=20 count=15054lb # [ 63.517063] niks3-server[1047]: 2026/09/20 09:34:15 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5055lb # [ 63.523965] niks3-server[1047]: 2026/09/20 09:34:15 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5056lb # [ 63.536133] niks3-server[1047]: 2026/09/20 09:34:15 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5057lb # [ 63.543058] niks3-server[1047]: 2026/09/20 09:34:15 INFO Completed upload id=205058worker2 # [ 63.570579] nix-grpc-daemon[901]: time=2026-09-20T09:34:15.646Z level=INFO msg="Upload complete. (260ms)"5059lb # [ 63.545846] niks3-server[1047]: 2026/09/20 09:34:15 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5060worker2 # [ 63.576577] nix-grpc-daemon[785]: ts=2026-09-20T09:34:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36824 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15061worker2 # [ 63.588872] nix-daemon[848]: reaped child process 1003, status = succeeded5062client # copying 1 paths...5063worker2 # [ 63.624832] nix-grpc-daemon[785]: ts=2026-09-20T09:34:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:368245064client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5065worker1 # [ 63.650745] nix-grpc-daemon[786]: ts=2026-09-20T09:34:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:466525066worker1 # [ 63.664476] nix-grpc-daemon[786]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5067worker1 # [ 63.678768] nix-grpc-daemon[786]: ts=2026-09-20T09:34:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_out=1205068client # [ 63.781542] nix-daemon[946]: reaped child process 1031, status = succeeded5069client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.93 seconds)5070client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&25071client # [ 63.978387] nix-daemon[946]: accepted connection from pid 1050, user root (trusted)5072client # this derivation will be built:5073client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5074worker1 # [ 64.169517] nix-grpc-daemon[786]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:466525075worker1 # [ 64.171952] nix-grpc-daemon[786]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466525076client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5077worker1 # [ 64.177719] nix-grpc-daemon[854]: time=2026-09-20T09:34:16.184Z level=INFO msg="All 1 paths already cached"5078worker2 # [ 64.187659] nix-grpc-daemon[785]: ts=2026-09-20T09:34:16Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05079worker1 # [ 64.197225] nix-grpc-daemon[786]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466525080worker2 # [ 64.192565] nix-grpc-daemon[785]: ts=2026-09-20T09:34:16Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15081worker1 # [ 64.204614] nix-grpc-daemon[786]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466525082client # copying 1 paths to 'lb:50051'...5083client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5084worker1 # [ 64.218026] nix-grpc-daemon[786]: ts=2026-09-20T09:34:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:466525085worker1 # [ 64.221907] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5086worker1 # [ 64.230764] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5)5087worker1 # [ 64.280254] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 194 ms (attempt 2/5)5088worker1 # [ 64.476846] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 190 ms (attempt 3/5)5089worker1 # [ 64.670432] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 552 ms (attempt 4/5)5090worker1 # [ 65.228091] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5091worker1 # [ 65.230450] nix-grpc-daemon[786]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5092worker1 # [ 65.396129] nix-daemon[844]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)5093worker1 # [ 65.407546] nix-daemon[844]: reaped child process 1167, status = succeeded5094worker1 # [ 65.566120] nix-daemon[844]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5095worker1 # [ 65.577675] nix-daemon[844]: reaped child process 1172, status = succeeded5096lb # [ 65.559804] niks3-server[1047]: 2026/09/20 09:34:17 INFO Received uploads request method=POST path=/api/pending_closures5097worker1 # [ 65.597469] nix-grpc-daemon[854]: time=2026-09-20T09:34:17.604Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5098worker1 # [ 65.598840] nix-grpc-daemon[854]: time=2026-09-20T09:34:17.604Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5099lb # [ 65.583410] niks3-server[1047]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5100lb # [ 65.596228] niks3-server[1047]: 2026/09/20 09:34:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5101worker1 # [ 65.631321] nix-grpc-daemon[854]: time=2026-09-20T09:34:17.638Z level=INFO msg="Uploading 1 narinfos"5102lb # [ 65.601645] niks3-server[1047]: 2026/09/20 09:34:17 INFO Signed narinfos id=21 count=15103lb # [ 65.608412] niks3-server[1047]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5104lb # [ 65.619955] niks3-server[1047]: 2026/09/20 09:34:17 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5105worker1 # [ 65.654750] nix-grpc-daemon[854]: time=2026-09-20T09:34:17.662Z level=INFO msg="Upload complete. (414ms)"5106lb # [ 65.625104] niks3-server[1047]: 2026/09/20 09:34:17 INFO Completed upload id=215107worker1 # [ 65.658064] nix-grpc-daemon[786]: ts=2026-09-20T09:34:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=1 paths=1 nar_bytes_in=5605108lb # [ 65.628856] niks3-server[1047]: 2026/09/20 09:34:17 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5109worker1 # [ 65.662387] nix-grpc-daemon[786]: ts=2026-09-20T09:34:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:466525110worker1 # [ 65.666898] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5111worker1 # [ 65.669038] nix-daemon[844]: reaped child process 1158, status = succeeded5112worker1 # [ 65.674322] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted)5113client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5114client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5115worker1 # [ 66.889406] nix-daemon[844]: reaped child process 1176, status = succeeded5116worker1 # [ 67.041424] nix-daemon[844]: accepted connection from pid 1191, user nix-grpc-daemon (trusted)5117worker1 # [ 67.052813] nix-daemon[844]: reaped child process 1194, status = succeeded5118lb # [ 67.040438] niks3-server[1047]: 2026/09/20 09:34:19 INFO Received uploads request method=POST path=/api/pending_closures5119worker1 # [ 67.076888] nix-grpc-daemon[854]: time=2026-09-20T09:34:19.083Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5120lb # [ 67.066462] niks3-server[1047]: 2026/09/20 09:34:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5121lb # [ 67.072473] niks3-server[1047]: 2026/09/20 09:34:19 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5122lb # [ 67.075907] niks3-server[1047]: 2026/09/20 09:34:19 INFO Signed narinfos id=22 count=15123worker1 # [ 67.107989] nix-grpc-daemon[854]: time=2026-09-20T09:34:19.115Z level=INFO msg="Uploading 1 narinfos"5124lb # [ 67.081173] niks3-server[1047]: 2026/09/20 09:34:19 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5125lb # [ 67.090259] niks3-server[1047]: 2026/09/20 09:34:19 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5126lb # [ 67.094111] niks3-server[1047]: 2026/09/20 09:34:19 INFO Completed upload id=225127worker1 # [ 67.125195] nix-grpc-daemon[854]: time=2026-09-20T09:34:19.132Z level=INFO msg="Upload complete. (243ms)"5128lb # [ 67.096222] niks3-server[1047]: 2026/09/20 09:34:19 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5129worker1 # [ 67.127996] nix-grpc-daemon[786]: ts=2026-09-20T09:34:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15130worker1 # [ 67.134247] nix-daemon[844]: reaped child process 1174, status = succeeded5131client # copying 1 paths...5132worker1 # [ 67.174773] nix-grpc-daemon[786]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:466525133client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5134worker2 # [ 67.196887] nix-grpc-daemon[785]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:368245135worker2 # [ 67.210070] nix-grpc-daemon[785]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5136worker2 # [ 67.223374] nix-grpc-daemon[785]: ts=2026-09-20T09:34:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36824 duration_s=0 paths=1 nar_bytes_out=1205137client # [ 67.316113] nix-daemon[946]: reaped child process 1053, status = succeeded5138client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.53 seconds)5139client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&25140client # [ 67.502471] nix-daemon[946]: accepted connection from pid 1072, user root (trusted)5141client # this derivation will be built:5142client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5143worker1 # [ 67.695811] nix-grpc-daemon[786]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:466525144client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux'...5145worker1 # [ 67.701680] nix-grpc-daemon[786]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:466525146worker1 # [ 67.706838] nix-grpc-daemon[854]: time=2026-09-20T09:34:19.714Z level=INFO msg="All 1 paths already cached"5147worker2 # [ 67.713082] nix-grpc-daemon[785]: ts=2026-09-20T09:34:19Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05148worker2 # [ 67.715573] nix-grpc-daemon[785]: ts=2026-09-20T09:34:19Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15149worker2 # [ 67.717859] nix-grpc-daemon[785]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368245150worker2 # [ 67.723838] nix-grpc-daemon[785]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368245151worker2 # [ 67.729806] nix-grpc-daemon[901]: time=2026-09-20T09:34:19.805Z level=INFO msg="All 1 paths already cached"5152client # copying 1 paths to 'lb:50051'...5153client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5154worker2 # [ 67.733357] nix-grpc-daemon[785]: ts=2026-09-20T09:34:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368245155worker2 # [ 67.738320] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5156worker2 # [ 67.883286] nix-daemon[848]: accepted connection from pid 1033, user nix-grpc-daemon (trusted)5157worker2 # [ 67.895465] nix-daemon[848]: reaped child process 1036, status = succeeded5158worker2 # [ 68.057651] nix-daemon[848]: accepted connection from pid 1038, user nix-grpc-daemon (trusted)5159worker2 # [ 68.069955] nix-daemon[848]: reaped child process 1041, status = succeeded5160lb # [ 68.059801] niks3-server[1047]: 2026/09/20 09:34:20 INFO Received uploads request method=POST path=/api/pending_closures5161worker2 # [ 68.095482] nix-grpc-daemon[901]: time=2026-09-20T09:34:20.171Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5162worker2 # [ 68.096826] nix-grpc-daemon[901]: time=2026-09-20T09:34:20.171Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5163lb # [ 68.086675] niks3-server[1047]: 2026/09/20 09:34:20 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5164lb # [ 68.096904] niks3-server[1047]: 2026/09/20 09:34:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5165worker2 # [ 68.129069] nix-grpc-daemon[901]: time=2026-09-20T09:34:20.205Z level=INFO msg="Uploading 1 narinfos"5166lb # [ 68.101672] niks3-server[1047]: 2026/09/20 09:34:20 INFO Signed narinfos id=23 count=15167lb # [ 68.107852] niks3-server[1047]: 2026/09/20 09:34:20 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5168lb # [ 68.118515] niks3-server[1047]: 2026/09/20 09:34:20 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5169worker2 # [ 68.151863] nix-grpc-daemon[901]: time=2026-09-20T09:34:20.228Z level=INFO msg="Upload complete. (409ms)"5170lb # [ 68.124775] niks3-server[1047]: 2026/09/20 09:34:20 INFO Completed upload id=235171worker2 # [ 68.153884] nix-grpc-daemon[785]: ts=2026-09-20T09:34:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36824 duration_s=0 paths=1 nar_bytes_in=5605172lb # [ 68.128524] niks3-server[1047]: 2026/09/20 09:34:20 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5173worker2 # [ 68.159628] nix-grpc-daemon[785]: ts=2026-09-20T09:34:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368245174worker2 # [ 68.169367] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5175worker2 # [ 68.171099] nix-daemon[848]: reaped child process 1031, status = succeeded5176client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5177worker2 # [ 68.176320] nix-daemon[848]: accepted connection from pid 785, user nix-grpc-daemon (trusted)5178client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5179lb # [ 69.134427] envoy[928]: conn peer=[2001:db8:1::1]:41274 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4285 tx=2201 ms=22495180lb # [ 69.137601] envoy[928]: conn peer=[2001:db8:1::1]:41280 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=535181lb # [ 69.140416] envoy[928]: conn peer=[2001:db8:1::1]:41296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3864 tx=1741 ms=24705182lb # [ 69.143311] envoy[928]: conn peer=[2001:db8:1::1]:38780 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=835183lb # [ 69.146053] envoy[928]: conn peer=[2001:db8:1::1]:38782 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3698 tx=1823 ms=31085184lb # [ 69.148816] envoy[928]: conn peer=[2001:db8:1::1]:38796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=329 ms=815185worker2 # [ 69.190808] nix-daemon[848]: reaped child process 1045, status = succeeded5186worker2 # [ 69.352625] nix-daemon[848]: accepted connection from pid 1060, user nix-grpc-daemon (trusted)5187worker2 # [ 69.364200] nix-daemon[848]: reaped child process 1063, status = succeeded5188lb # [ 69.354703] niks3-server[1047]: 2026/09/20 09:34:21 INFO Received uploads request method=POST path=/api/pending_closures5189worker2 # [ 69.390189] nix-grpc-daemon[901]: time=2026-09-20T09:34:21.465Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5190lb # [ 69.378303] niks3-server[1047]: 2026/09/20 09:34:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5191worker2 # [ 69.413664] nix-grpc-daemon[901]: time=2026-09-20T09:34:21.489Z level=INFO msg="Uploading 1 narinfos"5192lb # [ 69.386732] niks3-server[1047]: 2026/09/20 09:34:21 INFO Signed narinfos id=24 count=15193lb # [ 69.390342] niks3-server[1047]: 2026/09/20 09:34:21 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5194lb # [ 69.395141] niks3-server[1047]: 2026/09/20 09:34:21 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5195lb # [ 69.406242] niks3-server[1047]: 2026/09/20 09:34:21 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5196lb # [ 69.411908] niks3-server[1047]: 2026/09/20 09:34:21 INFO Completed upload id=245197worker2 # [ 69.439288] nix-grpc-daemon[901]: time=2026-09-20T09:34:21.515Z level=INFO msg="Upload complete. (255ms)"5198lb # [ 69.413042] niks3-server[1047]: 2026/09/20 09:34:21 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5199worker2 # [ 69.442367] nix-grpc-daemon[785]: ts=2026-09-20T09:34:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36824 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15200worker2 # [ 69.448553] nix-daemon[848]: reaped child process 1043, status = succeeded5201client # copying 1 paths...5202worker1 # [ 69.491445] nix-grpc-daemon[786]: ts=2026-09-20T09:34:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:466525203worker1 # [ 69.499429] nix-grpc-daemon[786]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5204client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5205worker1 # [ 69.527648] nix-grpc-daemon[786]: ts=2026-09-20T09:34:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:466525206worker1 # [ 69.529822] nix-grpc-daemon[786]: ts=2026-09-20T09:34:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46652 duration_s=0 paths=1 nar_bytes_out=1205207client # [ 69.631074] nix-daemon[946]: reaped child process 1075, status = succeeded5208client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.31 seconds)5209(finished: subtest: an input only one worker has still reaches the builder, in 9.52 seconds)5210subtest: upload whose reference the worker lacks is completed from the cache5211client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache5212client # this derivation will be built:5213client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5214client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5215client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A input --argstr tag viacache, in 0.42 seconds)5216client: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache5217client # this derivation will be built:5218client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5219client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5220client: (finished: must succeed: nix-build --no-out-link /nix/store/c7y5zg34h61m4kmsb9zg0afqx5zqqhrd-dep.nix -A referrer --argstr tag viacache, in 0.42 seconds)5221client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5222client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.19 seconds)5223worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5224worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.22 seconds)5225worker2 # [ 70.957679] systemd[1]: Stopping Nix worker-protocol over gRPC...5226worker2 # [ 71.419062] nix-grpc-daemon[785]: ts=2026-09-20T09:34:23Z level=info event=signal_exit5227worker2 # [ 71.674826] nix-grpc-daemon[785]: ts=2026-09-20T09:34:23Z level=info event=scheduler_yield5228worker2 # [ 71.676657] nix-grpc-daemon[785]: ts=2026-09-20T09:34:23Z level=info event=scheduler_restarting addr=lb:500515229worker2 # [ 71.680539] nix-grpc-daemon[785]: ts=2026-09-20T09:34:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:600445230worker2 # [ 71.681937] nix-grpc-daemon[785]: ts=2026-09-20T09:34:23Z level=info event=worker_gone addr=192.168.1.4:500515231lb # [ 71.654647] niks3-server[1047]: 2026/09/20 09:34:23 INFO lead: released remote=192.168.1.4:505165232worker2 # [ 71.685785] nix-grpc-daemon[785]: ts=2026-09-20T09:34:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:600445233worker1 # [ 71.686779] nix-grpc-daemon[786]: ts=2026-09-20T09:34:23Z level=info event=scheduler_restarting addr=lb:500515234worker2 # [ 71.687143] nix-grpc-daemon[785]: ts=2026-09-20T09:34:23Z level=info event=worker_gone addr=192.168.1.3:500515235worker2 # [ 71.877375] nix-grpc-daemon[785]: ts=2026-09-20T09:34:23Z level=info event=scheduler_connected addr=lb:500515236worker2 # [ 71.879222] nix-grpc-daemon[785]: ts=2026-09-20T09:34:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5237worker1 # [ 71.887994] nix-grpc-daemon[786]: ts=2026-09-20T09:34:23Z level=info event=scheduler_connected addr=lb:500515238worker1 # [ 71.891278] nix-grpc-daemon[786]: ts=2026-09-20T09:34:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5239worker2 # [ 72.080126] nix-grpc-daemon[785]: ts=2026-09-20T09:34:24Z level=info event=scheduler_connected addr=lb:500515240worker2 # [ 72.084224] nix-grpc-daemon[785]: ts=2026-09-20T09:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5241worker1 # [ 72.090154] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_connected addr=lb:500515242worker1 # [ 72.093362] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5243worker2 # [ 72.284575] nix-grpc-daemon[785]: ts=2026-09-20T09:34:24Z level=info event=scheduler_connected addr=lb:500515244worker2 # [ 72.286604] nix-grpc-daemon[785]: ts=2026-09-20T09:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5245worker1 # [ 72.291061] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_connected addr=lb:500515246worker1 # [ 72.294913] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5247worker2 # [ 72.487566] nix-grpc-daemon[785]: ts=2026-09-20T09:34:24Z level=info event=scheduler_connected addr=lb:500515248worker2 # [ 72.489334] nix-grpc-daemon[785]: ts=2026-09-20T09:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5249worker1 # [ 72.495856] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_connected addr=lb:500515250worker1 # [ 72.499707] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5251worker2 # [ 72.675657] nix-grpc-daemon[785]: ts=2026-09-20T09:34:24Z level=info event=scheduler_restarting5252worker1 # [ 72.701819] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_connected addr=lb:500515253worker1 # [ 72.705245] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5254worker2 # [ 72.703429] nix-grpc-daemon[785]: ts=2026-09-20T09:34:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5255worker2 # [ 72.714965] nix-daemon[848]: reaped child process 850, status = succeeded5256worker1 # [ 72.902407] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_connected addr=lb:500515257worker1 # [ 72.906196] nix-grpc-daemon[786]: ts=2026-09-20T09:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5258worker1 # [ 73.106891] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_connected addr=lb:500515259worker1 # [ 73.110954] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5260worker1 # [ 73.311705] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_connected addr=lb:500515261worker1 # [ 73.315649] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5262worker1 # [ 73.516485] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_connected addr=lb:500515263worker1 # [ 73.520318] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5264worker2 # [ 73.542744] nix-grpc-daemon[785]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5265worker2 # [ 73.546204] nix-grpc-daemon[785]: E0000 00:00:1789896865.617681 828 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5266worker2 # [ 73.550824] nix-grpc-daemon[785]: E0000 00:00:1789896865.618232 828 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5267worker1 # [ 73.720983] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_connected addr=lb:500515268worker1 # [ 73.724827] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5269worker2 # [ 73.727893] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5270worker2 # [ 73.731138] systemd[1]: Stopped Nix worker-protocol over gRPC.5271worker2 # [ 73.734707] systemd[1]: nix-grpc-daemon.service: Consumed 1.314s CPU time over 54.568s wall clock time, 52.7M memory peak, 55K incoming IP traffic, 104.6K outgoing IP traffic.5272worker2 # [ 73.745664] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5273worker2 # [ 73.749891] systemd[1]: Closed Nix worker-protocol over gRPC.5274worker2 # [ 73.753369] systemd[1]: nix-grpc-daemon.socket: Consumed 298us CPU time over 1min 3.384s wall clock time, 256K memory peak, 72.1K incoming IP traffic, 63K outgoing IP traffic.5275lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5276lb: must succeed: curl -sf localhost:9901/clusters5277lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5278lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.08 seconds)5279client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25280worker1 # [ 73.925858] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_connected addr=lb:500515281worker1 # [ 73.929761] nix-grpc-daemon[786]: ts=2026-09-20T09:34:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5282worker1 # [ 74.031620] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:466525283worker1 # [ 74.039221] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:466525284client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5285client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.23 seconds)5286worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5287worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.03 seconds)5288worker1 # [ 74.130558] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_connected addr=lb:500515289worker1 # [ 74.134447] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5290worker2 # [ 74.148714] systemd[1]: Listening on Nix worker-protocol over gRPC.5291worker1 # [ 74.210411] systemd[1]: Stopping Nix worker-protocol over gRPC...5292worker1 # [ 74.335372] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_connected addr=lb:500515293worker1 # [ 74.339254] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5294worker1 # [ 74.439446] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=signal_exit5295worker1 # [ 74.540006] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_connected addr=lb:500515296worker1 # [ 74.543897] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5297worker1 # [ 74.624605] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_take_over5298lb # [ 74.597473] niks3-server[1047]: 2026/09/20 09:34:26 INFO lead: acquired remote=192.168.1.3:587525299worker1 # [ 74.627730] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_yield5300lb # [ 74.600657] niks3-server[1047]: 2026/09/20 09:34:26 INFO lead: released remote=192.168.1.3:587525301worker2 # [ 74.715762] systemd[1]: Starting Nix worker-protocol over gRPC...5302worker1 # [ 74.745552] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_connected addr=lb:500515303worker1 # [ 74.749307] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5304worker2 # [ 74.885261] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5305worker2 # [ 74.888975] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15306worker2 # [ 74.894957] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5307worker2 # [ 74.900223] systemd[1]: Started Nix worker-protocol over gRPC.5308lb # [ 74.879664] niks3-server[1047]: 2026/09/20 09:34:26 INFO lead: acquired remote=192.168.1.4:581365309worker2 # [ 74.908131] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:26Z level=info event=scheduler_take_over5310worker2 # [ 74.927849] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:27Z level=info event=scheduler_connected addr=lb:500515311worker2 # [ 74.930933] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05312worker2 # [ 74.948265] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05313worker1 # [ 74.950321] nix-grpc-daemon[786]: ts=2026-09-20T09:34:26Z level=info event=scheduler_connected addr=lb:500515314worker1 # [ 75.625510] nix-grpc-daemon[786]: ts=2026-09-20T09:34:27Z level=info event=scheduler_restarting5315worker2 # [ 75.631545] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:417745316worker2 # [ 75.635447] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:27Z level=info event=worker_gone addr=192.168.1.3:500515317worker1 # [ 75.648758] nix-grpc-daemon[786]: ts=2026-09-20T09:34:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5318worker1 # [ 75.661945] nix-daemon[844]: reaped child process 846, status = succeeded5319worker1 # [ 76.568494] nix-grpc-daemon[786]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5320worker1 # [ 76.569883] nix-grpc-daemon[786]: E0000 00:00:1789896868.575039 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5321worker1 # [ 76.572717] nix-grpc-daemon[786]: E0000 00:00:1789896868.579991 829 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5322worker1 # [ 77.659440] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5323worker1 # [ 77.664606] systemd[1]: Stopped Nix worker-protocol over gRPC.5324worker1 # [ 77.666697] systemd[1]: nix-grpc-daemon.service: Consumed 2.520s CPU time over 58.501s wall clock time, 61M memory peak, 90.7K incoming IP traffic, 169.3K outgoing IP traffic.5325worker1 # [ 77.677527] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5326worker1 # [ 77.681520] systemd[1]: Closed Nix worker-protocol over gRPC.5327worker1 # [ 77.685291] systemd[1]: nix-grpc-daemon.socket: Consumed 303us CPU time over 1min 8.248s wall clock time, 256K memory peak, 77.6K incoming IP traffic, 61.8K outgoing IP traffic.5328lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5329lb: must succeed: curl -sf localhost:9901/clusters5330lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5331lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5332client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25333worker2 # [ 77.997585] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:417885334worker2 # [ 78.005472] nix-daemon[848]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5335worker2 # [ 78.018924] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 1/5)5336worker2 # [ 78.114771] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 109 ms (attempt 2/5)5337worker2 # [ 78.229687] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 3/5)5338worker2 # [ 78.320703] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 575 ms (attempt 4/5)5339worker2 # [ 78.899426] nix-grpc-daemon[1084]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5340client # copying 2 paths...5341client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5342worker2 # [ 78.921567] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:417885343client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5344worker2 # [ 78.933334] nix-daemon[848]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)5345worker2 # [ 78.953925] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:31Z level=info event=niks3_push_started pid=11405346lb # [ 79.135481] envoy[928]: conn peer=[2001:db8:1::1]:38812 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3699 tx=1889 ms=18855347lb # [ 79.139636] envoy[928]: conn peer=[2001:db8:1::1]:38822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=555348lb # [ 79.143198] envoy[928]: conn peer=[2001:db8:1::4]:56150 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=12427 tx=2434 ms=532325349lb # [ 79.147060] envoy[928]: conn peer=[2001:db8:1::1]:51494 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=235350lb # [ 79.150485] envoy[928]: conn peer=[2001:db8:1::3]:57192 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15877 tx=3188 ms=561535351worker2 # [ 79.316847] nix-daemon[848]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5352worker2 # [ 79.327620] nix-daemon[848]: reaped child process 1156, status = succeeded5353lb # [ 79.321140] niks3-server[1047]: 2026/09/20 09:34:31 INFO Received uploads request method=POST path=/api/pending_closures5354lb # [ 79.328487] niks3-server[1047]: 2026/09/20 09:34:31 INFO Received uploads request method=POST path=/api/pending_closures5355worker2 # [ 79.363179] nix-grpc-daemon[1140]: time=2026-09-20T09:34:31.439Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5356worker2 # [ 79.366136] nix-grpc-daemon[1140]: time=2026-09-20T09:34:31.440Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5357worker2 # [ 79.367679] nix-grpc-daemon[1140]: time=2026-09-20T09:34:31.441Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5358lb # [ 79.375248] niks3-server[1047]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5359lb # [ 79.387824] niks3-server[1047]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5360lb # [ 79.393405] niks3-server[1047]: 2026/09/20 09:34:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5361lb # [ 79.396475] niks3-server[1047]: 2026/09/20 09:34:31 INFO Signed narinfos id=26 count=25362lb # [ 79.400813] niks3-server[1047]: 2026/09/20 09:34:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5363lb # [ 79.405907] niks3-server[1047]: 2026/09/20 09:34:31 INFO Signed narinfos id=25 count=15364worker2 # [ 79.434074] nix-grpc-daemon[1140]: time=2026-09-20T09:34:31.509Z level=INFO msg="Uploading 3 narinfos"5365lb # [ 79.409913] niks3-server[1047]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5366lb # [ 79.424531] niks3-server[1047]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5367lb # [ 79.430945] niks3-server[1047]: 2026/09/20 09:34:31 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5368lb # [ 79.433220] niks3-server[1047]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5369lb # [ 79.436553] niks3-server[1047]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5370lb # [ 79.441178] niks3-server[1047]: 2026/09/20 09:34:31 INFO Completed upload id=265371lb # [ 79.442469] niks3-server[1047]: 2026/09/20 09:34:31 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5372lb # [ 79.447074] niks3-server[1047]: 2026/09/20 09:34:31 INFO Completed upload id=255373worker2 # [ 79.475806] nix-grpc-daemon[1140]: time=2026-09-20T09:34:31.551Z level=INFO msg="Upload complete. (325ms)"5374lb # [ 79.451492] niks3-server[1047]: 2026/09/20 09:34:31 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5375worker2 # [ 79.481852] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41788 duration_s=0 paths=2 nar_bytes_in=3125376worker2 # [ 79.494130] nix-daemon[848]: reaped child process 1138, status = succeeded5377client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.75 seconds)5378worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5379worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.03 seconds)5380worker1 # [ 79.581442] systemd[1]: Listening on Nix worker-protocol over gRPC.5381worker1 # [ 79.584730] systemd[1]: Starting Nix worker-protocol over gRPC...5382worker1 # [ 79.761144] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5383worker1 # [ 79.764955] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15384worker1 # [ 79.770932] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5385worker1 # [ 79.775392] systemd[1]: Started Nix worker-protocol over gRPC.5386lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5387lb: must succeed: curl -sf localhost:9901/clusters5388worker1 # [ 79.807221] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:31Z level=info event=scheduler_connected addr=lb:500515389worker2 # [ 79.808929] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05390lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5391lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.07 seconds)5392(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.21 seconds)5393subtest: scheduler down: the other node takes the lock and keeps it5394lb: must succeed: curl -sf localhost:9901/clusters5395lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5396lb: must succeed: curl -sf localhost:9901/clusters5397lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5398lb: must succeed: curl -sf localhost:9901/clusters5399lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5400lb: must succeed: curl -sf localhost:9901/clusters5401lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5402worker2 # [ 80.037463] systemd[1]: Stopping Nix worker-protocol over gRPC...5403worker2 # [ 80.905705] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:32Z level=info event=signal_exit5404worker2 # [ 81.113795] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=scheduler_yield5405worker1 # [ 81.120434] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:33Z level=info event=scheduler_restarting addr=lb:500515406lb # [ 81.091207] niks3-server[1047]: 2026/09/20 09:34:33 INFO lead: released remote=192.168.1.4:581365407worker2 # [ 81.118580] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:417745408worker2 # [ 81.121943] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=worker_gone addr=192.168.1.3:500515409worker2 # [ 81.127741] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=scheduler_restarting addr=lb:500515410worker2 # [ 81.130790] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:417745411worker2 # [ 81.134121] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=worker_gone addr=192.168.1.4:500515412worker1 # [ 81.321158] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:33Z level=info event=scheduler_connected addr=lb:500515413worker1 # [ 81.325152] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5414worker2 # [ 81.327714] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=scheduler_connected addr=lb:500515415worker2 # [ 81.331577] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5416worker1 # [ 81.525567] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:33Z level=info event=scheduler_connected addr=lb:500515417worker1 # [ 81.527373] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5418worker2 # [ 81.532452] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=scheduler_connected addr=lb:500515419worker2 # [ 81.536405] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5420worker1 # [ 81.728378] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:33Z level=info event=scheduler_connected addr=lb:500515421worker1 # [ 81.732342] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5422worker2 # [ 81.733946] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=scheduler_connected addr=lb:500515423worker2 # [ 81.735683] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5424worker1 # [ 81.933104] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:33Z level=info event=scheduler_connected addr=lb:500515425worker1 # [ 81.937299] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5426worker2 # [ 81.937531] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:34Z level=info event=scheduler_connected addr=lb:500515427worker2 # [ 81.940653] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5428worker2 # [ 82.117882] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:34Z level=info event=scheduler_restarting5429worker2 # [ 82.130259] nix-daemon[848]: reaped child process 1133, status = succeeded5430worker1 # [ 82.139419] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_connected addr=lb:500515431worker2 # [ 82.141741] nix-grpc-daemon[1084]: ts=2026-09-20T09:34:34Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5432worker1 # [ 82.142775] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5433worker1 # [ 82.340143] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_connected addr=lb:500515434worker1 # [ 82.344246] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5435worker1 # [ 82.545112] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_connected addr=lb:500515436worker1 # [ 82.549138] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5437worker1 # [ 82.749954] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_connected addr=lb:500515438worker1 # [ 82.753944] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5439worker2 # [ 82.923152] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5440worker2 # [ 82.926995] systemd[1]: Stopped Nix worker-protocol over gRPC.5441worker2 # [ 82.930909] systemd[1]: nix-grpc-daemon.service: Consumed 320ms CPU time over 8.210s wall clock time, 32.6M memory peak, 17.4K incoming IP traffic, 22.3K outgoing IP traffic.5442worker2 # [ 82.942923] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5443worker2 # [ 82.947200] systemd[1]: Closed Nix worker-protocol over gRPC.5444worker2 # [ 82.950831] systemd[1]: nix-grpc-daemon.socket: Consumed 278us CPU time over 8.802s wall clock time, 256K memory peak, 20.2K incoming IP traffic, 15.3K outgoing IP traffic.5445worker1 # [ 82.954589] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_connected addr=lb:500515446worker1 # [ 82.958395] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5447worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5448worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5449worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5450worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5451worker1 # [ 83.159175] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_connected addr=lb:500515452worker1 # [ 83.161850] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5453worker1 # [ 83.362616] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_connected addr=lb:500515454worker1 # [ 83.366590] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5455worker1 # [ 83.567102] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_connected addr=lb:500515456worker1 # [ 83.568817] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5457worker1 # [ 83.769907] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_connected addr=lb:500515458worker1 # [ 83.774392] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5459worker1 # [ 83.975254] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_connected addr=lb:500515460worker1 # [ 83.979605] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5461worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5462worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5463worker1 # [ 84.180695] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:36Z level=info event=scheduler_connected addr=lb:500515464worker1 # [ 84.185125] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5465worker1 # [ 84.385883] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:36Z level=info event=scheduler_connected addr=lb:500515466worker1 # [ 84.389736] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5467worker1 # [ 84.590617] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:36Z level=info event=scheduler_connected addr=lb:500515468worker1 # [ 84.594472] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5469lb # [ 84.753907] niks3-server[1047]: 2026/09/20 09:34:36 INFO lead: acquired remote=192.168.1.3:531025470worker1 # [ 84.787121] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:36Z level=info event=scheduler_take_over5471worker1 # [ 84.794857] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:36Z level=info event=scheduler_connected addr=lb:500515472worker1 # [ 84.802330] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05473worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5474worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5475client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&25476worker1 # [ 85.420460] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:533405477worker1 # [ 85.425418] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5478client # don't know how to build these paths:5479client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5480worker1 # [ 85.433107] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533405481worker1 # [ 85.438694] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)5482worker1 # [ 85.445964] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 2/5)5483worker1 # [ 85.458229] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 186 ms (attempt 3/5)5484worker1 # [ 85.646668] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 442 ms (attempt 4/5)5485worker1 # [ 86.091757] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5486worker1 # [ 86.121427] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:38Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05487worker1 # [ 86.127396] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:38Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15488worker1 # [ 86.166570] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533405489worker1 # [ 86.179927] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533405490client # copying 1 paths to 'lb:50051'...5491client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5492worker1 # [ 86.193259] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533405493worker1 # [ 86.204126] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5494worker1 # [ 86.222817] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:38Z level=info event=niks3_push_started pid=12975495worker1 # [ 86.584853] nix-daemon[844]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5496worker1 # [ 86.596312] nix-daemon[844]: reaped child process 1313, status = succeeded5497worker1 # [ 86.748209] nix-daemon[844]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5498worker1 # [ 86.759487] nix-daemon[844]: reaped child process 1318, status = succeeded5499lb # [ 86.746540] niks3-server[1047]: 2026/09/20 09:34:38 INFO Received uploads request method=POST path=/api/pending_closures5500worker1 # [ 86.782878] nix-grpc-daemon[1297]: time=2026-09-20T09:34:38.788Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5501worker1 # [ 86.784502] nix-grpc-daemon[1297]: time=2026-09-20T09:34:38.788Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5502lb # [ 86.767424] niks3-server[1047]: 2026/09/20 09:34:38 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5503lb # [ 86.771056] niks3-server[1047]: 2026/09/20 09:34:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5504lb # [ 86.773857] niks3-server[1047]: 2026/09/20 09:34:38 INFO Signed narinfos id=27 count=15505worker1 # [ 86.805306] nix-grpc-daemon[1297]: time=2026-09-20T09:34:38.812Z level=INFO msg="Uploading 1 narinfos"5506lb # [ 86.777603] niks3-server[1047]: 2026/09/20 09:34:38 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5507lb # [ 86.782439] niks3-server[1047]: 2026/09/20 09:34:38 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5508lb # [ 86.785550] niks3-server[1047]: 2026/09/20 09:34:38 INFO Completed upload id=275509worker1 # [ 86.816590] nix-grpc-daemon[1297]: time=2026-09-20T09:34:38.823Z level=INFO msg="Upload complete. (387ms)"5510lb # [ 86.787145] niks3-server[1047]: 2026/09/20 09:34:38 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5511worker1 # [ 86.818640] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53340 duration_s=0 paths=1 nar_bytes_in=5205512worker1 # [ 86.822137] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533405513worker1 # [ 86.827530] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5514worker1 # [ 86.830590] nix-daemon[844]: reaped child process 1295, status = succeeded5515client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5516client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5517worker1 # [ 86.835384] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5518client # farm-a> LOG-farm-a-on-standby5519worker1 # [ 87.859336] nix-daemon[844]: reaped child process 1322, status = succeeded5520worker1 # [ 87.997255] nix-daemon[844]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5521worker1 # [ 88.009474] nix-daemon[844]: reaped child process 1340, status = succeeded5522lb # [ 87.990231] niks3-server[1047]: 2026/09/20 09:34:40 INFO Received uploads request method=POST path=/api/pending_closures5523worker1 # [ 88.027866] nix-grpc-daemon[1297]: time=2026-09-20T09:34:40.034Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5524worker1 # [ 88.029400] nix-grpc-daemon[1297]: time=2026-09-20T09:34:40.036Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5525lb # [ 88.020611] niks3-server[1047]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5526lb # [ 88.036288] niks3-server[1047]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5527lb # [ 88.045615] niks3-server[1047]: 2026/09/20 09:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5528worker1 # [ 88.080473] nix-grpc-daemon[1297]: time=2026-09-20T09:34:40.087Z level=INFO msg="Uploading 1 narinfos"5529lb # [ 88.050611] niks3-server[1047]: 2026/09/20 09:34:40 INFO Signed narinfos id=28 count=15530lb # [ 88.056322] niks3-server[1047]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5531lb # [ 88.064331] niks3-server[1047]: 2026/09/20 09:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5532lb # [ 88.070273] niks3-server[1047]: 2026/09/20 09:34:40 INFO Completed upload id=285533worker1 # [ 88.101204] nix-grpc-daemon[1297]: time=2026-09-20T09:34:40.106Z level=INFO msg="Upload complete. (257ms)"5534lb # [ 88.073744] niks3-server[1047]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5535worker1 # [ 88.106702] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53340 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15536worker1 # [ 88.120164] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:40Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05537worker1 # [ 88.122584] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:40Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15538worker1 # [ 88.126569] nix-daemon[844]: reaped child process 1320, status = succeeded5539worker1 # [ 88.164686] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533405540worker1 # [ 88.176497] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533405541worker1 # [ 88.188206] nix-grpc-daemon[1297]: time=2026-09-20T09:34:40.195Z level=INFO msg="All 1 paths already cached"5542client # copying 1 paths to 'lb:50051'...5543client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5544worker1 # [ 88.196249] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533405545worker1 # [ 88.206154] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5546worker1 # [ 88.370054] nix-daemon[844]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5547worker1 # [ 88.381682] nix-daemon[844]: reaped child process 1349, status = succeeded5548worker1 # [ 88.550180] nix-daemon[844]: accepted connection from pid 1351, user nix-grpc-daemon (trusted)5549worker1 # [ 88.561524] nix-daemon[844]: reaped child process 1354, status = succeeded5550lb # [ 88.547632] niks3-server[1047]: 2026/09/20 09:34:40 INFO Received uploads request method=POST path=/api/pending_closures5551worker1 # [ 88.584651] nix-grpc-daemon[1297]: time=2026-09-20T09:34:40.591Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5552worker1 # [ 88.586379] nix-grpc-daemon[1297]: time=2026-09-20T09:34:40.593Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5553lb # [ 88.575585] niks3-server[1047]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5554lb # [ 88.586647] niks3-server[1047]: 2026/09/20 09:34:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5555lb # [ 88.592243] niks3-server[1047]: 2026/09/20 09:34:40 INFO Signed narinfos id=29 count=15556worker1 # [ 88.625197] nix-grpc-daemon[1297]: time=2026-09-20T09:34:40.632Z level=INFO msg="Uploading 1 narinfos"5557lb # [ 88.595497] niks3-server[1047]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5558lb # [ 88.609084] niks3-server[1047]: 2026/09/20 09:34:40 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5559worker1 # [ 88.644400] nix-grpc-daemon[1297]: time=2026-09-20T09:34:40.651Z level=INFO msg="Upload complete. (427ms)"5560lb # [ 88.614741] niks3-server[1047]: 2026/09/20 09:34:40 INFO Completed upload id=295561worker1 # [ 88.647081] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53340 duration_s=0 paths=1 nar_bytes_in=6405562lb # [ 88.618242] niks3-server[1047]: 2026/09/20 09:34:40 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5563worker1 # [ 88.652501] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533405564worker1 # [ 88.656334] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5565worker1 # [ 88.658520] nix-daemon[844]: reaped child process 1344, status = succeeded5566worker1 # [ 88.663780] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5567client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5568client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5569lb # [ 89.136975] envoy[928]: conn peer=[2001:db8:1::1]:51504 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=305 ms=14975570lb # [ 89.142341] envoy[928]: conn peer=[2001:db8:1::4]:35670 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1323 tx=349 ms=72145571client # farm-b> LOG-farm-b-on-standby5572worker1 # [ 89.676575] nix-daemon[844]: reaped child process 1358, status = succeeded5573worker1 # [ 89.803875] nix-daemon[844]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)5574worker1 # [ 89.815262] nix-daemon[844]: reaped child process 1376, status = succeeded5575lb # [ 89.797874] niks3-server[1047]: 2026/09/20 09:34:41 INFO Received uploads request method=POST path=/api/pending_closures5576worker1 # [ 89.832369] nix-grpc-daemon[1297]: time=2026-09-20T09:34:41.839Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5577worker1 # [ 89.833797] nix-grpc-daemon[1297]: time=2026-09-20T09:34:41.841Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5578lb # [ 89.830895] niks3-server[1047]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5579lb # [ 89.839309] niks3-server[1047]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5580lb # [ 89.848069] niks3-server[1047]: 2026/09/20 09:34:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5581worker1 # [ 89.882808] nix-grpc-daemon[1297]: time=2026-09-20T09:34:41.889Z level=INFO msg="Uploading 1 narinfos"5582lb # [ 89.852845] niks3-server[1047]: 2026/09/20 09:34:41 INFO Signed narinfos id=30 count=15583lb # [ 89.857572] niks3-server[1047]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5584lb # [ 89.868822] niks3-server[1047]: 2026/09/20 09:34:41 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5585worker1 # [ 89.903237] nix-grpc-daemon[1297]: time=2026-09-20T09:34:41.910Z level=INFO msg="Upload complete. (231ms)"5586lb # [ 89.873445] niks3-server[1047]: 2026/09/20 09:34:41 INFO Completed upload id=305587lb # [ 89.876151] niks3-server[1047]: 2026/09/20 09:34:41 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5588worker1 # [ 89.906375] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53340 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15589worker1 # [ 89.912079] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:41Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05590worker1 # [ 89.914385] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:41Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15591worker1 # [ 89.918260] nix-daemon[844]: reaped child process 1356, status = succeeded5592worker1 # [ 89.957576] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533405593worker1 # [ 89.965302] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533405594worker1 # [ 89.974354] nix-grpc-daemon[1297]: time=2026-09-20T09:34:41.981Z level=INFO msg="All 2 paths already cached"5595client # copying 1 paths to 'lb:50051'...5596client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5597worker1 # [ 89.978724] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533405598worker1 # [ 89.982534] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5599worker1 # [ 90.136261] nix-daemon[844]: accepted connection from pid 1382, user nix-grpc-daemon (trusted)5600worker1 # [ 90.148132] nix-daemon[844]: reaped child process 1385, status = succeeded5601worker1 # [ 90.310297] nix-daemon[844]: accepted connection from pid 1387, user nix-grpc-daemon (trusted)5602worker1 # [ 90.321615] nix-daemon[844]: reaped child process 1390, status = succeeded5603lb # [ 90.308912] niks3-server[1047]: 2026/09/20 09:34:42 INFO Received uploads request method=POST path=/api/pending_closures5604worker1 # [ 90.347306] nix-grpc-daemon[1297]: time=2026-09-20T09:34:42.354Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5605worker1 # [ 90.348632] nix-grpc-daemon[1297]: time=2026-09-20T09:34:42.354Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5606lb # [ 90.333123] niks3-server[1047]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5607lb # [ 90.345093] niks3-server[1047]: 2026/09/20 09:34:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5608lb # [ 90.349183] niks3-server[1047]: 2026/09/20 09:34:42 INFO Signed narinfos id=31 count=15609worker1 # [ 90.382620] nix-grpc-daemon[1297]: time=2026-09-20T09:34:42.389Z level=INFO msg="Uploading 1 narinfos"5610lb # [ 90.356136] niks3-server[1047]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5611lb # [ 90.366854] niks3-server[1047]: 2026/09/20 09:34:42 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5612worker1 # [ 90.403108] nix-grpc-daemon[1297]: time=2026-09-20T09:34:42.410Z level=INFO msg="Upload complete. (414ms)"5613lb # [ 90.373243] niks3-server[1047]: 2026/09/20 09:34:42 INFO Completed upload id=315614worker1 # [ 90.404988] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53340 duration_s=0 paths=1 nar_bytes_in=7685615lb # [ 90.376819] niks3-server[1047]: 2026/09/20 09:34:42 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5616worker1 # [ 90.411331] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533405617worker1 # [ 90.413047] nix-daemon[844]: reaped child process 1380, status = succeeded5618worker1 # [ 90.415975] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5619worker1 # [ 90.422447] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5620client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5621client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5622client # farm-top> LOG-farm-top-on-standby5623worker1 # [ 91.033238] nix-daemon[844]: reaped child process 1394, status = succeeded5624worker1 # [ 91.176431] nix-daemon[844]: accepted connection from pid 1409, user nix-grpc-daemon (trusted)5625worker1 # [ 91.187851] nix-daemon[844]: reaped child process 1412, status = succeeded5626lb # [ 91.174522] niks3-server[1047]: 2026/09/20 09:34:43 INFO Received uploads request method=POST path=/api/pending_closures5627worker1 # [ 91.208812] nix-grpc-daemon[1297]: time=2026-09-20T09:34:43.215Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5628worker1 # [ 91.210285] nix-grpc-daemon[1297]: time=2026-09-20T09:34:43.217Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"5629lb # [ 91.194188] niks3-server[1047]: 2026/09/20 09:34:43 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5630lb # [ 91.197677] niks3-server[1047]: 2026/09/20 09:34:43 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst5631lb # [ 91.200401] niks3-server[1047]: 2026/09/20 09:34:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5632lb # [ 91.202796] niks3-server[1047]: 2026/09/20 09:34:43 INFO Signed narinfos id=32 count=15633worker1 # [ 91.232889] nix-grpc-daemon[1297]: time=2026-09-20T09:34:43.240Z level=INFO msg="Uploading 1 narinfos"5634lb # [ 91.206163] niks3-server[1047]: 2026/09/20 09:34:43 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls5635lb # [ 91.210316] niks3-server[1047]: 2026/09/20 09:34:43 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5636lb # [ 91.212995] niks3-server[1047]: 2026/09/20 09:34:43 INFO Completed upload id=325637worker1 # [ 91.242857] nix-grpc-daemon[1297]: time=2026-09-20T09:34:43.250Z level=INFO msg="Upload complete. (213ms)"5638lb # [ 91.215708] niks3-server[1047]: 2026/09/20 09:34:43 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo5639worker1 # [ 91.245644] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53340 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=15640worker1 # [ 91.252065] nix-daemon[844]: reaped child process 1392, status = succeeded5641client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby >&2, in 6.08 seconds)5642client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath5643client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag on-standby outPath, in 0.21 seconds)5644worker2 # [ 91.553972] systemd[1]: Listening on Nix worker-protocol over gRPC.5645worker2 # [ 91.558261] systemd[1]: Starting Nix worker-protocol over gRPC...5646worker2 # [ 91.699397] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5647worker2 # [ 91.703297] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15648worker2 # [ 91.709937] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5649worker2 # [ 91.712660] systemd[1]: Started Nix worker-protocol over gRPC.5650lb: must succeed: curl -sf localhost:9901/clusters5651worker2 # [ 91.741697] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:43Z level=info event=scheduler_connected addr=lb:500515652worker1 # [ 91.745797] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05653lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5654worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5655worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5656lb: must succeed: curl -sf localhost:9901/clusters5657lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5658lb: must succeed: curl -sf localhost:9901/clusters5659lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5660worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5661worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5662(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.17 seconds)5663subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5664client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs5665client # [ 92.079421] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs.5666client # Running as unit: rs.service; invocation ID: 688a8524c49548458b2bd47f670919025667client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag rs, in 0.07 seconds)5668worker1 # [ 92.287884] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:533405669client # [ 92.287278] nix[1176]: don't know how to build these paths:5670client # [ 92.288269] nix[1176]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5671worker1 # [ 92.301398] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:44Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=05672worker1 # [ 92.306621] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:44Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=15673worker1 # [ 92.345336] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533405674worker1 # [ 92.357123] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:533405675client # [ 92.360226] nix[1176]: copying 1 paths to 'lb:50051'...5676worker1 # [ 92.368142] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:533405677client # [ 92.362292] nix[1176]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...5678worker1 # [ 92.378685] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5679worker1 # [ 92.563463] nix-daemon[844]: accepted connection from pid 1427, user nix-grpc-daemon (trusted)5680worker1 # [ 92.575529] nix-daemon[844]: reaped child process 1430, status = succeeded5681worker1 # [ 92.744476] nix-daemon[844]: accepted connection from pid 1432, user nix-grpc-daemon (trusted)5682worker1 # [ 92.755728] nix-daemon[844]: reaped child process 1435, status = succeeded5683lb # [ 92.743588] niks3-server[1047]: 2026/09/20 09:34:44 INFO Received uploads request method=POST path=/api/pending_closures5684worker1 # [ 92.779521] nix-grpc-daemon[1297]: time=2026-09-20T09:34:44.786Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5685worker1 # [ 92.781085] nix-grpc-daemon[1297]: time=2026-09-20T09:34:44.786Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"5686lb # [ 92.756540] niks3-server[1047]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst5687lb # [ 92.760726] niks3-server[1047]: 2026/09/20 09:34:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5688lb # [ 92.763404] niks3-server[1047]: 2026/09/20 09:34:44 INFO Signed narinfos id=33 count=15689worker1 # [ 92.793381] nix-grpc-daemon[1297]: time=2026-09-20T09:34:44.800Z level=INFO msg="Uploading 1 narinfos"5690lb # [ 92.766803] niks3-server[1047]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls5691lb # [ 92.770391] niks3-server[1047]: 2026/09/20 09:34:44 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5692lb # [ 92.773924] niks3-server[1047]: 2026/09/20 09:34:44 INFO Completed upload id=335693worker1 # [ 92.803979] nix-grpc-daemon[1297]: time=2026-09-20T09:34:44.811Z level=INFO msg="Upload complete. (411ms)"5694lb # [ 92.775698] niks3-server[1047]: 2026/09/20 09:34:44 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo5695worker1 # [ 92.806219] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53340 duration_s=0 paths=1 nar_bytes_in=4165696worker1 # [ 92.810073] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:533405697worker1 # [ 92.815529] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5698worker1 # [ 92.817802] nix-daemon[844]: reaped child process 1425, status = succeeded5699worker1 # [ 92.822901] nix-daemon[844]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5700client # [ 92.815313] nix[1176]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...5701lb: must succeed: curl -sf localhost:9901/clusters5702lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5703lb: must succeed: curl -sf localhost:9901/clusters5704lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5705lb: must succeed: curl -sf localhost:9901/clusters5706lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5707lb: must succeed: curl -sf localhost:9901/clusters5708lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5709worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5710worker2 # [ 94.860519] systemd-journald[289]: Received client request to rotate journal, rotating.5711worker2 # [ 94.879503] systemd-journald[289]: Vacuuming done, freed 0B of archived journals from /var/log/journal/9a634e25404d49cbad84007d85bce902.5712worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)5713worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5714worker1 # [ 94.800759] systemd[1]: Stopping Nix worker-protocol over gRPC...5715worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5716client: must succeed: sleep 5.05717worker1 # [ 95.783136] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:47Z level=info event=draining5718lb # [ 99.138958] envoy[928]: conn peer=[2001:db8:1::1]:59550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9564 tx=3862 ms=58425719client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5720worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5721worker1 # [ 100.253382] nix-daemon[844]: reaped child process 1439, status = succeeded5722worker1 # [ 100.411107] nix-daemon[844]: accepted connection from pid 1479, user nix-grpc-daemon (trusted)5723worker1 # [ 100.422207] nix-daemon[844]: reaped child process 1482, status = succeeded5724lb # [ 100.406716] niks3-server[1047]: 2026/09/20 09:34:52 INFO Received uploads request method=POST path=/api/pending_closures5725worker1 # [ 100.446946] nix-grpc-daemon[1297]: time=2026-09-20T09:34:52.454Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5726worker1 # [ 100.452045] nix-grpc-daemon[1297]: time=2026-09-20T09:34:52.455Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"5727lb # [ 100.450669] niks3-server[1047]: 2026/09/20 09:34:52 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5728lb # [ 100.460981] niks3-server[1047]: 2026/09/20 09:34:52 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv5729lb # [ 100.464825] niks3-server[1047]: 2026/09/20 09:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5730lb # [ 100.469320] niks3-server[1047]: 2026/09/20 09:34:52 INFO Signed narinfos id=34 count=15731worker1 # [ 100.501803] nix-grpc-daemon[1297]: time=2026-09-20T09:34:52.508Z level=INFO msg="Uploading 1 narinfos"5732lb # [ 100.478440] niks3-server[1047]: 2026/09/20 09:34:52 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls5733lb # [ 100.482227] niks3-server[1047]: 2026/09/20 09:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5734lb # [ 100.486879] niks3-server[1047]: 2026/09/20 09:34:52 INFO Completed upload id=345735worker1 # [ 100.517056] nix-grpc-daemon[1297]: time=2026-09-20T09:34:52.524Z level=INFO msg="Upload complete. (281ms)"5736lb # [ 100.488727] niks3-server[1047]: 2026/09/20 09:34:52 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo5737worker1 # [ 100.525388] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53340 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=15738worker1 # [ 100.535392] nix-daemon[844]: reaped child process 1437, status = succeeded5739client # [ 100.554593] systemd[1]: rs.service: Deactivated successfully.5740client # [ 100.558287] systemd[1]: rs.service: Consumed 110ms CPU time over 8.477s wall clock time, 37.1M memory peak, 8.4K incoming IP traffic, 9.1K outgoing IP traffic.5741worker1 # [ 100.784661] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:52Z level=info event=signal_exit5742worker1 # [ 101.002806] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_yield5743lb # [ 100.975239] niks3-server[1047]: 2026/09/20 09:34:53 INFO lead: released remote=192.168.1.3:531025744worker2 # [ 101.003966] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_restarting addr=lb:500515745worker1 # [ 101.010790] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:533285746worker1 # [ 101.015384] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=worker_gone addr=192.168.1.4:500515747worker1 # [ 101.018404] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_restarting addr=lb:500515748worker1 # [ 101.021407] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:533285749worker1 # [ 101.025455] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=worker_gone addr=192.168.1.3:500515750worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.09 seconds)5751client: waiting for success: ! systemctl is-active rs5752worker2 # [ 101.203793] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_connected addr=lb:500515753worker2 # [ 101.207954] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5754worker1 # [ 101.211862] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_connected addr=lb:500515755worker1 # [ 101.215563] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5756client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5757client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5758client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5759client: must fail: journalctl -u rs | grep -q 'reconnecting'5760client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)5761lb: must succeed: curl -sf localhost:9901/clusters5762lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5763worker2 # [ 101.409201] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_connected addr=lb:500515764worker2 # [ 101.413332] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5765worker1 # [ 101.416466] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_connected addr=lb:500515766worker1 # [ 101.420307] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5767worker2 # [ 101.614424] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_connected addr=lb:500515768worker2 # [ 101.618527] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5769worker1 # [ 101.622307] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_connected addr=lb:500515770worker1 # [ 101.625444] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5771lb # [ 101.692969] niks3-server[1047]: 2026/09/20 09:34:53 INFO lead: acquired remote=192.168.1.4:361665772worker2 # [ 101.721419] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_take_over5773worker1 # [ 101.822305] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_connected addr=lb:500515774worker2 # [ 101.819331] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_connected addr=lb:500515775worker1 # [ 101.823953] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5776worker2 # [ 101.823538] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5777worker1 # [ 102.006941] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:54Z level=info event=scheduler_restarting5778worker1 # [ 102.020054] nix-daemon[844]: reaped child process 1288, status = succeeded5779worker1 # [ 102.029408] nix-grpc-daemon[1218]: ts=2026-09-20T09:34:54Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5780worker2 # [ 102.026336] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:54Z level=info event=scheduler_connected addr=lb:500515781worker2 # [ 102.032579] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05782lb: must succeed: curl -sf localhost:9901/clusters5783lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5784worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5785worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5786worker1 # [ 102.815934] nix-grpc-daemon[1218]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5787worker1 # [ 102.817877] nix-grpc-daemon[1218]: E0000 00:00:1789896894.822601 1261 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5788worker1 # [ 102.820686] nix-grpc-daemon[1218]: E0000 00:00:1789896894.827922 1261 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5789lb: must succeed: curl -sf localhost:9901/clusters5790lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5791worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5792worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.07 seconds)5793worker1 # [ 103.891953] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5794worker1 # [ 103.895822] systemd[1]: Stopped Nix worker-protocol over gRPC.5795worker1 # [ 103.897895] systemd[1]: nix-grpc-daemon.service: Consumed 1.213s CPU time over 24.306s wall clock time, 35.8M memory peak, 47.3K incoming IP traffic, 91K outgoing IP traffic.5796worker1 # [ 103.913527] systemd[1]: Starting Nix worker-protocol over gRPC...5797worker1 # [ 104.078434] nix-grpc-daemon[1489]: ts=2026-09-20T09:34:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5798worker1 # [ 104.082321] nix-grpc-daemon[1489]: ts=2026-09-20T09:34:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15799worker1 # [ 104.087723] nix-grpc-daemon[1489]: ts=2026-09-20T09:34:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5800worker1 # [ 104.090491] systemd[1]: Started Nix worker-protocol over gRPC.5801worker1 # [ 104.111593] nix-grpc-daemon[1489]: ts=2026-09-20T09:34:56Z level=info event=scheduler_connected addr=lb:500515802worker2 # [ 104.111462] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05803lb: must succeed: curl -sf localhost:9901/clusters5804lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5805worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5806worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)5807(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.71 seconds)5808subtest: scheduler restart mid-build: build finishes, no double build5809client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid5810client # [ 104.785534] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid.5811client # Running as unit: mid.service; invocation ID: 5760d30070144b07a1d8a6f3791666475812client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag mid, in 0.07 seconds)5813worker2 # [ 105.005616] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333325814worker2 # [ 105.018069] nix-daemon[848]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5815client # [ 105.026224] nix[1212]: don't know how to build these paths:5816client # [ 105.027075] nix[1212]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5817worker2 # [ 105.038382] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:57Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05818worker2 # [ 105.042231] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:57Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15819worker2 # [ 105.087208] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333325820worker2 # [ 105.099486] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)5821worker2 # [ 105.132196] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 2/5)5822worker2 # [ 105.178664] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 3/5)5823worker2 # [ 105.269825] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 456 ms (attempt 4/5)5824worker2 # [ 105.732159] nix-grpc-daemon[1177]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5825worker2 # [ 105.752594] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333325826client # [ 105.758507] nix[1212]: copying 1 paths to 'lb:50051'...5827client # [ 105.762135] nix[1212]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...5828worker2 # [ 105.767408] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333325829worker2 # [ 105.778159] nix-daemon[848]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5830worker2 # [ 105.799275] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:57Z level=info event=niks3_push_started pid=13005831worker2 # [ 106.160567] nix-daemon[848]: accepted connection from pid 1317, user nix-grpc-daemon (trusted)5832worker2 # [ 106.174736] nix-daemon[848]: reaped child process 1320, status = succeeded5833worker2 # [ 106.344137] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5834worker2 # [ 106.357635] nix-daemon[848]: reaped child process 1325, status = succeeded5835lb # [ 106.348905] niks3-server[1047]: 2026/09/20 09:34:58 INFO Received uploads request method=POST path=/api/pending_closures5836worker2 # [ 106.389120] nix-grpc-daemon[1300]: time=2026-09-20T09:34:58.463Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5837worker2 # [ 106.390706] nix-grpc-daemon[1300]: time=2026-09-20T09:34:58.463Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"5838lb # [ 106.386530] niks3-server[1047]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst5839lb # [ 106.397801] niks3-server[1047]: 2026/09/20 09:34:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5840lb # [ 106.406484] niks3-server[1047]: 2026/09/20 09:34:58 INFO Signed narinfos id=35 count=15841worker2 # [ 106.437463] nix-grpc-daemon[1300]: time=2026-09-20T09:34:58.513Z level=INFO msg="Uploading 1 narinfos"5842lb # [ 106.429135] niks3-server[1047]: 2026/09/20 09:34:58 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5843lb # [ 106.435695] niks3-server[1047]: 2026/09/20 09:34:58 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls5844lb # [ 106.439364] niks3-server[1047]: 2026/09/20 09:34:58 INFO Completed upload id=355845worker2 # [ 106.466983] nix-grpc-daemon[1300]: time=2026-09-20T09:34:58.543Z level=INFO msg="Upload complete. (466ms)"5846worker2 # [ 106.474692] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33332 duration_s=0 paths=1 nar_bytes_in=4245847worker2 # [ 106.480542] nix-grpc-daemon[1300]: time=2026-09-20T09:34:58.547Z level=WARN msg="Failed to register uploaded object" key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo error="server returned 404: object is not pending upload\n"5848worker2 # [ 106.485744] nix-grpc-daemon[1177]: ts=2026-09-20T09:34:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333325849worker2 # [ 106.496119] nix-daemon[848]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5850worker2 # [ 106.500426] nix-daemon[848]: reaped child process 1298, status = succeeded5851worker2 # [ 106.505846] nix-daemon[848]: accepted connection from pid 1177, user nix-grpc-daemon (trusted)5852client # [ 106.501517] nix[1212]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5853lb: must succeed: curl -sf localhost:9901/clusters5854lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5855lb: must succeed: curl -sf localhost:9901/clusters5856lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5857worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5858worker2 # [ 108.544317] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1177 (nix-grpc-daemon) on client request.5859worker2 # [ 108.546305] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5860lb # [ 108.525665] niks3-server[1047]: 2026/09/20 09:35:00 INFO lead: released remote=192.168.1.4:361665861client # [ 108.555690] nix[1212]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5862worker1 # [ 108.562233] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5863worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)5864client: must succeed: sleep 5.05865client # [ 108.559623] nix[1212]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5866worker2 # [ 108.574617] nix-daemon[848]: reaped child process 1327, status = succeeded5867worker2 # [ 108.575836] nix-daemon[848]: reaped child process 1291, status = succeeded5868worker2 # [ 108.580108] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5869worker2 # [ 108.581657] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5870worker2 # [ 108.582946] systemd[1]: nix-grpc-daemon.service: Consumed 462ms CPU time over 17.021s wall clock time, 33M memory peak, 14.8K incoming IP traffic, 22.6K outgoing IP traffic.5871worker2 # [ 108.591524] nix-daemon[1329]: unexpected Nix daemon error: error: interrupted by the user5872worker2 # [ 108.596168] nix-daemon[848]: reaped child process 1329, status = failed with exit code 15873worker2 # [ 108.742563] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5874worker2 # [ 108.750792] systemd[1]: Starting Nix worker-protocol over gRPC...5875worker1 # [ 108.762721] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:00Z level=info event=scheduler_connected addr=lb:500515876worker2 # [ 108.945561] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5877worker2 # [ 108.947895] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15878worker2 # [ 108.953711] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5879worker2 # [ 108.956940] systemd[1]: Started Nix worker-protocol over gRPC.5880worker2 # [ 108.966169] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=info event=scheduler_take_over5881lb # [ 108.938195] niks3-server[1047]: 2026/09/20 09:35:00 INFO lead: acquired remote=192.168.1.4:504125882worker1 # [ 108.972404] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:00Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5883worker2 # [ 108.984152] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=info event=scheduler_connected addr=lb:500515884worker2 # [ 108.985869] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05885worker2 # [ 109.069079] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=05886worker2 # [ 109.074729] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=15887worker2 # [ 109.117149] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333545888worker2 # [ 109.128097] nix-daemon[848]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5889worker2 # [ 109.141609] nix-daemon[848]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5890client # [ 109.149123] nix[1212]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...5891worker2 # [ 109.154971] nix-daemon[848]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5892lb # [ 109.138707] envoy[928]: conn peer=[2001:db8:1::1]:59564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2947 tx=1324 ms=82495893lb # [ 109.142445] envoy[928]: conn peer=[2001:db8:1::3]:40368 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5767 tx=1343 ms=222255894worker1 # [ 109.173359] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:01Z level=info event=scheduler_connected addr=lb:500515895worker2 # [ 109.174433] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05896lb # [ 109.146957] envoy[928]: conn peer=[2001:db8:1::4]:47978 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1560 tx=483 ms=168225897client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5898client: waiting for success: ! systemctl is-active mid5899worker2 # [ 113.974183] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:06Z level=info event=niks3_push_started pid=14445900worker2 # [ 113.986421] nix-daemon[848]: reaped child process 1420, status = succeeded5901worker2 # [ 114.289079] nix-daemon[848]: accepted connection from pid 1457, user nix-grpc-daemon (trusted)5902worker2 # [ 114.299273] nix-daemon[848]: reaped child process 1460, status = succeeded5903lb # [ 114.289751] niks3-server[1047]: 2026/09/20 09:35:06 INFO Received uploads request method=POST path=/api/pending_closures5904worker2 # [ 114.325262] nix-grpc-daemon[1444]: time=2026-09-20T09:35:06.401Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5905worker2 # [ 114.327249] nix-grpc-daemon[1444]: time=2026-09-20T09:35:06.402Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"5906lb # [ 114.332952] niks3-server[1047]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5907lb # [ 114.344135] niks3-server[1047]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv5908lb # [ 114.351509] niks3-server[1047]: 2026/09/20 09:35:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5909lb # [ 114.356061] niks3-server[1047]: 2026/09/20 09:35:06 INFO Signed narinfos id=36 count=15910worker2 # [ 114.386546] nix-grpc-daemon[1444]: time=2026-09-20T09:35:06.462Z level=INFO msg="Uploading 1 narinfos"5911lb # [ 114.363473] niks3-server[1047]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls5912lb # [ 114.371853] niks3-server[1047]: 2026/09/20 09:35:06 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5913lb # [ 114.380331] niks3-server[1047]: 2026/09/20 09:35:06 INFO Completed upload id=365914worker2 # [ 114.407940] nix-grpc-daemon[1444]: time=2026-09-20T09:35:06.484Z level=INFO msg="Upload complete. (266ms)"5915lb # [ 114.382176] niks3-server[1047]: 2026/09/20 09:35:06 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo5916worker2 # [ 114.415138] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33354 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=15917worker2 # [ 114.428857] nix-daemon[848]: reaped child process 1418, status = succeeded5918client # [ 114.446788] systemd[1]: mid.service: Deactivated successfully.5919client # [ 114.448448] systemd[1]: mid.service: Consumed 120ms CPU time over 9.664s wall clock time, 39.3M memory peak, 9.6K incoming IP traffic, 10.8K outgoing IP traffic.5920client: (finished: waiting for success: ! systemctl is-active mid, in 1.09 seconds)5921client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5922client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5923(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.24 seconds)5924subtest: interrupting the client stops the build on the worker5925client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr5926client # [ 115.002592] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr.5927client # Running as unit: intr.service; invocation ID: 2019d44f85cc4f61bdc222e345641bda5928client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag intr, in 0.04 seconds)5929client # [ 115.194820] nix[1248]: don't know how to build these paths:5930client # [ 115.195802] nix[1248]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5931worker2 # [ 115.198616] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333545932worker2 # [ 115.209677] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:07Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=05933worker2 # [ 115.213734] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:07Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.3:50051 clients=15934worker1 # [ 115.265619] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546485935worker1 # [ 115.276336] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5936worker1 # [ 115.292919] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5)5937worker1 # [ 115.381870] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 2/5)5938worker1 # [ 115.418565] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 3/5)5939worker1 # [ 115.433227] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 116 ms (attempt 4/5)5940worker1 # [ 115.555366] nix-grpc-daemon[1489]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5941worker1 # [ 115.570188] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546485942client # [ 115.572254] nix[1248]: copying 1 paths to 'lb:50051'...5943worker1 # [ 115.579995] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546485944client # [ 115.574530] nix[1248]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...5945worker1 # [ 115.592129] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5946worker1 # [ 115.611208] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:07Z level=info event=niks3_push_started pid=15775947worker1 # [ 115.914038] nix-daemon[844]: accepted connection from pid 1590, user nix-grpc-daemon (trusted)5948worker1 # [ 115.924291] nix-daemon[844]: reaped child process 1593, status = succeeded5949worker1 # [ 116.069316] nix-daemon[844]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)5950worker1 # [ 116.080563] nix-daemon[844]: reaped child process 1598, status = succeeded5951lb # [ 116.064467] niks3-server[1047]: 2026/09/20 09:35:08 INFO Received uploads request method=POST path=/api/pending_closures5952worker1 # [ 116.102310] nix-grpc-daemon[1577]: time=2026-09-20T09:35:08.109Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5953worker1 # [ 116.105341] nix-grpc-daemon[1577]: time=2026-09-20T09:35:08.111Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"5954lb # [ 116.097847] niks3-server[1047]: 2026/09/20 09:35:08 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst5955lb # [ 116.104555] niks3-server[1047]: 2026/09/20 09:35:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5956lb # [ 116.109255] niks3-server[1047]: 2026/09/20 09:35:08 INFO Signed narinfos id=37 count=15957worker1 # [ 116.142162] nix-grpc-daemon[1577]: time=2026-09-20T09:35:08.149Z level=INFO msg="Uploading 1 narinfos"5958lb # [ 116.122731] niks3-server[1047]: 2026/09/20 09:35:08 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls5959lb # [ 116.128858] niks3-server[1047]: 2026/09/20 09:35:08 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5960lb # [ 116.133327] niks3-server[1047]: 2026/09/20 09:35:08 INFO Completed upload id=375961worker1 # [ 116.166438] nix-grpc-daemon[1577]: time=2026-09-20T09:35:08.172Z level=INFO msg="Upload complete. (392ms)"5962worker1 # [ 116.167672] nix-grpc-daemon[1577]: time=2026-09-20T09:35:08.173Z level=WARN msg="Failed to register uploaded object" key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo error="server returned 404: object is not pending upload\n"5963worker1 # [ 116.171046] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=4245964worker1 # [ 116.172894] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546485965worker1 # [ 116.179082] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5966worker1 # [ 116.188914] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5967client # [ 116.180701] nix[1248]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...5968worker1 # [ 116.191440] nix-daemon[844]: reaped child process 1575, status = succeeded5969client: must succeed: systemctl kill -s INT intr5970client # [ 117.415545] systemd[1]: intr.service: Sent signal SIGINT to main process 1248 (nix) on client request.5971worker1 # [ 117.428832] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:09Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv5972client # [ 117.422643] nix[1248]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5973client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)5974client # [ 117.429858] nix[1248]: error: interrupted by the user5975client # [ 117.462139] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5976client # [ 117.463537] systemd[1]: intr.service: Failed with result 'exit-code'.5977client # [ 117.465564] systemd[1]: intr.service: Consumed 111ms CPU time over 2.462s wall clock time, 39.3M memory peak, 7.6K incoming IP traffic, 8.9K outgoing IP traffic.5978worker1 # [ 117.607362] nix-daemon[844]: reaped child process 1604, status = succeeded5979worker1 # [ 117.612312] nix-daemon[1606]: unexpected Nix daemon error: error: interrupted by the user5980worker1 # [ 117.615777] nix-daemon[844]: reaped child process 1606, status = failed with exit code 15981(finished: subtest: interrupting the client stops the build on the worker, in 3.70 seconds)5982subtest: a deploy mid-build drains: the build finishes, the new generation takes over5983client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw5984client # [ 118.724877] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key" --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw.5985client # Running as unit: sw.service; invocation ID: c6d123faf5fb4a95bfdeb77f0e525fa45986client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/yggfr0llgkhpsm4k92sg65ggxk04755p-slow.nix --argstr tag sw, in 0.07 seconds)5987worker2 # [ 118.938908] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:333545988client # [ 118.940198] nix[1268]: don't know how to build these paths:5989client # [ 118.941135] nix[1268]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv5990worker2 # [ 118.953638] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:11Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=05991worker2 # [ 118.959243] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:11Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.4:50051 clients=15992worker2 # [ 118.997887] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333545993worker2 # [ 119.010277] nix-grpc-daemon[1365]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)5994worker2 # [ 119.061270] nix-grpc-daemon[1365]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 2/5)5995lb # [ 119.138476] envoy[928]: conn peer=[2001:db8:1::1]:51090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3835 tx=1662 ms=94345996lb # [ 119.142466] envoy[928]: conn peer=[2001:db8:1::1]:40648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2950 tx=729 ms=22525997worker2 # [ 119.221179] nix-grpc-daemon[1365]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 3/5)5998worker2 # [ 119.411318] nix-grpc-daemon[1365]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 107 ms (attempt 4/5)5999worker2 # [ 119.524532] nix-grpc-daemon[1365]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6000worker2 # [ 119.541748] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:333546001client # [ 119.546892] nix[1268]: copying 1 paths to 'lb:50051'...6002client # [ 119.550859] nix[1268]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6003worker2 # [ 119.556495] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:333546004worker2 # [ 119.567532] nix-daemon[848]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)6005worker2 # [ 119.730455] nix-daemon[848]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6006worker2 # [ 119.741785] nix-daemon[848]: reaped child process 1498, status = succeeded6007worker2 # [ 119.893442] nix-daemon[848]: accepted connection from pid 1500, user nix-grpc-daemon (trusted)6008worker2 # [ 119.904562] nix-daemon[848]: reaped child process 1503, status = succeeded6009lb # [ 119.894395] niks3-server[1047]: 2026/09/20 09:35:11 INFO Received uploads request method=POST path=/api/pending_closures6010worker2 # [ 119.927212] nix-grpc-daemon[1444]: time=2026-09-20T09:35:12.003Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6011worker2 # [ 119.928738] nix-grpc-daemon[1444]: time=2026-09-20T09:35:12.004Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6012lb # [ 119.911611] niks3-server[1047]: 2026/09/20 09:35:11 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6013lb # [ 119.914791] niks3-server[1047]: 2026/09/20 09:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6014lb # [ 119.916884] niks3-server[1047]: 2026/09/20 09:35:11 INFO Signed narinfos id=38 count=16015worker2 # [ 119.944980] nix-grpc-daemon[1444]: time=2026-09-20T09:35:12.020Z level=INFO msg="Uploading 1 narinfos"6016lb # [ 119.921573] niks3-server[1047]: 2026/09/20 09:35:11 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6017lb # [ 119.924073] niks3-server[1047]: 2026/09/20 09:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6018lb # [ 119.927130] niks3-server[1047]: 2026/09/20 09:35:11 INFO Completed upload id=386019worker2 # [ 119.954396] nix-grpc-daemon[1444]: time=2026-09-20T09:35:12.030Z level=INFO msg="Upload complete. (375ms)"6020lb # [ 119.929655] niks3-server[1047]: 2026/09/20 09:35:11 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6021worker2 # [ 119.956473] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33354 duration_s=0 paths=1 nar_bytes_in=4166022worker2 # [ 119.960204] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:333546023worker2 # [ 119.972026] nix-daemon[848]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)6024worker2 # [ 119.973147] nix-daemon[848]: reaped child process 1493, status = succeeded6025client # [ 119.975223] nix[1268]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6026worker2 # [ 119.980783] nix-daemon[848]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)6027worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26028worker2 # Checking switch inhibitors... done6029worker2 # [ 121.901510] nixos[1538]: switching to system configuration /nix/store/sk820hc3azag8pgag8zqa4r8rvx2cp9w-nixos-system-worker2-test6030worker2 # [ 121.905614] systemd[1]: Stopped target Local File Systems.6031worker2 # activating the configuration...6032worker2 # [ 121.907389] systemd[1]: Stopped target Remote File Systems.6033worker2 # setting up /etc...6034worker2 # [ 122.526570] systemd[1]: Reload requested from client PID 1538 ('.switch-to-conf') (unit backdoor.service)...6035worker2 # [ 122.528533] systemd[1]: Reloading...6036worker2 # [ 123.208839] systemd[1]: Reloading finished in 679 ms.6037worker2 # restarting sysinit-reactivation.target6038worker2 # [ 123.264410] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6039worker2 # [ 123.397275] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6040worker2 # [ 123.399159] systemd[1]: Reached target Reactivate sysinit units.6041worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6042worker2 # [ 123.404028] systemd[1]: Reloading D-Bus System Message Bus...6043worker2 # [ 123.406625] dbus-broker-launch[535]: Caught SIGHUP, trigger reload.6044worker2 # [ 123.409395] systemd[1]: Reloading Nix worker-protocol over gRPC...6045worker2 # [ 123.440616] dbus-broker-launch[535]: Looking up NSS user entry for 'systemd-timesync'...6046worker2 # [ 123.444146] dbus-broker-launch[535]: NSS returned no entry for 'systemd-timesync'6047worker2 # [ 123.445150] dbus-broker-launch[535]: Invalid user-name in /nix/store/nl02z4vnl25dc1df60lqifjnw6c89zpg-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6048worker2 # [ 123.454221] systemd[1]: Reloaded Nix worker-protocol over gRPC.6049worker2 # [ 123.476357] systemd[1]: Reloaded D-Bus System Message Bus.6050worker2 # [ 123.482911] systemd[1]: Reached target Local File Systems.6051worker2 # [ 123.484392] systemd[1]: Reached target Remote File Systems.6052worker2 # [ 123.491030] systemd[1]: Starting linger-users.service...6053worker2 # [ 123.494825] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6054worker2 # [ 123.499297] systemd[1]: Starting Create SUID/SGID Wrappers...6055worker2 # [ 123.500433] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6056worker2 # [ 123.502904] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676057worker2 # [ 123.506124] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6058worker2 # [ 123.507561] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6059worker2 # [ 123.508721] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6060worker2 # [ 123.514926] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6061worker2 # [ 123.516254] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6062worker2 # [ 123.558787] systemd[1]: linger-users.service: Deactivated successfully.6063worker2 # [ 123.560298] systemd[1]: Finished linger-users.service.6064worker2 # [ 123.903172] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6065worker2 # [ 123.904244] systemd[1]: Finished Create SUID/SGID Wrappers.6066worker2 # [ 123.961790] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:16Z level=info event=draining6067worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6068worker2 # [ 124.159965] nixos[1538]: finished switching to system configuration /nix/store/sk820hc3azag8pgag8zqa4r8rvx2cp9w-nixos-system-worker2-test6069worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 2.95 seconds)6070worker2: must succeed: pgrep -f 'read -t [9]0 x'6071worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6072client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&26073client # don't know how to build these paths:6074client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6075worker1 # [ 124.453115] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:546486076worker1 # [ 124.459215] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486077worker2 # [ 124.468765] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:16Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06078worker2 # [ 124.470603] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:16Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.3:50051 clients=16079worker1 # [ 124.515061] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486080worker1 # [ 124.525139] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486081client # copying 1 paths to 'lb:50051'...6082worker1 # [ 124.535605] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486083client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6084worker1 # [ 124.556765] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6085worker1 # [ 124.716466] nix-daemon[844]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6086worker1 # [ 124.728467] nix-daemon[844]: reaped child process 1655, status = succeeded6087worker1 # [ 124.856553] nix-daemon[844]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6088worker1 # [ 124.868371] nix-daemon[844]: reaped child process 1660, status = succeeded6089lb # [ 124.856386] niks3-server[1047]: 2026/09/20 09:35:16 INFO Received uploads request method=POST path=/api/pending_closures6090worker1 # [ 124.895249] nix-grpc-daemon[1577]: time=2026-09-20T09:35:16.902Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6091worker1 # [ 124.896837] nix-grpc-daemon[1577]: time=2026-09-20T09:35:16.904Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6092lb # [ 124.889556] niks3-server[1047]: 2026/09/20 09:35:16 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6093lb # [ 124.899581] niks3-server[1047]: 2026/09/20 09:35:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6094worker1 # [ 124.935809] nix-grpc-daemon[1577]: time=2026-09-20T09:35:16.942Z level=INFO msg="Uploading 1 narinfos"6095lb # [ 124.905604] niks3-server[1047]: 2026/09/20 09:35:16 INFO Signed narinfos id=39 count=16096lb # [ 124.916749] niks3-server[1047]: 2026/09/20 09:35:16 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6097lb # [ 124.923544] niks3-server[1047]: 2026/09/20 09:35:16 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6098worker1 # [ 124.960617] nix-grpc-daemon[1577]: time=2026-09-20T09:35:16.967Z level=INFO msg="Upload complete. (392ms)"6099lb # [ 124.931141] niks3-server[1047]: 2026/09/20 09:35:16 INFO Completed upload id=396100worker1 # [ 124.962806] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=5286101lb # [ 124.934254] niks3-server[1047]: 2026/09/20 09:35:16 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6102worker1 # [ 124.968773] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486103worker1 # [ 124.978879] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6104worker1 # [ 124.982229] nix-daemon[844]: reaped child process 1650, status = succeeded6105worker1 # [ 124.987239] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6106client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6107client # farm-a> node-a: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6108client # farm-a> LOG-farm-a-during-drain6109worker1 # [ 126.407903] nix-daemon[844]: reaped child process 1664, status = succeeded6110worker1 # [ 126.554321] nix-daemon[844]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6111worker1 # [ 126.565860] nix-daemon[844]: reaped child process 1682, status = succeeded6112lb # [ 126.552950] niks3-server[1047]: 2026/09/20 09:35:18 INFO Received uploads request method=POST path=/api/pending_closures6113worker1 # [ 126.587256] nix-grpc-daemon[1577]: time=2026-09-20T09:35:18.594Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6114worker1 # [ 126.588733] nix-grpc-daemon[1577]: time=2026-09-20T09:35:18.596Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6115lb # [ 126.595116] niks3-server[1047]: 2026/09/20 09:35:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6116lb # [ 126.599078] niks3-server[1047]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6117lb # [ 126.606771] niks3-server[1047]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6118worker1 # [ 126.641393] nix-grpc-daemon[1577]: time=2026-09-20T09:35:18.648Z level=INFO msg="Uploading 1 narinfos"6119lb # [ 126.611331] niks3-server[1047]: 2026/09/20 09:35:18 INFO Signed narinfos id=40 count=16120lb # [ 126.627333] niks3-server[1047]: 2026/09/20 09:35:18 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6121lb # [ 126.632527] niks3-server[1047]: 2026/09/20 09:35:18 INFO Completed upload id=406122worker1 # [ 126.662540] nix-grpc-daemon[1577]: time=2026-09-20T09:35:18.669Z level=INFO msg="Upload complete. (267ms)"6123lb # [ 126.633813] niks3-server[1047]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6124lb # [ 126.638478] niks3-server[1047]: 2026/09/20 09:35:18 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6125worker1 # [ 126.668402] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16126worker2 # [ 126.669389] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:18Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06127worker2 # [ 126.672261] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:18Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.3:50051 clients=16128worker1 # [ 126.679743] nix-daemon[844]: reaped child process 1662, status = succeeded6129worker1 # [ 126.721830] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486130worker1 # [ 126.734738] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486131worker1 # [ 126.748739] nix-grpc-daemon[1577]: time=2026-09-20T09:35:18.755Z level=INFO msg="All 1 paths already cached"6132client # copying 1 paths to 'lb:50051'...6133worker1 # [ 126.757635] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486134client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6135worker1 # [ 126.777172] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6136worker1 # [ 126.951357] nix-daemon[844]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6137worker1 # [ 126.962990] nix-daemon[844]: reaped child process 1689, status = succeeded6138worker1 # [ 127.118969] nix-daemon[844]: accepted connection from pid 1691, user nix-grpc-daemon (trusted)6139worker1 # [ 127.130439] nix-daemon[844]: reaped child process 1694, status = succeeded6140lb # [ 127.117549] niks3-server[1047]: 2026/09/20 09:35:19 INFO Received uploads request method=POST path=/api/pending_closures6141worker1 # [ 127.157758] nix-grpc-daemon[1577]: time=2026-09-20T09:35:19.164Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6142worker1 # [ 127.159172] nix-grpc-daemon[1577]: time=2026-09-20T09:35:19.164Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6143lb # [ 127.147607] niks3-server[1047]: 2026/09/20 09:35:19 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6144lb # [ 127.157076] niks3-server[1047]: 2026/09/20 09:35:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6145worker1 # [ 127.192478] nix-grpc-daemon[1577]: time=2026-09-20T09:35:19.199Z level=INFO msg="Uploading 1 narinfos"6146lb # [ 127.162948] niks3-server[1047]: 2026/09/20 09:35:19 INFO Signed narinfos id=41 count=16147lb # [ 127.169838] niks3-server[1047]: 2026/09/20 09:35:19 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6148lb # [ 127.178983] niks3-server[1047]: 2026/09/20 09:35:19 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6149lb # [ 127.187663] niks3-server[1047]: 2026/09/20 09:35:19 INFO Completed upload id=416150worker1 # [ 127.217676] nix-grpc-daemon[1577]: time=2026-09-20T09:35:19.224Z level=INFO msg="Upload complete. (428ms)"6151lb # [ 127.190682] niks3-server[1047]: 2026/09/20 09:35:19 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6152worker1 # [ 127.222153] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=6406153worker1 # [ 127.227182] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486154worker1 # [ 127.237060] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6155worker1 # [ 127.240500] nix-daemon[844]: reaped child process 1684, status = succeeded6156client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6157client # farm-b> node-a: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6158worker1 # [ 127.245549] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6159worker1 # [ 128.459179] nix-daemon[844]: reaped child process 1698, status = succeeded6160client # farm-b> LOG-farm-b-during-drain6161worker1 # [ 128.607077] nix-daemon[844]: accepted connection from pid 1713, user nix-grpc-daemon (trusted)6162worker1 # [ 128.619312] nix-daemon[844]: reaped child process 1716, status = succeeded6163lb # [ 128.604429] niks3-server[1047]: 2026/09/20 09:35:20 INFO Received uploads request method=POST path=/api/pending_closures6164worker1 # [ 128.640546] nix-grpc-daemon[1577]: time=2026-09-20T09:35:20.647Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6165worker1 # [ 128.642037] nix-grpc-daemon[1577]: time=2026-09-20T09:35:20.649Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6166lb # [ 128.628272] niks3-server[1047]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6167lb # [ 128.631600] niks3-server[1047]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6168lb # [ 128.633579] niks3-server[1047]: 2026/09/20 09:35:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6169lb # [ 128.636462] niks3-server[1047]: 2026/09/20 09:35:20 INFO Signed narinfos id=42 count=16170worker1 # [ 128.666355] nix-grpc-daemon[1577]: time=2026-09-20T09:35:20.673Z level=INFO msg="Uploading 1 narinfos"6171lb # [ 128.639645] niks3-server[1047]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6172lb # [ 128.645537] niks3-server[1047]: 2026/09/20 09:35:20 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6173lb # [ 128.649379] niks3-server[1047]: 2026/09/20 09:35:20 INFO Completed upload id=426174worker1 # [ 128.679285] nix-grpc-daemon[1577]: time=2026-09-20T09:35:20.686Z level=INFO msg="Upload complete. (226ms)"6175lb # [ 128.652067] niks3-server[1047]: 2026/09/20 09:35:20 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6176worker1 # [ 128.682225] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16177worker1 # [ 128.688607] nix-daemon[844]: reaped child process 1696, status = succeeded6178worker2 # [ 128.685795] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:20Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06179worker2 # [ 128.687615] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:20Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.3:50051 clients=16180worker1 # [ 128.733114] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486181worker1 # [ 128.744781] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486182worker1 # [ 128.758247] nix-grpc-daemon[1577]: time=2026-09-20T09:35:20.765Z level=INFO msg="All 2 paths already cached"6183client # copying 1 paths to 'lb:50051'...6184worker1 # [ 128.773289] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486185client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6186worker1 # [ 128.792765] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6187worker1 # [ 128.974050] nix-daemon[844]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6188worker1 # [ 128.987240] nix-daemon[844]: reaped child process 1723, status = succeeded6189worker1 # [ 129.152475] nix-daemon[844]: accepted connection from pid 1725, user nix-grpc-daemon (trusted)6190worker1 # [ 129.164200] nix-daemon[844]: reaped child process 1728, status = succeeded6191lb # [ 129.148162] niks3-server[1047]: 2026/09/20 09:35:21 INFO Received uploads request method=POST path=/api/pending_closures6192worker1 # [ 129.185675] nix-grpc-daemon[1577]: time=2026-09-20T09:35:21.192Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6193worker1 # [ 129.187513] nix-grpc-daemon[1577]: time=2026-09-20T09:35:21.194Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6194lb # [ 129.174747] niks3-server[1047]: 2026/09/20 09:35:21 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6195lb # [ 129.189143] niks3-server[1047]: 2026/09/20 09:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6196lb # [ 129.194212] niks3-server[1047]: 2026/09/20 09:35:21 INFO Signed narinfos id=43 count=16197worker1 # [ 129.227843] nix-grpc-daemon[1577]: time=2026-09-20T09:35:21.234Z level=INFO msg="Uploading 1 narinfos"6198lb # [ 129.200573] niks3-server[1047]: 2026/09/20 09:35:21 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6199lb # [ 129.212476] niks3-server[1047]: 2026/09/20 09:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6200lb # [ 129.220090] niks3-server[1047]: 2026/09/20 09:35:21 INFO Completed upload id=436201worker1 # [ 129.250185] nix-grpc-daemon[1577]: time=2026-09-20T09:35:21.257Z level=INFO msg="Upload complete. (442ms)"6202lb # [ 129.222541] niks3-server[1047]: 2026/09/20 09:35:21 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6203worker1 # [ 129.254172] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=7686204worker1 # [ 129.260261] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486205worker1 # [ 129.269079] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6206worker1 # [ 129.276065] nix-daemon[844]: reaped child process 1718, status = succeeded6207worker1 # [ 129.281078] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6208client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6209client # farm-top> node-a: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6210worker1 # [ 130.894884] nix-daemon[844]: reaped child process 1732, status = succeeded6211client # farm-top> LOG-farm-top-during-drain6212worker1 # [ 131.035383] nix-daemon[844]: accepted connection from pid 1747, user nix-grpc-daemon (trusted)6213worker1 # [ 131.047418] nix-daemon[844]: reaped child process 1750, status = succeeded6214lb # [ 131.032816] niks3-server[1047]: 2026/09/20 09:35:23 INFO Received uploads request method=POST path=/api/pending_closures6215worker1 # [ 131.067338] nix-grpc-daemon[1577]: time=2026-09-20T09:35:23.074Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6216worker1 # [ 131.068990] nix-grpc-daemon[1577]: time=2026-09-20T09:35:23.076Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6217lb # [ 131.053542] niks3-server[1047]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6218lb # [ 131.055580] niks3-server[1047]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6219lb # [ 131.058491] niks3-server[1047]: 2026/09/20 09:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6220lb # [ 131.060160] niks3-server[1047]: 2026/09/20 09:35:23 INFO Signed narinfos id=44 count=16221worker1 # [ 131.091577] nix-grpc-daemon[1577]: time=2026-09-20T09:35:23.098Z level=INFO msg="Uploading 1 narinfos"6222lb # [ 131.066320] niks3-server[1047]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6223lb # [ 131.068366] niks3-server[1047]: 2026/09/20 09:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6224lb # [ 131.071803] niks3-server[1047]: 2026/09/20 09:35:23 INFO Completed upload id=446225worker1 # [ 131.102946] nix-grpc-daemon[1577]: time=2026-09-20T09:35:23.110Z level=INFO msg="Upload complete. (214ms)"6226lb # [ 131.073439] niks3-server[1047]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6227worker1 # [ 131.105662] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16228worker1 # [ 131.111832] nix-daemon[844]: reaped child process 1730, status = succeeded6229client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain >&2, in 6.93 seconds)6230client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath6231client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag during-drain outPath, in 0.21 seconds)6232worker2: must succeed: pkill -f 'read -t [9]0 x'6233worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6234client: waiting for success: ! systemctl is-active sw6235worker2 # [ 131.618462] nix-daemon[848]: reaped child process 1511, status = succeeded6236worker2 # [ 131.776630] nix-daemon[848]: accepted connection from pid 1684, user nix-grpc-daemon (trusted)6237worker2 # [ 131.786859] nix-daemon[848]: reaped child process 1687, status = succeeded6238lb # [ 131.778451] niks3-server[1047]: 2026/09/20 09:35:23 INFO Received uploads request method=POST path=/api/pending_closures6239worker2 # [ 131.815103] nix-grpc-daemon[1444]: time=2026-09-20T09:35:23.891Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6240worker2 # [ 131.816842] nix-grpc-daemon[1444]: time=2026-09-20T09:35:23.892Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6241lb # [ 131.813162] niks3-server[1047]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6242lb # [ 131.825934] niks3-server[1047]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6243lb # [ 131.836898] niks3-server[1047]: 2026/09/20 09:35:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6244lb # [ 131.840634] niks3-server[1047]: 2026/09/20 09:35:23 INFO Signed narinfos id=45 count=16245worker2 # [ 131.871126] nix-grpc-daemon[1444]: time=2026-09-20T09:35:23.946Z level=INFO msg="Uploading 1 narinfos"6246lb # [ 131.847645] niks3-server[1047]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6247lb # [ 131.857519] niks3-server[1047]: 2026/09/20 09:35:23 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6248lb # [ 131.860828] niks3-server[1047]: 2026/09/20 09:35:23 INFO Completed upload id=456249worker2 # [ 131.888333] nix-grpc-daemon[1444]: time=2026-09-20T09:35:23.964Z level=INFO msg="Upload complete. (288ms)"6250lb # [ 131.862557] niks3-server[1047]: 2026/09/20 09:35:23 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6251worker2 # [ 131.894910] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33354 duration_s=11 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16252worker2 # [ 131.907941] nix-daemon[848]: reaped child process 1509, status = succeeded6253client # [ 131.928980] systemd[1]: sw.service: Deactivated successfully.6254client # [ 131.933759] systemd[1]: sw.service: Consumed 127ms CPU time over 13.207s wall clock time, 39.1M memory peak, 8.9K incoming IP traffic, 9.1K outgoing IP traffic.6255worker2 # [ 131.963395] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=signal_exit6256worker2 # [ 131.983589] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_yield6257worker2 # [ 131.987104] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:333366258worker1 # [ 131.988440] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:23Z level=info event=scheduler_restarting addr=lb:500516259lb # [ 131.959836] niks3-server[1047]: 2026/09/20 09:35:23 INFO lead: released remote=192.168.1.4:504126260worker2 # [ 131.988727] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=worker_gone addr=192.168.1.3:500516261worker2 # [ 131.990033] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_restarting addr=lb:500516262worker2 # [ 131.991625] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:333366263worker2 # [ 131.992945] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=worker_gone addr=192.168.1.4:500516264worker2 # [ 132.189406] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_connected addr=lb:500516265worker2 # [ 132.191322] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6266worker1 # [ 132.194367] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:24Z level=info event=scheduler_connected addr=lb:500516267worker1 # [ 132.197508] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6268worker2 # [ 132.391974] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_connected addr=lb:500516269worker2 # [ 132.394381] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6270worker1 # [ 132.395261] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:24Z level=info event=scheduler_connected addr=lb:500516271worker1 # [ 132.399170] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6272client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds)6273client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6274client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6275worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6276worker2 # [ 132.594932] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_connected addr=lb:500516277worker2 # [ 132.596660] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6278worker1 # [ 132.599834] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:24Z level=info event=scheduler_connected addr=lb:500516279worker1 # [ 132.603642] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6280worker2 # [ 132.797294] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_connected addr=lb:500516281worker2 # [ 132.799051] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6282worker1 # [ 132.804490] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:24Z level=info event=scheduler_connected addr=lb:500516283worker1 # [ 132.808316] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6284worker2 # [ 132.984747] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:25Z level=info event=scheduler_restarting6285worker2 # [ 132.996184] nix-daemon[848]: reaped child process 1416, status = succeeded6286worker2 # [ 133.003766] nix-grpc-daemon[1365]: ts=2026-09-20T09:35:25Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6287worker1 # [ 133.008871] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_connected addr=lb:500516288worker1 # [ 133.012720] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6289worker2 # [ 133.047098] nix-grpc-daemon[1365]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6290worker2 # [ 133.050374] nix-grpc-daemon[1365]: E0000 00:00:1789896925.122964 1408 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6291worker2 # [ 133.054968] nix-grpc-daemon[1365]: E0000 00:00:1789896925.123209 1408 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6292worker1 # [ 133.213521] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_connected addr=lb:500516293worker1 # [ 133.215297] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6294worker1 # [ 133.415784] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_connected addr=lb:500516295worker1 # [ 133.417602] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6296worker1 # [ 133.618052] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_connected addr=lb:500516297worker1 # [ 133.619930] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6298worker1 # [ 133.820903] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_connected addr=lb:500516299worker1 # [ 133.824919] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6300worker1 # [ 134.026063] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_connected addr=lb:500516301worker1 # [ 134.030136] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6302worker1 # [ 134.096787] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_take_over6303lb # [ 134.067290] niks3-server[1047]: 2026/09/20 09:35:26 INFO lead: acquired remote=192.168.1.3:575046304worker1 # [ 134.231841] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_connected addr=lb:500516305worker1 # [ 134.235580] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6306worker1 # [ 134.436475] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_connected addr=lb:500516307worker1 # [ 134.440327] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6308worker1 # [ 134.641061] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_connected addr=lb:500516309worker1 # [ 134.644908] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6310worker1 # [ 134.845855] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_connected addr=lb:500516311worker1 # [ 134.849782] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6312worker2 # [ 135.010523] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6313worker2 # [ 135.014621] systemd[1]: nix-grpc-daemon.service: Consumed 636ms CPU time over 26.429s wall clock time, 33.1M memory peak, 23.6K incoming IP traffic, 32.2K outgoing IP traffic.6314worker2 # [ 135.030489] systemd[1]: Starting Nix worker-protocol over gRPC...6315worker1 # [ 135.050540] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:27Z level=info event=scheduler_connected addr=lb:500516316worker1 # [ 135.059895] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06317worker2 # [ 135.199747] nix-grpc-daemon[1701]: ts=2026-09-20T09:35:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6318worker2 # [ 135.203569] nix-grpc-daemon[1701]: ts=2026-09-20T09:35:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16319worker2 # [ 135.209607] nix-grpc-daemon[1701]: ts=2026-09-20T09:35:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6320worker2 # [ 135.213125] systemd[1]: Started Nix worker-protocol over gRPC.6321worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.70 seconds)6322lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6323lb: must succeed: curl -sf localhost:9901/clusters6324worker2 # [ 135.236826] nix-grpc-daemon[1701]: ts=2026-09-20T09:35:27Z level=info event=scheduler_connected addr=lb:500516325worker1 # [ 135.242859] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06326lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6327lb: must succeed: curl -sf localhost:9901/clusters6328lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6329lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.11 seconds)6330lb: must succeed: curl -sf localhost:9901/clusters6331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6332worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true6333worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.04 seconds)6334(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.75 seconds)6335subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6336client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16337worker2 # [ 136.653414] nix-grpc-daemon[1701]: ts=2026-09-20T09:35:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:605506338worker2 # [ 136.658682] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6339worker2 # [ 136.665954] nix-grpc-daemon[1701]: ts=2026-09-20T09:35:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:605506340worker2 # [ 136.671542] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 1/5)6341worker2 # [ 136.704795] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 2/5)6342worker2 # [ 136.742291] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 132 ms (attempt 3/5)6343worker2 # [ 136.876889] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 687 ms (attempt 4/5)6344worker2 # [ 137.564769] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6345worker1 # [ 137.614827] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:29Z level=debug event=want client=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06346worker1 # [ 137.619919] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:29Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16347worker1 # [ 137.624707] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:29Z level=debug event=want client=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06348worker1 # [ 137.628521] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:29Z level=debug event=want client=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06349worker1 # [ 137.663462] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486350worker1 # [ 137.673833] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486351worker1 # [ 137.684476] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486352worker1 # [ 137.695296] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6353worker1 # [ 137.876137] nix-daemon[844]: accepted connection from pid 1761, user nix-grpc-daemon (trusted)6354worker1 # [ 137.887725] nix-daemon[844]: reaped child process 1764, status = succeeded6355worker1 # [ 138.056974] nix-daemon[844]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)6356worker1 # [ 138.068726] nix-daemon[844]: reaped child process 1769, status = succeeded6357lb # [ 138.056190] niks3-server[1047]: 2026/09/20 09:35:30 INFO Received uploads request method=POST path=/api/pending_closures6358worker1 # [ 138.092701] nix-grpc-daemon[1577]: time=2026-09-20T09:35:30.099Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6359worker1 # [ 138.094083] nix-grpc-daemon[1577]: time=2026-09-20T09:35:30.099Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6360lb # [ 138.072488] niks3-server[1047]: 2026/09/20 09:35:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6361lb # [ 138.074503] niks3-server[1047]: 2026/09/20 09:35:30 INFO Signed narinfos id=46 count=16362worker1 # [ 138.106487] nix-grpc-daemon[1577]: time=2026-09-20T09:35:30.113Z level=INFO msg="Uploading 1 narinfos"6363lb # [ 138.078052] niks3-server[1047]: 2026/09/20 09:35:30 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6364lb # [ 138.081592] niks3-server[1047]: 2026/09/20 09:35:30 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6365lb # [ 138.086259] niks3-server[1047]: 2026/09/20 09:35:30 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6366lb # [ 138.089831] niks3-server[1047]: 2026/09/20 09:35:30 INFO Completed upload id=466367worker1 # [ 138.119649] nix-grpc-daemon[1577]: time=2026-09-20T09:35:30.126Z level=INFO msg="Upload complete. (413ms)"6368lb # [ 138.091592] niks3-server[1047]: 2026/09/20 09:35:30 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6369worker1 # [ 138.121679] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=5046370worker1 # [ 138.125357] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486371worker1 # [ 138.131173] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6372worker1 # [ 138.133418] nix-daemon[844]: reaped child process 1759, status = succeeded6373worker1 # [ 138.139530] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6374worker1 # [ 138.951523] nix-daemon[844]: reaped child process 1773, status = succeeded6375worker1 # [ 139.085079] nix-daemon[844]: accepted connection from pid 1788, user nix-grpc-daemon (trusted)6376worker1 # [ 139.096771] nix-daemon[844]: reaped child process 1791, status = succeeded6377lb # [ 139.079344] niks3-server[1047]: 2026/09/20 09:35:31 INFO Received uploads request method=POST path=/api/pending_closures6378worker1 # [ 139.117911] nix-grpc-daemon[1577]: time=2026-09-20T09:35:31.124Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6379worker1 # [ 139.119391] nix-grpc-daemon[1577]: time=2026-09-20T09:35:31.126Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6380lb # [ 139.123421] niks3-server[1047]: 2026/09/20 09:35:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6381worker1 # [ 139.158830] nix-grpc-daemon[1577]: time=2026-09-20T09:35:31.165Z level=INFO msg="Uploading 1 narinfos"6382lb # [ 139.129125] niks3-server[1047]: 2026/09/20 09:35:31 INFO Signed narinfos id=47 count=16383lb # [ 139.138382] niks3-server[1047]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6384lb # [ 139.142911] niks3-server[1047]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6385lb # [ 139.147390] envoy[928]: conn peer=[2001:db8:1::1]:36598 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9585 tx=3820 ms=66786386lb # [ 139.150266] envoy[928]: conn peer=[2001:db8:1::1]:40650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1369 ms=129726387lb # [ 139.155085] envoy[928]: conn peer=[2001:db8:1::4]:60148 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1721 tx=539 ms=240206388lb # [ 139.159515] niks3-server[1047]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6389lb # [ 139.163458] niks3-server[1047]: 2026/09/20 09:35:31 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6390lb # [ 139.167611] niks3-server[1047]: 2026/09/20 09:35:31 INFO Completed upload id=476391worker1 # [ 139.197572] nix-grpc-daemon[1577]: time=2026-09-20T09:35:31.204Z level=INFO msg="Upload complete. (253ms)"6392lb # [ 139.169552] niks3-server[1047]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6393worker1 # [ 139.206802] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=1 outputs=16394worker1 # [ 139.218308] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:31Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16395worker1 # [ 139.221700] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:31Z level=debug event=want client=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06396worker1 # [ 139.229456] nix-daemon[844]: reaped child process 1771, status = succeeded6397worker1 # [ 139.259187] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486398worker1 # [ 139.270916] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486399worker1 # [ 139.282537] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486400worker1 # [ 139.290298] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6401worker1 # [ 139.458598] nix-daemon[844]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6402worker1 # [ 139.470225] nix-daemon[844]: reaped child process 1800, status = succeeded6403worker1 # [ 139.636103] nix-daemon[844]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6404worker1 # [ 139.647306] nix-daemon[844]: reaped child process 1805, status = succeeded6405lb # [ 139.635243] niks3-server[1047]: 2026/09/20 09:35:31 INFO Received uploads request method=POST path=/api/pending_closures6406worker1 # [ 139.673929] nix-grpc-daemon[1577]: time=2026-09-20T09:35:31.680Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6407worker1 # [ 139.675395] nix-grpc-daemon[1577]: time=2026-09-20T09:35:31.681Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6408lb # [ 139.659242] niks3-server[1047]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6409lb # [ 139.672168] niks3-server[1047]: 2026/09/20 09:35:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6410worker1 # [ 139.706513] nix-grpc-daemon[1577]: time=2026-09-20T09:35:31.713Z level=INFO msg="Uploading 1 narinfos"6411lb # [ 139.677112] niks3-server[1047]: 2026/09/20 09:35:31 INFO Signed narinfos id=48 count=16412lb # [ 139.683367] niks3-server[1047]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6413lb # [ 139.693887] niks3-server[1047]: 2026/09/20 09:35:31 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6414worker1 # [ 139.728840] nix-grpc-daemon[1577]: time=2026-09-20T09:35:31.736Z level=INFO msg="Upload complete. (425ms)"6415lb # [ 139.698968] niks3-server[1047]: 2026/09/20 09:35:31 INFO Completed upload id=486416worker1 # [ 139.731564] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=5046417lb # [ 139.704136] niks3-server[1047]: 2026/09/20 09:35:31 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6418worker1 # [ 139.738080] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486419worker1 # [ 139.746248] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6420worker1 # [ 139.749110] nix-daemon[844]: reaped child process 1795, status = succeeded6421worker1 # [ 139.754737] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6422worker1 # [ 140.566378] nix-daemon[844]: reaped child process 1809, status = succeeded6423worker1 # [ 140.720086] nix-daemon[844]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)6424worker1 # [ 140.731551] nix-daemon[844]: reaped child process 1827, status = succeeded6425lb # [ 140.720169] niks3-server[1047]: 2026/09/20 09:35:32 INFO Received uploads request method=POST path=/api/pending_closures6426worker1 # [ 140.757995] nix-grpc-daemon[1577]: time=2026-09-20T09:35:32.765Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6427worker1 # [ 140.759536] nix-grpc-daemon[1577]: time=2026-09-20T09:35:32.766Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6428lb # [ 140.750511] niks3-server[1047]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6429lb # [ 140.753098] niks3-server[1047]: 2026/09/20 09:35:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6430lb # [ 140.755459] niks3-server[1047]: 2026/09/20 09:35:32 INFO Signed narinfos id=49 count=16431worker1 # [ 140.785416] nix-grpc-daemon[1577]: time=2026-09-20T09:35:32.792Z level=INFO msg="Uploading 1 narinfos"6432lb # [ 140.759243] niks3-server[1047]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6433lb # [ 140.761857] niks3-server[1047]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6434lb # [ 140.765125] niks3-server[1047]: 2026/09/20 09:35:32 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6435lb # [ 140.768771] niks3-server[1047]: 2026/09/20 09:35:32 INFO Completed upload id=496436worker1 # [ 140.798654] nix-grpc-daemon[1577]: time=2026-09-20T09:35:32.805Z level=INFO msg="Upload complete. (239ms)"6437lb # [ 140.770267] niks3-server[1047]: 2026/09/20 09:35:32 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6438worker1 # [ 140.800975] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=2 outputs=16439worker1 # [ 140.806332] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:32Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16440worker1 # [ 140.808056] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486441worker1 # [ 140.811956] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:32Z level=debug event=want client=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06442worker1 # [ 140.816811] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486443worker1 # [ 140.819062] nix-daemon[844]: reaped child process 1807, status = succeeded6444worker1 # [ 140.822505] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486445worker1 # [ 140.826277] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6446worker1 # [ 140.986079] nix-daemon[844]: accepted connection from pid 1833, user nix-grpc-daemon (trusted)6447worker1 # [ 140.997844] nix-daemon[844]: reaped child process 1836, status = succeeded6448worker1 # [ 141.135327] nix-daemon[844]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6449worker1 # [ 141.146781] nix-daemon[844]: reaped child process 1841, status = succeeded6450lb # [ 141.132335] niks3-server[1047]: 2026/09/20 09:35:33 INFO Received uploads request method=POST path=/api/pending_closures6451worker1 # [ 141.169986] nix-grpc-daemon[1577]: time=2026-09-20T09:35:33.177Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6452worker1 # [ 141.171534] nix-grpc-daemon[1577]: time=2026-09-20T09:35:33.178Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6453lb # [ 141.159891] niks3-server[1047]: 2026/09/20 09:35:33 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6454lb # [ 141.171795] niks3-server[1047]: 2026/09/20 09:35:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6455lb # [ 141.176117] niks3-server[1047]: 2026/09/20 09:35:33 INFO Signed narinfos id=50 count=16456worker1 # [ 141.210056] nix-grpc-daemon[1577]: time=2026-09-20T09:35:33.217Z level=INFO msg="Uploading 1 narinfos"6457lb # [ 141.182060] niks3-server[1047]: 2026/09/20 09:35:33 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6458lb # [ 141.192609] niks3-server[1047]: 2026/09/20 09:35:33 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6459lb # [ 141.198599] niks3-server[1047]: 2026/09/20 09:35:33 INFO Completed upload id=506460worker1 # [ 141.228618] nix-grpc-daemon[1577]: time=2026-09-20T09:35:33.235Z level=INFO msg="Upload complete. (396ms)"6461lb # [ 141.202424] niks3-server[1047]: 2026/09/20 09:35:33 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6462worker1 # [ 141.232199] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=5046463worker1 # [ 141.235393] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486464worker1 # [ 141.241556] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6465worker1 # [ 141.245879] nix-daemon[844]: reaped child process 1831, status = succeeded6466worker1 # [ 141.251483] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6467worker1 # [ 142.464538] nix-daemon[844]: reaped child process 1845, status = succeeded6468worker1 # [ 142.600140] nix-daemon[844]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)6469worker1 # [ 142.611677] nix-daemon[844]: reaped child process 1863, status = succeeded6470lb # [ 142.598789] niks3-server[1047]: 2026/09/20 09:35:34 INFO Received uploads request method=POST path=/api/pending_closures6471worker1 # [ 142.633261] nix-grpc-daemon[1577]: time=2026-09-20T09:35:34.640Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6472worker1 # [ 142.634738] nix-grpc-daemon[1577]: time=2026-09-20T09:35:34.642Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6473lb # [ 142.618249] niks3-server[1047]: 2026/09/20 09:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6474worker1 # [ 142.652427] nix-grpc-daemon[1577]: time=2026-09-20T09:35:34.659Z level=INFO msg="Uploading 1 narinfos"6475lb # [ 142.623795] niks3-server[1047]: 2026/09/20 09:35:34 INFO Signed narinfos id=51 count=16476lb # [ 142.625856] niks3-server[1047]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6477lb # [ 142.628386] niks3-server[1047]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6478lb # [ 142.637768] niks3-server[1047]: 2026/09/20 09:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6479lb # [ 142.640597] niks3-server[1047]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6480lb # [ 142.642971] niks3-server[1047]: 2026/09/20 09:35:34 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6481lb # [ 142.645220] niks3-server[1047]: 2026/09/20 09:35:34 INFO Completed upload id=516482worker1 # [ 142.676206] nix-grpc-daemon[1577]: time=2026-09-20T09:35:34.683Z level=INFO msg="Upload complete. (219ms)"6483worker1 # [ 142.680344] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=3 outputs=16484worker1 # [ 142.683696] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:34Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16485worker1 # [ 142.685623] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486486worker1 # [ 142.689259] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:34Z level=debug event=want client=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06487worker1 # [ 142.694333] nix-daemon[844]: reaped child process 1843, status = succeeded6488worker1 # [ 142.695555] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486489worker1 # [ 142.701175] nix-grpc-daemon[1577]: time=2026-09-20T09:35:34.708Z level=INFO msg="All 1 paths already cached"6490worker1 # [ 142.704911] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486491worker1 # [ 142.709691] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6492worker1 # [ 142.855855] nix-daemon[844]: accepted connection from pid 1869, user nix-grpc-daemon (trusted)6493worker1 # [ 142.867508] nix-daemon[844]: reaped child process 1872, status = succeeded6494worker1 # [ 143.025866] nix-daemon[844]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6495worker1 # [ 143.037248] nix-daemon[844]: reaped child process 1877, status = succeeded6496lb # [ 143.023890] niks3-server[1047]: 2026/09/20 09:35:35 INFO Received uploads request method=POST path=/api/pending_closures6497worker1 # [ 143.057935] nix-grpc-daemon[1577]: time=2026-09-20T09:35:35.065Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6498worker1 # [ 143.059441] nix-grpc-daemon[1577]: time=2026-09-20T09:35:35.066Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6499lb # [ 143.041055] niks3-server[1047]: 2026/09/20 09:35:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6500lb # [ 143.043151] niks3-server[1047]: 2026/09/20 09:35:35 INFO Signed narinfos id=52 count=16501worker1 # [ 143.072967] nix-grpc-daemon[1577]: time=2026-09-20T09:35:35.080Z level=INFO msg="Uploading 1 narinfos"6502lb # [ 143.044181] niks3-server[1047]: 2026/09/20 09:35:35 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6503lb # [ 143.049041] niks3-server[1047]: 2026/09/20 09:35:35 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6504lb # [ 143.052635] niks3-server[1047]: 2026/09/20 09:35:35 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6505lb # [ 143.057953] niks3-server[1047]: 2026/09/20 09:35:35 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6506lb # [ 143.061095] niks3-server[1047]: 2026/09/20 09:35:35 INFO Completed upload id=526507worker1 # [ 143.090897] nix-grpc-daemon[1577]: time=2026-09-20T09:35:35.098Z level=INFO msg="Upload complete. (376ms)"6508worker1 # [ 143.092376] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=6246509worker1 # [ 143.098375] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486510worker1 # [ 143.102453] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6511worker1 # [ 143.104748] nix-daemon[844]: reaped child process 1867, status = succeeded6512worker1 # [ 143.109983] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6513worker1 # [ 144.526802] nix-daemon[844]: reaped child process 1881, status = succeeded6514worker1 # [ 144.678406] nix-daemon[844]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)6515worker1 # [ 144.689711] nix-daemon[844]: reaped child process 1899, status = succeeded6516lb # [ 144.676985] niks3-server[1047]: 2026/09/20 09:35:36 INFO Received uploads request method=POST path=/api/pending_closures6517worker1 # [ 144.718300] nix-grpc-daemon[1577]: time=2026-09-20T09:35:36.724Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6518worker1 # [ 144.719649] nix-grpc-daemon[1577]: time=2026-09-20T09:35:36.724Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6519lb # [ 144.731051] niks3-server[1047]: 2026/09/20 09:35:36 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6520lb # [ 144.736452] niks3-server[1047]: 2026/09/20 09:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6521lb # [ 144.739616] niks3-server[1047]: 2026/09/20 09:35:36 INFO Signed narinfos id=53 count=16522worker1 # [ 144.772442] nix-grpc-daemon[1577]: time=2026-09-20T09:35:36.779Z level=INFO msg="Uploading 1 narinfos"6523lb # [ 144.742300] niks3-server[1047]: 2026/09/20 09:35:36 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6524lb # [ 144.748236] niks3-server[1047]: 2026/09/20 09:35:36 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6525lb # [ 144.756752] niks3-server[1047]: 2026/09/20 09:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6526lb # [ 144.762205] niks3-server[1047]: 2026/09/20 09:35:36 INFO Completed upload id=536527lb # [ 144.763465] niks3-server[1047]: 2026/09/20 09:35:36 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6528worker1 # [ 144.792442] nix-grpc-daemon[1577]: time=2026-09-20T09:35:36.799Z level=INFO msg="Upload complete. (274ms)"6529worker1 # [ 144.801869] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=4 outputs=16530worker1 # [ 144.811136] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:36Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16531worker1 # [ 144.814882] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486532worker1 # [ 144.818215] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:36Z level=debug event=want client=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06533worker1 # [ 144.827529] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486534worker1 # [ 144.830742] nix-daemon[844]: reaped child process 1879, status = succeeded6535worker1 # [ 144.835043] nix-grpc-daemon[1577]: time=2026-09-20T09:35:36.842Z level=INFO msg="All 1 paths already cached"6536worker1 # [ 144.839259] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486537worker1 # [ 144.844125] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6538worker1 # [ 144.992447] nix-daemon[844]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6539worker1 # [ 145.006538] nix-daemon[844]: reaped child process 1908, status = succeeded6540worker1 # [ 145.150993] nix-daemon[844]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6541worker1 # [ 145.163334] nix-daemon[844]: reaped child process 1913, status = succeeded6542lb # [ 145.144540] niks3-server[1047]: 2026/09/20 09:35:37 INFO Received uploads request method=POST path=/api/pending_closures6543worker1 # [ 145.181909] nix-grpc-daemon[1577]: time=2026-09-20T09:35:37.189Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6544worker1 # [ 145.183713] nix-grpc-daemon[1577]: time=2026-09-20T09:35:37.190Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6545lb # [ 145.171685] niks3-server[1047]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6546lb # [ 145.180585] niks3-server[1047]: 2026/09/20 09:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6547worker1 # [ 145.217243] nix-grpc-daemon[1577]: time=2026-09-20T09:35:37.224Z level=INFO msg="Uploading 1 narinfos"6548lb # [ 145.185916] niks3-server[1047]: 2026/09/20 09:35:37 INFO Signed narinfos id=54 count=16549lb # [ 145.194581] niks3-server[1047]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6550lb # [ 145.207095] niks3-server[1047]: 2026/09/20 09:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6551lb # [ 145.211305] niks3-server[1047]: 2026/09/20 09:35:37 INFO Completed upload id=546552worker1 # [ 145.243695] nix-grpc-daemon[1577]: time=2026-09-20T09:35:37.250Z level=INFO msg="Upload complete. (394ms)"6553lb # [ 145.214752] niks3-server[1047]: 2026/09/20 09:35:37 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6554worker1 # [ 145.246619] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=6246555worker1 # [ 145.251243] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486556worker1 # [ 145.261104] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6557worker1 # [ 145.265046] nix-daemon[844]: reaped child process 1903, status = succeeded6558worker1 # [ 145.269903] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6559worker1 # [ 146.082344] nix-daemon[844]: reaped child process 1917, status = succeeded6560worker1 # [ 146.210481] nix-daemon[844]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6561worker1 # [ 146.222118] nix-daemon[844]: reaped child process 1935, status = succeeded6562lb # [ 146.205482] niks3-server[1047]: 2026/09/20 09:35:38 INFO Received uploads request method=POST path=/api/pending_closures6563worker1 # [ 146.243147] nix-grpc-daemon[1577]: time=2026-09-20T09:35:38.250Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6564worker1 # [ 146.244644] nix-grpc-daemon[1577]: time=2026-09-20T09:35:38.251Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6565lb # [ 146.234638] niks3-server[1047]: 2026/09/20 09:35:38 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6566lb # [ 146.251145] niks3-server[1047]: 2026/09/20 09:35:38 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6567lb # [ 146.260683] niks3-server[1047]: 2026/09/20 09:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6568worker1 # [ 146.295338] nix-grpc-daemon[1577]: time=2026-09-20T09:35:38.302Z level=INFO msg="Uploading 1 narinfos"6569lb # [ 146.265270] niks3-server[1047]: 2026/09/20 09:35:38 INFO Signed narinfos id=55 count=16570lb # [ 146.268703] niks3-server[1047]: 2026/09/20 09:35:38 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6571lb # [ 146.279118] niks3-server[1047]: 2026/09/20 09:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6572lb # [ 146.285436] niks3-server[1047]: 2026/09/20 09:35:38 INFO Completed upload id=556573lb # [ 146.287172] niks3-server[1047]: 2026/09/20 09:35:38 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6574worker1 # [ 146.315517] nix-grpc-daemon[1577]: time=2026-09-20T09:35:38.322Z level=INFO msg="Upload complete. (239ms)"6575worker1 # [ 146.322768] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=5 outputs=16576worker1 # [ 146.329248] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:38Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16577worker1 # [ 146.337948] nix-daemon[844]: reaped child process 1915, status = succeeded6578worker1 # [ 146.366149] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486579worker1 # [ 146.373600] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:38Z level=debug event=want client=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06580worker1 # [ 146.381822] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486581worker1 # [ 146.392570] nix-grpc-daemon[1577]: time=2026-09-20T09:35:38.399Z level=INFO msg="All 1 paths already cached"6582worker1 # [ 146.398837] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486583worker1 # [ 146.405963] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6584worker1 # [ 146.554371] nix-daemon[844]: accepted connection from pid 1941, user nix-grpc-daemon (trusted)6585worker1 # [ 146.565782] nix-daemon[844]: reaped child process 1944, status = succeeded6586worker1 # [ 146.731138] nix-daemon[844]: accepted connection from pid 1946, user nix-grpc-daemon (trusted)6587worker1 # [ 146.742624] nix-daemon[844]: reaped child process 1949, status = succeeded6588lb # [ 146.730506] niks3-server[1047]: 2026/09/20 09:35:38 INFO Received uploads request method=POST path=/api/pending_closures6589worker1 # [ 146.768413] nix-grpc-daemon[1577]: time=2026-09-20T09:35:38.775Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6590worker1 # [ 146.769891] nix-grpc-daemon[1577]: time=2026-09-20T09:35:38.777Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6591lb # [ 146.762183] niks3-server[1047]: 2026/09/20 09:35:38 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6592lb # [ 146.769884] niks3-server[1047]: 2026/09/20 09:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6593lb # [ 146.776435] niks3-server[1047]: 2026/09/20 09:35:38 INFO Signed narinfos id=56 count=16594worker1 # [ 146.809602] nix-grpc-daemon[1577]: time=2026-09-20T09:35:38.816Z level=INFO msg="Uploading 1 narinfos"6595lb # [ 146.781627] niks3-server[1047]: 2026/09/20 09:35:38 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6596lb # [ 146.798886] niks3-server[1047]: 2026/09/20 09:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6597lb # [ 146.805324] niks3-server[1047]: 2026/09/20 09:35:38 INFO Completed upload id=566598worker1 # [ 146.835193] nix-grpc-daemon[1577]: time=2026-09-20T09:35:38.842Z level=INFO msg="Upload complete. (418ms)"6599lb # [ 146.807787] niks3-server[1047]: 2026/09/20 09:35:38 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6600worker1 # [ 146.838940] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=6246601worker1 # [ 146.844283] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486602worker1 # [ 146.854133] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6603worker1 # [ 146.858837] nix-daemon[844]: reaped child process 1939, status = succeeded6604worker1 # [ 146.863923] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6605worker1 # [ 148.088726] nix-daemon[844]: reaped child process 1953, status = succeeded6606worker1 # [ 148.248240] nix-daemon[844]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)6607worker1 # [ 148.259599] nix-daemon[844]: reaped child process 1971, status = succeeded6608lb # [ 148.246878] niks3-server[1047]: 2026/09/20 09:35:40 INFO Received uploads request method=POST path=/api/pending_closures6609worker1 # [ 148.284715] nix-grpc-daemon[1577]: time=2026-09-20T09:35:40.291Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6610worker1 # [ 148.286256] nix-grpc-daemon[1577]: time=2026-09-20T09:35:40.293Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6611lb # [ 148.281526] niks3-server[1047]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6612lb # [ 148.292483] niks3-server[1047]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6613lb # [ 148.298376] niks3-server[1047]: 2026/09/20 09:35:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6614lb # [ 148.303635] niks3-server[1047]: 2026/09/20 09:35:40 INFO Signed narinfos id=57 count=16615worker1 # [ 148.336491] nix-grpc-daemon[1577]: time=2026-09-20T09:35:40.343Z level=INFO msg="Uploading 1 narinfos"6616lb # [ 148.310744] niks3-server[1047]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6617lb # [ 148.320593] niks3-server[1047]: 2026/09/20 09:35:40 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6618lb # [ 148.326455] niks3-server[1047]: 2026/09/20 09:35:40 INFO Completed upload id=576619lb # [ 148.328544] niks3-server[1047]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6620worker1 # [ 148.357128] nix-grpc-daemon[1577]: time=2026-09-20T09:35:40.363Z level=INFO msg="Upload complete. (284ms)"6621worker1 # [ 148.366884] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=6 outputs=16622worker1 # [ 148.375105] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:40Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16623worker1 # [ 148.379219] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486624worker1 # [ 148.382445] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:40Z level=debug event=want client=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06625worker1 # [ 148.392510] nix-daemon[844]: reaped child process 1951, status = succeeded6626worker1 # [ 148.418492] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486627worker1 # [ 148.431213] nix-grpc-daemon[1577]: time=2026-09-20T09:35:40.438Z level=INFO msg="All 2 paths already cached"6628worker1 # [ 148.438990] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486629worker1 # [ 148.448143] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6630worker1 # [ 148.611426] nix-daemon[844]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)6631worker1 # [ 148.623235] nix-daemon[844]: reaped child process 1980, status = succeeded6632worker1 # [ 148.782980] nix-daemon[844]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6633worker1 # [ 148.794772] nix-daemon[844]: reaped child process 1985, status = succeeded6634lb # [ 148.780241] niks3-server[1047]: 2026/09/20 09:35:40 INFO Received uploads request method=POST path=/api/pending_closures6635worker1 # [ 148.818421] nix-grpc-daemon[1577]: time=2026-09-20T09:35:40.825Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6636worker1 # [ 148.820288] nix-grpc-daemon[1577]: time=2026-09-20T09:35:40.827Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"6637lb # [ 148.809993] niks3-server[1047]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst6638lb # [ 148.818379] niks3-server[1047]: 2026/09/20 09:35:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6639lb # [ 148.822913] niks3-server[1047]: 2026/09/20 09:35:40 INFO Signed narinfos id=58 count=16640worker1 # [ 148.855737] nix-grpc-daemon[1577]: time=2026-09-20T09:35:40.862Z level=INFO msg="Uploading 1 narinfos"6641lb # [ 148.834731] niks3-server[1047]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls6642lb # [ 148.841688] niks3-server[1047]: 2026/09/20 09:35:40 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6643lb # [ 148.848231] niks3-server[1047]: 2026/09/20 09:35:40 INFO Completed upload id=586644worker1 # [ 148.880689] nix-grpc-daemon[1577]: time=2026-09-20T09:35:40.887Z level=INFO msg="Upload complete. (422ms)"6645lb # [ 148.851295] niks3-server[1047]: 2026/09/20 09:35:40 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo6646worker1 # [ 148.884460] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=7526647worker1 # [ 148.889259] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486648worker1 # [ 148.899105] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6649worker1 # [ 148.906117] nix-daemon[844]: reaped child process 1975, status = succeeded6650worker1 # [ 148.910423] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6651worker1 # [ 149.723545] nix-daemon[844]: reaped child process 1989, status = succeeded6652worker1 # [ 149.870676] nix-daemon[844]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)6653worker1 # [ 149.882352] nix-daemon[844]: reaped child process 2007, status = succeeded6654lb # [ 149.871264] niks3-server[1047]: 2026/09/20 09:35:41 INFO Received uploads request method=POST path=/api/pending_closures6655worker1 # [ 149.909474] nix-grpc-daemon[1577]: time=2026-09-20T09:35:41.915Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6656worker1 # [ 149.910837] nix-grpc-daemon[1577]: time=2026-09-20T09:35:41.915Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"6657lb # [ 149.904844] niks3-server[1047]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv6658lb # [ 149.911385] niks3-server[1047]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst6659lb # [ 149.926107] niks3-server[1047]: 2026/09/20 09:35:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6660worker1 # [ 149.962652] nix-grpc-daemon[1577]: time=2026-09-20T09:35:41.969Z level=INFO msg="Uploading 1 narinfos"6661lb # [ 149.932287] niks3-server[1047]: 2026/09/20 09:35:41 INFO Signed narinfos id=59 count=16662lb # [ 149.936603] niks3-server[1047]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls6663lb # [ 149.944876] niks3-server[1047]: 2026/09/20 09:35:41 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6664lb # [ 149.947766] niks3-server[1047]: 2026/09/20 09:35:41 INFO Completed upload id=596665worker1 # [ 149.977684] nix-grpc-daemon[1577]: time=2026-09-20T09:35:41.984Z level=INFO msg="Upload complete. (261ms)"6666lb # [ 149.950468] niks3-server[1047]: 2026/09/20 09:35:41 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo6667worker1 # [ 149.980733] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=7 outputs=16668worker1 # [ 149.983105] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:41Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=16669worker1 # [ 149.986117] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486670worker1 # [ 149.991989] nix-daemon[844]: reaped child process 1987, status = succeeded6671worker1 # [ 149.994422] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486672worker1 # [ 150.000673] nix-grpc-daemon[1577]: time=2026-09-20T09:35:42.007Z level=INFO msg="All 2 paths already cached"6673worker1 # [ 150.004571] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486674worker1 # [ 150.009290] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6675worker1 # [ 150.158862] nix-daemon[844]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)6676worker1 # [ 150.170527] nix-daemon[844]: reaped child process 2014, status = succeeded6677worker1 # [ 150.339605] nix-daemon[844]: accepted connection from pid 2016, user nix-grpc-daemon (trusted)6678worker1 # [ 150.350974] nix-daemon[844]: reaped child process 2019, status = succeeded6679lb # [ 150.338749] niks3-server[1047]: 2026/09/20 09:35:42 INFO Received uploads request method=POST path=/api/pending_closures6680worker1 # [ 150.376902] nix-grpc-daemon[1577]: time=2026-09-20T09:35:42.383Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6681worker1 # [ 150.378363] nix-grpc-daemon[1577]: time=2026-09-20T09:35:42.383Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"6682lb # [ 150.364616] niks3-server[1047]: 2026/09/20 09:35:42 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst6683lb # [ 150.376195] niks3-server[1047]: 2026/09/20 09:35:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6684lb # [ 150.382833] niks3-server[1047]: 2026/09/20 09:35:42 INFO Signed narinfos id=60 count=16685worker1 # [ 150.415746] nix-grpc-daemon[1577]: time=2026-09-20T09:35:42.422Z level=INFO msg="Uploading 1 narinfos"6686lb # [ 150.386167] niks3-server[1047]: 2026/09/20 09:35:42 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls6687lb # [ 150.398479] niks3-server[1047]: 2026/09/20 09:35:42 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6688worker1 # [ 150.433209] nix-grpc-daemon[1577]: time=2026-09-20T09:35:42.440Z level=INFO msg="Upload complete. (419ms)"6689lb # [ 150.403420] niks3-server[1047]: 2026/09/20 09:35:42 INFO Completed upload id=606690worker1 # [ 150.435126] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=7526691lb # [ 150.408414] niks3-server[1047]: 2026/09/20 09:35:42 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo6692worker1 # [ 150.442624] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486693worker1 # [ 150.449261] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6694worker1 # [ 150.451497] nix-daemon[844]: reaped child process 2009, status = succeeded6695worker1 # [ 150.456779] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6696worker1 # [ 151.881752] nix-daemon[844]: reaped child process 2023, status = succeeded6697worker1 # [ 152.044336] nix-daemon[844]: accepted connection from pid 2038, user nix-grpc-daemon (trusted)6698worker1 # [ 152.054531] nix-daemon[844]: reaped child process 2041, status = succeeded6699lb # [ 152.042421] niks3-server[1047]: 2026/09/20 09:35:44 INFO Received uploads request method=POST path=/api/pending_closures6700worker1 # [ 152.080528] nix-grpc-daemon[1577]: time=2026-09-20T09:35:44.087Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6701worker1 # [ 152.082147] nix-grpc-daemon[1577]: time=2026-09-20T09:35:44.089Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"6702lb # [ 152.069603] niks3-server[1047]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst6703lb # [ 152.073089] niks3-server[1047]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv6704lb # [ 152.075936] niks3-server[1047]: 2026/09/20 09:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6705lb # [ 152.078582] niks3-server[1047]: 2026/09/20 09:35:44 INFO Signed narinfos id=61 count=16706worker1 # [ 152.109819] nix-grpc-daemon[1577]: time=2026-09-20T09:35:44.116Z level=INFO msg="Uploading 1 narinfos"6707lb # [ 152.081802] niks3-server[1047]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls6708lb # [ 152.086054] niks3-server[1047]: 2026/09/20 09:35:44 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6709lb # [ 152.089772] niks3-server[1047]: 2026/09/20 09:35:44 INFO Completed upload id=616710worker1 # [ 152.120910] nix-grpc-daemon[1577]: time=2026-09-20T09:35:44.128Z level=INFO msg="Upload complete. (256ms)"6711lb # [ 152.091615] niks3-server[1047]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo6712worker1 # [ 152.124129] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=8 outputs=16713worker1 # [ 152.127572] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:44Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=16714worker1 # [ 152.129289] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486715worker1 # [ 152.135288] nix-daemon[844]: reaped child process 2021, status = succeeded6716worker1 # [ 152.136532] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486717worker1 # [ 152.142199] nix-grpc-daemon[1577]: time=2026-09-20T09:35:44.149Z level=INFO msg="All 2 paths already cached"6718worker1 # [ 152.146033] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:546486719worker1 # [ 152.150729] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6720worker1 # [ 152.296209] nix-daemon[844]: accepted connection from pid 2045, user nix-grpc-daemon (trusted)6721worker1 # [ 152.307842] nix-daemon[844]: reaped child process 2048, status = succeeded6722worker1 # [ 152.453057] nix-daemon[844]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)6723worker1 # [ 152.464276] nix-daemon[844]: reaped child process 2053, status = succeeded6724lb # [ 152.447419] niks3-server[1047]: 2026/09/20 09:35:44 INFO Received uploads request method=POST path=/api/pending_closures6725worker1 # [ 152.481835] nix-grpc-daemon[1577]: time=2026-09-20T09:35:44.488Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6726worker1 # [ 152.483203] nix-grpc-daemon[1577]: time=2026-09-20T09:35:44.488Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"6727lb # [ 152.460624] niks3-server[1047]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst6728lb # [ 152.462815] niks3-server[1047]: 2026/09/20 09:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6729lb # [ 152.464864] niks3-server[1047]: 2026/09/20 09:35:44 INFO Signed narinfos id=62 count=16730worker1 # [ 152.494790] nix-grpc-daemon[1577]: time=2026-09-20T09:35:44.501Z level=INFO msg="Uploading 1 narinfos"6731lb # [ 152.468670] niks3-server[1047]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls6732lb # [ 152.472248] niks3-server[1047]: 2026/09/20 09:35:44 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6733lb # [ 152.475227] niks3-server[1047]: 2026/09/20 09:35:44 INFO Completed upload id=626734worker1 # [ 152.505122] nix-grpc-daemon[1577]: time=2026-09-20T09:35:44.512Z level=INFO msg="Upload complete. (349ms)"6735lb # [ 152.477842] niks3-server[1047]: 2026/09/20 09:35:44 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo6736worker1 # [ 152.507709] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=0 paths=1 nar_bytes_in=7526737worker1 # [ 152.510093] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:546486738worker1 # [ 152.516282] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6739worker1 # [ 152.518525] nix-daemon[844]: reaped child process 2043, status = succeeded6740worker1 # [ 152.525044] nix-daemon[844]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)6741worker1 # [ 153.548692] nix-daemon[844]: reaped child process 2057, status = succeeded6742worker1 # [ 153.724074] nix-daemon[844]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)6743worker1 # [ 153.735692] nix-daemon[844]: reaped child process 2075, status = succeeded6744lb # [ 153.726474] niks3-server[1047]: 2026/09/20 09:35:45 INFO Received uploads request method=POST path=/api/pending_closures6745worker1 # [ 153.764859] nix-grpc-daemon[1577]: time=2026-09-20T09:35:45.771Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6746worker1 # [ 153.766329] nix-grpc-daemon[1577]: time=2026-09-20T09:35:45.773Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"6747lb # [ 153.765109] niks3-server[1047]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst6748lb # [ 153.773205] niks3-server[1047]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv6749lb # [ 153.783422] niks3-server[1047]: 2026/09/20 09:35:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6750lb # [ 153.789499] niks3-server[1047]: 2026/09/20 09:35:45 INFO Signed narinfos id=63 count=16751worker1 # [ 153.821986] nix-grpc-daemon[1577]: time=2026-09-20T09:35:45.829Z level=INFO msg="Uploading 1 narinfos"6752lb # [ 153.792090] niks3-server[1047]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls6753lb # [ 153.802795] niks3-server[1047]: 2026/09/20 09:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6754lb # [ 153.807816] niks3-server[1047]: 2026/09/20 09:35:45 INFO Completed upload id=636755worker1 # [ 153.839079] nix-grpc-daemon[1577]: time=2026-09-20T09:35:45.846Z level=INFO msg="Upload complete. (307ms)"6756lb # [ 153.810510] niks3-server[1047]: 2026/09/20 09:35:45 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo6757worker1 # [ 153.846658] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54648 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=9 outputs=16758worker1 # [ 153.858986] nix-daemon[844]: reaped child process 2055, status = succeeded6759client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.47 seconds)6760worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1'6761worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1', in 0.07 seconds)6762client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16763worker2 # [ 154.166794] nix-grpc-daemon[1701]: ts=2026-09-20T09:35:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:605506764worker1 # [ 154.174780] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:546486765worker1 # [ 154.191891] nix-grpc-daemon[1489]: ts=2026-09-20T09:35:46Z level=debug event=want client=2 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=06766lb # [ 159.143931] envoy[928]: conn peer=[2001:db8:1::1]:38572 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27055 tx=10627 ms=172106767client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.26 seconds)6768(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.80 seconds)6769subtest: low disk drains a worker and builds go to the other6770worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6771worker1 # [ 185.115914] nix-grpc-daemon[1489]: ts=2026-09-20T09:36:17Z level=info event=unhealthy reason=min_free6772worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.95 seconds)6773worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6774worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.04 seconds)6775client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&26776worker2 # [ 185.436707] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:605506777client # don't know how to build these paths:6778client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6779worker2 # [ 185.445396] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:605506780worker1 # [ 185.467843] nix-grpc-daemon[1489]: ts=2026-09-20T09:36:17Z level=debug event=want client=3 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=06781worker1 # [ 185.469673] nix-grpc-daemon[1489]: ts=2026-09-20T09:36:17Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=16782worker2 # [ 185.471954] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:605506783worker2 # [ 185.483700] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:605506784client # copying 1 paths to 'lb:50051'...6785client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...6786worker2 # [ 185.494664] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:605506787worker2 # [ 185.503798] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6788worker2 # [ 185.522893] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:17Z level=info event=niks3_push_started pid=17576789worker2 # [ 185.851084] nix-daemon[848]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6790worker2 # [ 185.861453] nix-daemon[848]: reaped child process 1773, status = succeeded6791worker2 # [ 186.015203] nix-daemon[848]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)6792worker2 # [ 186.026389] nix-daemon[848]: reaped child process 1778, status = succeeded6793lb # [ 186.015074] niks3-server[1047]: 2026/09/20 09:36:18 INFO Received uploads request method=POST path=/api/pending_closures6794worker2 # [ 186.052825] nix-grpc-daemon[1757]: time=2026-09-20T09:36:18.128Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6795worker2 # [ 186.056060] nix-grpc-daemon[1757]: time=2026-09-20T09:36:18.130Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"6796lb # [ 186.054340] niks3-server[1047]: 2026/09/20 09:36:18 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst6797lb # [ 186.062964] niks3-server[1047]: 2026/09/20 09:36:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6798worker2 # [ 186.099453] nix-grpc-daemon[1757]: time=2026-09-20T09:36:18.175Z level=INFO msg="Uploading 1 narinfos"6799lb # [ 186.072065] niks3-server[1047]: 2026/09/20 09:36:18 INFO Signed narinfos id=64 count=16800lb # [ 186.080862] niks3-server[1047]: 2026/09/20 09:36:18 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls6801lb # [ 186.094512] niks3-server[1047]: 2026/09/20 09:36:18 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6802lb # [ 186.098962] niks3-server[1047]: 2026/09/20 09:36:18 INFO Completed upload id=646803worker2 # [ 186.126456] nix-grpc-daemon[1757]: time=2026-09-20T09:36:18.202Z level=INFO msg="Upload complete. (422ms)"6804lb # [ 186.100774] niks3-server[1047]: 2026/09/20 09:36:18 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo6805worker2 # [ 186.129070] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60550 duration_s=0 paths=1 nar_bytes_in=5126806worker2 # [ 186.132921] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:605506807worker2 # [ 186.139468] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6808worker2 # [ 186.141729] nix-daemon[848]: reaped child process 1755, status = succeeded6809worker2 # [ 186.146830] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6810client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...6811client # farm-a> worker2-next: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6812client # farm-a> LOG-farm-a-drain6813worker2 # [ 187.173363] nix-daemon[848]: reaped child process 1782, status = succeeded6814worker2 # [ 187.291197] nix-daemon[848]: accepted connection from pid 1797, user nix-grpc-daemon (trusted)6815worker2 # [ 187.302994] nix-daemon[848]: reaped child process 1800, status = succeeded6816lb # [ 187.286898] niks3-server[1047]: 2026/09/20 09:36:19 INFO Received uploads request method=POST path=/api/pending_closures6817worker2 # [ 187.323543] nix-grpc-daemon[1757]: time=2026-09-20T09:36:19.399Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6818worker2 # [ 187.324984] nix-grpc-daemon[1757]: time=2026-09-20T09:36:19.401Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"6819lb # [ 187.325986] niks3-server[1047]: 2026/09/20 09:36:19 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6820lb # [ 187.336680] niks3-server[1047]: 2026/09/20 09:36:19 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv6821lb # [ 187.339907] niks3-server[1047]: 2026/09/20 09:36:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6822lb # [ 187.343858] niks3-server[1047]: 2026/09/20 09:36:19 INFO Signed narinfos id=65 count=16823worker2 # [ 187.373922] nix-grpc-daemon[1757]: time=2026-09-20T09:36:19.449Z level=INFO msg="Uploading 1 narinfos"6824lb # [ 187.353474] niks3-server[1047]: 2026/09/20 09:36:19 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6825lb # [ 187.358743] niks3-server[1047]: 2026/09/20 09:36:19 INFO Completed upload id=656826worker2 # [ 187.386115] nix-grpc-daemon[1757]: time=2026-09-20T09:36:19.462Z level=INFO msg="Upload complete. (232ms)"6827lb # [ 187.360791] niks3-server[1047]: 2026/09/20 09:36:19 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls6828lb # [ 187.363474] niks3-server[1047]: 2026/09/20 09:36:19 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo6829worker2 # [ 187.390111] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60550 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=10 outputs=16830worker1 # [ 187.398984] nix-grpc-daemon[1489]: ts=2026-09-20T09:36:19Z level=debug event=want client=3 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=06831worker1 # [ 187.400920] nix-grpc-daemon[1489]: ts=2026-09-20T09:36:19Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=16832worker2 # [ 187.400052] nix-daemon[848]: reaped child process 1780, status = succeeded6833worker2 # [ 187.439701] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:605506834worker2 # [ 187.451767] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:605506835worker2 # [ 187.464206] nix-grpc-daemon[1757]: time=2026-09-20T09:36:19.540Z level=INFO msg="All 1 paths already cached"6836client # copying 1 paths to 'lb:50051'...6837client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...6838worker2 # [ 187.471749] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:605506839worker2 # [ 187.479867] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6840worker2 # [ 187.617849] nix-daemon[848]: accepted connection from pid 1804, user nix-grpc-daemon (trusted)6841worker2 # [ 187.629557] nix-daemon[848]: reaped child process 1807, status = succeeded6842worker2 # [ 187.789493] nix-daemon[848]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6843worker2 # [ 187.800920] nix-daemon[848]: reaped child process 1812, status = succeeded6844lb # [ 187.789745] niks3-server[1047]: 2026/09/20 09:36:19 INFO Received uploads request method=POST path=/api/pending_closures6845worker2 # [ 187.826574] nix-grpc-daemon[1757]: time=2026-09-20T09:36:19.902Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6846worker2 # [ 187.828069] nix-grpc-daemon[1757]: time=2026-09-20T09:36:19.904Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"6847lb # [ 187.821708] niks3-server[1047]: 2026/09/20 09:36:19 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst6848lb # [ 187.830687] niks3-server[1047]: 2026/09/20 09:36:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6849lb # [ 187.836631] niks3-server[1047]: 2026/09/20 09:36:19 INFO Signed narinfos id=66 count=16850worker2 # [ 187.866491] nix-grpc-daemon[1757]: time=2026-09-20T09:36:19.942Z level=INFO msg="Uploading 1 narinfos"6851lb # [ 187.842645] niks3-server[1047]: 2026/09/20 09:36:19 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls6852lb # [ 187.855167] niks3-server[1047]: 2026/09/20 09:36:19 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6853lb # [ 187.861667] niks3-server[1047]: 2026/09/20 09:36:19 INFO Completed upload id=666854worker2 # [ 187.891314] nix-grpc-daemon[1757]: time=2026-09-20T09:36:19.967Z level=INFO msg="Upload complete. (399ms)"6855lb # [ 187.864930] niks3-server[1047]: 2026/09/20 09:36:19 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo6856worker2 # [ 187.893907] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60550 duration_s=0 paths=1 nar_bytes_in=6246857worker2 # [ 187.899072] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:605506858worker2 # [ 187.909335] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6859worker2 # [ 187.916263] nix-daemon[848]: reaped child process 1802, status = succeeded6860worker2 # [ 187.921793] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6861client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...6862client # farm-b> worker2-next: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6863client # farm-b> LOG-farm-b-drain6864worker2 # [ 188.545875] nix-daemon[848]: reaped child process 1816, status = succeeded6865worker2 # [ 188.702711] nix-daemon[848]: accepted connection from pid 1831, user nix-grpc-daemon (trusted)6866worker2 # [ 188.713410] nix-daemon[848]: reaped child process 1834, status = succeeded6867lb # [ 188.705160] niks3-server[1047]: 2026/09/20 09:36:20 INFO Received uploads request method=POST path=/api/pending_closures6868worker2 # [ 188.741405] nix-grpc-daemon[1757]: time=2026-09-20T09:36:20.816Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6869worker2 # [ 188.742920] nix-grpc-daemon[1757]: time=2026-09-20T09:36:20.816Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"6870lb # [ 188.742309] niks3-server[1047]: 2026/09/20 09:36:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6871worker2 # [ 188.775031] nix-grpc-daemon[1757]: time=2026-09-20T09:36:20.851Z level=INFO msg="Uploading 1 narinfos"6872lb # [ 188.747922] niks3-server[1047]: 2026/09/20 09:36:20 INFO Signed narinfos id=67 count=16873lb # [ 188.758197] niks3-server[1047]: 2026/09/20 09:36:20 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst6874lb # [ 188.765482] niks3-server[1047]: 2026/09/20 09:36:20 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv6875lb # [ 188.775309] niks3-server[1047]: 2026/09/20 09:36:20 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6876lb # [ 188.780815] niks3-server[1047]: 2026/09/20 09:36:20 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls6877lb # [ 188.783384] niks3-server[1047]: 2026/09/20 09:36:20 INFO Completed upload id=676878worker2 # [ 188.810959] nix-grpc-daemon[1757]: time=2026-09-20T09:36:20.887Z level=INFO msg="Upload complete. (279ms)"6879lb # [ 188.785081] niks3-server[1047]: 2026/09/20 09:36:20 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo6880worker2 # [ 188.816495] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60550 duration_s=0 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=11 outputs=16881worker1 # [ 188.822819] nix-grpc-daemon[1489]: ts=2026-09-20T09:36:20Z level=debug event=want client=3 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=06882worker1 # [ 188.824783] nix-grpc-daemon[1489]: ts=2026-09-20T09:36:20Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=16883worker2 # [ 188.824946] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:605506884worker2 # [ 188.835480] nix-daemon[848]: reaped child process 1814, status = succeeded6885worker2 # [ 188.837183] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:605506886worker2 # [ 188.842938] nix-grpc-daemon[1757]: time=2026-09-20T09:36:20.919Z level=INFO msg="All 2 paths already cached"6887client # copying 1 paths to 'lb:50051'...6888worker2 # [ 188.847907] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:605506889client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...6890worker2 # [ 188.853066] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6891worker2 # [ 189.005705] nix-daemon[848]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6892worker2 # [ 189.017470] nix-daemon[848]: reaped child process 1841, status = succeeded6893lb # [ 189.144607] envoy[928]: conn peer=[2001:db8:1::1]:56928 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=498 ms=300376894worker2 # [ 189.179305] nix-daemon[848]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6895worker2 # [ 189.190714] nix-daemon[848]: reaped child process 1846, status = succeeded6896lb # [ 189.180072] niks3-server[1047]: 2026/09/20 09:36:21 INFO Received uploads request method=POST path=/api/pending_closures6897worker2 # [ 189.216831] nix-grpc-daemon[1757]: time=2026-09-20T09:36:21.292Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6898worker2 # [ 189.218212] nix-grpc-daemon[1757]: time=2026-09-20T09:36:21.292Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"6899lb # [ 189.205158] niks3-server[1047]: 2026/09/20 09:36:21 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst6900lb # [ 189.217609] niks3-server[1047]: 2026/09/20 09:36:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6901lb # [ 189.222984] niks3-server[1047]: 2026/09/20 09:36:21 INFO Signed narinfos id=68 count=16902worker2 # [ 189.254169] nix-grpc-daemon[1757]: time=2026-09-20T09:36:21.330Z level=INFO msg="Uploading 1 narinfos"6903lb # [ 189.227576] niks3-server[1047]: 2026/09/20 09:36:21 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls6904lb # [ 189.241115] niks3-server[1047]: 2026/09/20 09:36:21 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6905worker2 # [ 189.273936] nix-grpc-daemon[1757]: time=2026-09-20T09:36:21.350Z level=INFO msg="Upload complete. (415ms)"6906lb # [ 189.246936] niks3-server[1047]: 2026/09/20 09:36:21 INFO Completed upload id=686907worker2 # [ 189.275812] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60550 duration_s=0 paths=1 nar_bytes_in=7526908lb # [ 189.250627] niks3-server[1047]: 2026/09/20 09:36:21 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo6909worker2 # [ 189.281891] nix-daemon[848]: reaped child process 1836, status = succeeded6910worker2 # [ 189.283356] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:605506911worker2 # [ 189.287963] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6912worker2 # [ 189.293434] nix-daemon[848]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6913client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...6914client # farm-top> worker2-next: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6915client # farm-top> LOG-farm-top-drain6916worker2 # [ 190.105568] nix-daemon[848]: reaped child process 1850, status = succeeded6917worker2 # [ 190.251291] nix-daemon[848]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6918worker2 # [ 190.262919] nix-daemon[848]: reaped child process 1868, status = succeeded6919lb # [ 190.252771] niks3-server[1047]: 2026/09/20 09:36:22 INFO Received uploads request method=POST path=/api/pending_closures6920worker2 # [ 190.288929] nix-grpc-daemon[1757]: time=2026-09-20T09:36:22.364Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6921worker2 # [ 190.290469] nix-grpc-daemon[1757]: time=2026-09-20T09:36:22.366Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"6922lb # [ 190.287781] niks3-server[1047]: 2026/09/20 09:36:22 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst6923lb # [ 190.302782] niks3-server[1047]: 2026/09/20 09:36:22 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv6924lb # [ 190.306660] niks3-server[1047]: 2026/09/20 09:36:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6925lb # [ 190.311488] niks3-server[1047]: 2026/09/20 09:36:22 INFO Signed narinfos id=69 count=16926worker2 # [ 190.341757] nix-grpc-daemon[1757]: time=2026-09-20T09:36:22.417Z level=INFO msg="Uploading 1 narinfos"6927lb # [ 190.321156] niks3-server[1047]: 2026/09/20 09:36:22 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls6928lb # [ 190.326598] niks3-server[1047]: 2026/09/20 09:36:22 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6929lb # [ 190.332727] niks3-server[1047]: 2026/09/20 09:36:22 INFO Completed upload id=696930worker2 # [ 190.360300] nix-grpc-daemon[1757]: time=2026-09-20T09:36:22.436Z level=INFO msg="Upload complete. (259ms)"6931lb # [ 190.335285] niks3-server[1047]: 2026/09/20 09:36:22 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo6932worker2 # [ 190.366557] nix-grpc-daemon[1701]: ts=2026-09-20T09:36:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60550 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=12 outputs=16933worker2 # [ 190.379413] nix-daemon[848]: reaped child process 1848, status = succeeded6934client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ca.pem&client-cert=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.pem&client-key=/nix/store/hh35p8dmc1g9fk16aw3d6myi3l612ls9-farm-certs/ci.key' --eval-store auto -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain >&2, in 5.19 seconds)6935client: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath6936client: (finished: must succeed: nix eval --raw -f /nix/store/kh0qcgyv9dmrj3hks77rdgj728ar81vy-job.nix --argstr tag drain outPath, in 0.20 seconds)6937worker1: must succeed: rm /nix/.rw-store/fill6938worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.11 seconds)6939worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6940worker1 # [ 191.119303] nix-grpc-daemon[1489]: ts=2026-09-20T09:36:23Z level=info event=healthy reason=min_free6941worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)6942(finished: subtest: low disk drains a worker and builds go to the other, in 7.63 seconds)6943(finished: run the VM test script, in 192.71 seconds)6944test script finished in 192.92s6945cleanup6946kill QemuMachine (pid 49)6947client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6948kill QemuMachine (pid 50)6949lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6950kill QemuMachine (pid 51)6951worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6952kill QemuMachine (pid 52)6953worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/b5bpi6zfajzzrwwpgba2q6li3nnya4bs-python3-3.14.7/bin/python3.14)6954(finished: cleanup, in 0.91 seconds)6955additionally exposed symbols:6956 client, lb, worker1, worker2,6957 vlan1,6958 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh